How to evaluate the Classification Performance on Cityscapes?

[ShihuaHuang95]

Thanks for the awesome work.
And I have realized the FCN-score on Cityscapes mentioned in the pix2pix and cyclegan papers. Regarding to Classification Performance stated in papers, I think the main work is replacing the color pixels of generated label with trainId, so for each generated label, I use two loops (W=2048 and H=1024), however it requires huge computation resource which is unrealistic. I am wondering how can you evaluate the Classification Performance with a easy way. I am very appreciated if you will share me a copy of the code.
Best,

[junyanz]

Please see these two discussions 1 and 2 @tinghuiz

[ShihuaHuang95]

Thanks for the reply.
Yes, I have read those discussions and it is very helpful for my realization of FCN-score, however I encounter problem with Classification Performance on generated color segmentation labels that is how to convert those color label into gray label (or TrainId) easily. The evaluation code provided in pix2pix repository excludes the implementation of converting color segmentation label to gray label. And I am wondering if you could share some important details about the implementation and the related code would be better, for which I will be very appreciated.
Best,

[junyanz]

I think we just assigned it to its nearest neighbor in color space. @tinghuiz @phillipi

[junyanz]

@phillipi could you provide a script for that?

[xuwh15]

@junyanz Which parameters did you use for training FCN-8s on cityscape dataset? I what to reproduce this result on pytorch. Thank you.

[junyanz]

We do have a pretrained FCN model. Please see here for more details. @tinghuiz

[lx7555]

I want to evaluate cityscapes datasets,because I try to reproduce the fcn score results from pip2pip by using https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix code.
I try many ways.I can not get any results when I run the scripts. All parameters are zero.
I still want to know how to configure the cityspaces dataset
There are three folders with gtFine, originals image, and predictions.
Are the gtFine and predictions color or grayscale? And what is the size of these three types of pictures?
I use python2,caffe.I Is that possible that it was because i was using python2.7 but not python3 ?
@hshihua

[ZhangCZhen]

@hshihua Hello, I have the same question as you. Have you solved it? How to convert those color label into gray label (or TrainId) easily? Thanks a lot. Looking forward to your reply.

[HuXiaokai12138]

I want to evaluate cityscapes datasets,because I try to reproduce the fcn score results from pip2pip by using https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix code. I try many ways.I can not get any results when I run the scripts. All parameters are zero. I still want to know how to configure the cityspaces dataset There are three folders with gtFine, originals image, and predictions. Are the gtFine and predictions color or grayscale? And what is the size of these three types of pictures? I use python2,caffe.I Is that possible that it was because i was using python2.7 but not python3 ? @hshihua

I wonder, did you solve it? I have the same issue on my side and look forward to your reply