Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Many incorrectly labeled images #8

Open
kylemcdonald opened this issue Sep 16, 2020 · 4 comments
Open

Many incorrectly labeled images #8

kylemcdonald opened this issue Sep 16, 2020 · 4 comments

Comments

@kylemcdonald
Copy link

kylemcdonald commented Sep 16, 2020

The paper reads "We further refined the annotations by training a model from the initial ground truth annotations and applying back to the dataset. We then manually re-verified the annotations for images whose annotations differ from model predictions."

But I tried to do this check myself, and I found many examples of annotations that do not match the model predictions. I do not have an estimate for how many images have this problem. But here are a few samples from the validation dataset that are annotated as "White", along with the model's predictions for these images. I would guess 2-3 of these 64 people would self-identify as "White":

Filename Predicted Race
val/1914.jpg Indian
val/5302.jpg Black
val/8590.jpg Southeast Asian
val/8963.jpg Black
val/9763.jpg Southeast Asian
val/9377.jpg Southeast Asian
val/7653.jpg Southeast Asian
val/2173.jpg Black
val/6261.jpg Indian
val/2698.jpg Black
val/322.jpg Southeast Asian
val/7489.jpg East Asian
val/2865.jpg Black
val/7394.jpg Southeast Asian
val/6331.jpg Black
val/8906.jpg East Asian
val/3797.jpg East Asian
val/5689.jpg Latino_Hispanic
val/7191.jpg East Asian
val/1312.jpg Indian
val/1399.jpg Indian
val/5204.jpg Latino_Hispanic
val/1758.jpg East Asian
val/7019.jpg Black
val/5771.jpg Southeast Asian
val/3903.jpg Latino_Hispanic
val/3204.jpg Middle Eastern
val/10556.jpg Latino_Hispanic
val/8838.jpg Southeast Asian
val/9757.jpg Latino_Hispanic
val/6590.jpg Latino_Hispanic
val/144.jpg Southeast Asian
val/10507.jpg Latino_Hispanic
val/1554.jpg Latino_Hispanic
val/7518.jpg Indian
val/5563.jpg Black
val/209.jpg Indian
val/10349.jpg Latino_Hispanic
val/8969.jpg Black
val/8475.jpg Black
val/5485.jpg Latino_Hispanic
val/4649.jpg Latino_Hispanic
val/68.jpg Southeast Asian
val/1286.jpg East Asian
val/2777.jpg Latino_Hispanic
val/397.jpg Latino_Hispanic
val/6448.jpg East Asian
val/1173.jpg Indian
val/10222.jpg Southeast Asian
val/4156.jpg Southeast Asian
val/7783.jpg Latino_Hispanic
val/10794.jpg East Asian
val/1309.jpg Latino_Hispanic
val/5787.jpg East Asian
val/9198.jpg Latino_Hispanic
val/4890.jpg Southeast Asian
val/6822.jpg Latino_Hispanic
val/8659.jpg Latino_Hispanic
val/953.jpg East Asian
val/10843.jpg Latino_Hispanic
val/8177.jpg East Asian
val/8870.jpg Latino_Hispanic
val/9399.jpg Latino_Hispanic
val/3652.jpg Latino_Hispanic
@MandoraCC
Copy link

Excuse me, may I ask you that how you get the FairFace dataset? I found that I can not find the download link of FairFace.

@cpshaheen
Copy link

Excuse me, may I ask you that how you get the FairFace dataset? I found that I can not find the download link of FairFace.

https://drive.google.com/file/d/1Z1RqRo0_JiavaZw2yzZG6WETdZQ8qX86/view

if you scroll to the bottom of the readme the link for this dataset (padding 0.25) and another are available

Best of luck on your endeavor!

@joojs
Copy link

joojs commented Feb 19, 2021

The annotations may not be perfect due to the subjective nature of the task. The paper is about perceived, not self-identified, race (and also gender, age), like most existing face attribute classification papers/datasets.

@ufousoumo
Copy link

Excuse me,may I ask you that how you get the pretrained models? I could not find the download link of the pretrained models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants