Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Fix Open Images testunit to avoid error in Windows CI #7252

Merged
merged 71 commits into from
Feb 25, 2022
Merged
Show file tree
Hide file tree
Changes from 70 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
f9691af
[Feature] support openimage group of eval
BIGWangYuDong Oct 20, 2021
7372cce
[Feature] support openimage group of eval
BIGWangYuDong Oct 20, 2021
f5dab26
support openimage dataset
BIGWangYuDong Oct 21, 2021
5fbd45d
support openimage challenge dataset
BIGWangYuDong Oct 26, 2021
3e51829
fully support OpenImages-V6 and OpenImages Challenge 2019
BIGWangYuDong Nov 17, 2021
0e6e84d
Fix some logic error
BIGWangYuDong Nov 17, 2021
1b3be42
update config file
BIGWangYuDong Nov 17, 2021
e4f820b
fix get data_infos error
BIGWangYuDong Nov 18, 2021
d8b9c33
fully support OpenImages evaluation
BIGWangYuDong Nov 29, 2021
90a5f81
update OpenImages config files
BIGWangYuDong Nov 29, 2021
432be4a
[Feature] support OpenImages datasets
BIGWangYuDong Nov 29, 2021
b5ff0fd
fix bug
BIGWangYuDong Nov 29, 2021
4d43cbd
support load image metas from pipeline
BIGWangYuDong Nov 30, 2021
e02d82d
Merge branch 'dev-v2.20.0' into dev_openimage
BIGWangYuDong Nov 30, 2021
47592f9
fix bug
BIGWangYuDong Dec 1, 2021
742e71c
fix get classes logic error
BIGWangYuDong Dec 1, 2021
7190b18
update code
BIGWangYuDong Dec 7, 2021
23ef5ba
support get image metas
BIGWangYuDong Dec 21, 2021
ae95fb3
support openimags
BIGWangYuDong Dec 21, 2021
68be391
support collect image metas
BIGWangYuDong Dec 21, 2021
b6f8bfc
support Open Images
BIGWangYuDong Dec 21, 2021
2bddac4
fix openimages logic
BIGWangYuDong Dec 21, 2021
c89ee72
minor fix
BIGWangYuDong Dec 21, 2021
47b4f9f
add a new function to compute openimages tpfp
BIGWangYuDong Dec 21, 2021
7c75d45
minor fix
BIGWangYuDong Dec 21, 2021
f344d2d
Merge branch 'master' into dev_openimage
BIGWangYuDong Dec 21, 2021
1bf805b
fix ci error
BIGWangYuDong Dec 21, 2021
cc9d20d
minor fix
BIGWangYuDong Dec 22, 2021
33d6f75
fix indication
BIGWangYuDong Dec 22, 2021
d2b7d54
minor fix
BIGWangYuDong Dec 23, 2021
ee11b0a
fix returns
BIGWangYuDong Dec 23, 2021
4599dc1
fix returns
BIGWangYuDong Dec 23, 2021
3ef86eb
fix returns
BIGWangYuDong Dec 23, 2021
fc435d5
fix returns
BIGWangYuDong Dec 23, 2021
751c8f5
fix returns
BIGWangYuDong Dec 23, 2021
369d8ee
minor fix
BIGWangYuDong Dec 23, 2021
d7367aa
update readme
BIGWangYuDong Dec 23, 2021
087d2ee
support loading image level labels and fix some logic
BIGWangYuDong Dec 23, 2021
fcf3ca5
minor fix
BIGWangYuDong Dec 23, 2021
67991cd
minor fix
BIGWangYuDong Dec 23, 2021
47d6cc4
add class names
BIGWangYuDong Dec 23, 2021
a64d9a0
minor fix
BIGWangYuDong Dec 23, 2021
2e2d415
minor fix
BIGWangYuDong Dec 27, 2021
b284ed4
minor fix
BIGWangYuDong Dec 27, 2021
a147a2e
add openimages test unit
BIGWangYuDong Dec 27, 2021
0b9086f
minor fix
BIGWangYuDong Dec 27, 2021
c98c440
minor fix
BIGWangYuDong Dec 27, 2021
399e58f
fix test unit
BIGWangYuDong Dec 27, 2021
830f1d7
minor fix
BIGWangYuDong Dec 28, 2021
196284d
fix logic error
BIGWangYuDong Dec 29, 2021
0ffc753
minor fix
BIGWangYuDong Jan 5, 2022
7320454
fully support openimages
BIGWangYuDong Jan 9, 2022
02e9d6e
minor fix
BIGWangYuDong Feb 15, 2022
2bd4512
fix docstring
BIGWangYuDong Feb 15, 2022
c79d2ac
fix docstrings in readthedocs
BIGWangYuDong Feb 16, 2022
1ca725e
update get image metas script
BIGWangYuDong Feb 16, 2022
4b54e54
label_description_file -> label_file
BIGWangYuDong Feb 16, 2022
dee9f23
update openimages readme
BIGWangYuDong Feb 16, 2022
f0607d8
fix test unit
BIGWangYuDong Feb 16, 2022
4547803
fix test unit
BIGWangYuDong Feb 16, 2022
89da9da
Merge branch 'master' into dev_openimage
BIGWangYuDong Feb 17, 2022
93e7501
merge dev branch
BIGWangYuDong Feb 17, 2022
7246b7e
minor fix
BIGWangYuDong Feb 22, 2022
5adab34
update readme file
BIGWangYuDong Feb 22, 2022
a221195
Update get_image_metas.py
BIGWangYuDong Feb 22, 2022
e6b5af7
fix oid testunit to avoid some error in windows
BIGWangYuDong Feb 25, 2022
322c584
merge dev branch
BIGWangYuDong Feb 25, 2022
7ba7333
minor fix to avoid some error in windows
BIGWangYuDong Feb 25, 2022
1c539e8
minor fix
BIGWangYuDong Feb 25, 2022
9c8e84e
add comments in oid test unit
BIGWangYuDong Feb 25, 2022
d45cbb5
minor fix
BIGWangYuDong Feb 25, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions mmdet/datasets/openimages.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,10 +262,10 @@ def get_meta_from_file(self, meta_file=''):
metas = mmcv.load(meta_file)
assert len(metas) == len(self)
for i in range(len(metas)):
file_name = metas[i]['filename'].split('/')[-1]
file_name = osp.split(metas[i]['filename'])[-1]
img_info = self.data_infos[i].get('img_info', None)
if img_info is not None:
assert file_name == img_info['filename'].split('/')[-1]
assert file_name == osp.split(img_info['filename'])[-1]
else:
assert file_name == self.data_infos[i]['filename']
hw = metas[i]['ori_shape'][:2]
Expand All @@ -286,10 +286,10 @@ def get_img_shape(self, metas):
"""Set images original shape into data_infos."""
assert len(metas) == len(self)
for i in range(len(metas)):
file_name = metas[i].data['filename'].split('/')[-1]
file_name = osp.split(metas[i].data['ori_filename'])[-1]
img_info = self.data_infos[i].get('img_info', None)
if img_info is not None:
assert file_name == img_info['filename'].split('/')[-1]
assert file_name == osp.split(img_info['filename'])[-1]
else:
assert file_name == self.data_infos[i]['filename']
hw = metas[i].data['ori_shape'][:2]
Expand Down Expand Up @@ -519,7 +519,7 @@ def get_image_level_ann(self, image_level_ann_file):
img_info = self.data_infos[i].get('img_info', None)
if img_info is not None:
# for Open Images Challenges
img_id = img_info['filename'].split('/')[-1][:-4]
img_id = osp.split(img_info['filename'])[-1][:-4]
else:
# for Open Images v6
img_id = self.data_infos[i]['img_id']
Expand Down
29 changes: 20 additions & 9 deletions tests/test_data/test_datasets/test_openimages_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def _create_ids_error_oid_csv(
fake_csv_file,
):
label_description = ['/m/000002', 'Football']
with open(label_file, 'w') as f:
with open(label_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerow(label_description)

Expand All @@ -31,7 +31,9 @@ def _create_ids_error_oid_csv(
'000595fe6fee6369', 'xclick', '/m/000000', '1', '0',
'1', '0', '1', '0', '0', '1', '0', '0'
]]
with open(fake_csv_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(fake_csv_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(annotations)
Expand All @@ -41,7 +43,7 @@ def _create_oid_style_ann(label_file, csv_file, label_level_file):
label_description = [['/m/000000', 'Sports equipment'],
['/m/000001', 'Ball'], ['/m/000002', 'Football'],
['/m/000004', 'Bicycle']]
with open(label_file, 'w') as f:
with open(label_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerows(label_description)

Expand All @@ -60,15 +62,19 @@ def _create_oid_style_ann(label_file, csv_file, label_level_file):
0, 0, 0, 0
],
]
with open(csv_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(csv_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(annotations)

header = ['ImageID', 'Source', 'LabelName', 'Confidence']
annotations = [['color', 'xclick', '/m/000002', '1'],
['color', 'xclick', '/m/000004', '0']]
with open(label_level_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(label_level_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(annotations)
Expand Down Expand Up @@ -129,22 +135,27 @@ def _creat_oid_challenge_style_ann(txt_file, label_file, label_level_file):
'1 0.0333333 0.1 0.0333333 0.1 1\n',
'1 0.1 0.166667 0.1 0.166667 0\n',
]

with open(txt_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(txt_file, 'w', newline='') as f:
f.writelines(bboxes)
f.close()

label_description = [['/m/000000', 'Sports equipment', 1],
['/m/000001', 'Ball', 2],
['/m/000002', 'Football', 3],
['/m/000004', 'Bicycle', 4]]
with open(label_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(label_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerows(label_description)

header = ['ImageID', 'LabelName', 'Confidence']
annotations = [['color', '/m/000001', '1'], ['color', '/m/000000', '0']]
with open(label_level_file, 'w') as f:
# `newline=''` is used to avoid index error of out of bounds
# in Windows system
with open(label_level_file, 'w', newline='') as f:
f_csv = csv.writer(f)
f_csv.writerow(header)
f_csv.writerows(annotations)
Expand Down