[Feature] Support loading annotation from file for video SR datasets #423
Conversation
|
|
||
| with open(self.ann_file, 'r') as fin: | ||
| for line in fin: | ||
| key, max_frame_num = line.strip().split(' ') |
There was a problem hiding this comment.
use rsplit, and restrict the split num to 2.
This way, spaces in paths are supported.
There was a problem hiding this comment.
If using .split(' ', 2), for path 1/000.png 10, it will become ['path', 1/000.png', '10'] Need to join it back if we want to support spaces in paths.
There was a problem hiding this comment.
the number is num of splits. So 1 split splits to 2 items
note the "r" in rsplit
There was a problem hiding this comment.
Oh I missed rsplit. It should work then.
| with open(self.ann_file, 'r') as fin: | ||
| for line in fin: | ||
| key, max_frame_num = line.strip().split(' ') | ||
| sequence = key.split('/')[0] |
There was a problem hiding this comment.
Do not hardcode path separator.
| gt_paths = sorted( | ||
| glob.glob(osp.join(self.gt_folder, key, '*.png'))) | ||
| lq_paths = [ | ||
| osp.join(self.lq_folder, key, f'im{i}.png') |
There was a problem hiding this comment.
Unrelated to this pr, but the hardcoded png format is a headache also.
There was a problem hiding this comment.
It is the official format of Vimeo-90K, I think it is reasonable to take this format (?)
There was a problem hiding this comment.
Not really. Our codebase is for developers, who will use their own format (whatever their dataset happens uses)
There was a problem hiding this comment.
This class is for Vimeo-90K only. SRFolderVideoDataset or SRFolderMultipleGTDataset should be used for general purposes.
Actually we could support start_idx and filename_tmpl. But this way cannot support general cases. For instance, it fails when the files are named im01.png, im03.png, im05.png, im07.png, im09.png, img11.png, img13.png.
There was a problem hiding this comment.
Okay lets keep it as is for now.
Codecov Report
@@ Coverage Diff @@
## master #423 +/- ##
==========================================
+ Coverage 80.64% 80.72% +0.07%
==========================================
Files 188 189 +1
Lines 10107 10071 -36
Branches 1485 1495 +10
==========================================
- Hits 8151 8130 -21
+ Misses 1741 1723 -18
- Partials 215 218 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| data_infos = [] | ||
|
|
||
| with open(self.ann_file, 'r') as fin: | ||
| for line in fin: |
| self.folders = {} | ||
| data_infos = [] | ||
|
|
||
| with open(self.ann_file, 'r') as fin: |
| ] | ||
| gt_paths = [ | ||
| osp.join(self.gt_folder, key, f'im{i}.png') | ||
| for i in range(1, 8) |
There was a problem hiding this comment.
8 is a magic number. Can it be made into a variable at least?
There was a problem hiding this comment.
Yes it can be done. I will update this in the afternoon.
…pen-mmlab#423) * Support loading annotation from file * Update
Motivation
As raised in #395, some datasets use
globorscandirwhen loading annotations. These functions are not compatible with ceph. As a result, it is unable to train some models when ceph is used.Modifications
SRFolderMultipleGTDatasetandSRFolderVideoDataset, an argumentann_fileis added. If it is provided, annotations will be loaded from the file.SRVimeo90KMultipleGTDataset,globis removed, and the file paths are replaced by the standard Vimeo-90K file names.