Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Fix bug in conversion of potsdam #1279

Merged
merged 1 commit into from
Feb 11, 2022
Merged

Conversation

SBCV
Copy link
Contributor

@SBCV SBCV commented Feb 9, 2022

Motivation

Currently, the success of the conversion of the potsdam dataset depends on the order of the files returned by os.listdir(tmp_dir)[0]. This works correctly for the first zip file, but can potentially break the processing of the second zip file.

Modification

Ensure that a new temporary directory is created for each zip file. This way each temporary directory contains only one zip file. In this case os.listdir(tmp_dir)[0] will return the correct string.

@CLAassistant
Copy link

CLAassistant commented Feb 9, 2022

CLA assistant check
All committers have signed the CLA.

@Junjun2016
Copy link
Collaborator

Hi @SBCV
Thanks for your contribution.
Please sign the CLA.

@MengzhangLI
Copy link
Contributor

Hi, thanks for your warm hearted pr.

By the way, in what circumstance would the potential error happen? In current code, all zip files are used in tempfile.TemporaryDirectory, which would be elapsed after all codes within it being executed.

Could you give certain examples or error snapshot to help me figure out your explanation? Thanks in advance.

Best,

@SBCV
Copy link
Contributor Author

SBCV commented Feb 9, 2022

Let's assume that we have a folder containing

  • 2_Ortho_RGB.zip
  • 5_Labels_all_noBoundary.zip

Now let us assume that
zipp_list = glob.glob(os.path.join(dataset_path, '*.zip'))
contains
zipp_list = ["path/to/5_Labels_all_noBoundary.zip", "path/to/2_Ortho_RGB.zip"]

Note: The ORDER of zipp_list is ESSENTIAL for the bug to appear!

After the SECOND execution of

for zipp in zipp_list:
    zip_file = zipfile.ZipFile(zipp)
    zip_file.extractall(tmp_dir)

the temporary folder contains

2_Ortho_RGB	                             
	top_potsdam_2_10_RGB.tif          <-- these files are in a subfolder
	top_potsdam_2_10_RGB.tfw
	top_potsdam_2_11_RGB.tif
	top_potsdam_2_11_RGB.tfw
	...						
top_potsdam_2_10_label_noBoundary.tif     <-- these files are at the top level of the temp dir
top_potsdam_2_11_label_noBoundary.tif
....

Because the temporary folder already contains

top_potsdam_2_10_label_noBoundary.tif
top_potsdam_2_11_label_noBoundary.tif
...

the following if case (in the second iteration) is never executed

src_path_list = glob.glob(os.path.join(tmp_dir, '*.tif'))
if not len(src_path_list):

But it should, since the tif files in 2_Ortho_RGB.zip are located in a subdirectory.

Therefore, the created dataset directory does not contain the RGB images of the potsdam dataset (only the labels)

@Junjun2016 Junjun2016 changed the title Fix bug in conversion of potsdam [Fix] Fix bug in conversion of potsdam Feb 10, 2022
@MengzhangLI
Copy link
Contributor

OK, got it. Thanks for your nice explanation!

@MeowZheng MeowZheng merged commit 0934a57 into open-mmlab:master Feb 11, 2022
bowenroom pushed a commit to bowenroom/mmsegmentation that referenced this pull request Feb 25, 2022
@SBCV SBCV deleted the potsdam branch June 10, 2022 07:07
aravind-h-v pushed a commit to aravind-h-v/mmsegmentation that referenced this pull request Mar 27, 2023
@jason102811
Copy link

sbcv,您好!您在MMSeg项目中给我们提的PR非常重要,感谢您付出私人时间帮助改进开源项目,相信很多开发者会从你的PR中受益。
我们非常期待与您继续合作,OpenMMLab专门成立了贡献者组织MMSIG,为贡献者们提供开源证书、荣誉体系和专享好礼,可通过添加微信:openmmlabwx 联系我们(请备注mmsig+GitHub id),由衷希望您能加入!
Dear sbcv,
First of all, we want to express our gratitude for your significant PR in the MMSeg project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.
We are looking forward to continuing our collaboration with you. OpenMMLab has established a special contributors' organization called MMSIG, which provides contributors with open-source certificates, a recognition system, and exclusive rewards. You can contact us by adding our WeChat(if you have WeChat): openmmlabwx, or join in our discord: https://discord.gg/qH9fysxPDW. We sincerely hope you will join us!
Best regards! @SBCV

wjkim81 pushed a commit to wjkim81/mmsegmentation that referenced this pull request Dec 3, 2023
* add smoothnet

* refactor smothnet model and filter

* Add unit test for SmoothNetFilter

* support root-index

* add docstring

* add md and update ReadMe

* update readme

* allow targets share filter

* remove devug code

* Update smoothnet_h36m.md

* fix lint

* fix unittest

Co-authored-by: ly015 <liyining0712@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants