Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support download dataset #14

Merged
merged 3 commits into from
Jul 11, 2024
Merged

support download dataset #14

merged 3 commits into from
Jul 11, 2024

Conversation

ganisback
Copy link
Collaborator

@ganisback ganisback commented Jul 10, 2024 •
edited
Loading

@ganisback ganisback requested review from SeanHH86 and Rader July 10, 2024 09:08
Rader
Rader reviewed Jul 10, 2024
View reviewed changes
pycsghub/file_download.py Outdated Show resolved Hide resolved
@SeanHH86
Copy link
Collaborator

对于没有指定下载路径，不同类型的是不是要放到本地不同的目录下, 以防止repo_id重复
/home/.cache/hub/csg/mode
/home/.cache/hub/csg/dataset
/home/.cache/hub/csg/space

@ganisback
Copy link
Collaborator Author

对于没有指定下载路径，不同类型的是不是要放到本地不同的目录下, 以防止repo_id重复 /home/.cache/hub/csg/mode /home/.cache/hub/csg/dataset /home/.cache/hub/csg/space

这个是follow huggingface 的规则。https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
image

@SeanHH86
Copy link
Collaborator

对于没有指定下载路径，不同类型的是不是要放到本地不同的目录下, 以防止repo_id重复 /home/.cache/hub/csg/mode /home/.cache/hub/csg/dataset /home/.cache/hub/csg/space

这个是follow huggingface 的规则。https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables image

同时从csg下载model和dataset, 不同考虑repo_id重复吗？会覆盖

@ganisback ganisback requested a review from Rader July 10, 2024 14:35
@ganisback
Copy link
Collaborator Author

对于没有指定下载路径，不同类型的是不是要放到本地不同的目录下, 以防止repo_id重复 /home/.cache/hub/csg/mode /home/.cache/hub/csg/dataset /home/.cache/hub/csg/space

这个是follow huggingface 的规则。https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables image

同时从csg下载model和dataset, 不同考虑repo_id重复吗？会覆盖

added subdir dataset

SeanHH86
SeanHH86 approved these changes Jul 11, 2024
View reviewed changes
Copy link
Collaborator

@SeanHH86 SeanHH86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@ganisback ganisback removed the request for review from Rader July 11, 2024 03:02
@ganisback ganisback self-assigned this Jul 11, 2024
@ganisback ganisback requested a review from Rader July 11, 2024 03:03
@ganisback ganisback merged commit 3b43a2c into main Jul 11, 2024
@ganisback ganisback deleted the support-download-dataset branch July 11, 2024 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SeanHH86 SeanHH86 SeanHH86 approved these changes

@Rader Rader Awaiting requested review from Rader

Assignees

@ganisback ganisback

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ganisback @SeanHH86 @Rader