-
Notifications
You must be signed in to change notification settings - Fork 580
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not remove None values in RepoCardData serialization #2626
Do not remove None values in RepoCardData serialization #2626
Conversation
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! ruff formatting tests/test_repocard_data.py
+ the fixes made in #2621 should make the CI green ✅
Thanks for the quick reviews :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change makes sense to me 👍
Thanks for fixing this! - any plans for a new release with the fix? |
This fix will be shipped in next release but can't commit on when it'll be. In the meantime, I advice you to install |
More details in huggingface/datasets#7243 (comment) and especially huggingface/datasets#7243 (comment).
When serializing repo cards metadata, we are currently removing any none values from any dictionary, set, list or tuple. This is done since the module has been introduced in #940 and never caused problems until huggingface/datasets#7243 was reported. I do find automatically removing None values inside values to be pretty clunky as it is values set by the user and not default ones.
This PR updates this behavior to remove only top-level None values, i.e. values that are None by default and that therefore shouldn't be serialized (as not set by the users themselves). The
_remove_none
helper is still kept to remove None values from the model-index (where it's rightfully necessary), but that's all.