Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove logic converting empty or falsy YAML to empty dict #103912

Merged
merged 5 commits into from
Dec 5, 2023

Conversation

emontnemery
Copy link
Contributor

@emontnemery emontnemery commented Nov 13, 2023

Proposed change

Remove logic from yaml.loader.load_yaml converting "falsy" files to empty dict

Rationale: Empty files, or files containing falsy data such as the number 0, False, an empty list etc. are valid yaml and there's no reason why they should be converted to an empty dict.

It's however a common case that we do want to load configuration files which must be dictionaries and which we do allow to be empty.
Add a new helper yaml.loader.load_yaml_dict for this case which:

  • Returns an empty dict if the file is empty
  • Raises if the top level structure is not a dict
  • Is typed and guaranteed to return a dict, so the caller does not need to check this

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@emontnemery emontnemery requested a review from a team as a code owner November 13, 2023 12:25
@home-assistant home-assistant bot added cla-signed core small-pr PRs with less than 30 lines. labels Nov 13, 2023
Comment on lines 197 to 198
if result is None:
return NodeDictClass()
Copy link
Contributor Author

@emontnemery emontnemery Nov 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like quite the hack, shouldn't the consumer handle this? We could also make our own extensions, i.e. !include and friends, do this conversion and document it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kinda odd behavior in general, why would we map an empty files to an empty dict in the first place?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's to make !include and friends more user friendly by treating an empty yaml file as an empty dictionary.

I think we should handle this in config.py though, not in our yaml loader.

Let's make this PR a draft until that's been clarified.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did this indeed to ensure that the top-level data structure returend from a yaml.load is always a dictionary. Let's just say that we didn't always had config validation …

Copy link
Contributor

@jbouwh jbouwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch,
Thnx @emontnemery 👍

@emontnemery emontnemery marked this pull request as draft November 13, 2023 14:23
@emontnemery emontnemery changed the title Correct logic converting empty YAML to empty dict Remove logic converting empty YAML to empty dict Dec 4, 2023
@emontnemery emontnemery marked this pull request as ready for review December 4, 2023 16:44
homeassistant/util/yaml/loader.py Outdated Show resolved Hide resolved
@home-assistant
Copy link

home-assistant bot commented Dec 5, 2023

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@home-assistant home-assistant bot marked this pull request as draft December 5, 2023 13:00
@emontnemery emontnemery marked this pull request as ready for review December 5, 2023 15:13
@MartinHjelmare
Copy link
Member

I'm missing the reason for the refactor in the PR description.

@emontnemery emontnemery changed the title Remove logic converting empty YAML to empty dict Remove logic converting empty or falsy YAML to empty dict Dec 5, 2023
@emontnemery
Copy link
Contributor Author

I'm missing the reason for the refactor in the PR description.

PR description updated to motivate the changes

Copy link
Member

@MartinHjelmare MartinHjelmare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@MartinHjelmare MartinHjelmare merged commit 5b55c7d into dev Dec 5, 2023
53 checks passed
@MartinHjelmare MartinHjelmare deleted the yaml_loader_fix_none_to_dict branch December 5, 2023 17:08
@github-actions github-actions bot locked and limited conversation to collaborators Dec 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed core small-pr PRs with less than 30 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants