Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in utils.load_json_file when pyjson5 encounters errors #102

Merged
merged 1 commit into from
Jul 18, 2024

Conversation

MHendricks
Copy link
Member

Some pyjson5 exceptions include set result to None, in this case convert it into a dict and add the filename for debugging corrupt json files.

Ensure the filename is shown if an illegal character is encountered. For example a file gets converted to a bunch of null values.

Checklist

  • I have read the CONTRIBUTING.md document
  • I formatted my changes with black
  • I linted my changes with flake8
  • I have added documentation regarding my changes where necessary
  • Any pre-existing tests continue to pass
  • Additional tests were made covering my changes

Types of Changes

  • Bugfix (change that fixes an issue)
  • New Feature (change that adds functionality)
  • Documentation Update (if none of the other choices apply)

Proposed Changes

This is the original error I got when encountering a bad .host.json file using a command like hab -v dump -.

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\hab\utils.py", line 297, in load_json_file
    data = json.load(fle)
  File "src/_legacy.pyx", line 53, in pyjson5.load
  File "src/_exports.pyx", line 286, in pyjson5.decode_io
  File "src/_decoder.pyx", line 913, in pyjson5._decode_callback
  File "src/_decoder.pyx", line 815, in pyjson5._decode_all
pyjson5.Json5IllegalCharacter: ("Expected b'JSON5Value' near 1, found U+0000", None, '\x00')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python39\lib\site-packages\hab\__main__.py", line 13, in <module>
    sys.exit(hab.cli.cli(prog_name="python -m hab"))
  File "C:\Program Files\Python39\lib\site-packages\hab\cli.py", line 752, in cli
    return _cli(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\click\decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\hab\cli.py", line 651, in dump
    ret = resolver.resolve(uri)
  File "C:\Program Files\Python39\lib\site-packages\hab\resolver.py", line 311, in resolve
    return context.reduced(self, uri=uri)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\hab_base.py", line 642, in reduced
    return FlatConfig(self, resolver, uri=uri)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\flat_config.py", line 27, in __init__
    self._collect_values(self.original_node)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\hab_base.py", line 95, in _collect_values
    if getattr(self, attrname) != NotSet:
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\flat_config.py", line 164, in environment
    for version in self.versions:
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\flat_config.py", line 226, in versions
    reqs = self.resolver.resolve_requirements(distros)
  File "C:\Program Files\Python39\lib\site-packages\hab\resolver.py", line 325, in resolve_requirements
    return solver.resolve()
  File "C:\Program Files\Python39\lib\site-packages\hab\solvers.py", line 190, in resolve
    return self._resolve(self.requirements)
  File "C:\Program Files\Python39\lib\site-packages\hab\solvers.py", line 141, in _resolve
    dist = self.resolver.distros[name]
  File "C:\Program Files\Python39\lib\site-packages\hab\resolver.py", line 180, in distros
    self._distros = self.parse_distros(self.distro_paths)
  File "C:\Program Files\Python39\lib\site-packages\hab\resolver.py", line 302, in parse_distros
    DistroVersion(forest, self, path, root_paths=set((dirname,)))
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\distro_version.py", line 19, in __init__
    super().__init__(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\hab_base.py", line 65, in __init__
    self.load(filename)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\distro_version.py", line 115, in load
    data = self._load(filename)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\distro_version.py", line 103, in _load
    ret = super()._load(filename, cached=cached)
  File "C:\Program Files\Python39\lib\site-packages\hab\parsers\hab_base.py", line 560, in _load
    return utils.load_json_file(self.filename)
  File "C:\Program Files\Python39\lib\site-packages\hab\utils.py", line 302, in load_json_file
    e.result["filename"] = str(filename)
TypeError: 'NoneType' object does not support item assignment

The new traceback is still very long in verbose mode, but now ends with the json error and that exception includes the json filename being parsed.

  File "C:\blur\dev\hab_\hab\utils.py", line 297, in load_json_file
    data = json.load(fle)
  File "src/_legacy.pyx", line 53, in pyjson5.load
  File "src/_exports.pyx", line 286, in pyjson5.decode_io
  File "src/_decoder.pyx", line 913, in pyjson5._decode_callback
  File "src/_decoder.pyx", line 815, in pyjson5._decode_all
pyjson5.Json5IllegalCharacter: ("Expected b'JSON5Value' near 1, found U+0000", {'filename': 'C:\\blur\\hab\\distros\\maya2024\\2024.2.0.1186\\.hab.json'}, '\x00')

Some pyjson5 exceptions include set result to None, in this case convert
it into a dict and add the filename for debugging corrupt json files.

Ensure the filename is shown if an illegal character is encountered. For
example a file gets converted to a bunch of null values.
@MHendricks MHendricks merged commit ae4e1c4 into main Jul 18, 2024
31 checks passed
@MHendricks MHendricks deleted the mikeh/json_bin_error branch July 18, 2024 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant