Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Unresolvable JSON pointer" when jsonschema>=4.18.1 #19

Closed
EliahKagan opened this issue Jul 15, 2023 · 2 comments
Closed

"Unresolvable JSON pointer" when jsonschema>=4.18.1 #19

EliahKagan opened this issue Jul 15, 2023 · 2 comments

Comments

@EliahKagan
Copy link
Owner

EliahKagan commented Jul 15, 2023 •
edited
Loading

When pylint-sarif-unofficial (this software) is used with jsonschema at version 4.18.1 or higher--but not at 4.17.* and not at 4.18.0--it gives the error:

jsonschema.exceptions._RefResolutionError: Unresolvable JSON pointer: 'definitions/messageClass'

Here's an example of full error output, running pylint2sarif on itself:

(pylint-sarif-unofficial-py3.11) ek@Glub:~/repos-wsl/pylint-sarif (develop +%=)$ pylint2sarif src/pylint2sarif.py
Traceback (most recent call last):
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/jsonschema/validators.py", line 1147, in resolve_fragment
    document = document[part]
               ~~~~~~~~^^^^^^
KeyError: 'messageClass'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/bin/pylint2sarif", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/ek/repos-wsl/pylint-sarif/src/pylint2sarif.py", line 41, in main
    p2s = Pylint2Sarif(args)
          ^^^^^^^^^^^^^^^^^^
  File "/home/ek/repos-wsl/pylint-sarif/src/pylint2sarif.py", line 137, in __init__
    self.sarif = builder.build_classes()
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/python_jsonschema_objects/__init__.py", line 135, in build_classes
    builder.construct(uri, defn, **kw)
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/python_jsonschema_objects/classbuilder.py", line 497, in construct
    ret = self._construct(uri, *args, **kw)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/python_jsonschema_objects/classbuilder.py", line 584, in _construct
    self.resolved[uri] = self._build_object(uri, clsdata, parent, **kw)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/python_jsonschema_objects/classbuilder.py", line 665, in _build_object
    typ = self.resolve_type(ref, ".".join([nm, prop]))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/python_jsonschema_objects/classbuilder.py", line 485, in resolve_type
    with self.resolver.resolving(ref) as resolved:
  File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/jsonschema/validators.py", line 1034, in resolving
    url, resolved = self.resolve(ref)
                    ^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/jsonschema/validators.py", line 1081, in resolve
    return url, self._remote_cache(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/jsonschema/validators.py", line 1099, in resolve_from_url
    return self.resolve_fragment(document, fragment)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ek/.cache/pypoetry/virtualenvs/pylint-sarif-unofficial-4ksIlfgF-py3.11/lib/python3.11/site-packages/jsonschema/validators.py", line 1149, in resolve_fragment
    raise exceptions._RefResolutionError(
jsonschema.exceptions._RefResolutionError: Unresolvable JSON pointer: 'definitions/messageClass'

jsonschema 4.18 changes how $ref is handled and that appears related to this, but 4.18.0 does work. The patch releases 4.18.1, 4.18.2, and 4.18.3 address regressions and shouldn't break anything, but I'm unsure if the bug is really in jsonschema or in pylint-sarif-unofficial.

Because I haven't yet dropped support for Python 3.7, but jsonschema 4.18 has, this bug is not observed when cloning this repository, installing it with poetry install, and testing it, and is further not observed even after poetry update, poetry up, or poetry up --latest. I first noticed this problem on another project's CI (for dmvassallo/EmbeddingScratchwork#208). I reproduced it as shown above by running poetry shell and, in the poetry-managed virtual environment, pip install -U jsonschema. I verified which versions are affected by running commands like pip install -U jsonschema==4.18.0 and testing again.

Because 4.18.1, 4.18.2, and 4.18.3 address regressions present in 4.18.0, it is somewhat undesirable to pin jsonschema at 4.18.0, even as a temporary stopgap. The best thing to do would be for me to find the cause of the incompatibility in pylint-sarif-unofficial and fix it, and/or open a bug report for jsonschema. However, in the short term (which would become long term if I never get around to doing that), it might be better to pin jsonschema at 4.17.* and modify this repository and PyPI package's readme to advocate installing pylint-sarif-unofficial with pipx rather than in the project virtual environment. (That is itself because, if pinned, tools like Dependabot typically won't update it.)
EliahKagan added a commit that referenced this issue Jul 15, 2023
@EliahKagan
Update dependencies
e890299 
This neither fixes nor exacerbates #19. Because I am still giving
the 3.7.1 as the earliest version to support, poetry installs an
early enough version of jsonschema to avoid #19, even though that
bug is conceptually unrelated to what version of Python is used.
Unfortunately, this only applies to the "poetry install" usage,
which is not a significant use case other than for development.
The problem in #19 will usually (perhaps almost always) happen in
actual use.
@EliahKagan EliahKagan mentioned this issue Jul 15, 2023
Merged
EliahKagan added a commit that referenced this issue Jul 15, 2023
@EliahKagan
Update dependencies (#20)
80e51f5 
This neither fixes nor exacerbates #19. Because I am still giving
the 3.7.1 as the earliest version to support, poetry installs an
early enough version of jsonschema to avoid #19, even though that
bug is conceptually unrelated to what version of Python is used.
Unfortunately, this only applies to the "poetry install" usage,
which is not a significant use case other than for development.
The problem in #19 will usually (perhaps almost always) happen in
actual use.
EliahKagan added a commit that referenced this issue Jul 16, 2023
@EliahKagan
Keep jsonschema at 4.17.* to work around #19
63e87e0 
This also updates the readme accordingly, to recommending
installing via pipx instead of as a dev dependency of a project
that might otherwise depend on jsonschema and benefit from using a
later version.

It also adds a link to PyPI. This will look a little weird as long
as the main (repo root) readme and PyPI readme are the same file,
but I think it's better than omitting it, and I don't want to add a
badge for it instead until I've thought through what other badges
(if any) ought to be shown.
EliahKagan added a commit that referenced this issue Jul 16, 2023
@EliahKagan
Hold jsonschema at 4.17.* to work around #19
dbd1551 
This also updates the readme accordingly, to recommending
installing via pipx instead of as a dev dependency of a project
that might otherwise depend on jsonschema and benefit from using a
later version.

It also adds a link to PyPI. This will look a little weird as long
as the main (repo root) readme and PyPI readme are the same file,
but I think it's better than omitting it, and I don't want to add a
badge for it instead until I've thought through what other badges
(if any) ought to be shown.
@EliahKagan EliahKagan mentioned this issue Jul 16, 2023
Merged
EliahKagan added a commit that referenced this issue Jul 16, 2023
@EliahKagan
Hold jsonschema at 4.17.* to work around #19
dc37c79 
This also updates the readme accordingly, to link #19 and recommend
installing via pipx instead of as a dev dependency of a project
that might otherwise depend on jsonschema and benefit from using a
later version.

It also adds a link to PyPI. This will look a little weird as long
as the main (repo root) readme and PyPI readme are the same file,
but I think it's better than omitting it, and I don't want to add a
badge for it instead until I've thought through what other badges
(if any) ought to be shown.
EliahKagan added a commit that referenced this issue Jul 16, 2023
@EliahKagan
Hold jsonschema at 4.17.* to work around #19 (#21)
66a85a8 
This also updates the readme accordingly, to link #19 and recommend
installing via pipx instead of as a dev dependency of a project
that might otherwise depend on jsonschema and benefit from using a
later version.

It also adds a link to PyPI. This will look a little weird as long
as the main (repo root) readme and PyPI readme are the same file,
but I think it's better than omitting it, and I don't want to add a
badge for it instead until I've thought through what other badges
(if any) ought to be shown.
EliahKagan added a commit to EliahKagan/EmbeddingScratchwork that referenced this issue Jul 16, 2023
@EliahKagan
Install pylint-sarif-unofficial via pipx
899f7e0 
And use the new version.

This change--both installing it with pipx on CI instead of having
it as a development dependency, and using the new version--is to
work around EliahKagan/pylint-sarif#19.

This also upgrades (other) direct and indirect dependencies.
@EliahKagan EliahKagan mentioned this issue Jul 16, 2023
Merged
EliahKagan added a commit to EliahKagan/EmbeddingScratchwork that referenced this issue Jul 16, 2023
@EliahKagan
Install pylint-sarif-unofficial via pipx
d8e3f3f 
And use the new version.

This change--both installing it with pipx on CI instead of having
it as a development dependency, and using the new version--is to
work around EliahKagan/pylint-sarif#19.

This also upgrades (other) direct and indirect dependencies.
EliahKagan added a commit to dmvassallo/EmbeddingScratchwork that referenced this issue Jul 16, 2023
@EliahKagan
Install pylint-sarif-unofficial via pipx (#209)
b7425f7 
Without this change, pylint-sarif-unofficial should still start
working again, because version 0.2.0 holds back the version of
jsonschema. However, we have other dependencies that pull in
jsonschema as an indirect dependency, and I'd rather not keep those
from benefiting from improvements.

* Install pylint-sarif-unofficial via pipx

And use the new version.

This change--both installing it with pipx on CI instead of having
it as a development dependency, and using the new version--is to
work around EliahKagan/pylint-sarif#19.

This also upgrades (other) direct and indirect dependencies.

* Use stable 0.2.0

(This version is actually the same code as the alpha.)

* Don't specify version

0.2.0 is available when given explicitly, as the previous commit
verifies. It should be selected automatically when no version is
given, as long as no later stable version is available.
@EliahKagan EliahKagan mentioned this issue Jul 17, 2023
Closed
@EliahKagan
Copy link
Owner Author

This project uses jsonschema indirectly through python-jsonschema-objects, which does not currently support version 4.18.* of jsonschema.

Since the status of python-jsonschema-objects with jsonschema 4.18 is now tracked in cwacek/python-jsonschema-objects#235, and versions >0.4.2 of python-jsonschema-objects pin jsonschema to be below 4.18 such that this project no longer has to (#26), I'm closing this issue for now. It can be reopened if it turns out this project has any incompatibilities beyond those that are found to affect python-jsonschema-objects in general (or a new issue can be opened).

@AngellusMortis
Copy link

This has been resolved now if you want to test pylint-sarif with newer versions of both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AngellusMortis @EliahKagan