Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Error while reading version specifier in poetry.lock #702

Closed
Neplex opened this issue Mar 28, 2024 · 11 comments · Fixed by #703
Closed

[BUG] Error while reading version specifier in poetry.lock #702

Neplex opened this issue Mar 28, 2024 · 11 comments · Fixed by #703
Labels
bug Something isn't working source: poetry

Comments

@Neplex
Copy link

Neplex commented Mar 28, 2024 •
edited
Loading

Describe the bug

Hi, I use cyclonedx to generate a BOM from my Poetry dependencies. I recently upgrade a python dependency and regenerate my poetry.lock file.

While generating the BOM I now have the following error :

$ cyclonedx-py poetry --no-dev --validate --output-format XML --outfile sbom.xml
CRITICAL | CDX > Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^

To Reproduce

You can generate the Poetry lockfile with ray[serve]@2.10.0 as dependency

Expected behavior

It should not have issue with the given syntax as Poetry have no issue with it

Screenshots or output-paste

$ cyclonedx-py poetry --no-dev --validate --output-format XML --outfile sbom.xml
CRITICAL | CDX > Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^

Environment

  • cyclonedx-py version: 4.1.3
  • Python version: 3.10.13
  • Poetry version: 1.7.1
  • OS: Debian Bookworm

Additional context

Add any other context about the problem here.
@Neplex Neplex added the bug Something isn't working label Mar 28, 2024
@jkowalleck
Copy link
Member

jkowalleck commented Mar 28, 2024 •
edited
Loading

tried to reproduce:

$ cd /tmp/.....
$ poetry init 

$ # same version like in the issue desctiotion
$ poetry add ray==2.10.0
Package operations: 19 installs, 0 updates, 0 removals

  - Installing attrs (23.2.0)
  - Installing rpds-py (0.18.0)
  - Installing referencing (0.34.0)
  - Installing certifi (2024.2.2)
  - Installing charset-normalizer (3.3.2)
  - Installing frozenlist (1.4.1)
  - Installing idna (3.6)
  - Installing jsonschema-specifications (2023.12.1)
  - Installing urllib3 (2.2.1)
  - Installing aiosignal (1.3.1)
  - Installing click (8.1.7)
  - Installing filelock (3.13.3)
  - Installing jsonschema (4.21.1)
  - Installing msgpack (1.0.8)
  - Installing packaging (24.0)
  - Installing protobuf (5.26.1)
  - Installing pyyaml (6.0.1)
  - Installing requests (2.31.0)
  - Installing ray (2.10.0)

Writing lock file

$ # same version like in the issue desctiotion
$ poetry add --dev cyclonedx-bom==4.1.3

$ # same command like in the issue description but with debug outut
$ poetry run cyclonedx-py poetry --no-dev --validate --output-format XML --outfile sbom.xml -vvv
DEBUG    | CDX > args: {'command': 'poetry', 'groups_without': [], 'groups_with': [], 'groups_only': [], 'no_dev': True, 'extras': [], 'all_extras': False, 'mc_type': <ComponentType.APPLICATION: 'application'>, 'project_directory': '.', 'short_purls': False, 'outfile': <_io.TextIOWrapper name='sbom.xml' mode='wt' encoding='utf8'>, 'schema_version': <SchemaVersion.V1_5: (1, 5)>, 'output_format': <OutputFormat.XML: 2>, 'output_reproducible': False, 'should_validate': True, '_bbc': <class 'cyclonedx_py._internal.poetry.PoetryBB'>}
INFO     | CDX > Generating SBOM ...
DEBUG    | CDX.PoetryBB > use_groups: frozenset({'main'})
DEBUG    | CDX.PoetryBB > use_extras: frozenset()
DEBUG    | CDX.PoetryBB > root-component: <Component bom-ref=<BomRef 'testing-issue702' id=140308289511120>, group=None, name=testing-issue702, version=0.1.0, type=ComponentType.APPLICATION>
DEBUG    | CDX.PoetryBB > lock_version: (2, 0)
DEBUG    | CDX.PoetryBB > root-component depends on python
DEBUG    | CDX.PoetryBB > root-component depends on ray
INFO     | CDX.PoetryBB > add component for package 'ray'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'ray@2.10.0' id=140308284713040>, group=None, name=ray, version=2.10.0, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'aiosignal'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'aiosignal@1.3.1' id=140308289517520>, group=None, name=aiosignal, version=1.3.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'frozenlist'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'frozenlist@1.4.1' id=140308286104400>, group=None, name=frozenlist, version=1.4.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'click'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'click@8.1.7' id=140308286056528>, group=None, name=click, version=8.1.7, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'colorama'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'colorama@0.4.6' id=140308286063440>, group=None, name=colorama, version=0.4.6, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'filelock'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'filelock@3.13.3' id=140308286090704>, group=None, name=filelock, version=3.13.3, type=ComponentType.LIBRARY>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'frozenlist@1.4.1' id=140308286104400>, group=None, name=frozenlist, version=1.4.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'jsonschema'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'jsonschema@4.21.1' id=140308286326480>, group=None, name=jsonschema, version=4.21.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'attrs'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'attrs@23.2.0' id=140308285812048>, group=None, name=attrs, version=23.2.0, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'jsonschema-specifications'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'jsonschema-specifications@2023.12.1' id=140308286333968>, group=None, name=jsonschema-specifications, version=2023.12.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'referencing'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'referencing@0.34.0' id=140308284756944>, group=None, name=referencing, version=0.34.0, type=ComponentType.LIBRARY>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'attrs@23.2.0' id=140308285812048>, group=None, name=attrs, version=23.2.0, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'rpds-py'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'rpds-py@0.18.0' id=140308284800976>, group=None, name=rpds-py, version=0.18.0, type=ComponentType.LIBRARY>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'referencing@0.34.0' id=140308284756944>, group=None, name=referencing, version=0.34.0, type=ComponentType.LIBRARY>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'rpds-py@0.18.0' id=140308284800976>, group=None, name=rpds-py, version=0.18.0, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'msgpack'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'msgpack@1.0.8' id=140308286500304>, group=None, name=msgpack, version=1.0.8, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'packaging'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'packaging@23.2' id=140308286603536>, group=None, name=packaging, version=23.2, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'protobuf'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'protobuf@5.26.1' id=140308286617232>, group=None, name=protobuf, version=5.26.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'pyyaml'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pyyaml@6.0.1' id=140308284589456>, group=None, name=pyyaml, version=6.0.1, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'requests'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'requests@2.31.0' id=140308284763920>, group=None, name=requests, version=2.31.0, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'certifi'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'certifi@2024.2.2' id=140308285858896>, group=None, name=certifi, version=2024.2.2, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'charset-normalizer'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'charset-normalizer@3.3.2' id=140308285872336>, group=None, name=charset-normalizer, version=3.3.2, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'idna'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'idna@3.6' id=140308286273296>, group=None, name=idna, version=3.6, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'urllib3'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'urllib3@2.2.1' id=140308285039696>, group=None, name=urllib3, version=2.2.1, type=ComponentType.LIBRARY>
INFO     | CDX > Serializing SBOM: 1.5/XML
INFO     | CDX > Validating result to schema: 1.5/XML
DEBUG    | CDX > result is schema-valid
INFO     | CDX > Writing to: sbom.xml
DEBUG    | CDX > Wrote 167979 bytes to sbom.xml

work data and the resulting SBOM xml file: issues702.zip

conclusion: could not reproduce.

@jkowalleck
Copy link
Member

@Neplex i was unable to reproduce the event.
could you share a reproducible setup for debugging purposes?

@jkowalleck jkowalleck added the question Further information is requested label Mar 28, 2024
@Neplex
Copy link
Author

Neplex commented Mar 28, 2024 •
edited
Loading

Hi @jkowalleck thanks for your quick reply. I forgot an extra in my description. Could you try again with ray[serve]@2.10.0 ?

Here the complete debug log

DEBUG    | CDX > args: {'command': 'poetry', 'groups_without': [], 'groups_with': [], 'groups_only': [], 'no_dev': True, 'extras': [], 'all_extras': False, 'mc_type': <ComponentType.APPLICATION: 'application'>, 'project_directory': '.', 'short_purls': False, 'outfile': <_io.TextIOWrapper name='sbom.xml' mode='wt' encoding='utf8'>, 'schema_version': <SchemaVersion.V1_5: (1, 5)>, 'output_format': <OutputFormat.XML: 2>, 'output_reproducible': False, 'should_validate': True, '_bbc': <class 'cyclonedx_py._internal.poetry.PoetryBB'>}
INFO     | CDX > Generating SBOM ...
DEBUG    | CDX.PoetryBB > use_groups: frozenset({'main'})
DEBUG    | CDX.PoetryBB > use_extras: frozenset()
DEBUG    | CDX.PoetryBB > root-component: <Component bom-ref=<BomRef 'test' id=1812141888112>, group=None, name=test, version=0.1.0, type=application>
DEBUG    | CDX.PoetryBB > lock_version: (2, 0)
DEBUG    | CDX.PoetryBB > root-component depends on python
DEBUG    | CDX.PoetryBB > root-component depends on ray
INFO     | CDX.PoetryBB > add component for package 'ray'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'ray@2.10.0' id=1812145128400>, group=None, name=ray, version=2.10.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'aiosignal'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'aiosignal@1.3.1' id=1812142962672>, group=None, name=aiosignal, version=1.3.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'frozenlist'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'frozenlist@1.4.1' id=1812143370448>, group=None, name=frozenlist, version=1.4.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'click'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'click@8.1.7' id=1812143255712>, group=None, name=click, version=8.1.7, type=library>
INFO     | CDX.PoetryBB > add component for package 'colorama'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'colorama@0.4.6' id=1812143257200>, group=None, name=colorama, version=0.4.6, type=library>
INFO     | CDX.PoetryBB > add component for package 'filelock'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'filelock@3.13.3' id=1812143367472>, group=None, name=filelock, version=3.13.3, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'frozenlist@1.4.1' id=1812143370448>, group=None, name=frozenlist, version=1.4.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'jsonschema'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'jsonschema@4.21.1' id=1812143889888>, group=None, name=jsonschema, version=4.21.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'attrs'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'attrs@23.2.0' id=1812141894592>, group=None, name=attrs, version=23.2.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'jsonschema-specifications'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'jsonschema-specifications@2023.12.1' id=1812143891376>, group=None, name=jsonschema-specifications, version=2023.12.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'referencing'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'referencing@0.34.0' id=1812145136800>, group=None, name=referencing, version=0.34.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'attrs@23.2.0' id=1812141894592>, group=None, name=attrs, version=23.2.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'rpds-py'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'rpds-py@0.18.0' id=1812145142752>, group=None, name=rpds-py, version=0.18.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'referencing@0.34.0' id=1812145136800>, group=None, name=referencing, version=0.34.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'rpds-py@0.18.0' id=1812145142752>, group=None, name=rpds-py, version=0.18.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'msgpack'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'msgpack@1.0.8' id=1812144088992>, group=None, name=msgpack, version=1.0.8, type=library>
INFO     | CDX.PoetryBB > add component for package 'packaging'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'packaging@23.2' id=1812144511664>, group=None, name=packaging, version=23.2, type=library>
INFO     | CDX.PoetryBB > add component for package 'protobuf'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'protobuf@5.26.1' id=1812144517616>, group=None, name=protobuf, version=5.26.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'pyyaml'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pyyaml@6.0.1' id=1812144911744>, group=None, name=pyyaml, version=6.0.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'requests'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'requests@2.31.0' id=1812145138288>, group=None, name=requests, version=2.31.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'certifi'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'certifi@2024.2.2' id=1812143053488>, group=None, name=certifi, version=2024.2.2, type=library>
INFO     | CDX.PoetryBB > add component for package 'charset-normalizer'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'charset-normalizer@3.3.2' id=1812143056464>, group=None, name=charset-normalizer, version=3.3.2, type=library>
INFO     | CDX.PoetryBB > add component for package 'idna'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'idna@3.6' id=1812143885424>, group=None, name=idna, version=3.6, type=library>
INFO     | CDX.PoetryBB > add component for package 'urllib3'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'urllib3@2.2.1' id=1812145669360>, group=None, name=urllib3, version=2.2.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'aiohttp'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'aiohttp@3.9.3' id=1812141899152>, group=None, name=aiohttp, version=3.9.3, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'aiosignal@1.3.1' id=1812142962672>, group=None, name=aiosignal, version=1.3.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'async-timeout'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'async-timeout@4.0.3' id=1812143050608>, group=None, name=async-timeout, version=4.0.3, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'attrs@23.2.0' id=1812141894592>, group=None, name=attrs, version=23.2.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'frozenlist@1.4.1' id=1812143370448>, group=None, name=frozenlist, version=1.4.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'multidict'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'multidict@6.0.5' id=1812144209584>, group=None, name=multidict, version=6.0.5, type=library>
INFO     | CDX.PoetryBB > add component for package 'yarl'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'yarl@1.9.4' id=1812146330864>, group=None, name=yarl, version=1.9.4, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'idna@3.6' id=1812143885424>, group=None, name=idna, version=3.6, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'multidict@6.0.5' id=1812144209584>, group=None, name=multidict, version=6.0.5, type=library>
INFO     | CDX.PoetryBB > add component for package 'aiohttp-cors'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'aiohttp-cors@0.7.0' id=1812142961184>, group=None, name=aiohttp-cors, version=0.7.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'aiohttp@3.9.3' id=1812141899152>, group=None, name=aiohttp, version=3.9.3, type=library>
INFO     | CDX.PoetryBB > add component for package 'colorful'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'colorful@0.5.6' id=1812143258688>, group=None, name=colorful, version=0.5.6, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'colorama@0.4.6' id=1812143257200>, group=None, name=colorama, version=0.4.6, type=library>
INFO     | CDX.PoetryBB > add component for package 'fastapi'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'fastapi@0.110.0' id=1812143365984>, group=None, name=fastapi, version=0.110.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'pydantic'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pydantic@2.6.4' id=1812144694336>, group=None, name=pydantic, version=2.6.4, type=library>
INFO     | CDX.PoetryBB > add component for package 'annotated-types'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'annotated-types@0.6.0' id=1812142964160>, group=None, name=annotated-types, version=0.6.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'pydantic-core'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pydantic-core@2.16.3' id=1812144695824>, group=None, name=pydantic-core, version=2.16.3, type=library>
INFO     | CDX.PoetryBB > add component for package 'typing-extensions'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'typing-extensions@4.10.0' id=1812145502480>, group=None, name=typing-extensions, version=4.10.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'typing-extensions@4.10.0' id=1812145502480>, group=None, name=typing-extensions, version=4.10.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'starlette'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'starlette@0.36.3' id=1812145498016>, group=None, name=starlette, version=0.36.3, type=library>
INFO     | CDX.PoetryBB > add component for package 'anyio'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'anyio@4.3.0' id=1812143047632>, group=None, name=anyio, version=4.3.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'exceptiongroup'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'exceptiongroup@1.2.0' id=1812143364496>, group=None, name=exceptiongroup, version=1.2.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'idna@3.6' id=1812143885424>, group=None, name=idna, version=3.6, type=library>
INFO     | CDX.PoetryBB > add component for package 'sniffio'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'sniffio@1.3.1' id=1812145495040>, group=None, name=sniffio, version=1.3.1, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'typing-extensions@4.10.0' id=1812145502480>, group=None, name=typing-extensions, version=4.10.0, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'typing-extensions@4.10.0' id=1812145502480>, group=None, name=typing-extensions, version=4.10.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'grpcio'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'grpcio@1.62.1' id=1812143601984>, group=None, name=grpcio, version=1.62.1, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'grpcio@1.62.1' id=1812143601984>, group=None, name=grpcio, version=1.62.1, type=library>
INFO     | CDX.PoetryBB > add component for package 'opencensus'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'opencensus@0.11.4' id=1812144507200>, group=None, name=opencensus, version=0.11.4, type=library>
INFO     | CDX.PoetryBB > add component for package 'google-api-core'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'google-api-core@2.8.0' id=1812143597520>, group=None, name=google-api-core, version=2.8.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'google-auth'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'google-auth@2.29.0' id=1812143599008>, group=None, name=google-auth, version=2.29.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'cachetools'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'cachetools@5.3.3' id=1812143052000>, group=None, name=cachetools, version=5.3.3, type=library>
INFO     | CDX.PoetryBB > add component for package 'pyasn1-modules'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pyasn1-modules@0.4.0' id=1812144692848>, group=None, name=pyasn1-modules, version=0.4.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'pyasn1'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'pyasn1@0.6.0' id=1812144691360>, group=None, name=pyasn1, version=0.6.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'rsa'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'rsa@4.9' id=1812145490576>, group=None, name=rsa, version=4.9, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'pyasn1@0.6.0' id=1812144691360>, group=None, name=pyasn1, version=0.6.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'googleapis-common-protos'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'googleapis-common-protos@1.56.1' id=1812143600496>, group=None, name=googleapis-common-protos, version=1.56.1, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'protobuf@5.26.1' id=1812144517616>, group=None, name=protobuf, version=5.26.1, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'protobuf@5.26.1' id=1812144517616>, group=None, name=protobuf, version=5.26.1, type=library>
DEBUG    | CDX.PoetryBB > existing component: <Component bom-ref=<BomRef 'requests@2.31.0' id=1812145138288>, group=None, name=requests, version=2.31.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'opencensus-context'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'opencensus-context@0.1.3' id=1812144508688>, group=None, name=opencensus-context, version=0.1.3, type=library>
INFO     | CDX.PoetryBB > add component for package 'six'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'six@1.16.0' id=1812145492064>, group=None, name=six, version=1.16.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'prometheus-client'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'prometheus-client@0.20.0' id=1812144516128>, group=None, name=prometheus-client, version=0.20.0, type=library>
INFO     | CDX.PoetryBB > add component for package 'py-spy'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'py-spy@0.3.14' id=1812144687952>, group=None, name=py-spy, version=0.3.14, type=library>
DEBUG    | CDX > Error: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^
Traceback (most recent call last):
  File "C:\tmp\test\.venv\lib\site-packages\packaging\requirements.py", line 35, in __init__
    parsed = _parse_requirement(requirement_string)
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_parser.py", line 82, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_parser.py", line 120, in _parse_requirement_details
    specifier = _parse_specifier(tokenizer)
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_parser.py", line 210, in _parse_specifier
    with tokenizer.enclosing_tokens(
  File "C:\Users\nhiot\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 142, in __exit__
    next(self.gen)
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_tokenizer.py", line 187, in enclosing_tokens
    self.raise_syntax_error(
  File "C:\tmp\test\.venv\lib\site-packages\packaging\_tokenizer.py", line 165, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\cli.py", line 281, in run
    Command(**args, logger=logger)(**args)
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\cli.py", line 245, in __call__
    bom = self._make_bom(**kwargs)
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\cli.py", line 240, in _make_bom
    return self._bbc(**self._clean_kwargs(kwargs))
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\poetry.py", line 216, in __call__
    return self._make_bom(
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\poetry.py", line 287, in _make_bom
    self.__add_dep(bom, lock_entry, dep_spec.get('extras', ()), lock_data)
  File "C:\tmp\test\.venv\lib\site-packages\cyclonedx_py\_internal\poetry.py", line 326, in __add_dep
    for req in map(
  File "C:\tmp\test\.venv\lib\site-packages\packaging\requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^
CRITICAL | CDX > Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
    pydantic (<2.0.dev0 || >=2.5.dev0,<3)
             ~~~~~~~~~~~^

@jkowalleck
Copy link
Member

re: #702 (comment)
was able to reproduce. used the following setup: issue702.zip

@jkowalleck
Copy link
Member

jkowalleck commented Mar 28, 2024 •
edited
Loading

issue seams to be caused when running

cyclonedx-python/cyclonedx_py/_internal/poetry.py

Lines 326 to 328 in 7ae2145

for req in map(
Requirement,
chain.from_iterable(es for en, es in lock_entry.extras.items() if en in use_extras)

which is a map calling the constructor of packaging.requirements.Requirement for each item of lock_entry.extras

research showed, that the verison constraint poetry does at this point is no longer the original one ("pydantic!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3"),
but something poetry-specific: "pydantic (<2.0.dev0 || >=2.5.dev0,<3)"

other examples for version rework are:

  • protobuf >= 3.15.3, != 3.19.5 --> protobuf = ">=3.15.3,<3.19.5 || >3.19.5"
  • ... to be continued

anyway, the poetry versions are not working as expected.
furthermore, they are not needed for the for the implementation in this very tool.
this means, they could be stripped out, before parsing ... if possible...
need to investigate further.

possibple solution: remove (.+) from the requireemnt - to strip the version entirely

@jkowalleck jkowalleck added bug Something isn't working and removed bug Something isn't working question Further information is requested labels Mar 28, 2024
@jkowalleck
Copy link
Member

@Neplex, thank you for the report.

I was able to reproduce and isolate the issue.
A fix shall be published soo.

@Neplex
Copy link
Author

Neplex commented Mar 28, 2024

@jkowalleck Happy to helps and thanks for your time. Hope to see the fix soon

@jkowalleck
Copy link
Member

jkowalleck commented Mar 28, 2024 •
edited
Loading

Possible solution:
dont depend on packaging to parse package info, but do it in custom implementation.
string format is ^(?<package-name>.+)(?:\[(?<extras>.+)\])?(?: \((?<version-constraint>.+)\))?$
regex might be just slow, maybe a simpler method might be utlized .... need to think about it ...

@Neplex
Copy link
Author

Neplex commented Mar 28, 2024

Is it necessary to parse these fields ? Given the built BOM, only the version and the package name seems required. And for the dependency graph same thing apply, we just need to know that Package A depends on Package B the version used to build the purl can be get from the pinned version of Package B. Am I wrong ?

@jkowalleck
Copy link
Member

jkowalleck commented Mar 28, 2024 •
edited
Loading

this issue occurred when parsing the extras of a given package, to sort out which optional dependencies were actually used. At this very point of determination, the version of a package does not matter(since it is stated at a whole other area), but we care for the name of the optional dependency, and a possible extra of that optional dependency.

PS: no worries. Thanks to your help, I will be able to craft a test bed for proper (regression/unit/integration) testing the needed fix.

@jkowalleck jkowalleck mentioned this issue Mar 28, 2024
Merged
@jkowalleck
Copy link
Member

fix was released via https://github.com/CycloneDX/cyclonedx-python/releases/tag/v4.1.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working source: poetry
Projects
None yet
Milestone
No milestone
2 participants
@jkowalleck @Neplex