Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue 456 add rdf parser #469

Merged
merged 43 commits into from
Feb 22, 2023
Merged

Issue 456 add rdf parser #469

merged 43 commits into from
Feb 22, 2023

Conversation

meretp
Copy link
Collaborator

@meretp meretp commented Feb 9, 2023

fixes #456

Signed-off-by: Meret Behrens meret.behrens@tngtech.com

meretp added 3 commits February 9, 2023 15:06
@meretp
delete rdflib as optional dependency as it is required
18e7063 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
delete installation from rdflib as the package is a required dependency
cf5ee6a 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
move helper methods to also use it for the rdf parser
298255e 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp meretp marked this pull request as draft February 9, 2023 14:08
@meretp
move actor_parser.py as it is format-agnostic and will be used for js…
9729b96 
…on, yaml, xml and rdf parsing

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp meretp force-pushed the issue-456-add-rdf-parser branch 3 times, most recently from f163c45 to ef61acf Compare February 9, 2023 14:56
@meretp meretp marked this pull request as ready for review February 9, 2023 14:58
@meretp meretp force-pushed the issue-456-add-rdf-parser branch from 158a28a to e578450 Compare February 14, 2023 15:05
armintaenzertng
armintaenzertng requested changes Feb 20, 2023
View reviewed changes
Copy link
Collaborator

@armintaenzertng armintaenzertng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort! :)
Looks very good to me overall, but I have a few details to go over again.

src/spdx/parser/rdf/graph_parsing_functions.py Outdated Show resolved Hide resolved
src/spdx/casing_tools.py Outdated Show resolved Hide resolved
tests/spdx/parser/rdf/test_rdf_parser.py Outdated Show resolved Hide resolved
tests/spdx/parser/rdf/test_annotation_parser.py Outdated Show resolved Hide resolved
tests/spdx/parser/rdf/test_creation_info_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/package_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/package_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/package_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/package_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/snippet_parser.py Outdated Show resolved Hide resolved
@meretp meretp force-pushed the issue-456-add-rdf-parser branch 2 times, most recently from c99aa82 to 868461b Compare February 20, 2023 12:42
armintaenzertng
armintaenzertng approved these changes Feb 21, 2023
View reviewed changes
Copy link
Collaborator

@armintaenzertng armintaenzertng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the issues :)
I have two small remarks left

src/spdx/parser/rdf/creation_info_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/extracted_licensing_info_parser.py
parsing_method=str)
cross_references = []
for (_, _, cross_reference_node) in graph.triples((extracted_licensing_info_node, RDFS.seeAlso, None)):
cross_references.append(cross_reference_node.toPython())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, let's keep it that way :)

src/spdx/parser/rdf/snippet_parser.py Outdated Show resolved Hide resolved
@meretp meretp force-pushed the issue-456-add-rdf-parser branch from 51bfd7c to c7860ad Compare February 21, 2023 13:05
armintaenzertng
armintaenzertng requested changes Feb 21, 2023
View reviewed changes
Copy link
Collaborator

@armintaenzertng armintaenzertng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, but there is still an issue with the range parser...

src/spdx/parser/rdf/snippet_parser.py Outdated Show resolved Hide resolved
src/spdx/parser/rdf/snippet_parser.py Outdated Show resolved Hide resolved
armintaenzertng
armintaenzertng approved these changes Feb 22, 2023
View reviewed changes
Copy link
Collaborator

@armintaenzertng armintaenzertng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well then, off we go! :D

meretp added 13 commits February 22, 2023 11:01
@meretp
[issue-456] add creation_info parser
96498c0 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add graph parsing functions
024fa89 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add snippet parser
b1a47f9 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add checksum parser
8466d2e 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add external document ref parser
6d1e93e 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add file parser
7087e36 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add annotation parser
1eb803b 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add package parser
6c8fdee 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add package verification code parser
625e2a9 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add helper method in casing_tools and move the module as …
198f2a6 
…it is used in the parser and writer layer

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add parser for ExternalPackageRefs
b3e274b 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] use enum helper method for primary package purpose
cea02ae 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] use enum helper method for annotation type
d0238be 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
meretp added 25 commits February 22, 2023 11:02
@meretp
[issue-456] add relationship parser
58783f9 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] use NamespaceManager to translate external document names…
da33b14 
…paces prefix to the short identifier

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add extracted licensing info parser
69d16aa 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] allow no assertion in package download location
6652e4f 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456, refactor] extract method to get a unique value, sort methods
ce17ca6 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456, refactor] replace str_or_no_assertion_or_none with parse …
9737fde 
…helper method

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add license expression parser
9df9980 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456, refactor]
7b61c8f 
extract method to apply parsing method or log error, change order of parameters, rename method_to_apply -> parsing_method

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] parse list of license expressions
9ecc373 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] also parse URIRefs with the documents namespace as prefix…
1b8cf82 
… as license expression (could be ListedLicense, ExtractedLicensingInfo or simple reference)

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add rdf parser to cli tool
432a39f 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] delete RDFExample from specVersion 2.1 as the tool curren…
7a27e88 
…tly only supports SPDX 2.3

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456, refactor] use generic helper methods instead of individua…
03ee53c 
…l helper method, rename argument

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] move actor_parser.py as it is format-agnostic and used fo…
446757c 
…r json,yaml,xml and rdf parsing

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456, refactor] adapt type hints, delete unused imports, rename…
86bc68c 
…, reformat

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] add helper method to remove prefix to support Python 3.7 …
47e7db0 
…and Python 3.8

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] allow multiple members in ConjunctiveLicenseSet and Disju…
9681339 
…nctiveLicenseSet

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] strip parsed values by default to prevent new lines and l…
8f9259b 
…eading/ trailing whitespaces in string values

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[issue-456] fix parsing of externalPackageRef
9eb1206 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
fix helper method
4ec5b3c 
The return value of err.args[0] is a string, so we need to use append to add this string to the list of messages in the logger.

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[review] use helper method for enums also for FileType
c4d04d4 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[review, refactor] strip copyright texts, delete trailing comma
c787280 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[review] rewrite parse_ranges
6b70dda 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[review, refactor] let Constructor handle None values and use the mor…
1d728b7 
…e general helper method

Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp
[review] refactor helper methods
1cbe629 
Signed-off-by: Meret Behrens <meret.behrens@tngtech.com>
@meretp meretp force-pushed the issue-456-add-rdf-parser branch from b34f727 to 1cbe629 Compare February 22, 2023 10:17
@meretp meretp merged commit 872ab59 into spdx:refactor-python-tools Feb 22, 2023
@meretp meretp deleted the issue-456-add-rdf-parser branch February 22, 2023 10:21
This was referenced Feb 22, 2023
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@armintaenzertng armintaenzertng armintaenzertng approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@meretp @armintaenzertng