Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scancode-toolkit update breaks Docker pipelines #438

Closed
JonoYang opened this issue May 12, 2022 · 3 comments
Closed

scancode-toolkit update breaks Docker pipelines #438

JonoYang opened this issue May 12, 2022 · 3 comments

Comments

@JonoYang
Copy link
Member

Several package related functions used by the docker pipeline have been removed in the latest version of scancode-toolkit. Debian-related package handling, especially code that deals with handling the status files and such, work slightly differently than the way it used to. Functions that the pipelines relied on, like get_installed_packages, have been removed.
JonoYang added a commit that referenced this issue May 12, 2022
@JonoYang
Update package_getter #434 #438
fe612f1 
    * Adapt code from previous version of scancode-toolkit for use in the debian pipeline

Signed-off-by: Jono Yang <jyang@nexb.com>
@JonoYang
Copy link
Member Author

I brought in the code for get_installed_packages and its helper functions from a previous version of scancode and put it in scanpipe/pipes/debian.py, but there seems to be a problem with some part of license detection when I run the docker pipeline on docker://ubuntu:

'AND' object has no attribute 'iscanonical'

Traceback:
  File "/home/jono/nexb/src/scancode.io/scanpipe/pipelines/__init__.py", line 115, in execute
    step(self)
  File "/home/jono/nexb/src/scancode.io/scanpipe/pipelines/docker.py", line 93, in collect_and_create_system_packages
    docker.scan_image_for_system_packages(self.project, image)
  File "/home/jono/nexb/src/scancode.io/scanpipe/pipes/docker.py", line 166, in scan_image_for_system_packages
    for i, (purl, package, layer) in enumerate(installed_packages):
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/container_inspector/image.py", line 446, in get_installed_packages
    for purl, package in layer.get_installed_packages(packages_getter):
  File "/home/jono/nexb/src/scancode.io/scanpipe/pipes/debian.py", line 76, in package_getter
    for package in packages:
  File "/home/jono/nexb/src/scancode.io/scanpipe/pipes/debian.py", line 101, in get_installed_packages
    dc = debian_copyright.parse_copyright_file(copyright_location)
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 198, in parse_copyright_file
    dc = StructuredCopyrightProcessor.from_file(
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 453, in from_file
    dc.detect_license()
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 678, in detect_license
    debian_licensing = DebianLicensing.from_license_paragraphs(
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 934, in from_license_paragraphs
    result = DebianLicensing.parse_paras_with_license_text(
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 1060, in parse_paras_with_license_text
    if not have_consistent_licenses_with_match(
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/packagedcode/debian_copyright.py", line 1728, in have_consistent_licenses_with_match
    if not reference_match.same_licensing(match):
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py", line 346, in same_licensing
    return self.rule.same_licensing(other.rule)
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py", line 1546, in same_licensing
    return self.licensing.is_equivalent(
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py", line 301, in is_equivalent
    ex2 = self._parse_and_simplify(expression2, **kwargs)
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py", line 326, in _parse_and_simplify
    return expression.simplify()
  File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/boolean/boolean.py", line 1281, in simplify
    if self.iscanonical:

@JonoYang
Copy link
Member Author

Skipping the license detection part allows the docker pipeline to complete, but we get a lot of errors:

image

This error shows up for the codebase resource created for the binaries in /usr/bin/:

ERROR: for scanner: licenses: ERROR: Unknown error: Traceback (most recent call last): File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/interrupt.py", line 91, in interruptible return NO_ERROR, func(*(args or ()), **(kwargs or {})) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py", line 176, in get_licenses matches = idx.match( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 925, in match return self.match_query( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 996, in match_query matched = matcher( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 668, in get_exact_matches matches, _discarded = match.refine_matches( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py", line 2596, in refine_matches matches, discarded = filter_invalid_matches_to_single_word_gibberish(matches) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py", line 1769, in filter_invalid_matches_to_single_word_gibberish rule_text = rule.text().strip() File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py", line 1656, in text raise InvalidRule( licensedcode.models.InvalidRule: Inconsistent rule text for: gpl-3.0-plus_28.RULE file:///home/pombreda/w421/scancode-toolkit-licenses/src/licensedcode/data/rules/gpl-3.0-plus_28.RULE
--

[ERROR: for scanner: licenses: ERROR: Unknown error: Traceback (most recent call last): File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/interrupt.py", line 91, in interruptible return NO_ERROR, func(*(args or ()), **(kwargs or {})) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py", line 176, in get_licenses matches = idx.match( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 925, in match return self.match_query( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 996, in match_query matched = matcher( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py", line 668, in get_exact_matches matches, _discarded = match.refine_matches( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py", line 2596, in refine_matches matches, discarded = filter_invalid_matches_to_single_word_gibberish(matches) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py", line 1769, in filter_invalid_matches_to_single_word_gibberish rule_text = rule.text().strip() File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py", line 1656, in text raise InvalidRule( licensedcode.models.InvalidRule: Inconsistent rule text for: gpl-3.0-plus_28.RULE file:///home/pombreda/w421/scancode-toolkit-licenses/src/licensedcode/data/rules/gpl-3.0-plus_28.RULE](http://127.0.0.1:8001/project/d80e10c3-e811-4d23-8e5c-f24c945f0e99/errors/?message=ERROR:%20for%20scanner:%20licenses:ERROR:%20Unknown%20error:Traceback%20(most%20recent%20call%20last):%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/interrupt.py%22,%20line%2091,%20in%20interruptible%20%20%20%20return%20NO_ERROR,%20func(*(args%20or%20()),%20**(kwargs%20or%20{}))%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py%22,%20line%20176,%20in%20get_licenses%20%20%20%20matches%20=%20idx.match(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py%22,%20line%20925,%20in%20match%20%20%20%20return%20self.match_query(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py%22,%20line%20996,%20in%20match_query%20%20%20%20matched%20=%20matcher(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/index.py%22,%20line%20668,%20in%20get_exact_matches%20%20%20%20matches,%20_discarded%20=%20match.refine_matches(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py%22,%20line%202596,%20in%20refine_matches%20%20%20%20matches,%20discarded%20=%20filter_invalid_matches_to_single_word_gibberish(matches)%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/match.py%22,%20line%201769,%20in%20filter_invalid_matches_to_single_word_gibberish%20%20%20%20rule_text%20=%20rule.text().strip()%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py%22,%20line%201656,%20in%20text%20%20%20%20raise%20InvalidRule(licensedcode.models.InvalidRule:%20Inconsistent%20rule%20text%20for:%20gpl-3.0-plus_28.RULEfile:///home/pombreda/w421/scancode-toolkit-licenses/src/licensedcode/data/rules/gpl-3.0-plus_28.RULE)

This error shows up on copyright files from /usr/share/doc/:

[ERROR: for scanner: licenses: ERROR: Unknown error: Traceback (most recent call last): File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/interrupt.py", line 91, in interruptible return NO_ERROR, func(*(args or ()), **(kwargs or {})) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py", line 192, in get_licenses _licenses_data_from_match( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py", line 240, in _licenses_data_from_match for license_key in match.rule.license_keys(): File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py", line 1536, in license_keys return self.licensing.license_keys( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py", line 411, in license_keys symbols = self.license_symbols( File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py", line 356, in license_symbols symbols = (s for s in expression.get_literals() if isinstance(s, BaseSymbol)) File "/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/boolean/boolean.py", line 674, in get_literals if self.isliteral: AttributeError: 'AND' object has no attribute 'isliteral'](http://127.0.0.1:8001/project/d80e10c3-e811-4d23-8e5c-f24c945f0e99/errors/?message=ERROR:%20for%20scanner:%20licenses:ERROR:%20Unknown%20error:Traceback%20(most%20recent%20call%20last):%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/interrupt.py%22,%20line%2091,%20in%20interruptible%20%20%20%20return%20NO_ERROR,%20func(*(args%20or%20()),%20**(kwargs%20or%20{}))%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py%22,%20line%20192,%20in%20get_licenses%20%20%20%20_licenses_data_from_match(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/scancode/api.py%22,%20line%20240,%20in%20_licenses_data_from_match%20%20%20%20for%20license_key%20in%20match.rule.license_keys():%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/licensedcode/models.py%22,%20line%201536,%20in%20license_keys%20%20%20%20return%20self.licensing.license_keys(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py%22,%20line%20411,%20in%20license_keys%20%20%20%20symbols%20=%20self.license_symbols(%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/license_expression/__init__.py%22,%20line%20356,%20in%20license_symbols%20%20%20%20symbols%20=%20(s%20for%20s%20in%20expression.get_literals()%20if%20isinstance(s,%20BaseSymbol))%20%20File%20%22/home/jono/nexb/src/scancode.io/lib/python3.9/site-packages/boolean/boolean.py%22,%20line%20674,%20in%20get_literals%20%20%20%20if%20self.isliteral:AttributeError:%20%27AND%27%20object%20has%20no%20attribute%20%27isliteral%27)

@JonoYang
Copy link
Member Author

Running scancode --reindex-licenses while in the scancode.io virtual environment seemed to have resolved the license errors, leaving the DiscoveredPackage errors where no version is provided when making a package.

JonoYang added a commit that referenced this issue May 12, 2022
@JonoYang
Allow packages to be created without versions #438
29a72b3 
    * Update DiscoveredPackage.create_from_data to create packages without a version

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue May 18, 2022
@JonoYang
Update package_getter #434 #438
bcca2d7 
    * Adapt code from previous version of scancode-toolkit for use in the debian pipeline

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue May 18, 2022
@JonoYang
Allow packages to be created without versions #438
7917727 
    * Update DiscoveredPackage.create_from_data to create packages without a version

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue May 25, 2022
@JonoYang
Get file info and packages in initial scan #438
befe574 
    * Remove step for scanning application packages

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 7, 2022
@JonoYang
Revert changes to docker pipes and pipeline #438
093a52e 
    * Check for existence of installed_file attribute before using it

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 8, 2022
@JonoYang
Use generic package_getter for all distros #438
c58771a 
    * Ensure both installed_file and codebase_resource have the same checksum field before comparing them

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 8, 2022
@JonoYang
Use get_path() with strip_root to get paths #438
4a8713d 
    * Update mappings_keys_by_fieldname
    * Look for package data in package_data field instead of packages in save_scan_package_results

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Remove distro specific pipes #438
a0705c3 
    * Move get_installed_packages to rootfs.py
    * Use get_package_data instead of get_package_info
    * Rename all instances of packages to package_data when scanning for application packages
    * Update test docker images and test results
    * Add test for basic rootfs

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Use list comprehension for key_file_packages #438
0c26b7c 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Add test docker image for Ubuntu #438
faf6fba 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Add test docker image for Ubuntu #438
552bdb8 
    * Update expected test results
    * Remove old ubuntu.tar

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Update formatting #411 #438
155fe97 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Use smaller rpm docker image for testing #438
18cc997 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Replace ubuntu docker test image #438
23dc0e1 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Use purl data in update_or_create_packages #438
feca62b 
    * Add package_uid to test package data
    * Update expected test result

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 9, 2022
@JonoYang
Use purl data in update_or_create_packages #438
b3f4656 
    * Add package_uid to test package data
    * Update expected test result

Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 13, 2022
@JonoYang
Bump scancode version to v31.0.0rc1 #438 #411
1592cd3 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 13, 2022
@JonoYang
Consider all PURL fields when ordering Packages #411 #438
942e06d 
Signed-off-by: Jono Yang <jyang@nexb.com>31.0.0rc1
JonoYang added a commit that referenced this issue Jun 13, 2022
@JonoYang
Consider all PURL fields when ordering Packages #411 #438
9fcec67 
Signed-off-by: Jono Yang <jyang@nexb.com>
JonoYang added a commit that referenced this issue Jun 14, 2022
@JonoYang
Create Packages before Resources #411 #438
20ffbe0 
    * In the LoadInventory pipeline, create the DiscoveredPackages from a scan before creating the CodebaseResources

Signed-off-by: Jono Yang <jyang@nexb.com>
tdruez added a commit that referenced this issue Jun 14, 2022
@JonoYang @tdruez
Upgrade ScanCode-toolkit to version v31 #411
ba4695a 
* Upgrade scancode-toolkit to latest beta release #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Add a test class to regen test data #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Upgrade container_inspector to latest 31.0.0 version #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Handle new scan format in scancode pipes #411

Signed-off-by: Jono Yang <jyang@nexb.com>

* Handle package_uids for DiscoveredPackages #411

    * Remove create_discovered_packages2 and create_codebase_resources2

Signed-off-by: Jono Yang <jyang@nexb.com>

* Update deprecated code #411

    * Normalize package_uids before comparing results in tests
    * Update expected test results

Signed-off-by: Jono Yang <jyang@nexb.com>

* Regenerate asgiref 3.3.0 test data #411

    * Mark ProjectCodebase tests with expectedFailure
    * We will revisit ProjectCodebase and update it to fit our current models

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add asgiref-3.3.0_scancode_scan.json #411

    * We are using a scancode scan results for tests since asgiref-3.3.0_scan.json is not exactly the same format as scancode's json output

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add asgiref-3.3.0_walk_test_fixtures.json #411

    * Update regen_test_data.py to generate asgiref-3.3.0_walk_test_fixtures.json

Signed-off-by: Jono Yang <jyang@nexb.com>

* Signed-off-by: Jono Yang <jyang@nexb.com>

* Update make_results_summary() #411

    * No need to explicity get license_clarity_score in make_results_summary()
    * Update expected test results

Signed-off-by: Jono Yang <jyang@nexb.com>

* Exclude system_environment from diff #411

    * Add .vscode to .gitignore

Signed-off-by: Jono Yang <jyang@nexb.com>

* Upgrade scancode-toolkit and extractcode to latest version #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Update package_getter #434 #438

    * Adapt code from previous version of scancode-toolkit for use in the debian pipeline

Signed-off-by: Jono Yang <jyang@nexb.com>

* Allow packages to be created without versions #438

    * Update DiscoveredPackage.create_from_data to create packages without a version

Signed-off-by: Jono Yang <jyang@nexb.com>

* Update expected test results

Signed-off-by: Jono Yang <jyang@nexb.com>

* Report DiscoveredPackage correctly in summary #411

    * Ensure that DiscoveredPackages are reported one time in the scan_package pipeline summary
   * Add test to check key_file_packages field in the summary output

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add test for docker pipeline for alpine #411

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add docker pipeline test for rpm images #411

Signed-off-by: Jono Yang <jyang@nexb.com>

* Track package_uids in make_results_summary #435

    * Avoid checking if package_data dictionary is already in the key_files_packages list
    * Keep track of package_uids instead

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add truncated ubuntu docker image for testing #435

Signed-off-by: Jono Yang <jyang@nexb.com>

* Bump scancode and commoncode versions #435

Signed-off-by: Jono Yang <jyang@nexb.com>

* Update docker pipeline #435

    * We now run scancode-toolkit on the docker image resources using the new --system-package option
    * This gives us the installed system packages in the returned scan
    * We use the scan to create the DiscoveredPackages and CodebaseResources
    * The rest of the pipeline is unchanged

Signed-off-by: Jono Yang <jyang@nexb.com>

* Fix code validity #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Simplify the filtering of key_files_packages using a QuerySet #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Remove copied code from docker.py #411 #435

    * Create Docker pipeline from combining the rootfs pipeline and scan_package pipeline

Signed-off-by: Jono Yang <jyang@nexb.com>

* Update alpine test image and results #411 #435

    * TODO: create smaller test images for ubuntu and redhat docker image tests

Signed-off-by: Jono Yang <jyang@nexb.com>

* Properly create multiple package instances #411

    * Do not attempt to combine multiple instances of the same package
    * Store package_uid in extra data by itself
    * Add test for multiple package instances

Signed-off-by: Jono Yang <jyang@nexb.com>

* Sort packages in JSON output by type and name #411

    * Normalize package_uid in extra_data fields

Signed-off-by: Jono Yang <jyang@nexb.com>

* Get file info and packages in initial scan #438

    * Remove step for scanning application packages

Signed-off-by: Jono Yang <jyang@nexb.com>

* Revert changes to docker pipes and pipeline #438

    * Check for existence of installed_file attribute before using it

Signed-off-by: Jono Yang <jyang@nexb.com>

* Use generic package_getter for all distros #438

    * Ensure both installed_file and codebase_resource have the same checksum field before comparing them

Signed-off-by: Jono Yang <jyang@nexb.com>

* Use get_path() with strip_root to get paths #438

    * Update mappings_keys_by_fieldname
    * Look for package data in package_data field instead of packages in save_scan_package_results

Signed-off-by: Jono Yang <jyang@nexb.com>

* Remove distro specific pipes #438

    * Move get_installed_packages to rootfs.py
    * Use get_package_data instead of get_package_info
    * Rename all instances of packages to package_data when scanning for application packages
    * Update test docker images and test results
    * Add test for basic rootfs

Signed-off-by: Jono Yang <jyang@nexb.com>

* Use list comprehension for key_file_packages #438

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add package_uid field to DiscoveredPackage #411

    * Update expected test results

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add test docker image for Ubuntu #438

    * Update expected test results
    * Remove old ubuntu.tar

Signed-off-by: Jono Yang <jyang@nexb.com>

* Update formatting #411 #438

Signed-off-by: Jono Yang <jyang@nexb.com>

* Use smaller rpm docker image for testing #438

Signed-off-by: Jono Yang <jyang@nexb.com>

* Replace ubuntu docker test image #438

Signed-off-by: Jono Yang <jyang@nexb.com>

* Use purl data in update_or_create_packages #438

    * Add package_uid to test package data
    * Update expected test result

Signed-off-by: Jono Yang <jyang@nexb.com>

* Bump scancode version to v31.0.0rc1 #438 #411

Signed-off-by: Jono Yang <jyang@nexb.com>

* Consider all PURL fields when ordering Packages #411 #438

Signed-off-by: Jono Yang <jyang@nexb.com>

* Create Packages before Resources  #411 #438

    * In the LoadInventory pipeline, create the DiscoveredPackages from a scan before creating the CodebaseResources

Signed-off-by: Jono Yang <jyang@nexb.com>

* Add test for load_inventory pipeline #411

Signed-off-by: Jono Yang <jyang@nexb.com>

* Code cleanups and formatting #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Upgrade dependencies #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refactor create_inventory_from_scan to remove duplicated code #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Add changelog entry #411

Signed-off-by: Thomas Druez <tdruez@nexb.com>

Co-authored-by: Thomas Druez <tdruez@nexb.com>
@tdruez tdruez closed this as completed Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tdruez @JonoYang