v34.7.1

Changelog

• Add pipeline step selection for a run execution.
  This allows to run a pipeline in an advanced mode allowing to skip some steps,
  or restart from a step, like the last failed step.
  The steps can be edited from the Run "status" modal using the "Select steps" button.
  This is an advanced feature and should we used with caution. #1303
• Display the resolved_to_package as link in the dependencies tab. #1314
• Add support for multiple instances of a PackageURL in the CycloneDX outputs.
  The package_uid is now included in each BOM Component as a property. #1316
• Add administration interface. Can be enabled with the SCANCODEIO_ENABLE_ADMIN_SITE
  setting.
  Add --admin and --super options to the create-user management command. #1323
• Add results_url and summary_url on the API ProjectSerializer. #1325

What's Changed

• Add pipeline step selection for a run execution #1303 by @tdruez in #1310
• Display the resolved_to_package as link in the dependencies tab by @tdruez in #1314
• Add support for multiple instances of a PURL in the CycloneDX outputs… by @tdruez in #1317
• Refactor the Webhook.get_payload to use Serializers #1325 by @tdruez in #1326
• Display sizes in bytes and humanized #1322 by @tdruez in #1324
• Add administration site for main scanpipe models by @tdruez in #1323

Full Changelog: v34.7.0...v34.7.1

v34.7.0

Changelog

• Add all "classify" plugin fields from scancode-toolkit on the CodebaseResource model. #1275
• Refine the extraction errors reporting to include the resource path for rendering
  link to the related resources in the UI. #1273
• Add a flush-projects management command, to Delete all project data and their
  related work directories created more than a specified number of days ago. #1289
• Update the inspect_packages pipeline to have an optional StaticResolver
  group to create resolved packages and dependency relationships from lockfiles
  and manifests having pre-resolved dependencies. Also update this pipeline to
  perform package assembly from multiple manifests and files to create
  discovered packages. Also update the resolve_dependencies pipeline to have
  the same StaticResolver group and mode the dynamic resolution part to a new
  optional DynamicResolver group. #1244
• Add a new attribute is_direct to the DiscoveredDependency model and two new
  attributes is_private and is_virtual to the DiscoveredPackage model.
  Also update the UIs to show these attributes and show the package_data field
  contents for CodebaseResources in the extra_data tab. #1244
• Update scancode-toolkit to version 32.2.1. For the complete list of updates
  and improvements see https://github.com/nexB/scancode-toolkit/releases/tag/v32.2.0
  and https://github.com/nexB/scancode-toolkit/releases/tag/v32.2.1
• Add support for providing pipeline "selected_groups" in the run entry point. #1306

What's Changed

• Add all "classify" plugin fields on the CodebaseResource model #1275 by @tdruez in #1286
• Add a flush-projects management command for bulk deletion #1289 by @tdruez in #1291
• Refine the extraction errors reporting include the resource path #1273 by @tdruez in #1276
• Cleanup and re-organise unit test data by @tdruez in #1296
• Add tutorial for end-to-end scanning to DejaCode #1280 by @pombredanne in #1295
• Resolve dependencies from lockfiles by @AyanSinhaMahapatra in #1244
• Update scancode-toolkit to v32.2.1 by @AyanSinhaMahapatra in #1305
• Add support for pipeline "selected_groups" in the run cli #1306 by @tdruez in #1307

Full Changelog: v34.6.3...v34.7.0

v34.6.3

Changelog

• Use the --option=value syntax for args entries in place of --option value
  for fetching Docker images using skopeo through run_command_safely calls. #1257
• Fix an issue in the d2d JavaScript mapper. #1274
• Add support for a ignored_vulnerabilities field on the Project configuration. #1271

What's Changed

• Use the --option=value syntax for run_command_safely args #1257 by @tdruez in #1270
• Fix an issue in the d2d JavaScript mapper by @tdruez in #1274
• Add ignored_vulnerabilities field on the Project configuration #1271 by @tdruez in #1281

Full Changelog: v34.6.2...v34.6.3

v34.6.2

Changelog

• Store SBOMs headers in the Project.extra_data field during the load_sboms
  pipeline. #1253
• Add support for fetching Git repository as Project input. #921
• Enhance the logging and reporting of input fetch exceptions. #1257

What's Changed

• Do not sys.exit in execute_project function by @tdruez in #1265
• Store SBOMs headers in the Project.extra_data field #1253 by @tdruez in #1266
• UI enhancements by @tdruez in #1267
• Add support for fetching git repo as Project input #921 by @tdruez in #1254
• Enhance the logging and reporting of input fetch exceptions #1257 by @tdruez in #1269

Full Changelog: v34.6.1...v34.6.2

v34.6.1

Changelog

• Remove print statements from migration files.
• Display full traceback on error in the execute management command.
• Log the Project message creation.
• Refactor the get_env_from_config_file to support empty config file.

What's Changed

• Release 34.6.1 by @tdruez in #1260

Full Changelog: v34.6.0...v34.6.1

v34.6.0

Changelog

• Add a new scan_for_virus add-on pipeline based on ClamAV scan.
  Found viruses are stored as "error" Project messages and on their related codebase
  resource instance using the extra_data field. #1182
• Add ability to filter by tag on the resource list view. #1217
• Use "unknown" as the Package URL default type when no values are provided for that
  field. This allows to create a discovered package instance instead of raising a
  Project error message. #1249
• Rename DiscoveredDependency resolved_to to resolved_to_package, and
  resolved_dependencies to resolved_from_dependencies for clarity and
  consistency.
  Add children_packages and parent_packages ManyToMany field on the
  DiscoveredPackage model.
  Add full dependency tree in the CycloneDX output. #1066
• Add a new run entry point for executing pipeline as a single command. #1256
• Generate a DiscoveredPackage.package_uid in create_from_data when not provided. #1256

What's Changed

• Add ability to filter by tag on the resource list view #1217 by @tdruez in #1247
• Increase size of CodebaseResource.status from 30 to 50 by @JonoYang in #1248
• Implement a ScanForVirus Pipeline #1182 by @tdruez in #1193
• Include virus report in the resource extra_data field by @keshav-space in #1250
• Use "unknown" as the Package URL default type for missing data #1249 by @tdruez in #1251
• Add children_packages m2m and rename resolved_to_package #1066 by @tdruez in #1252
• Add an entry point for executing pipeline as a single command by @tdruez in #1256
• Generate a package_uid in create_from_data when not provided #1256 by @tdruez in #1258
• Release 34.6.0 by @tdruez in #1259

Full Changelog: v34.5.0...v34.6.0

v34.5.0

Changelog

• Display the current path location in the "Codebase" panel as a navigation breadcrumbs. #1158
• Fix a rendering issue in the dependency details view when for_package or
  datafile_resource fields do not have a value. #1177
• Add a new CollectPygmentsSymbolsAndStrings pipeline (addon) for collecting source
  symbol, string and comments using Pygments. #1179
• Workaround an issue with the cyclonedx-python-lib that does not allow to load
  SBOMs that contains properties with no values.
  Also, a few fixes pre-validation are applied before deserializing thr SBOM for
  maximum compatibility. #1185 #1230
• Add a new CollectTreeSitterSymbolsAndStrings pipeline (addon) for collecting source
  symbol and string using tree-sitter. #1181
• Fix inspect_packages pipeline to properly link discovered packages and dependencies to
  codebase resources of package manifests where they were found. Also correctly assign
  the datasource_ids attribute for packages and dependencies. #1180
• Add "Product name" and "Product version" as new project settings. #1197
• Add "Product name" and "Product version" as new project settings. #1197
• Raise the minimum RAM required per CPU code in the docs.
  A good rule of thumb is to allow 2 GB of memory per CPU.
  For example, if Docker is configured for 8 CPUs, a minimum of 16 GB of memory is
  required. #1191
• Add value validation for the search complex query syntax. #1183
• Bump matchcode-toolkit version to v5.0.0.
• Fix the content of the package_url field in CycloneDX outputs. #1224
• Enhance support for encoded package_url during the conversion to model fields. #1171
• Remove the scancode_license_score option from the Project configuration. #1231
• Remove the extract_recursively option from the Project configuration. #1236
• Add support for a ignored_dependency_scopes field on the Project configuration. #1197
• Add support for storing the scancode-config.yml file in codebase.
  The scancode-config.yml file can be provided as a project input, or can be located
  in the codebase/ immediate subdirectories. This allows to provide the configuration
  file as part of an input archive or a git clone for example. #1236
• Provide a downloadable YAML scancode-config.yml template in the documentation. #1197
• Add support for CycloneDX SBOM component properties as generated by external tools.
  For example, the ResolvedUrl generated by cdxgen is now imported as the package
  download_url.

What's Changed

• Display the current path location in the "Codebase" panel #1158 by @tdruez in #1173
• Add D2D for ELFs and Go binaries #1113 #1114 by @TG1999 in #1170
• Fix a rendering issue in the dependency details view #1177 by @tdruez in #1178
• Addon pipeline to collect pygments symbols by @keshav-space in #1179
• Workaround a loading issue with cyclonedx-python-lib #1185 by @tdruez in #1186
• Addon pipeline to collect tree-sitter symbols by @keshav-space in #1181
• Populate package and dependency attributes in inspect_packages by @AyanSinhaMahapatra in #1180
• Increase scancodeio version length by @TG1999 in #1202
• Add "Product name" and "Product version" as new project settings #1197 by @tdruez in #1204
• Skip source-inspector installation on darwin arm64 (not compatible) by @tdruez in #1205
• Raise the minimum RAM required per CPU in the docs #1191 by @tdruez in #1192
• Mock download get requests #1206 by @JonoYang in #1209
• Add value validation for the search complex query syntax #1183 by @tdruez in #1210
• Add tutorial for symbol and string collection by @keshav-space in #1198
• Bump matchcode-toolkit to v5.0.0 by @JonoYang in #1221
• Rename symbols pipelines by @keshav-space in #1222
• Add requires-review tag for resources not mapped by @TG1999 in #1218
• Fix the content of the package_url field in CycloneDX outputs #1224 by @tdruez in #1225
• Add support for the empty lists in delete_empty_properties #1185 by @tdruez in #1226
• Enhance support for encoded package_url in the conversion to fields by @tdruez in #1227
• Bump matchcode-toolkit version to v5.1.0 by @JonoYang in #1228
• Bump source-inspector to v0.5.1 by @keshav-space in #1233
• Improve the CycloneDX SBOM pre-validation fixes #1230 by @tdruez in #1232
• Enhance help text documentation for Project settings form #1197 by @tdruez in #1229
• Remove the license_score option from Project configuration #1231 by @tdruez in #1234
• Add new flag for approximate file matches in scanpipe.pipes.flag by @JonoYang in #1239
• Add support for CycloneDX SBOM component properties from external tools by @tdruez in #1241
• Add new resolved_to field on DiscoveredDependency #1066 by @tdruez in #1240
• Bump container-inspector and commoncode versions by @JonoYang in #1242
• Add support for scancode-config.yml in codebase #1236 by @tdruez in #1243
• Add support for ignored_dependency_scopes field for configuration by @tdruez in #1235
• Provide a downloadable scancode-config.yml template in docs #1197 by @tdruez in #1245
• Release 34.5.0 by @tdruez in #1246

Full Changelog: v34.4.0...v34.5.0

v34.4.0

Changelog

• Upgrade Gunicorn to v22.0.0 security release.
• Display the list of fields available for the advanced search syntax in the modal UI. #1164
• Add support for CycloneDX 1.6 outputs and inputs.
  Also, the CycloneDX outputs can be downloaded as 1.6, 1.5, and 1.4 spec versions. #1165
• Update matchcode-toolkit to v4.1.0
• Add a new function
  scanpipe.pipes.matchcode.fingerprint_codebase_resources(), which computes
  approximate file matching fingerprints for text files using the new
  get_file_fingerprint_hashes function from matchcode-toolkit.
• Rename the purldb-scan-queue-worker management command to purldb-scan-worker.
• Add docker-compose.purldb-scan-worker.yml to run ScanCode.io as a PurlDB
  scan worker service.

What's Changed

• Add support for CycloneDX 1.6 outputs and inputs by @tdruez in #1165
• Display the list of fields for the advanced search syntax #1164 by @tdruez in #1167
• Update docker-compose.yml by @JonoYang in #1133
• Fingerprint codebase resources by @JonoYang in #1163
• Workaround the unsupported new tools format in cyclonedx #1171 by @tdruez in #1172

Full Changelog: v34.3.0...v34.4.0

v34.3.0

Changelog

• Associate resolved packages with their source codebase resource. #1140

• Add a new CollectSourceStrings pipeline (addon) for collecting source string using xgettext. #1160

Full Changelog: v34.2.0...v34.3.0

v34.2.0

Changelog

• Add support for Python 3.12 and upgrade to Python 3.12 in the Dockerfile. #1138
• Add support for CycloneDX XML inputs. #1136
• Upgrade the SPDX schema to v2.3.1 #1130

Full Changelog: v34.1.0...v34.2.0