license_info_in_file is mandatory in SPDX-2.2

[tpodowd]

What happened:
When I try to validate the spdx-2.2 json file using python-tools command pyspdxtools, it outputs a number of different issues one of them being for each File, it says the license_info_in_file is mandatory.

For example, it says the following for the /etc directory that is listed.

file name must not be an absolute path starting with "/", but is: /etc
license_info_in_file is mandatory in SPDX-2.2

In the spdx file, it has:

  {
   "fileName": "/etc",
   "SPDXID": "SPDXRef-File-etc-c0bccd0a3289c2a9",
   "fileTypes": [
    "OTHER"
   ],
   "checksums": [
    {
     "algorithm": "SHA1",
     "checksumValue": "0000000000000000000000000000000000000000"
    }
   ],
   "licenseConcluded": "NOASSERTION",
   "licenseInfoInFiles": null,
   "copyrightText": ""
  },

What you expected to happen:
Using microsoft's sbom-tool, it has the following for each file which does validate:

      "licenseInfoInFiles": [
        "NOASSERTION"
      ],

Steps to reproduce the issue:

$ mkdir delme/etc
$ SYFT_FILE_METADATA_CATALOGER_ENABLED=true /tmp/syft packages dir:delme --base-path delme -o spdx-json@2.2 --file file_issue.spdx.json
 ✔ Indexed file system                                                                                                                                                                                                     delme
 ✔ Cataloged file digests
 ✔ Cataloged packages              [0 packages]
 ✔ Cataloged file metadata
[0000]  WARN no explicit name and version provided for directory source, deriving artifact ID from the given path (which is not ideal)
$ pyspdxtools -i file_issue.spdx.json
ERROR:root:The document is invalid. The following issues have been found:
verification_code must be None if files_analyzed is False, but is: PackageVerificationCode(value='', excluded_files=[])
license_concluded is mandatory in SPDX-2.2
license_declared is mandatory in SPDX-2.2
license_info_in_file is mandatory in SPDX-2.2
file name must not be an absolute path starting with "/", but is: /etc
license_info_in_file is mandatory in SPDX-2.2

There are other validation issues also... I guess I will write a bug for each one?

Anything else we need to know?:
If I manually edit the json and change the file entry fro null to the array with NOASSERTION, then that particular error goes away.

Environment:

• Output of syft version:

$ /tmp/syft version
Application:     syft
Version:         0.91.0
BuildDate:       2023-09-20T19:42:04Z
GitCommit:       b7fa75d7f82a6816d307805ac07e6965c799e938
GitDescription:  v0.91.0
Platform:        linux/amd64
GoVersion:       go1.21.1
Compiler:        gc

• OS (e.g: cat /etc/os-release or similar):

$ cat /etc/redhat-release
Rocky Linux release 9.1 (Blue Onyx)

[kzantow]

NOTE: "file name must not be an absolute path starting with "/", but is: /etc" is handled by issue: #2093

[kzantow]

This error is not according to the spec: verification_code must be None if files_analyzed is False. The spec says about the package verification code:

(must be omitted) if FilesAnalyzed is false.

This error is also not according to the spec: license_info_in_file is mandatory in SPDX-2.2. The spec says, also:

(shall be omitted) if FilesAnalyzed is false.

[tpodowd]

Hi @kzantow - Thanks for looking at this. I realised that my reproduction instructions were not accurate as I had a syft.yaml file in the directory. I removed this file and updated the instructions in my original comment to include the environment variable SYFT_FILE_METADATA_CATALOGER_ENABLED=true also.

Let me address your comments above also.

This error is not according to the spec: verification_code must be None if files_analyzed is False. The spec says about the package verification code:

(must be omitted) if FilesAnalyzed is false.

This is complaining about this section of the generated file:

 "packages": [
  {
   "name": "delme",
   "SPDXID": "SPDXRef-DocumentRoot-Directory-delme",
   "supplier": "NOASSERTION",
   "downloadLocation": "NOASSERTION",
   "filesAnalyzed": false,
   "packageVerificationCode": {
    "packageVerificationCodeValue": ""
   },
   "licenseConcluded": "",
   "licenseInfoFromFiles": null,
   "licenseDeclared": "",
   "copyrightText": ""
  }

This contains the following:

   "packageVerificationCode": {
    "packageVerificationCodeValue": ""
   },

I believe this is not the same as "must be omitted" as it is explicitly set to an empty string. If I manually edit the json file and update it to read the following, that error disappears.

   "packageVerificationCode": null,

Actually, I could also remove (ie, omit) the key packageVerificationCode altogether, but the spec says that is is required so not sure about that.

[tpodowd]

with respect to:

This error is also not according to the spec: license_info_in_file is mandatory in SPDX-2.2. The spec says, also:

(shall be omitted) if FilesAnalyzed is false.

I think the above is for licenseInfoFromFiles which is different to the error license_info_in_file is mandatory in SPDX-2.2 which is complaining about the 2 of the files in the files section.

The section would be this:

 "files": [
  {
   "fileName": "",
   "SPDXID": "SPDXRef-File--5567c94c988a1a09",
   "fileTypes": [
    "OTHER"
   ],
   "checksums": [
    {
     "algorithm": "SHA1",
     "checksumValue": "0000000000000000000000000000000000000000"
    }
   ],
   "licenseConcluded": "NOASSERTION",
   "licenseInfoInFiles": null,
   "copyrightText": ""
  },
  {
   "fileName": "/etc",
   "SPDXID": "SPDXRef-File-etc-c0bccd0a3289c2a9",
   "fileTypes": [
    "OTHER"
   ],
   "checksums": [
    {
     "algorithm": "SHA1",
     "checksumValue": "0000000000000000000000000000000000000000"
    }
   ],
   "licenseConcluded": "NOASSERTION",
   "licenseInfoInFiles": null,
   "copyrightText": ""
  }
 ],

The errors for this being:

license_info_in_file is mandatory in SPDX-2.2
file name must not be an absolute path starting with "/", but is: /etc
license_info_in_file is mandatory in SPDX-2.2

I think the relevant specification section is:
https://spdx.github.io/spdx-spec/v2.2.2/file-information/#86-license-information-in-file-field

I think the issue is that "null" does not match the required cardinality of 1..*. In the above case I think a good fix is:

"licenseInfoInFiles": [
  "NONE"
],

as one is a directory "/etc" and the other one is "" (actually not sure what file that is matching?). But "NOASSERTION" is also good too if the tool does not check.

[kzantow]

About the

 "packageVerificationCode": {
    "packageVerificationCodeValue": ""
   },

issue, I've filed an upstream PR to fix this: spdx/tools-golang#223

And you're right about the licenseInfoInFiles, sorry for confusing that. The relative path issue is tracked separately. A PR to fix the rest is here: #2168

[tpodowd]

Thanks @kzantow . Will look forward to these changes.

[wagoodman]

Putting this back to in progress since there isn't anything to review until the upstream PR is merged