Update to Latest NTIA-conformance-checker version

[jspeed-meyers]

v0.2.0 now available: https://github.com/spdx/ntia-conformance-checker/releases/tag/v0.2.0

My tentative suggestion is to use the GitHub release again this time. But for the NEXT update, you could switch to the PyPI release. If you like to live dangerously though (🗡️!!), there is also a PyPI release, but its newness makes it untested.

PyPI release link: https://pypi.org/project/ntia-conformance-checker/#history

[goneall]

With the changes in the ntia-conformance-checker, it looks like we'll need to change the implementation.

Current call to the library is at

spdx-online-tools/src/app/core.py

Line 210 in 43e330c

retval = check_anything.check_minimum_elements(str(settings.APP_DIR + uploaded_file_url)).messages

I'm thinking we'll want to change this to use the SbomChecker passing in the file and calling the printMissingComponents function and displaying the output as is.

The only thing I'm not completely clear on is the opening the file - it looks like I should pass in the filename str(settings.APP_DIR + uploaded_file_url) based on the current code, but I'm not sure if there isn't a more straightforward approach.

@jspeed-meyers @linynjosh - Let me know your thoughts.

[jspeed-meyers]

@goneall -- I thought the new changes might break something. :/ But hopefully the new printing mode is worth it :)

My intention was for both sbom.print_table_output() and sbom.print_components_missing_info() to be used inside spdx-online-tools. There are two ways (at least):

One is to treat ntia-conformance-checker as a CLI app and to call it with subprocess. For instance:

result = subprocess.run(
    ["ntia-checker", "--output", "print", "--verbose", "--file", 
     settings.APP_DIR + uploaded_file_url],
    capture_output=True,
    text=True
)
print(result.stdout)

This way the table view is printed and any missing components are printed too. And the CLI entrypoint takes care of all of this and you don't have to use ntia-conformance-checker as a library.

It probably is also possible to use ntia-conformance-checker as a library, but there will (I think) need to be some small changes. I think spdx/ntia-conformance-checker#81 will actually break the ability of ntia-conformance-checker to be a library. But a 10-minute code change should fix that.

I would therefore recommend the former path, but if you prefer the latter path, just say so and I can put in a new PR.

Regardless, I'm glad to debug this with you further if you get stuck.

[goneall]

@jspeed-meyers - I attempted using the command line with little success, if you could take a look at PR #432

I suspect it is a setup issue - probably need to install the conformance checker as an app.

[jspeed-meyers]

@goneall - Sorry for this trouble.

Upon second thought, I suspect that using the CLI app directly is not the way to go for two reasons. First, the security implications are actually a concern as the Lift static analysis tool results suggest, especially because this function is to take user input, specifically a filename. Given that, staying away from subprocess seems prudent. Second, I actually do think it should be possible to call ntia-conformance-checker as a library still. And if there is a problem, I'll work on PRs to ntia-conformance-checker to make sure it can work as a library.

On the library route: I do think sbom.print_table_output() and sbom.print_components_missing_info() should both be called after instantiating SbomChecker. That way a user gets a table view and detailed output on any nonconformant components.

If you want to pair program at all, let me know. I'm not too skilled at actually developing an deploying web applications, so I admit to being a little turned around. Again, sorry for any trouble!

[goneall]

@jspeed-meyers - no worries - I just attempted using the SbomChecker and ran into an import problem. I'm not really much of a Python programmer and I have to admit I spend most of my time with these import issues.

Let me know if you see an easy fix. I'm wondering if the ntia-conformance-checker needs to export the class somehow.

The attempt is at PR #433

[jspeed-meyers]

@goneall - I come bearing good news, derived from turning ignorance into some sort of fragile, limited knowledge.

There is a fix (I think). You're right that the class needs to be exported. I put in a PR to ntia-conformance-checker to export the class. See PR #434

The code should NOW be able to be used in this manner:

import ntia_conformance_checker as ntia

sbom = ntia.SbomChecker(settings.APP_DIR + uploaded_file_url)
sbom.print_table_output()
sbom.print_components_missing_info()

So take 3! And I'll keep debugging if there are more problems.

[KEY CAVEAT: This code will work once the PR above is merged in and we cut a new release. If you hit approve, I have time to hit the release button tomorrow :) ]

[goneall]

Thanks @jspeed-meyers - I think the PR is ready for review.

The output has a bit of an issue with formatting - it seems to be missing newlines, but at least it's producing output.

I'm also not sure the approach I'm using to capture the prints is the best approach. It might be better to allow an optional parameter to the print functions in the SbomChecker so it can print directly.

Below is a screenshot of the output:

[jspeed-meyers]

@goneall, well, it's progress :)

I opened an issue in ntia-conformance-checker to create a print option that is optimized ntia-conformance-checker. We can target that feature for the next release.

The crux of the problem, from a conceptual perspective, is that the current printing functionality has been optimized for a terminal, but the UI in spdx-online-tools doesn't appear to operate like a terminal. I suspect this won't be hard to improve, but I'll need to investigate.

I'll reviewed the PR right now too.

[goneall]

Note there is a new HTML output option: spdx/ntia-conformance-checker#102

On the next release of the NTIA conformance checker, we can update the online tools to output HTML.