Use ATEX from PyPI + compress uploaded files

[comps]

Description:

This extends #14203 by using atex from PyPI directly (instead of using git), hopefully resulting in a more stable version.

It also takes advantage of the (new) LZMAJSONAggregator that transparently on-the-fly compresses both the results.json.xz and all files uploaded by tests. This is less critical now (with just STIG), but once we start extending the workflow to all profiles and (later) to run the full "daily productization" set of tests, it will be super useful to keep Testing Farm artifact storage within reasonable limits.

Yes, mixing gz / xz in the code is a bit weird now (eg. using .gz for results.sqlite.gz), but that's just a result of the project separation:

• *.xz comes from an Aggregator (results + uploaded files)
• .sqlite.gz comes from atex-html, a separate project that requires Gzip compression, because web browsers don't support LZMA

Also note atex-html/json2db.py has been updated to support .xz inputs, and (from my testing) seems to work well with all of .json / .json.gz / json.xz.

Rationale:

Some rough Gzip / LZMA estimates show about 20% to 400% improvements in compressed size differences, indicating that LZMA does make sense, especially when RHSecurityCompliance/contest#497 is merged and ALL test uploaded files are uncompressed.

Review Hints:

Note that I am intentionally keeping https://github.com/RHSecurityCompliance/atex (main branch) behind the current 0.11 ATEX release (on PyPI, and tagged on Github) to prevent existing CaC/content PRs breaking on the API change.
But a few days after this PR gets merged, I'll update Github too, so hopefully everyone has rebased by then.

Unfortunately, as indicated by @ggbecker , I don't think we can test this PR using existing Github workflows, so review + blind merge might be needed here.

[github-actions]

ATEX Test Results

Test artifacts have been submitted to Testing Farm.

Results: View Test Results
Workflow Run: View Workflow Details

This comment was automatically generated by the ATEX workflow.

[Mab879]

Does this also need to point to PyPi?

[comps]

Good catch, fixed, thanks!

[openshift-ci]

@comps: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-openshift-node-compliance	dc67da7	link	true	/test e2e-aws-openshift-node-compliance

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.