feat: add --image flag in gator test|expand

[davis-haba]

Signed-off-by: davis-haba davishaba@google.com

Adds --image flag to gator test|expand, which supports pulling OCI Artifacts containing policy bundles for ingestion by gator. #2333

The flag is mixable with the existing --filename flag, as well as with stdin in the case of gator test. For example:

gator test --image=localhost:5000/gator/my-bundle:v1 --filename=my-manifest.yaml

The documentation has been updated with the new --image flag, as well as some information on creating OCI Artifacts with policy bundles.

This PR also introduces duplicate-detection logic in gator test|expand, which will log a warning if resources with the same GKNN are found across any of the input methods (--filename, --image, or stdin). In the case of duplicates, the command does not fail closed, and we cannot make any guarantees about which copy will be used.

Some example warnings:

WARNING - Resource named 'my-resource' (from file: 'file.yaml', image: 'gcr.io/this/that:latest') is already defined in file: 'another.yaml', image:'gcr.io/that/this:v1'

WARNING - Resource named 'my-resource' (from stdin) is already defined in file: 'file.yaml'

WARNING - Resource named 'my-resource' (from file: 'file.yaml') is already defined in stdin

WARNING - Resource named 'my-resource' (from file: 'file.yaml') is already defined in file: 'another.yaml', image: 'gcr.io/this/that:latest'

The gator tests have been updated to run in a container, which is executable via make test-gator-containerized`. The approach creates a "test runner" container to run the tests, and another container inside the test runner to act as the image registry.

The Gator CLI code has also been refactored such that code for each command has its own package, and common code is in gator/reader and gator/. #1779

[codecov-commenter]

Codecov Report

Base: 53.52% // Head: 53.38% // Decreases project coverage by -0.14% ⚠️

Coverage data is based on head (a7aade3) compared to base (92841ff).
Patch coverage: 27.30% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2398      +/-   ##
==========================================
- Coverage   53.52%   53.38%   -0.15%     
==========================================
  Files         117      115       -2     
  Lines       10281    10170     -111     
==========================================
- Hits         5503     5429      -74     
+ Misses       4354     4326      -28     
+ Partials      424      415       -9     

Flag	Coverage Δ
unittests	53.38% <27.30%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/gator/reader/filereader.go	0.00% <0.00%> (ø)
pkg/gator/reader/read_constraints.go	12.50% <0.00%> (ø)
pkg/gator/verify/printer_go.go	46.78% <ø> (ø)
pkg/gator/verify/result.go	72.72% <ø> (ø)
pkg/gator/reader/conflictdetector.go	66.66% <66.66%> (ø)
pkg/gator/verify/runner.go	85.54% <90.32%> (ø)
pkg/gator/test/test.go	61.90% <100.00%> (ø)
pkg/gator/verify/assertion.go	100.00% <100.00%> (ø)
pkg/gator/verify/filter.go	100.00% <100.00%> (ø)
pkg/gator/verify/read_suites.go	88.34% <100.00%> (ø)
... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[acpana]

Thanks for working on this 💯 ; left some nits and questions, nothing blocking from me

[maxsmythe]

Nice refactor!

Basically LGTM modulo some nits, and a more substantial comment that I think we might be able to consolidate Dockerfiles to have only one test image.

[acpana]

some light nits, UX questions and string change callouts;

otherwise LGTM! Thanks for following up here.

[maxsmythe]

LGTM

[sozercan]

this is going to make artifacts created by this (like gator binary) owned by root. can we do this without privileged?

[davis-haba]

The test needs to run docker-inside-docker in order to spin up a local container registry to push to test bundles to. Unfortunately, --privileged is necessary to allow the inner container (the container registry) to be able to access the file system of the outer container (the test runner).

I'm not sure I understand why this would make the gator binary owned by root, as this privileged execution is just for a test image. Is there someway we can exclude it? Otherwise, if this is a problem, we might have to explore other methods of e2e testing this, like perhaps using an external public registry for the test images instead of spinning up a local one.

[sozercan]

@davis-haba overall LGTM, added a few comments and looks like there are merge conflicts now

[acpana]

LGTM !! (modulo merge conflicts)