Build a comprehensive container with required tools #655
Assignees
Comments
57 tasks
aj-stein
pushed a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
5 tasks
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 6, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 7, 2024
aj-stein-gsa
added a commit
to aj-stein-gsa/fedramp-automation
that referenced
this issue
Sep 7, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 9, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 9, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 9, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 9, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 9, 2024
aj-stein-gsa
added a commit
that referenced
this issue
Sep 11, 2024
…write perms dropping (#665) * Add initial OCI spec for container for #655 * Now add FR constraint files for #655 * Woops, fix typo in clone path for fd_data_dl scratch container * Constraints in /opt/fedramp sudir, make it WORKDIR * Switch to Alpine Maven scratch image not Debian * Switch to Node for final image, install oscaljs * Add checkout data to final image * Fix missed parameterization of git image * Add non-default OCI image build target for make * Verify GPG signaure of oscal-cli build * Add clean target for OCI image builds * Allow for TLS bypass and proxy in Makefile Disable cert-checking for the local version that is built on laptops for GSA staff who make use of a VPN/proxy solution that intercept all TLS communication for security monitoring. This includes not just Docker, but also the containers as they build an image. Since production images will be made in GitHub Actions without the Makefile, these directives will be ignored. * Do not do slow git clone, use local COPY instead For speed, ease of access, and leave commit metadata from the container ID linked to the commit hash itself, just copy from the outside context of the image build. * Add publish target to Makefile with useful tags Also try docker push to GHCR to start before moving on the "in pipeline" build with GitHub Actions. * Fix repeat docker commands for correct tag-n-push * Correct the org.opencontainers.image.source label * Actions: perms for writing packages (ghcr.io) * Actions: follow GH tutorial, more perms added * Actions: build, sign, push, attest and OCI image This workflow change is the first attempt at building, pushing, and signing the validation-tools image to push to the ghcr.io registry. * Actions: ref_name for image tags problematic For both PRs and non-PR branches, that seems to cause problems for tags that we ought to avoid for now. * Actions: use action correctly, no manual labels * Actions: remove metadata from Dockerfile, use GHA * Actions: woops, forgot explicit checkout path Our GHA CI/CD checks out to `./git-content`, `.` by default so the action directive looking for context did not find the Dockerfile. * Actions: check if least privilege perms block push See more details in this reply and the larger context from others who cannot push a built container to ghcr.io. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: scratch that, `write-all` blocked by org The github.com/GSA organization still blocks the write to an org-level package in very permissive move. Tips from the discussions posts did not help here. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: add metadata action SHA options We need to force SHA1 long (not seven-digit short version to avoid collisions), remove both `sha-` prefix and remove suffix explicitly. * Actions, sigh, really remove `sha256` prefix again It seems that didn't stick the last time, so I will try this config again and follow the official custom hash label strategy from the action example from the official README. * Support MVP platforms, arm64 and amd64 If not we will only support modern Apple computers with modern M1 chips, not Intel environments for PC and older Macs. We need broad support for these top platforms. * Explicit platform option for buildx too for #656 It seems this may be needed because I still get similar but different warnings on multi-platform docker builds when using on macOS on an Apple laptop with a M1 processor and amd64 processor for personal computers with Windows and Linux operating systems respectively. > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested * Pin metadata action and update configs for #656 - Had a slightly wrong version of docker/metadata-action that could not use annotations properly, hence no annotations on image. - Use annotations instead of custom override labels with that action. - Update docker/build-push-registry action to retrieve those labels as well. - Change subject name for attestation to end with `-attestation` suffix to make the GHCR registry entries less confusing. * Woops, attestation subject === image name for #656 I re-read the dogs. Attestations will be uploaded to Sigstore but I will not busy up the registry with them every moment as it will make it even more confusing for novice users and advanced developers what data they are looking for by content-addressable git commit hash ID. * Explanatory comments on Dockerfile lint for #656 For future analysis or assessment, I am leaving information in the Dockerfile as comments to address warning output in docker build and push flagging a potential finding re secrets based on variable names. ```sh 4 warnings found (use docker --debug to expand): - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20) - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43) ``` They are IDs to secrets, not actually secrets, now I have documented it. * Attestations need explicit reg push off for #656 Just removing it may not have done the trick.
github-project-automation
bot
moved this from 👀 In review
to ✅ Done
in FedRAMP Automation
aj-stein-gsa
added a commit
that referenced
this issue
Sep 24, 2024
…write perms dropping (#665) * Add initial OCI spec for container for #655 * Now add FR constraint files for #655 * Woops, fix typo in clone path for fd_data_dl scratch container * Constraints in /opt/fedramp sudir, make it WORKDIR * Switch to Alpine Maven scratch image not Debian * Switch to Node for final image, install oscaljs * Add checkout data to final image * Fix missed parameterization of git image * Add non-default OCI image build target for make * Verify GPG signaure of oscal-cli build * Add clean target for OCI image builds * Allow for TLS bypass and proxy in Makefile Disable cert-checking for the local version that is built on laptops for GSA staff who make use of a VPN/proxy solution that intercept all TLS communication for security monitoring. This includes not just Docker, but also the containers as they build an image. Since production images will be made in GitHub Actions without the Makefile, these directives will be ignored. * Do not do slow git clone, use local COPY instead For speed, ease of access, and leave commit metadata from the container ID linked to the commit hash itself, just copy from the outside context of the image build. * Add publish target to Makefile with useful tags Also try docker push to GHCR to start before moving on the "in pipeline" build with GitHub Actions. * Fix repeat docker commands for correct tag-n-push * Correct the org.opencontainers.image.source label * Actions: perms for writing packages (ghcr.io) * Actions: follow GH tutorial, more perms added * Actions: build, sign, push, attest and OCI image This workflow change is the first attempt at building, pushing, and signing the validation-tools image to push to the ghcr.io registry. * Actions: ref_name for image tags problematic For both PRs and non-PR branches, that seems to cause problems for tags that we ought to avoid for now. * Actions: use action correctly, no manual labels * Actions: remove metadata from Dockerfile, use GHA * Actions: woops, forgot explicit checkout path Our GHA CI/CD checks out to `./git-content`, `.` by default so the action directive looking for context did not find the Dockerfile. * Actions: check if least privilege perms block push See more details in this reply and the larger context from others who cannot push a built container to ghcr.io. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: scratch that, `write-all` blocked by org The github.com/GSA organization still blocks the write to an org-level package in very permissive move. Tips from the discussions posts did not help here. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: add metadata action SHA options We need to force SHA1 long (not seven-digit short version to avoid collisions), remove both `sha-` prefix and remove suffix explicitly. * Actions, sigh, really remove `sha256` prefix again It seems that didn't stick the last time, so I will try this config again and follow the official custom hash label strategy from the action example from the official README. * Support MVP platforms, arm64 and amd64 If not we will only support modern Apple computers with modern M1 chips, not Intel environments for PC and older Macs. We need broad support for these top platforms. * Explicit platform option for buildx too for #656 It seems this may be needed because I still get similar but different warnings on multi-platform docker builds when using on macOS on an Apple laptop with a M1 processor and amd64 processor for personal computers with Windows and Linux operating systems respectively. > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested * Pin metadata action and update configs for #656 - Had a slightly wrong version of docker/metadata-action that could not use annotations properly, hence no annotations on image. - Use annotations instead of custom override labels with that action. - Update docker/build-push-registry action to retrieve those labels as well. - Change subject name for attestation to end with `-attestation` suffix to make the GHCR registry entries less confusing. * Woops, attestation subject === image name for #656 I re-read the dogs. Attestations will be uploaded to Sigstore but I will not busy up the registry with them every moment as it will make it even more confusing for novice users and advanced developers what data they are looking for by content-addressable git commit hash ID. * Explanatory comments on Dockerfile lint for #656 For future analysis or assessment, I am leaving information in the Dockerfile as comments to address warning output in docker build and push flagging a potential finding re secrets based on variable names. ```sh 4 warnings found (use docker --debug to expand): - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20) - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43) ``` They are IDs to secrets, not actually secrets, now I have documented it. * Attestations need explicit reg push off for #656 Just removing it may not have done the trick.
aj-stein-gsa
added a commit
that referenced
this issue
Sep 25, 2024
…write perms dropping (#665) * Add initial OCI spec for container for #655 * Now add FR constraint files for #655 * Woops, fix typo in clone path for fd_data_dl scratch container * Constraints in /opt/fedramp sudir, make it WORKDIR * Switch to Alpine Maven scratch image not Debian * Switch to Node for final image, install oscaljs * Add checkout data to final image * Fix missed parameterization of git image * Add non-default OCI image build target for make * Verify GPG signaure of oscal-cli build * Add clean target for OCI image builds * Allow for TLS bypass and proxy in Makefile Disable cert-checking for the local version that is built on laptops for GSA staff who make use of a VPN/proxy solution that intercept all TLS communication for security monitoring. This includes not just Docker, but also the containers as they build an image. Since production images will be made in GitHub Actions without the Makefile, these directives will be ignored. * Do not do slow git clone, use local COPY instead For speed, ease of access, and leave commit metadata from the container ID linked to the commit hash itself, just copy from the outside context of the image build. * Add publish target to Makefile with useful tags Also try docker push to GHCR to start before moving on the "in pipeline" build with GitHub Actions. * Fix repeat docker commands for correct tag-n-push * Correct the org.opencontainers.image.source label * Actions: perms for writing packages (ghcr.io) * Actions: follow GH tutorial, more perms added * Actions: build, sign, push, attest and OCI image This workflow change is the first attempt at building, pushing, and signing the validation-tools image to push to the ghcr.io registry. * Actions: ref_name for image tags problematic For both PRs and non-PR branches, that seems to cause problems for tags that we ought to avoid for now. * Actions: use action correctly, no manual labels * Actions: remove metadata from Dockerfile, use GHA * Actions: woops, forgot explicit checkout path Our GHA CI/CD checks out to `./git-content`, `.` by default so the action directive looking for context did not find the Dockerfile. * Actions: check if least privilege perms block push See more details in this reply and the larger context from others who cannot push a built container to ghcr.io. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: scratch that, `write-all` blocked by org The github.com/GSA organization still blocks the write to an org-level package in very permissive move. Tips from the discussions posts did not help here. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: add metadata action SHA options We need to force SHA1 long (not seven-digit short version to avoid collisions), remove both `sha-` prefix and remove suffix explicitly. * Actions, sigh, really remove `sha256` prefix again It seems that didn't stick the last time, so I will try this config again and follow the official custom hash label strategy from the action example from the official README. * Support MVP platforms, arm64 and amd64 If not we will only support modern Apple computers with modern M1 chips, not Intel environments for PC and older Macs. We need broad support for these top platforms. * Explicit platform option for buildx too for #656 It seems this may be needed because I still get similar but different warnings on multi-platform docker builds when using on macOS on an Apple laptop with a M1 processor and amd64 processor for personal computers with Windows and Linux operating systems respectively. > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested * Pin metadata action and update configs for #656 - Had a slightly wrong version of docker/metadata-action that could not use annotations properly, hence no annotations on image. - Use annotations instead of custom override labels with that action. - Update docker/build-push-registry action to retrieve those labels as well. - Change subject name for attestation to end with `-attestation` suffix to make the GHCR registry entries less confusing. * Woops, attestation subject === image name for #656 I re-read the dogs. Attestations will be uploaded to Sigstore but I will not busy up the registry with them every moment as it will make it even more confusing for novice users and advanced developers what data they are looking for by content-addressable git commit hash ID. * Explanatory comments on Dockerfile lint for #656 For future analysis or assessment, I am leaving information in the Dockerfile as comments to address warning output in docker build and push flagging a potential finding re secrets based on variable names. ```sh 4 warnings found (use docker --debug to expand): - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20) - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43) ``` They are IDs to secrets, not actually secrets, now I have documented it. * Attestations need explicit reg push off for #656 Just removing it may not have done the trick.
brian-ruf
pushed a commit
to brian-ruf/fedramp-automation
that referenced
this issue
Nov 8, 2024
…write perms dropping (GSA#665) * Add initial OCI spec for container for GSA#655 * Now add FR constraint files for GSA#655 * Woops, fix typo in clone path for fd_data_dl scratch container * Constraints in /opt/fedramp sudir, make it WORKDIR * Switch to Alpine Maven scratch image not Debian * Switch to Node for final image, install oscaljs * Add checkout data to final image * Fix missed parameterization of git image * Add non-default OCI image build target for make * Verify GPG signaure of oscal-cli build * Add clean target for OCI image builds * Allow for TLS bypass and proxy in Makefile Disable cert-checking for the local version that is built on laptops for GSA staff who make use of a VPN/proxy solution that intercept all TLS communication for security monitoring. This includes not just Docker, but also the containers as they build an image. Since production images will be made in GitHub Actions without the Makefile, these directives will be ignored. * Do not do slow git clone, use local COPY instead For speed, ease of access, and leave commit metadata from the container ID linked to the commit hash itself, just copy from the outside context of the image build. * Add publish target to Makefile with useful tags Also try docker push to GHCR to start before moving on the "in pipeline" build with GitHub Actions. * Fix repeat docker commands for correct tag-n-push * Correct the org.opencontainers.image.source label * Actions: perms for writing packages (ghcr.io) * Actions: follow GH tutorial, more perms added * Actions: build, sign, push, attest and OCI image This workflow change is the first attempt at building, pushing, and signing the validation-tools image to push to the ghcr.io registry. * Actions: ref_name for image tags problematic For both PRs and non-PR branches, that seems to cause problems for tags that we ought to avoid for now. * Actions: use action correctly, no manual labels * Actions: remove metadata from Dockerfile, use GHA * Actions: woops, forgot explicit checkout path Our GHA CI/CD checks out to `./git-content`, `.` by default so the action directive looking for context did not find the Dockerfile. * Actions: check if least privilege perms block push See more details in this reply and the larger context from others who cannot push a built container to ghcr.io. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: scratch that, `write-all` blocked by org The github.com/GSA organization still blocks the write to an org-level package in very permissive move. Tips from the discussions posts did not help here. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: add metadata action SHA options We need to force SHA1 long (not seven-digit short version to avoid collisions), remove both `sha-` prefix and remove suffix explicitly. * Actions, sigh, really remove `sha256` prefix again It seems that didn't stick the last time, so I will try this config again and follow the official custom hash label strategy from the action example from the official README. * Support MVP platforms, arm64 and amd64 If not we will only support modern Apple computers with modern M1 chips, not Intel environments for PC and older Macs. We need broad support for these top platforms. * Explicit platform option for buildx too for GSA#656 It seems this may be needed because I still get similar but different warnings on multi-platform docker builds when using on macOS on an Apple laptop with a M1 processor and amd64 processor for personal computers with Windows and Linux operating systems respectively. > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested * Pin metadata action and update configs for GSA#656 - Had a slightly wrong version of docker/metadata-action that could not use annotations properly, hence no annotations on image. - Use annotations instead of custom override labels with that action. - Update docker/build-push-registry action to retrieve those labels as well. - Change subject name for attestation to end with `-attestation` suffix to make the GHCR registry entries less confusing. * Woops, attestation subject === image name for GSA#656 I re-read the dogs. Attestations will be uploaded to Sigstore but I will not busy up the registry with them every moment as it will make it even more confusing for novice users and advanced developers what data they are looking for by content-addressable git commit hash ID. * Explanatory comments on Dockerfile lint for GSA#656 For future analysis or assessment, I am leaving information in the Dockerfile as comments to address warning output in docker build and push flagging a potential finding re secrets based on variable names. ```sh 4 warnings found (use docker --debug to expand): - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20) - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43) ``` They are IDs to secrets, not actually secrets, now I have documented it. * Attestations need explicit reg push off for GSA#656 Just removing it may not have done the trick.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: