Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add registry1 zarf flavors to uds core #63

Merged
merged 47 commits into from
Jan 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d3665be
add flavors
zachariahmiller Dec 11, 2023
5315889
fix promtail chart image tag ref and make validate-all standalone
zachariahmiller Dec 11, 2023
5098de0
playing with composition
jeff-mccoy Dec 12, 2023
3331ab8
composition ignores required
jeff-mccoy Dec 12, 2023
67f738a
partial flavors composition refactor
zachariahmiller Jan 11, 2024
59c5ca6
Update grafana to new pattern/package structure
zachariahmiller Jan 11, 2024
94383d7
refactor flavors with common component pattern
zachariahmiller Jan 12, 2024
f2469ed
refactor flavors with common component pattern
zachariahmiller Jan 12, 2024
2ffc41b
merge main into flavors, resolve conflicts, fix missed values deletions
zachariahmiller Jan 12, 2024
74f4b82
Update CI, remove missed, extraneous, values in flavor specific value…
zachariahmiller Jan 12, 2024
3ebfe8d
fix missed merge conflict in settings.json
zachariahmiller Jan 12, 2024
75e3c6b
run pepr format
zachariahmiller Jan 12, 2024
07cf53f
fix missing security context in neuvector monitor values
zachariahmiller Jan 12, 2024
683a3bf
update CI to add running (only) a full test for registry1 flavor
zachariahmiller Jan 12, 2024
af83a83
add registry1 login
zachariahmiller Jan 12, 2024
490a244
chore: refactor istio flavors (#108)
mjnagel Jan 16, 2024
e066e21
Update .yamllint
zachariahmiller Jan 16, 2024
640f51a
chore: linting fixes
mjnagel Jan 16, 2024
5747a62
rename UDS_FLAVOR to FLAVOR and update bundle publish path to support…
zachariahmiller Jan 17, 2024
36cee94
chore: merge main into flavors and resolve conflicts
zachariahmiller Jan 17, 2024
da674eb
fix: disable neuvector monitor chart until securitycontext is exposed…
zachariahmiller Jan 17, 2024
c6cc82d
fix: re-add required: true on istio control plane so single capabilit…
zachariahmiller Jan 17, 2024
7dfaf37
chore: resolve yaml lint issues after merge, pepr format
zachariahmiller Jan 17, 2024
580614c
chore: add missing default task at top level taskfile, update upgrade…
zachariahmiller Jan 17, 2024
e02ed11
chore: update uds cli version to properly set the FLAVOR variable for…
zachariahmiller Jan 17, 2024
e52949e
chore: add regitry1 login for upgrade job
zachariahmiller Jan 17, 2024
efbca27
chore: fix settings.json uds version, refactor test apps to zarf package
zachariahmiller Jan 17, 2024
c0509d6
fix: add neuvector peer auth exception
zachariahmiller Jan 18, 2024
e8c014f
Merge branch 'main' into 60-add-registry1-zarf-flavors-to-uds-core
zachariahmiller Jan 18, 2024
ed9de1d
fix: remove deployment strategy
zachariahmiller Jan 18, 2024
225d400
refactor workflow matrix
zachariahmiller Jan 18, 2024
4f4225d
remove single capability conditional
zachariahmiller Jan 18, 2024
d3d5724
chore: update concurrency group name
zachariahmiller Jan 18, 2024
dc19b05
chore: refactor how we temporarily disable the neuvector exporter
zachariahmiller Jan 18, 2024
8d1b5ae
chore: uncomment monitor chart in neuvector common
zachariahmiller Jan 18, 2024
2a92191
docs: update readme to reflect flavor usage and update registry paths
zachariahmiller Jan 18, 2024
e2b814f
ci: fix upgrade test on release please PRs (#113)
mjnagel Jan 18, 2024
b7a41e8
fix: update publish to ignore arm64 for registry1 flavor, remove extr…
zachariahmiller Jan 18, 2024
bac065f
chore: remove extra package.json dev dependency
zachariahmiller Jan 19, 2024
6925d78
ci: refactor to avoid skipped job ugliness (#115)
mjnagel Jan 19, 2024
8e83760
fix: add retry action action on registry1 docker login to mitigate in…
zachariahmiller Jan 19, 2024
f943b57
fix: pass secrets as inputs to setup action
zachariahmiller Jan 19, 2024
be707d0
ci: add formatting checks before pipeline execution
zachariahmiller Jan 19, 2024
7355de9
fix: add missing node setup to format check
zachariahmiller Jan 19, 2024
8b52893
ci: add missing deps to formatting check task
zachariahmiller Jan 19, 2024
5cb4089
chore: fix formatting per checks
zachariahmiller Jan 19, 2024
3d87a9c
ci: update step name
zachariahmiller Jan 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion .github/actions/setup/action.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
# action.yml
name: "Setup Environment"
description: "UDS Environment Setup"
inputs:
gh_token:
description: 'GITHUB_TOKEN'
required: true
ib_user:
description: 'IRON_BANK_ROBOT_USERNAME'
required: true
ib_password:
description: 'IRON_BANK_ROBOT_PASSWORD'
required: true

runs:
using: "composite"
Expand All @@ -27,4 +37,21 @@ runs:
- name: Install UDS CLI
shell: bash
# renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
run: brew install defenseunicorns/tap/uds@0.5.3
run: brew install defenseunicorns/tap/uds@0.6.1

- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: dummy
password: ${{ inputs.gh_token }}

# Retries intermittent registry1 login action
- uses: Wandalen/wretry.action@v1
with:
attempt_limit: 3
action: docker/login-action@v3
with: |
registry: registry1.dso.mil
username: ${{ inputs.ib_user }}
password: ${{ inputs.ib_password }}
38 changes: 38 additions & 0 deletions .github/workflows/pull-request-conditionals.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,32 @@ defaults:
run:
shell: bash -e -o pipefail {0} # Ensures that scripts fail on error and pipefail is set.

# Abort prior jobs in the same workflow / PR
concurrency:
group: test-${{ github.ref }}
cancel-in-progress: true

jobs:
lint-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Use Node.js latest
uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65 # v4.0.0
with:
node-version: 20
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
- name: Install UDS CLI
shell: bash
# renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
run: brew install defenseunicorns/tap/uds@0.6.1
- name: Run Formatting Checks
run: uds run lint-check

# This job checks if there are changes in specific paths source packages.
check-paths:
needs: lint-check
runs-on: ubuntu-latest
name: Select Jobs
outputs:
Expand All @@ -41,7 +64,22 @@ jobs:
strategy:
matrix:
package: ${{ fromJSON(needs.check-paths.outputs.packages) }}
flavor: [upstream, registry1]
uses: ./.github/workflows/test.yaml
with:
package: ${{ matrix.package }}
flavor: ${{ matrix.flavor }}
secrets: inherit # Inherits all secrets from the parent workflow.

# This job triggers a separate workflow for each flavor core package.
run-package-upgrade-test:
needs: check-paths
name: Schedule
strategy:
matrix:
package: [all]
flavor: [upstream, registry1]
uses: ./.github/workflows/test-upgrade.yaml
with:
flavor: ${{ matrix.flavor }}
secrets: inherit # Inherits all secrets from the parent workflow.
14 changes: 7 additions & 7 deletions .github/workflows/tag-and-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ jobs:
run: echo "release_created=${{ steps.tag.outputs.release_created || false }}" >> $GITHUB_OUTPUT

publish-uds-core:
strategy:
matrix:
flavor: [upstream, registry1]
needs: tag-new-version
if: ${{ needs.tag-new-version.outputs.release_created == 'true'}}
runs-on: ubuntu-latest
Expand All @@ -36,16 +39,13 @@ jobs:

- name: Environment setup
uses: ./.github/actions/setup

- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: dummy
password: ${{ secrets.GITHUB_TOKEN }}
gh_token: ${{ secrets.GITHUB_TOKEN }}
ib_user: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
ib_password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}

- name: Publish UDS Zarf Package
run: uds run -f tasks/publish.yaml packages
run: uds run -f tasks/publish.yaml packages --set FLAVOR=${{ matrix.flavor }}

- name: Publish UDS Bundles
run: uds run -f tasks/publish.yaml bundles
Expand Down
50 changes: 50 additions & 0 deletions .github/workflows/test-upgrade.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Test Upgrade of UDS Core

on:
# Manual trigger
workflow_dispatch:
inputs:
flavor:
type: string
description: "Flavor of the core package to test"
required: true
# Triggered by pull-request-conditionals.yaml
workflow_call:
inputs:
flavor:
type: string
description: "Flavor of the core package to test"
required: true

permissions:
contents: read

jobs:
test-upgrade:
runs-on: "uds-ubuntu-big-boy-8-core"
timeout-minutes: 30
name: Test Upgrade

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Environment setup
uses: ./.github/actions/setup
with:
gh_token: ${{ secrets.GITHUB_TOKEN }}
ib_user: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
ib_password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}

- name: Test upgrade
run: uds run test-uds-core-upgrade --set FLAVOR=${{ inputs.flavor }}

- name: Debug Output
if: ${{ !cancelled() }}
run: |
kubectl get all -A
kubectl get package -A

- name: Save logs
if: always()
uses: ./.github/actions/save-logs
51 changes: 16 additions & 35 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,21 @@ on:
type: string
description: "The name of the source package to test"
required: true
flavor:
type: string
description: "Flavor of the source package to test"
required: true
# Triggered by pull-request-conditionals.yaml
workflow_call:
inputs:
package:
type: string
description: "The name of the source package to test"
required: true

# Abort prior jobs in the same workflow / PR
concurrency:
group: test-${{ github.ref }}-${{ inputs.package }}
cancel-in-progress: true
flavor:
type: string
description: "Flavor of the source package to test"
required: true

permissions:
contents: read
Expand All @@ -28,7 +31,7 @@ jobs:
test:
runs-on: "${{ inputs.package == 'all' && 'uds-ubuntu-big-boy-8-core' || 'ubuntu-latest'}}"
timeout-minutes: 30
name: Test
name: Test Install
env:
UDS_PKG: ${{ inputs.package }}

Expand All @@ -38,43 +41,21 @@ jobs:

- name: Environment setup
uses: ./.github/actions/setup
with:
gh_token: ${{ secrets.GITHUB_TOKEN }}
ib_user: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
ib_password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}

- name: Test a single source package
if: ${{ inputs.package != 'all' }}
run: uds run test-single-package
run: uds run test-single-package --set FLAVOR=${{ inputs.flavor }}

- name: Test UDS Core
if: ${{ inputs.package == 'all' }}
run: uds run test-uds-core

- name: Debug Output
if: ${{ !cancelled() }}
run: |
kubectl get all -A
kubectl get package -A

- name: Save logs
if: always()
uses: ./.github/actions/save-logs

test-upgrade:
if: ${{ inputs.package == 'all' }}
runs-on: "uds-ubuntu-big-boy-8-core"
timeout-minutes: 30
name: Test Upgrade

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Environment setup
uses: ./.github/actions/setup

- name: Test upgrade
run: uds run test-uds-core-upgrade
run: uds run test-uds-core --set FLAVOR=${{ inputs.flavor }}

- name: Debug Output
if: ${{ !cancelled() }}
if: ${{ inputs.package != 'all' && !cancelled() }}
run: |
kubectl get all -A
kubectl get package -A
Expand Down
4 changes: 2 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@
},
"yaml.schemas": {
// renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
"https://raw.githubusercontent.com/defenseunicorns/uds-cli/v0.5.3/uds.schema.json": [
"https://raw.githubusercontent.com/defenseunicorns/uds-cli/v0.6.1/uds.schema.json": [
"uds-bundle.yaml"
],

// renovate: datasource=github-tags depName=defenseunicorns/uds-cli versioning=semver
"https://raw.githubusercontent.com/defenseunicorns/uds-cli/v0.5.3/tasks.schema.json": [
"https://raw.githubusercontent.com/defenseunicorns/uds-cli/v0.6.1/tasks.schema.json": [
"tasks.yaml",
"tasks/**/*.yaml",
"src/**/validate.yaml"
Expand Down
39 changes: 39 additions & 0 deletions .yamllint
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
yaml-files:
jeff-mccoy marked this conversation as resolved.
Show resolved Hide resolved
- '**/*.y*ml'
- '.yamllint'

# Ignore files from upstream
ignore:
- 'k3d/local/manifests/metallb/metallb-native.yaml'
- '**/.terraform/**'
- '**/chart/templates**'
zachariahmiller marked this conversation as resolved.
Show resolved Hide resolved
- 'node_modules/**'

rules:
anchors: enable
braces: enable
brackets: enable
colons: enable
commas: enable
comments:
level: warning
comments-indentation:
level: warning
document-end: disable
document-start:
level: warning
empty-lines: enable
empty-values: disable
float-values: disable
hyphens: enable
indentation: enable
key-duplicates: enable
key-ordering: disable
line-length: disable
new-line-at-end-of-file: enable
new-lines: enable
octal-values: disable
quoted-strings: disable
trailing-spaces: enable
truthy:
level: warning
23 changes: 19 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The core applications are:
| Dependency | Minimum Version |
| -------------------------------------------------------------- | --------------- |
| [Zarf](https://github.com/defenseunicorns/zarf/releases) | 0.32.1 |
| [UDS CLI](https://github.com/defenseunicorns/uds-cli/releases) | 0.5.3 |
| [UDS CLI](https://github.com/defenseunicorns/uds-cli/releases) | 0.6.1 |
| [NodeJS](https://nodejs.org/en/download/) | LTS or Current |

<!-- endtable -->
zachariahmiller marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -41,10 +41,10 @@ A common need is bootstrapping a new UDS Core environment for development or tes

```bash
# ARM version
uds deploy oci://ghcr.io/defenseunicorns/packages/uds/bundles/k3d-core:arm64
uds deploy oci://ghcr.io/defenseunicorns/packages/uds/bundles/upstream/k3d-core:arm64

# AMD version
uds deploy oci://ghcr.io/defenseunicorns/packages/uds/bundles/k3d-core:amd64
uds deploy oci://ghcr.io/defenseunicorns/packages/uds/bundles/upstream/k3d-core:amd64
```

The bundle includes the uds.dev certs by default. You can use the UDS environment variables to override the default values. E.g.
Expand All @@ -58,7 +58,15 @@ UDS_TENANT_TLS_KEY=$(cat tenant.key)

UDS_DOMAIN=example.com

uds deploy oci://ghcr.io/defenseunicorns/package/uds/bundles/k3d-core:amd64
uds deploy oci://ghcr.io/defenseunicorns/package/uds/bundles/upstream/k3d-core:amd64
```

Note: In order to deploy using Iron Bank images, update the registry path replaceing ```upstream``` with ```registry1``` For example:

```bash
# ARM version
uds deploy oci://ghcr.io/defenseunicorns/packages/uds/bundles/registry1/k3d-core:arm64

```

### UDS Core Packages
Expand Down Expand Up @@ -114,3 +122,10 @@ UDS_PKG=neuvector uds run deploy-single-package
```bash
uds run -f src/neuvector/tasks/validate.yaml run
```

Note: The run tasks above default to targetting the upstream (oss) package flavor.
In order to run any of the above commands against the registry1 flavor the command must be updated with ```--set FLAVOR=registry1``` For example:

```bash
uds run test-uds-core --set FLAVOR=registry1
```
Loading
Loading