Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/Doc Request: zarf.yaml currently requires docker.io/ prefix to be specified for images from docker hub #194

Closed
neoakris opened this issue Dec 8, 2021 · 1 comment · Fixed by #196

Comments

@neoakris
Copy link

neoakris commented Dec 8, 2021

Short Description of Minor Issue:

I'm investigating tooling / workflow for an Ephemeral Transport Registry (for air gap registry imports with a good UX) and Zarf looks like a great candidate, I discovered If I have this:

kind: ZarfPackageConfig
metadata:
  name: ephemeral-transport-registry
components:
- name: images
  required: true
  images:
  - docker.io/busybox
  - docker.io/rancher/backup-restore-operator:v1.2.0
  - rancher/cis-operator:v1.0.6
  - rancher/configmap-reload:v0.3.0-rancher4
  - rancher/coredns-coredns:1.6.2

Then only the docker.io/... will get uploaded
and the imperative url will be ignored
(so only 2/5 images will get loaded into zarf's registry)


So my feature request / ask is for either:

  1. zarf cli to get smart enough to add docker.io/ prefix to images from docker hub
  2. or document the limitation / requirement that images use full domain names.

The following is additional details of an interesting use case / example usage:

What I'm trying to do is as follows:

  1. generate an arbitrary images.txt list of container images
  2. imperatively generate a declarative zarf.yaml config from images.txt
  3. use zarf to declaratively populate an ephemeral registry
# provision fresh centos7 and ssh in

# become root
sudo su - 

# install zarf 0.13.3 and zarf's dependencies artifact tar
cd ~
curl -L https://zarf-public.s3-us-gov-west-1.amazonaws.com/release/v0.13.3/zarf > zarf-cli
chmod +x zarf-cli
sudo mv zarf-cli /usr/bin/zarf
curl -L https://zarf-public.s3-us-gov-west-1.amazonaws.com/release/v0.13.3/zarf-init.tar.zst > zarf-init.tar.zst #538mb, takes about 1m20sec

# init zarf
cd ~
zarf init --components=management --host=127.0.0.1 --confirm

# wait for zarf registry to become healthy
time while [[ $(kubectl get pods -l app=docker-registry -n=registry -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for registry to be ready (about 2 mins)..." && sleep 5; done; echo "Registry is ready"



# Create test data (that won't require registry auth for the sake of convenience)
cat > images.txt <<EOF
docker.io/busybox
docker.io/rancher/backup-restore-operator:v1.2.0
rancher/cis-operator:v1.0.6
rancher/configmap-reload:v0.3.0-rancher4
rancher/coredns-coredns:1.6.2
EOF

# If images in images.txt required auth could use this commented out stuff
# Pre-login to Authenticated Registry
# export REGISTRY1_USERNAME=REPLACE_ME
# export REGISTRY1_PASSWORD=REPLACE_ME
# zarf tools registry login registry1.dso.mil -u $REGISTRY1_USERNAME -p $REGISTRY1_PASSWORD

# Imperatively generate declarative zarf.yaml config from images.txt
TEMPLATIZED_IMAGES_LIST=$(cat images.txt | sed 's/^/  - /' )
cat > zarf.yaml <<EOF
kind: ZarfPackageConfig
metadata:
  name: ephemeral-transport-registry
components:
- name: images
  required: true
  images:
$TEMPLATIZED_IMAGES_LIST
EOF

# Eyeball validation of zarf config
head zarf.yaml

time zarf package create --confirm
time zarf package deploy zarf-package-ephemeral-transport-registry.tar.zst --confirm
# gives info about the 2 that worked, warn about the 3 the failed

# verify contents of registry
zarf tools registry catalog 127.0.0.1
@jeff-mccoy
Copy link
Contributor

I think this is a duplicate of #44. This is currently fixed in a PR that's turned into a giant PR so I'm currently breaking it down into a smaller one to get this fix among others in sooner than later.

jeff-mccoy added a commit that referenced this issue Dec 11, 2021
)

### Breaking Changes:
* `localhost` is no longer a valid option for cluster ingress when initializing a zarf cluster. Instead you have to use a `127.0.0.1` or some other local ip found via `ifconfig`

### Fixes:
* No longer depends on 127.0.0.1 local bindings for the registry / gitops service
    * should fix #193 
* Resolve outstanding issues with image hostname swapping and
    * fixes #18
    * fixes #44
    * fixes #194

### Features:
* Adds `before` and `after` script options when defining a `zarf.yaml` with an optional retry flag
* Add symlink to ZarfFile for creating links to places files
* Add template boolean to ZarfFile to allow injection of zarf variables into text files
* Adds a new `zarf tool` command to print out config schema and commit the output to the repo (will need to make a git hook or something later on)
* Changes `zarf destroy` command to run any script that starts with `zarf-clean` instead of only running the k3s-remove script
* Add new ZarfState and `.zarf-state.yaml` for persisting host information from `zarf init` to `zarf package deploy`
* Remove all hard-coded logic for k3s install, now uses only standard zarf component features like everything else
* Add user prompt with host/IP address suggestions for ingress

#### Misc:
* Upgrades k3s from v1.21.2 to v1.21.6
* Adds optional regex filter for when performing RecursiveFileList()
* Adds more description to the components in zarf.yaml
* Renames type ZarfConfig to ZarfPackage in the config pkg
* Handful of general code organizing changes (moving yaml related functions to the `...../utils/yaml.go`, etc.)
* Expose execCommand() with stdout control
* Move traefik to standalone component and drop the internal k3s install of traefik
* Use the airgap tarball of K3s instead of manually listing images
* Cleanup init prompt logic
jeff-mccoy added a commit that referenced this issue Feb 8, 2022
)

### Breaking Changes:
* `localhost` is no longer a valid option for cluster ingress when initializing a zarf cluster. Instead you have to use a `127.0.0.1` or some other local ip found via `ifconfig`

### Fixes:
* No longer depends on 127.0.0.1 local bindings for the registry / gitops service
    * should fix #193
* Resolve outstanding issues with image hostname swapping and
    * fixes #18
    * fixes #44
    * fixes #194

### Features:
* Adds `before` and `after` script options when defining a `zarf.yaml` with an optional retry flag
* Add symlink to ZarfFile for creating links to places files
* Add template boolean to ZarfFile to allow injection of zarf variables into text files
* Adds a new `zarf tool` command to print out config schema and commit the output to the repo (will need to make a git hook or something later on)
* Changes `zarf destroy` command to run any script that starts with `zarf-clean` instead of only running the k3s-remove script
* Add new ZarfState and `.zarf-state.yaml` for persisting host information from `zarf init` to `zarf package deploy`
* Remove all hard-coded logic for k3s install, now uses only standard zarf component features like everything else
* Add user prompt with host/IP address suggestions for ingress

#### Misc:
* Upgrades k3s from v1.21.2 to v1.21.6
* Adds optional regex filter for when performing RecursiveFileList()
* Adds more description to the components in zarf.yaml
* Renames type ZarfConfig to ZarfPackage in the config pkg
* Handful of general code organizing changes (moving yaml related functions to the `...../utils/yaml.go`, etc.)
* Expose execCommand() with stdout control
* Move traefik to standalone component and drop the internal k3s install of traefik
* Use the airgap tarball of K3s instead of manually listing images
* Cleanup init prompt logic

Signed-off-by: Jeff McCoy <code@jeffm.us>
Noxsios pushed a commit that referenced this issue Mar 8, 2023
)

### Breaking Changes:
* `localhost` is no longer a valid option for cluster ingress when initializing a zarf cluster. Instead you have to use a `127.0.0.1` or some other local ip found via `ifconfig`

### Fixes:
* No longer depends on 127.0.0.1 local bindings for the registry / gitops service
    * should fix #193
* Resolve outstanding issues with image hostname swapping and
    * fixes #18
    * fixes #44
    * fixes #194

### Features:
* Adds `before` and `after` script options when defining a `zarf.yaml` with an optional retry flag
* Add symlink to ZarfFile for creating links to places files
* Add template boolean to ZarfFile to allow injection of zarf variables into text files
* Adds a new `zarf tool` command to print out config schema and commit the output to the repo (will need to make a git hook or something later on)
* Changes `zarf destroy` command to run any script that starts with `zarf-clean` instead of only running the k3s-remove script
* Add new ZarfState and `.zarf-state.yaml` for persisting host information from `zarf init` to `zarf package deploy`
* Remove all hard-coded logic for k3s install, now uses only standard zarf component features like everything else
* Add user prompt with host/IP address suggestions for ingress

#### Misc:
* Upgrades k3s from v1.21.2 to v1.21.6
* Adds optional regex filter for when performing RecursiveFileList()
* Adds more description to the components in zarf.yaml
* Renames type ZarfConfig to ZarfPackage in the config pkg
* Handful of general code organizing changes (moving yaml related functions to the `...../utils/yaml.go`, etc.)
* Expose execCommand() with stdout control
* Move traefik to standalone component and drop the internal k3s install of traefik
* Use the airgap tarball of K3s instead of manually listing images
* Cleanup init prompt logic

Signed-off-by: Jeff McCoy <code@jeffm.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants