Toifixes (#304)

* helmdemo: allow merging user helm values, more comments * helminstaller+helmdemo: fix makefiles after removed VERSION file * Add readmes to the examples in components * helmdemo: stricter config scheme, add -X keeplocalblob=true * add missing file ocmconfig to toi doc * helminstaller add link to doc * fix typo in bootstrapping doc * fix components makefiles to use new versions * fix exampple packagespec.yaml * add image mapping to readme of helmdemo * update generated doc after rebase
open-component-model · Apr 4, 2023 · 48b1ee8 · 48b1ee8
1 parent 456b644
commit 48b1ee8
Show file tree

Hide file tree

Showing 17 changed files with 290 additions and 87 deletions.
diff --git a/README.md b/README.md
@@ -38,12 +38,16 @@ The `ocm` CLI documentation can be found [here](<(https://github.com/open-compon
 
 The code for the CLI can be found in [package `cmds/ocm`](https://github.com/open-component-model/ocm/blob/main/cmds/ocm).
 
-An example of how to use the `ocm` CLI in a Makefile can be found in [`examples/make`](https://github.com/open-component-model/ocm/blob/main/examples/make/Makefile).
-More comprehensive examples can be taken from the [`components`](https://github.com/open-component-model/ocm/tree/main/components) contained in this repository. Here a complete component build including a multi-arch image is done and finally packaged into a CTF archive which can be tranported into an OCI repository.
-
 The OCI and OCM support can be found in packages
 [`pkg/contexts/oci`](pkg/contexts/oci) and [`pkg/contexts/ocm`](pkg/contexts/ocm).
 
+## Examples
+An example of how to use the `ocm` CLI in a Makefile can be found in [`examples/make`](https://github.com/open-component-model/ocm/blob/main/examples/make/Makefile).
+
+More comprehensive examples can be taken from the [`components`](https://github.com/open-component-model/ocm/tree/main/components) contained in this repository. [Here](components/helmdemo/README.md) a complete component build including a multi-arch image is done and finally packaged into a CTF archive which can be tranported into an OCI repository. See the readme files for details.
+
+
+
 ## Contributing
 
 Code contributions, feature requests, bug reports, and help requests are very welcome. Please refer to the [Contributing Guide in the Community repository](https://github.com/open-component-model/community/blob/main/CONTRIBUTING.md) for more information on how to contribute to OCM.

diff --git a/cmds/ocm/topics/toi/bootstrapping/topic.go b/cmds/ocm/topics/toi/bootstrapping/topic.go
@@ -28,7 +28,7 @@ executors:
         name: installerimage
     config:
       level: info
-#   parameterMapping:  # optional spiff mapping of Package configuration to 
+#   parameterMapping:  # optional spiff mapping of Package configuration to
 #      ....            # executor parameters
     outputs:
        test: bla
@@ -104,18 +104,18 @@ to the caller.
 
 ### The <code>` + toi.TypeTOIPackage + `</code> Resource
 
-This resource describes an installable software package, whose content is 
+This resource describes an installable software package, whose content is
 contained in the component version, which contains the package resource.
 
 It is a plain yaml resource with the media types media type <code>` + mime.MIME_YAML + `</code>,
-<code>` + mime.MIME_YAML_ALT + `</code> or 
+<code>` + mime.MIME_YAML_ALT + `</code> or
 <code>` + toi.PackageSpecificationMimeType + `</code>) containing
 information required to control the instantiation of an executor.
 
 It has the following format:
 
 - **<code>description</code>** (optional) *string*
-  
+
   A short description of the installation package and some configuration hints.
 
 - **<code>executors</code>** *[]ExecutorSpecification*
@@ -241,7 +241,7 @@ defines additional identity attributes, the complete set must be specified.
 Instead of directly describing an image resource i the package file, it is
 possible to refer to a resource of type ` + toi.TypeTOIExecutor + `. This
 is a yaml file with the media type <code>` + mime.MIME_YAML + `</code>,
-<code>` + mime.MIME_YAML_ALT + `</code> or 
+<code>` + mime.MIME_YAML_ALT + `</code> or
 <code>` + toi.PackageSpecificationMimeType + `</code>) containing
 common information about the executor executor. If used by the package,
 this information is used to validate settings in the package specification.
@@ -274,7 +274,7 @@ It has the following format:
 
   Here the executor may request the provisioning of some credentials with a
   dedicated name/purpose and structure. If specified it will be propagated
-  to a using package. It this uses an own credentials section, this one 
+  to a using package. It this uses an own credentials section, this one
   will be filtered and checked for the actual executor.
 
 - **<code>outputs</code>** (optional) *map[string]OutputSpecification*
@@ -360,7 +360,7 @@ execution and reading provided executor outputs after the execution.
     ├── outputs
     │   ├── &lt;out>       any number of arbitrary output data provided
     │   │               by executor
-    │   └── ...         
+    │   └── ...
     └── run             good practice: typical location for the executed command
 </pre>
 

diff --git a/components/demoplugin/README.md b/components/demoplugin/README.md
@@ -0,0 +1,4 @@
+# Example demoplugin
+
+This folder contains an example how to build a plugin for the ocm CLI
+
diff --git a/components/helmdemo/Makefile b/components/helmdemo/Makefile
@@ -8,9 +8,10 @@ OCMREPO   ?= ghcr.io/$(GITHUBORG)/ocm
 HELMINSTCOMP = $(PROVIDER)/toi/installers/helminstaller
 
 REPO_ROOT                                      := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))/../..
-VERSION                                        = $(shell git describe --tags --exact-match 2>/dev/null|| echo "$$(cat $(REPO_ROOT)/VERSION)")
+VERSION                                        := $(shell go run $(REPO_ROOT)/pkg/version/generate/release_generate.go print-rc-version)
 COMMIT                                         = $(shell git rev-parse HEAD)
 EFFECTIVE_VERSION                              = $(VERSION)-$(COMMIT)
+HELMINSTVERSION ?= $(VERSION)
 
 CREDS := $(shell $(REPO_ROOT)/hack/githubcreds.sh)
 OCM = go run $(REPO_ROOT)/cmds/ocm $(CREDS)
@@ -23,8 +24,8 @@ CHARTSRCS=$(shell find echoserver -type f)
 ctf: $(GEN)/ctf
 
 $(GEN)/ctf: $(GEN)/ca
-	@rm -rf $(GEN)/ctf
-	$(OCM) transfer ca $(GEN)/ca $(GEN)/ctf
+	@rm -f $(GEN)/ctf
+	$(OCM) -X keeplocalblob=true transfer ca $(GEN)/ca $(GEN)/ctf
 	touch $(GEN)/ctf
 
 .PHONY: version
@@ -38,7 +39,7 @@ $(GEN)/ca: $(GEN)/.exists sources.yaml resources.yaml references.yaml $(CHARTSRC
 	$(OCM) create ca -f $(COMPONENT) "$(VERSION)" --provider $(PROVIDER) --file $(GEN)/ca
 	$(OCM) add sources $(GEN)/ca VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" sources.yaml
 	$(OCM) add resources $(GEN)/ca VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" resources.yaml
-	$(OCM) add references  $(GEN)/ca VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" HELMINSTCOMP=$(HELMINSTCOMP) references.yaml
+	$(OCM) add references  $(GEN)/ca VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" HELMINSTCOMP=$(HELMINSTCOMP) HELMINSTVERSION=$(HELMINSTVERSION) references.yaml
 	@touch $(GEN)/ca
 
 .PHONY: eval-resources
@@ -49,18 +50,18 @@ eval-resources:
 push: $(GEN)/ctf $(GEN)/push.$(NAME)
 
 $(GEN)/push.$(NAME): $(GEN)/ctf
-	$(OCM) transfer ctf -f $(GEN)/ctf $(OCMREPO)
+	$(OCM) -X keeplocalblob=true transfer ctf -f $(GEN)/ctf $(OCMREPO)
 	@touch $(GEN)/push.$(NAME)
 
 .PHONY: plain-push
 plain-push: $(GEN)
-	$(OCM) transfer ctf -f $(GEN)/ctf $(OCMREPO)
+	$(OCM) -X keeplocalblob=true transfer ctf -f $(GEN)/ctf $(OCMREPO)
 	@touch $(GEN)/push.$(NAME)
 
 .PHONY: transport
 transport:
 ifneq ($(TARGETREPO),)
-	$(OCM) transfer component -Vc  $(OCMREPO)//$(COMPONENT):$(VERSION) $(TARGETREPO)
+	$(OCM) -X keeplocalblob=true transfer component -Vc  $(OCMREPO)//$(COMPONENT):$(VERSION) $(TARGETREPO)
 endif
 
 $(GEN)/.exists:
@@ -73,6 +74,7 @@ info:
 	@echo "VERSION:  $(VERSION)"
 	@echo "COMMIT:   $(COMMIT)"
 	@echo "CREDS:    $(CREDS)"
+	@echo "version for helminstaller:  $(HELMINSTVERSION)"
 
 .PHONY: describe
 describe: $(GEN)/ctf

diff --git a/components/helmdemo/README.md b/components/helmdemo/README.md
@@ -0,0 +1,95 @@
+# Example echoserver
+
+This folder contains an example how to use the ocm CLI to build and upload a component version to an OCI registry. It refers to a public image and creates a component-descriptor with an appropriate access method. Additionally it embeds a helm chart for installation. Finally it adds a [toi package](../../docs/reference/ocm_toi-bootstrapping.md) specification and a reference to the [helminstaller](../helminstaller/README.md). This allows installing the component with the toi installer.
+
+# Building
+
+You can use `make` to build this component. You will have to adjust the variables at the top of the [makefile](Makefile) to your environment (at least `OCMREPO`). By default all artifacts are built in the `gen` folder of this project.
+
+The main targets are:
+
+* `make ca`: builds a component archive
+* `make ctf`: builds a common transport archive
+* `make push`: stores the component version in an OCI registry
+* `make transport`: transfers the component version between two registries
+* `make descriptor`: displays the component descriptor
+* `make describe`: displays the component and it dependencies in a tree structure
+
+You can find more information [here](../../cmds/helminstaller/README.md).
+
+# Using transported images (image mapping)
+
+When transferring a component version between registries or from a local common transport archive to an OCI registry often images are included. When installing the applications all images in a deployment need to be adjusted so that their url points to the new location. The toi installer can perform this mapping and will dynamically adjust the helm values. This needs to be configured in the `packagespec.yaml` file. The `packagespec.yaml` inserts the content of another file `helmconfig.yaml` where the image mapping is defined.
+
+```yaml
+imageMapping:
+  - tag: image.tag
+    repository: image.repository
+    resource:
+      name: image
+```
+
+This image mapping instructs the helm-installer to replace the tags
+
+```
+image:
+  repository: <add reference to image from component version here>
+  tag:        <add tag from component version here>
+```
+
+with the value it finds in the resource named "image" of the current component version:
+
+```yaml
+---
+name: image
+type: ociImage
+version: "1.0"
+access:
+  type: ociArtifact
+  imageReference: gcr.io/google_containers/echoserver:1.10
+```
+
+## Notes
+
+For a successful image mapping ensure that all you image resources have a globalAccess and relation external in their component descriptor. Only with global access Kubernetes is able to pull an image. Ensure that your components are transferred with the --copy-resources flag.
+
+# Installation
+
+You can use the `ocm bootstrap` command to install this component with the [toi installer](../../docs/reference/ocm_toi-bootstrapping.md) on a Kubernetes cluster.
+
+You will need a credentials file containing the kubeconfig for the target cluster:
+
+`credentials.yaml`:
+
+```yaml
+credentials:
+  target:
+    credentials:
+      KUBECONFIG: (( read("/<path-to-kuebconfig.yaml>", "text") ))
+
+```
+
+To set parameters for the target installation a configuration file can be created:
+
+`params.yaml`:
+
+```yaml
+# Namespace in target cluster to install the component:
+namespace: echo
+
+# Name of the helm release to be created:
+release: myechoserver
+
+# User provided helm values:
+replicaCount: 1
+ingress:
+  enabled: true
+```
+
+You can then install the echoserver with the command (`OCMREPO` and `VERSION` need to be adjusted):
+
+```shell
+OCMREPO=ghcr.io/open-component-model
+VERSION=0.2.0
+ocm bootstrap component install -p params.yaml -c credentials.yaml ${OCMREPO}//ocm.software/toi/demo/helmdemo:${VERSION}
+```
diff --git a/components/helmdemo/helmconfig.yaml b/components/helmdemo/helmconfig.yaml
@@ -11,3 +11,10 @@ imageMapping:
 kubeConfigName: target
 release: echoserver
 createNamespace: true
+
+# default helm values can be provided here. They will override any values given in the
+# configTemplate from the packagespec.yaml
+values:
+  replicaCount: 2
+  ingress:
+    enabled: false
diff --git a/components/helmdemo/packagespec.yaml b/components/helmdemo/packagespec.yaml
@@ -4,29 +4,67 @@ description: |
 executors:
   - resourceRef:
       resource:
-        #name: toiimage
         name: toiexecutor
       referencePath:
         - name: installer
     config: (( read("helmconfig.yaml") ))
 credentials:
   target:
     description: Kubeconfig for installation
-configTemplate:
-  namespace: default
-  release: echoserver
-configScheme:
-  type: object
-  additionalProperties: false
-  properties:
-    release:
-      type: string
-    namespace:
-      type: string
 additionalResources:
   configFile:
     resource:
       name: config-example
   credentialsFile:
     resource:
-      name: creds-example
+      name: creds-example
+# template used for generating the values file passed to helm
+# merges the values from executors/config/values
+# default helm values can be provided here. Do not forget merging the client
+# parameters. Otherwise no client values can be provided.
+# Can be customized on demand, default is to merge all user provided parameters
+# (ocm bootstrap ... -p )
+# configTemplate:
+#   namespace: default
+#   release: echoserver
+#   <<: (( merge ))     # merge all user provided values with configured defaults
+
+# Optionally provide a schema for configuration against user provided parameters are validatet
+# configScheme:
+#   type: object
+#   additionalProperties: false  # set to true to allow arbitrary values
+#   properties:
+#     release:
+#       type: string
+#     namespace:
+#       type: string
+#     imagePullSecrets:
+#       type: array
+#     nameOverride:
+#       type: object
+#     fullnameOverride:
+#       type: object
+#     replicaCount:
+#       type: integer
+#     ingress:
+#       type: object
+#     resources:
+#       type: object
+#     autoscaling:
+#       type: object
+#     nodeSelector:
+#       type: object
+#     tolerations:
+#       type: array
+#     affinity:
+#       type: object
+#     serviceAccount:
+#       type: object
+#     podAnnotations:
+#       type: object
+#     podSecurityContext:
+#       type: object
+#     securityContext:
+#       type: object
+#     service:
+#       type: object
diff --git a/components/helmdemo/references.yaml b/components/helmdemo/references.yaml
@@ -1,5 +1,4 @@
 ---
 name: installer
 componentName: ${HELMINSTCOMP}
-version: ${VERSION}
-
+version: ${HELMINSTVERSION}
diff --git a/components/helminstaller/Makefile b/components/helminstaller/Makefile
@@ -8,7 +8,7 @@ MULTI     ?= true
 PLATFORMS = linux/amd64 linux/arm64
 
 REPO_ROOT                                      := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))/../..
-VERSION                                        := $(shell git describe --tags --exact-match 2>/dev/null|| echo "$$(cat $(REPO_ROOT)/VERSION)")
+VERSION                                        := $(shell go run $(REPO_ROOT)/pkg/version/generate/release_generate.go print-rc-version)
 COMMIT                                         := $(shell git rev-parse --verify HEAD)
 EFFECTIVE_VERSION                              := $(VERSION)-$(COMMIT)
 GIT_TREE_STATE                                 := $(shell [ -z "$$(git status --porcelain 2>/dev/null)" ] && echo clean || echo dirty)
@@ -22,7 +22,7 @@ GEN = $(REPO_ROOT)/gen/$(NAME)
 CMDSRCS=$(shell find $(REPO_ROOT)/cmds/$(NAME) -type f)
 OCMSRCS=$(shell find $(REPO_ROOT)/pkg -type f) $(REPO_ROOT)/go.*
 
-ATTRIBUTES = VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI) 
+ATTRIBUTES = VERSION="$(VERSION)" COMMIT="$(COMMIT)" IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI)
 
 ifeq ($(MULTI),true)
 FLAGSUF     = .multi
@@ -85,7 +85,7 @@ $(GEN)/image.$(NAME).multi: $(GEN)/.exists Dockerfile $(CMDSRCS) $(OCMSRCS)
 	for i in $(PLATFORMS); do \
 	tag=$$(echo $$i | sed -e s:/:-:g); \
         echo building platform $$i; \
-	docker buildx build -t $(IMAGE):$(VERSION)-$$tag --platform $$i --file Dockerfile $(REPO_ROOT) \
+	docker buildx build --load -t $(IMAGE):$(VERSION)-$$tag --platform $$i --file Dockerfile $(REPO_ROOT) \
           --build-arg COMMIT=$(COMMIT) \
           --build-arg EFFECTIVE_VERSION=$(EFFECTIVE_VERSION) \
           --build-arg GIT_TREE_STATE=$(GIT_TREE_STATE); \
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,4 @@
		# Example demoplugin

		This folder contains an example how to build a plugin for the ocm CLI