From 3458bf493adfdb988ac2486df9e347808acbc0b8 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Thu, 30 May 2024 16:58:05 +0200 Subject: [PATCH 01/11] arch: add adr on autopilot basic auth + oras support adds an adr on adding oras and basic auth support on autopilot. Signed-off-by: Ricardo Maraschini --- .../adr-001-autopilot-oras-support.md | 100 ++++++++++++++++++ 1 file changed, 100 insertions(+) create mode 100644 docs/architecture/adr-001-autopilot-oras-support.md diff --git a/docs/architecture/adr-001-autopilot-oras-support.md b/docs/architecture/adr-001-autopilot-oras-support.md new file mode 100644 index 000000000000..0a24b45f54f1 --- /dev/null +++ b/docs/architecture/adr-001-autopilot-oras-support.md @@ -0,0 +1,100 @@ +# ADR 1: Add Support for ORAS Protocol and Basic Authentication in Autopilot for Artifact Retrieval + +## Context + +Registries are increasingly being used as generic artifact stores, expanding beyond their traditional role of hosting container images. To align with this trend, it is beneficial for Autopilot to support pulling artifacts directly from registries. Currently, Autopilot's capabilities are limited to downloading artifacts via the HTTP[S] protocols. + +Enhancing Autopilot to pull artifacts directly from registries will streamline workflows and improve efficiency by allowing integration and deployment of diverse artifacts without relying solely on HTTP[S] endpoints. This update will enable Autopilot to handle registry-specific protocols and authentication mechanisms, aligning it with modern deployment practices. + +## Decision + +Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) protocol. Additionally, add support for basic authentication to ensure secure access to artifacts over HTTP. + +## Solution + +Starting with the current `PlanResourceURL` struct: + +```go +type PlanResourceURL struct { + // URL is the URL of a downloadable resource. + URL string `json:"url"` + + // Sha256 provides an optional SHA256 hash of the URL's content for verification. + Sha256 string `json:"sha256,omitempty"` +} +``` + +We must specify to Autopilot where to access credentials for remote artifact pulls. This will be achieved by adjusting the struct as follows: + +```go +type PlanResourceURL struct { + // URL is the URL of a downloadable resource. + URL string `json:"url"` + + // Sha256 provides an optional SHA256 hash of the URL's content for verification. + Sha256 string `json:"sha256,omitempty"` + + // ArtifactPullSecrets holds a reference to a secret where Docker or Basic Auth + // credentials are stored. We use these credentials when pulling the artifacts from + // the URL. + ArtifactPullSecret *ArtifactPullSecret `json:"artifactPullSecret,omitempty"` + + // Insecure indicates whether certificates in the remote URL (if using TLS) can + // be ignored, aka InsecureSkipVerify. + Insecure bool `json:"insecure,omitempty"` +} +``` + +The `ArtifactPullSecret` property will be added and its struct will be defined as follow: + +```go +type ArtifactPullSecret struct { + // Namespace of the secret. + Namespace string `json:"namespace"` + + // Name of the secret. + Name string `json:"name"` +} +``` + +The secret pointed to by the provided `ArtifactPullSecret` property is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` (see below) or of type `kubernetes.io/basic-auth` if protocols `http://` or `https://` are used . + +Example configuration for ORAS: + +```yaml +url: oci://my.registry/binaries/k0s:v1.30.1+k0s.0 +sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 +artifactPullSecret: + namespace: kube-system + name: artifacts-registry +``` + +Example configuration for HTTPS: + +```yaml +url: https://my.file.server/binaries/k0s-v1.30.1+k0s.0 +sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 +artifactPullSecret: + namespace: kube-system + name: artifacts-basic-auth +``` + +### Additional Details + +- The `Insecure` property is equivalent of defining `InsecureSkipVerify` on a Go HTTP client. +- The `Insecure` property will be valid for both `oci://` and `https://` protocols. +- If no protocol is defined, HTTP is used. +- If no `ArtifactPullSecret` is defined, access will be anonymous (no authentication). + +## Status + +Proposed + +## Consequences + +- Users will have an additional protocol to be aware of. +- Introduction of a new type (`ArtifactPullSecret`) when ideally existing types could be reused. +- If the Secret referenced by `ArtifactPullSecret` does not exist, the download will fail. +- Users need to be notified about different failure types (e.g., unreadable secret, invalid secret). +- Additional configuration is required to handle basic authentication, ensuring secure access to resources. +- We will allow downloads from remote places using self-signed certificates if requested to. From 9516548c6bea42871485a8b4fb1bbcf1a5705e1d Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Fri, 31 May 2024 09:42:48 +0200 Subject: [PATCH 02/11] chore: addressing feedback on adr addressing some issues reported on the adr. Signed-off-by: Ricardo Maraschini --- ...pport.md => adr-001-autopilot-oci-basic-auth-support.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename docs/architecture/{adr-001-autopilot-oras-support.md => adr-001-autopilot-oci-basic-auth-support.md} (93%) diff --git a/docs/architecture/adr-001-autopilot-oras-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md similarity index 93% rename from docs/architecture/adr-001-autopilot-oras-support.md rename to docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index 0a24b45f54f1..49c02bedee7f 100644 --- a/docs/architecture/adr-001-autopilot-oras-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -1,4 +1,4 @@ -# ADR 1: Add Support for ORAS Protocol and Basic Authentication in Autopilot for Artifact Retrieval +# ADR 1: Add Support for OCI Registry and Basic Authentication ## Context @@ -8,7 +8,7 @@ Enhancing Autopilot to pull artifacts directly from registries will streamline w ## Decision -Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) protocol. Additionally, add support for basic authentication to ensure secure access to artifacts over HTTP. +Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) client. Additionally, add support for basic authentication to ensure secure access to artifacts over HTTP. ## Solution @@ -59,7 +59,7 @@ type ArtifactPullSecret struct { The secret pointed to by the provided `ArtifactPullSecret` property is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` (see below) or of type `kubernetes.io/basic-auth` if protocols `http://` or `https://` are used . -Example configuration for ORAS: +Example configuration for OCI: ```yaml url: oci://my.registry/binaries/k0s:v1.30.1+k0s.0 From db61167c2e0dfcc6599e9aad6a82823811e9372f Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Tue, 4 Jun 2024 11:21:22 +0200 Subject: [PATCH 03/11] chore: add support for authorization header as per pr comments it would be nice to also support the authorization header. this commit covers this possibility to the adr. Signed-off-by: Ricardo Maraschini --- ...dr-001-autopilot-oci-basic-auth-support.md | 35 ++++++++++++++++--- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index 49c02bedee7f..bee3e658ecd2 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -1,4 +1,4 @@ -# ADR 1: Add Support for OCI Registry and Basic Authentication +# ADR 1: Add Support for OCI Registry and HTTP Authentication ## Context @@ -6,9 +6,11 @@ Registries are increasingly being used as generic artifact stores, expanding bey Enhancing Autopilot to pull artifacts directly from registries will streamline workflows and improve efficiency by allowing integration and deployment of diverse artifacts without relying solely on HTTP[S] endpoints. This update will enable Autopilot to handle registry-specific protocols and authentication mechanisms, aligning it with modern deployment practices. +Currently, Autopilot does not support the retrieval of artifacts via the HTTP protocol when authentication is required. Implementing this feature to accommodate such authentication methods would be a valuable enhancement. + ## Decision -Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) client. Additionally, add support for basic authentication to ensure secure access to artifacts over HTTP. +Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) client. Additionally, add support for HTTP authentication to ensure secure access to artifacts. ## Solution @@ -57,7 +59,7 @@ type ArtifactPullSecret struct { } ``` -The secret pointed to by the provided `ArtifactPullSecret` property is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` (see below) or of type `kubernetes.io/basic-auth` if protocols `http://` or `https://` are used . +The secret pointed to by the provided `ArtifactPullSecret` property is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` (see below) or of type `Opaque` if protocols `http://` or `https://` are used. Example configuration for OCI: @@ -79,6 +81,31 @@ artifactPullSecret: name: artifacts-basic-auth ``` +### Secrets Layout + +For secrets of type `kubernetes.io/dockerconfigjson` the format is the default for Docker authentications, equal to what is used in a Pod's pull secret. For further details you can refer to the [official documentation](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). + +When it comes to the `Opaque` secret layout (used for HTTP requests) Autopilot will accept the following entries: + +- `username` and `password`: if both are set then Autopilot will attempt to pull the artifacts using [Basic Authentication](https://www.ibm.com/docs/en/cics-ts/6.1?topic=concepts-http-basic-authentication). +- `authorization`: if this property is set then the `Authorization` [header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization) will be set to its value when pulling the artifacts. + +No other property will be parsed and used. For sake of defining the expected behaviour in case of conflicting configurations: + +> In the case where the three properties are set (`username`, `password`, and `authorization`) Autopilot will ignore `username` and `password`, i.e. `authorization` takes precedence over username and password. + +The `authorization` entry is used as is, its content is placed directly into the `Authorization` header. For example a secret like the following will make Autopilot to set the `Authorization` header to `Bearer abc123def456ghi789jkl0`: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: creds + namespace: kube-system +data: + authorization: "Bearer abc123def456ghi789jkl0" +``` + ### Additional Details - The `Insecure` property is equivalent of defining `InsecureSkipVerify` on a Go HTTP client. @@ -96,5 +123,5 @@ Proposed - Introduction of a new type (`ArtifactPullSecret`) when ideally existing types could be reused. - If the Secret referenced by `ArtifactPullSecret` does not exist, the download will fail. - Users need to be notified about different failure types (e.g., unreadable secret, invalid secret). -- Additional configuration is required to handle basic authentication, ensuring secure access to resources. +- Additional configuration is required to handle authentication, ensuring secure access to resources. - We will allow downloads from remote places using self-signed certificates if requested to. From 50173599c00cb1e0adadfe6d5763c2088af8d7d1 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Fri, 14 Jun 2024 12:29:29 +0200 Subject: [PATCH 04/11] doc: be more specific about http protocol be more explicit by mentioning the secret may be used for http as well as for oci artifacts retrieval. Signed-off-by: Ricardo Maraschini --- ...adr-001-autopilot-oci-basic-auth-support.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index bee3e658ecd2..8c3b7502afce 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -36,9 +36,9 @@ type PlanResourceURL struct { // Sha256 provides an optional SHA256 hash of the URL's content for verification. Sha256 string `json:"sha256,omitempty"` - // ArtifactPullSecrets holds a reference to a secret where Docker or Basic Auth - // credentials are stored. We use these credentials when pulling the artifacts from - // the URL. + // ArtifactPullSecrets holds a reference to a secret where the credentials are + // stored. We use these credentials when pulling the artifacts from the provided + // URL using any of the supported protocols (http, https, and oci). ArtifactPullSecret *ArtifactPullSecret `json:"artifactPullSecret,omitempty"` // Insecure indicates whether certificates in the remote URL (if using TLS) can @@ -59,7 +59,7 @@ type ArtifactPullSecret struct { } ``` -The secret pointed to by the provided `ArtifactPullSecret` property is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` (see below) or of type `Opaque` if protocols `http://` or `https://` are used. +The secret pointed by the provided `ArtifactPullSecret` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). Example configuration for OCI: @@ -81,6 +81,16 @@ artifactPullSecret: name: artifacts-basic-auth ``` +Example configuration for HTTP: + +```yaml +url: http://my.file.server/binaries/k0s-v1.30.1+k0s.0 +sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 +artifactPullSecret: + namespace: kube-system + name: artifacts-token-based-auth +``` + ### Secrets Layout For secrets of type `kubernetes.io/dockerconfigjson` the format is the default for Docker authentications, equal to what is used in a Pod's pull secret. For further details you can refer to the [official documentation](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). From 76d9606dd334bac19385f1ee9853ecbc4d1ce907 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Wed, 3 Jul 2024 21:19:04 +0200 Subject: [PATCH 05/11] doc: rename "insecure" property use InsecureSkipTLSVerify instead. Signed-off-by: Ricardo Maraschini --- .../adr-001-autopilot-oci-basic-auth-support.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index 8c3b7502afce..a249f64c4689 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -41,9 +41,9 @@ type PlanResourceURL struct { // URL using any of the supported protocols (http, https, and oci). ArtifactPullSecret *ArtifactPullSecret `json:"artifactPullSecret,omitempty"` - // Insecure indicates whether certificates in the remote URL (if using TLS) can - // be ignored, aka InsecureSkipVerify. - Insecure bool `json:"insecure,omitempty"` + // InsecureSkipTLSVerify indicates whether certificates in the remote URL (if using + // TLS) can be ignored. + InsecureSkipTLSVerify bool `json:"insecureSkipTLSVerify,omitempty"` } ``` @@ -118,8 +118,8 @@ data: ### Additional Details -- The `Insecure` property is equivalent of defining `InsecureSkipVerify` on a Go HTTP client. -- The `Insecure` property will be valid for both `oci://` and `https://` protocols. +- The `InsecureSkipTLSVerify` property is equivalent of defining `InsecureSkipTLSVerify` on a Go HTTP client. +- The `InsecureSkipTLSVerify` property will be valid for both `oci://` and `https://` protocols. - If no protocol is defined, HTTP is used. - If no `ArtifactPullSecret` is defined, access will be anonymous (no authentication). From f2612b39bc83a971eee3a4aa4e144af6e628541e Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Wed, 3 Jul 2024 21:22:42 +0200 Subject: [PATCH 06/11] doc: using k8s.io/api/core/v1.SecretReference use an existing type instead of definining a new one. Signed-off-by: Ricardo Maraschini --- .../adr-001-autopilot-oci-basic-auth-support.md | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index a249f64c4689..60666fa09548 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -39,7 +39,7 @@ type PlanResourceURL struct { // ArtifactPullSecrets holds a reference to a secret where the credentials are // stored. We use these credentials when pulling the artifacts from the provided // URL using any of the supported protocols (http, https, and oci). - ArtifactPullSecret *ArtifactPullSecret `json:"artifactPullSecret,omitempty"` + ArtifactPullSecret *corev1.SecretReference `json:"artifactPullSecret,omitempty"` // InsecureSkipTLSVerify indicates whether certificates in the remote URL (if using // TLS) can be ignored. @@ -47,19 +47,7 @@ type PlanResourceURL struct { } ``` -The `ArtifactPullSecret` property will be added and its struct will be defined as follow: - -```go -type ArtifactPullSecret struct { - // Namespace of the secret. - Namespace string `json:"namespace"` - - // Name of the secret. - Name string `json:"name"` -} -``` - -The secret pointed by the provided `ArtifactPullSecret` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). +`ArtifactPullSecret` property is of type `SecretReference` as defined by `k8s.io/api/core/v1` package. The secret pointed by the provided `ArtifactPullSecret` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). Example configuration for OCI: @@ -130,7 +118,6 @@ Proposed ## Consequences - Users will have an additional protocol to be aware of. -- Introduction of a new type (`ArtifactPullSecret`) when ideally existing types could be reused. - If the Secret referenced by `ArtifactPullSecret` does not exist, the download will fail. - Users need to be notified about different failure types (e.g., unreadable secret, invalid secret). - Additional configuration is required to handle authentication, ensuring secure access to resources. From c3062aa2351ca866642ae48d48d98b2cdcf20725 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Wed, 3 Jul 2024 21:33:38 +0200 Subject: [PATCH 07/11] doc: define default protocol to https Signed-off-by: Ricardo Maraschini --- docs/architecture/adr-001-autopilot-oci-basic-auth-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index 60666fa09548..f1ddaff33bb7 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -108,7 +108,7 @@ data: - The `InsecureSkipTLSVerify` property is equivalent of defining `InsecureSkipTLSVerify` on a Go HTTP client. - The `InsecureSkipTLSVerify` property will be valid for both `oci://` and `https://` protocols. -- If no protocol is defined, HTTP is used. +- If no protocol is defined, HTTPS is used. - If no `ArtifactPullSecret` is defined, access will be anonymous (no authentication). ## Status From fff4dccaf2ca4891a56e9a6fd2eea83ba5dfe39e Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Wed, 3 Jul 2024 21:58:08 +0200 Subject: [PATCH 08/11] doc: renamed ArtifactPullSecret to SecretRef Signed-off-by: Ricardo Maraschini --- .../adr-001-autopilot-oci-basic-auth-support.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index f1ddaff33bb7..2265c4be5907 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -36,10 +36,10 @@ type PlanResourceURL struct { // Sha256 provides an optional SHA256 hash of the URL's content for verification. Sha256 string `json:"sha256,omitempty"` - // ArtifactPullSecrets holds a reference to a secret where the credentials are - // stored. We use these credentials when pulling the artifacts from the provided - // URL using any of the supported protocols (http, https, and oci). - ArtifactPullSecret *corev1.SecretReference `json:"artifactPullSecret,omitempty"` + // SecretRef holds a reference to a secret where the credentials are stored. We + // use these credentials when pulling the artifacts from the provided URL using + // any of the supported protocols (http, https, and oci). + SecretRef *corev1.SecretReference `json:"secretRef,omitempty"` // InsecureSkipTLSVerify indicates whether certificates in the remote URL (if using // TLS) can be ignored. @@ -47,7 +47,7 @@ type PlanResourceURL struct { } ``` -`ArtifactPullSecret` property is of type `SecretReference` as defined by `k8s.io/api/core/v1` package. The secret pointed by the provided `ArtifactPullSecret` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). +`SecretRef` property is of type `SecretReference` as defined by `k8s.io/api/core/v1` package. The secret pointed by the provided `SecretRef` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). Example configuration for OCI: @@ -109,7 +109,7 @@ data: - The `InsecureSkipTLSVerify` property is equivalent of defining `InsecureSkipTLSVerify` on a Go HTTP client. - The `InsecureSkipTLSVerify` property will be valid for both `oci://` and `https://` protocols. - If no protocol is defined, HTTPS is used. -- If no `ArtifactPullSecret` is defined, access will be anonymous (no authentication). +- If no `SecretRef` is defined, access will be anonymous (no authentication). ## Status @@ -118,7 +118,7 @@ Proposed ## Consequences - Users will have an additional protocol to be aware of. -- If the Secret referenced by `ArtifactPullSecret` does not exist, the download will fail. +- If the Secret referenced by `SecretRef` does not exist, the download will fail. - Users need to be notified about different failure types (e.g., unreadable secret, invalid secret). - Additional configuration is required to handle authentication, ensuring secure access to resources. - We will allow downloads from remote places using self-signed certificates if requested to. From d6ea3d1a82f576ac23bb3198a942cb3801197455 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Mon, 5 Aug 2024 15:59:08 +0200 Subject: [PATCH 09/11] chore: standardise adr on 80 columns as per pr review comment. Signed-off-by: Ricardo Maraschini --- ...dr-001-autopilot-oci-basic-auth-support.md | 95 +++++++++++++------ 1 file changed, 68 insertions(+), 27 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index 2265c4be5907..e48843ac504a 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -2,15 +2,28 @@ ## Context -Registries are increasingly being used as generic artifact stores, expanding beyond their traditional role of hosting container images. To align with this trend, it is beneficial for Autopilot to support pulling artifacts directly from registries. Currently, Autopilot's capabilities are limited to downloading artifacts via the HTTP[S] protocols. - -Enhancing Autopilot to pull artifacts directly from registries will streamline workflows and improve efficiency by allowing integration and deployment of diverse artifacts without relying solely on HTTP[S] endpoints. This update will enable Autopilot to handle registry-specific protocols and authentication mechanisms, aligning it with modern deployment practices. - -Currently, Autopilot does not support the retrieval of artifacts via the HTTP protocol when authentication is required. Implementing this feature to accommodate such authentication methods would be a valuable enhancement. +Registries are increasingly being used as generic artifact stores, expanding +beyond their traditional role of hosting container images. To align with this +trend, it is beneficial for Autopilot to support pulling artifacts directly +from registries. Currently, Autopilot's capabilities are limited to downloading +artifacts via the HTTP[S] protocols. + +Enhancing Autopilot to pull artifacts directly from registries will streamline +workflows and improve efficiency by allowing integration and deployment of +diverse artifacts without relying solely on HTTP[S] endpoints. This update will +enable Autopilot to handle registry-specific protocols and authentication +mechanisms, aligning it with modern deployment practices. + +Currently, Autopilot does not support the retrieval of artifacts via the HTTP +protocol when authentication is required. Implementing this feature to +accommodate such authentication methods would be a valuable enhancement. ## Decision -Implement support in Autopilot for pulling artifacts, such as k0s binaries and image bundles, directly from a registry using the [ORAS](https://oras.land/docs/) client. Additionally, add support for HTTP authentication to ensure secure access to artifacts. +Implement support in Autopilot for pulling artifacts, such as k0s binaries and +image bundles, directly from a registry using the +[ORAS](https://oras.land/docs/) client. Additionally, add support for HTTP +authentication to ensure secure access to artifacts. ## Solution @@ -21,33 +34,42 @@ type PlanResourceURL struct { // URL is the URL of a downloadable resource. URL string `json:"url"` - // Sha256 provides an optional SHA256 hash of the URL's content for verification. + // Sha256 provides an optional SHA256 hash of the URL's content for + // verification. Sha256 string `json:"sha256,omitempty"` } ``` -We must specify to Autopilot where to access credentials for remote artifact pulls. This will be achieved by adjusting the struct as follows: +We must specify to Autopilot where to access credentials for remote artifact +pulls. This will be achieved by adjusting the struct as follows: ```go type PlanResourceURL struct { // URL is the URL of a downloadable resource. URL string `json:"url"` - // Sha256 provides an optional SHA256 hash of the URL's content for verification. + // Sha256 provides an optional SHA256 hash of the URL's content for + // verification. Sha256 string `json:"sha256,omitempty"` - // SecretRef holds a reference to a secret where the credentials are stored. We - // use these credentials when pulling the artifacts from the provided URL using + // SecretRef holds a reference to a secret where the credentials are + // stored. We use these credentials when pulling the artifacts from the + // provided URL using // any of the supported protocols (http, https, and oci). SecretRef *corev1.SecretReference `json:"secretRef,omitempty"` - // InsecureSkipTLSVerify indicates whether certificates in the remote URL (if using - // TLS) can be ignored. + // InsecureSkipTLSVerify indicates whether certificates in the remote + // URL (if using TLS) can be ignored. InsecureSkipTLSVerify bool `json:"insecureSkipTLSVerify,omitempty"` } ``` -`SecretRef` property is of type `SecretReference` as defined by `k8s.io/api/core/v1` package. The secret pointed by the provided `SecretRef` will be used for pulling artifacts using either HTTP[S] or OCI protocols and is expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used (see below for details on the Secret layout). +`SecretRef` property is of type `SecretReference` as defined by +`k8s.io/api/core/v1` package. The secret pointed by the provided `SecretRef` +will be used for pulling artifacts using either HTTP[S] or OCI protocols and is +expected to by of type `kubernetes.io/dockerconfigjson` if the protocol in use +is `oci://` or of type `Opaque` if protocols `http://` or `https://` are used +(see below for details on the Secret layout). Example configuration for OCI: @@ -81,18 +103,31 @@ artifactPullSecret: ### Secrets Layout -For secrets of type `kubernetes.io/dockerconfigjson` the format is the default for Docker authentications, equal to what is used in a Pod's pull secret. For further details you can refer to the [official documentation](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). +For secrets of type `kubernetes.io/dockerconfigjson` the format is the default +for Docker authentications, equal to what is used in a Pod's pull secret. For +further details you can refer to the [official +documentation](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). -When it comes to the `Opaque` secret layout (used for HTTP requests) Autopilot will accept the following entries: +When it comes to the `Opaque` secret layout (used for HTTP requests) Autopilot +will accept the following entries: -- `username` and `password`: if both are set then Autopilot will attempt to pull the artifacts using [Basic Authentication](https://www.ibm.com/docs/en/cics-ts/6.1?topic=concepts-http-basic-authentication). -- `authorization`: if this property is set then the `Authorization` [header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization) will be set to its value when pulling the artifacts. +- `username` and `password`: if both are set then Autopilot will attempt to + pull the artifacts using [Basic + Authentication](https://www.ibm.com/docs/en/cics-ts/6.1?topic=concepts-http-basic-authentication). +- `authorization`: if this property is set then the `Authorization` + [header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization) + will be set to its value when pulling the artifacts. -No other property will be parsed and used. For sake of defining the expected behaviour in case of conflicting configurations: +No other property will be parsed and used. For sake of defining the expected +behaviour in case of conflicting configurations: -> In the case where the three properties are set (`username`, `password`, and `authorization`) Autopilot will ignore `username` and `password`, i.e. `authorization` takes precedence over username and password. +> In the case where the three properties are set (`username`, `password`, and +> `authorization`) Autopilot will ignore `username` and `password`, i.e. +> `authorization` takes precedence over username and password. -The `authorization` entry is used as is, its content is placed directly into the `Authorization` header. For example a secret like the following will make Autopilot to set the `Authorization` header to `Bearer abc123def456ghi789jkl0`: +The `authorization` entry is used as is, its content is placed directly into +the `Authorization` header. For example a secret like the following will make +Autopilot to set the `Authorization` header to `Bearer abc123def456ghi789jkl0`: ```yaml apiVersion: v1 @@ -106,8 +141,10 @@ data: ### Additional Details -- The `InsecureSkipTLSVerify` property is equivalent of defining `InsecureSkipTLSVerify` on a Go HTTP client. -- The `InsecureSkipTLSVerify` property will be valid for both `oci://` and `https://` protocols. +- The `InsecureSkipTLSVerify` property is equivalent of defining + `InsecureSkipTLSVerify` on a Go HTTP client. +- The `InsecureSkipTLSVerify` property will be valid for both `oci://` and + `https://` protocols. - If no protocol is defined, HTTPS is used. - If no `SecretRef` is defined, access will be anonymous (no authentication). @@ -118,7 +155,11 @@ Proposed ## Consequences - Users will have an additional protocol to be aware of. -- If the Secret referenced by `SecretRef` does not exist, the download will fail. -- Users need to be notified about different failure types (e.g., unreadable secret, invalid secret). -- Additional configuration is required to handle authentication, ensuring secure access to resources. -- We will allow downloads from remote places using self-signed certificates if requested to. +- If the Secret referenced by `SecretRef` does not exist, the download will + fail. +- Users need to be notified about different failure types (e.g., unreadable + secret, invalid secret). +- Additional configuration is required to handle authentication, ensuring + secure access to resources. +- We will allow downloads from remote places using self-signed certificates if + requested to. From b5899a17ef91416d5f546eaa7f5aad6a29a3a606 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Mon, 5 Aug 2024 16:00:52 +0200 Subject: [PATCH 10/11] chore: remove references to artifactPullSecret remove some dangling references to the old artifactPullSecret field. Signed-off-by: Ricardo Maraschini --- .../adr-001-autopilot-oci-basic-auth-support.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index e48843ac504a..c4ad287cf724 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -76,7 +76,7 @@ Example configuration for OCI: ```yaml url: oci://my.registry/binaries/k0s:v1.30.1+k0s.0 sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 -artifactPullSecret: +secretRef: namespace: kube-system name: artifacts-registry ``` @@ -86,7 +86,7 @@ Example configuration for HTTPS: ```yaml url: https://my.file.server/binaries/k0s-v1.30.1+k0s.0 sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 -artifactPullSecret: +secretRef: namespace: kube-system name: artifacts-basic-auth ``` @@ -96,7 +96,7 @@ Example configuration for HTTP: ```yaml url: http://my.file.server/binaries/k0s-v1.30.1+k0s.0 sha256: e95603f167cce6e3cffef5594ef06785b3c1c00d3e27d8e4fc33824fe6c38a99 -artifactPullSecret: +secretRef: namespace: kube-system name: artifacts-token-based-auth ``` From 98ede82b443e5a932c4f8cb7a4aab80c6ee960b0 Mon Sep 17 00:00:00 2001 From: Ricardo Maraschini Date: Mon, 5 Aug 2024 16:03:43 +0200 Subject: [PATCH 11/11] feat: signal error if no or invalid protocol is provided if no protocol is provided (http, https, or oci) or if an invalid one is provided an error should be signalled on the plan object. Signed-off-by: Ricardo Maraschini --- docs/architecture/adr-001-autopilot-oci-basic-auth-support.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md index c4ad287cf724..a8c3b58bb103 100644 --- a/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md +++ b/docs/architecture/adr-001-autopilot-oci-basic-auth-support.md @@ -145,7 +145,8 @@ data: `InsecureSkipTLSVerify` on a Go HTTP client. - The `InsecureSkipTLSVerify` property will be valid for both `oci://` and `https://` protocols. -- If no protocol is defined, HTTPS is used. +- If a protocol is not specified or an incorrect one is provided, an error + state should be activated. - If no `SecretRef` is defined, access will be anonymous (no authentication). ## Status