Replies: 8 comments 17 replies
-
|
https://github.com/gardener/component-spec/blob/master/doc/proposal/02-component-descriptor.md#references Third: Dependencies are not mentioned at all. When do I declare a dependency as a resource and when as component reference then? Can't we give a real example here? I feel this is entirely meaningless ...? |
Beta Was this translation helpful? Give feedback.
-
|
With |
Beta Was this translation helpful? Give feedback.
-
|
I defined the following component desc in folder component:
componentReferences:
- name: dependency
componentName: test.ref/ # additional name component missing
version: 0.0.1
name: skarlso.test/
provider: skarlso
repositoryContexts: []
resources:
- access:
imageReference: docker.io/skarlso/ocm:example-0.0.1
type: ociRegistry
name: ocm-hello-server
relation: external
type: ociImage
version: 0.0.1
- access:
imageReference: docker.io/skarlso/ocm:example-0.0.1
type: ociRegistry
name: ocm-hello-1
relation: external
type: ociImage
version: 0.0.1
sources: []
version: 0.0.1
meta:
schemaVersion: v2tree: Then test-ref is another component: component:
componentReferences: []
name: test.ref/
provider: skarlso
repositoryContexts: []
resources:
- access:
imageReference: docker.io/skarlso/ocm:example-0.0.1
type: ociRegistry
name: ocm-hello-db
relation: external
type: ociImage
version: 0.0.1
sources: []
version: 0.0.1
meta:
schemaVersion: v2tree: Then I ran a recursive download in the first component by defining the local ref component as a lookup option: That worked. :) |
Beta Was this translation helpful? Give feedback.
-
Okay but: |
Beta Was this translation helpful? Give feedback.
-
|
componentReference is a component reference, you can specify only ocm components there. In the nginx case: |
Beta Was this translation helpful? Give feedback.
-
|
Like if a helm chart is depend on something, it has to be deployed with that helm chart or another helm chart is added as dependency. You can't just add nginx as dependency to your helm chart if it does not have a helm chart. You can create an nginx helm chart or you can embed nginx in your helm chart directly. |
Beta Was this translation helpful? Give feedback.
-
|
I see it the same way as Balazs said. If it is a dependency, I would wrap it around a component descriptor, however........ Argument for Explicit dependency declarationRight now, the references are only traversed and then added as a list that is then processed in that order. There is no topological order in between them, so you don't know which component depends on which. The only thing that happens right now in It's also used in some other circumstances but there is no dependency declaration and the order is not guaranteed. There is an argument that the dependencies should be taken care of outside the schema definition but I disagree on that. I believe as Paul put it, it would be beneficial to define explicit dependencies as other specs do such as cyclonedx's dependency graph for example. What would we needWe would need something that declares dependencies. Right now, references in code are called dependencies, but they are only such in the sense of references and declaration. And not further processing. We have two options: Renamed
|
Beta Was this translation helpful? Give feedback.
-
|
I would add a new field to the |
Beta Was this translation helpful? Give feedback.
-
|
So a LAMP-(ish) stack would be modelled something like the following: erDiagram
NGINX-COMPONENT ||--|| NGINX-BINARY : "has-resource"
NGINX-COMPONENT ||--|| NGINX-CONF : "has-resource"
NGINX-COMPONENT ||--|| APPLICATION-CODE : "has-resource"
NGINX-COMPONENT ||--|| GITHUB-APP-REPOSITORY : "has-source"
PHP-COMPONENT ||--|| PHP-FPM-BINARY : "has-resource"
PHP-COMPONENT ||--|| INI-FILE : "has-resource"
MySQL-COMPONENT ||--|| MYSQL-BINARY : "has-resource"
LINUX-COMPONENT ||--|| UBUNTU-OCI-IMAGE : "has-resource"
LAMP-STACK-COMPONENT ||--|| NGINX-COMPONENT : "has-component-reference"
NGINX-COMPONENT ||--|| PHP-COMPONENT : "has-component-reference"
NGINX-COMPONENT ||--|| LINUX-COMPONENT : "has-component-reference"
PHP-COMPONENT ||--|| MySQL-COMPONENT : "has-component-reference"
PHP-COMPONENT ||--|| LINUX-COMPONENT : "has-component-reference"
MySQL-COMPONENT ||--|| LINUX-COMPONENT : "has-component-reference"
NGINX-COMPONENT {
version v1-0-0
}
PHP-COMPONENT {
version v1-0-0
}
MySQL-COMPONENT {
version v1-0-0
}
LINUX-COMPONENT {
version v1-0-0
}
|
Beta Was this translation helpful? Give feedback.
-
|
I'm not sure I understand all parts of this graph. I don't think the MySQL component would and PHP has a dependency connection between them, and I don't know what is the Linux Component with Ubuntu. I try to approach this from the point I understand (which may not be true): # assume that's the ubuntu container there
ApplicationComponent:
resources: ubuntu-oci-image
componentReferences:
- NginxComponent
- PHPComponent
- MySQLComponent
NginxComponent:
resources: [all the listed resources]
# I don't think it depends on the PHP component, the application does, nginx is happy alone
componentReferences: []
# i don't know what's the point of this, it can be part of the application container
# in a bare metal environment, it's just php installation and configuration, no dependencies,
# php can run without the application, nginx, or mysql
PHPComponent:
resources: [all the listed resources]
# it can depend on mysql libs, but I think it can be handled here
componentReferences: []
MySQLComponent:
resources: [all the listed resources]
# no dependencies because mysql is happy to run, the application depends on the database
componentReferences: []With this, when I try to deploy
At the same time, some of the resources of components can be part of the ApplicationComponent, like host specific nginx config, or app specific php-fpm pool defintion. Footnotes
|
Beta Was this translation helpful? Give feedback.
-
|
It is crucial to understand that the OCM itself does not care about the sequencing of deployments, in fact, it does not care about the deployment at all. That is something the Landscaper (or any other deployment machinery) has to take care of. We believe this does not belong to the Component Model, but is another model on top (hence "blueprint" artifacts for the Landscaper). The OCM provides a way to describe all required resources which are part of a component, either by having artifacts in the list of resources or by referencing other requirement components. The deployment (and therefore also its sequence) is something else, which the OCM does not know anything of. At least that is our vision, and it relates to our strong argument of separation of concerns: The OCM takes care about providing a unique identity for components, describing all artifacts required for a component and how to access these (and if required copy everything over to other OCI registries, making sure everything is properly signed and can be verified at the point of deployment), but how exactly the components "come to life" ( = are getting deployed), must be the task of someone else, Landscaper or whatever other deployment controller. |
Beta Was this translation helpful? Give feedback.
-
|
Scoping the OCM spec in this way is fine but my concern is that this potentially makes it just another SBOM specification. My vision is that it can be a SBOM, a better one than some of the existing ones but also much more. Having looked at this and done some deep thinking about this over the last two days I am of the view that the OCM spec is basically fine as it is in so far as it goes to describe components in terms of the resources and associated source code. It it provides the consumer with details of the artefacts and sources they might want to copy, deploy, scan etc. However I think it could still provide that information as well as going much further to provide a mechanism for defining how to build and deploy a component. As mentioned above I think an optional dependsOn field should be added to resources and component references within a component descriptor such that the provider can specify the order in which resources should be deployed and executed. I am uncomfortable about is the proliferation of configuration data outside the component descriptor, per the slides Uwe presented last Friday that described the idea of a provider publishing a component descriptor via an OCI repository and separate configuration data in a git repository. The consumer's sync tool will copy the component descriptor to their own trusted OCI repository, optionally fetching external artefacts as well if desired. Then the consumer will merge the configuration data from the provider with their own environment specific configuration data. I'd be more comfortable providing configuration data that relates to a component or resource as properties within the component descriptor. The consumer should be able define environment specific overlays to this configuration data, possible via an environment specific git repository or even a component descriptor that describes the target environment and optionally the cloud or bare metal resources it comprises and the applications to deploy to it. I'd envisage such a component descriptor having a component reference to the component published by the provider. Component level properties in the consumers environment component descriptor could provide environment specific values for configuration items defined by provider's component descriptor. I have been struggling with is how to provide within the component descriptor a mechanism to describe how to deploy a resource. We talked about resource types and a catalog of resource deployers that can be accessed to obtain a list of tools that can be used to deploy a resource of that type, i.e. helm command line tool, Flux helm controller and so on for helm chart. However how does the consumer know how to deploy those tools? Surely they are like resources that maybe should be automatically deployable, so they could be resources in a component descriptor that have a type which in turn has a deployer (brew, apt, yum, flux cli etc) and so we are into the chicken and egg loop! Effectively a consumer needs a OCM bootstrap process. We should provide a a cli tool that can be used to deploy component descriptors on the local machine, other machines or cloud resources. We would also provide a component descriptor that described a Kubernetes controller based OCM deployer solution. For example the cli tool when deployed on a Debian server would be deployed using apt and thus would be able to rely on apt being available to deploy resources on that machine, it could specify some other prerequisites too if needed but my vision is that a user should be able to deploy the OCM command line tool and then use it to deploy a provider's component, i.e. We would also provide a Kubernetes OCM manager component descriptor that they could deploy. The default version of this would deploy a Kubernetes cluster using Kind on the local machine and deploy Flux, Landscaper and OCM controller etc into it. We could provide versions of this for EKS, AKS etc. Alternatively the user could provide the url of their own component descriptor that defines the environment they want to build (or already have) and reference the Kubernetes OCM manager component descriptor as a component reference, effectively deploying it in an existing environment. From this point the user can deploy any component descriptor that defines resources that are deployed using Kubernetes controllers like Landscaper including any resource described using Terraform, via a Terraform controller. Also, maybe controversially, in my opinion the component descriptor, resources and sources should be defined as Kubernetes custom resource definitions. This enables contributors and users to leverage the extensive range of tooling that exist for creating, transforming and deploying Kubernetes objects. Happy to discuss these ideas further, maybe next week when Uwe is back, but also happy to pursue the original vision based on OCM as a mechanism for describing components and Landscaper as the deployment tool. |
Beta Was this translation helpful? Give feedback.
-
Already talked to Uwe about this. We should definitely have a meeting to talk about all these points, ideally already next week. |
Beta Was this translation helpful? Give feedback.
-
|
The idea of the dependsOn is the order of deployment of items within a component and as I understand it components can contain resources and references to other components. My thinking is that a component descriptor will contain a number of resources which are part of the component and references to other components that it requires. The order in which these are processed might be important. |
Beta Was this translation helpful? Give feedback.
-
|
I think if you have a resource which is a dependency, you should wrap that resource into a component and reference it. |
Beta Was this translation helpful? Give feedback.
-
|
My thinking was that a component might be something like a LAMP stack. In this example the resources might be a database, a webserver, frontend and backend applications etc. It might be that the database is shared between a number of application and thus is a component rather than a resource so in my component it is a component reference and that component might have some number of resources that if done in sequence deploys and populates or upgrades and migrates DDL/DML. |
Beta Was this translation helpful? Give feedback.
-
|
Sure, I think that's evident, that the depending component's resource would have to be processed first, including its depending resources and their resources. :) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Skarlso, I'm not sure if I really understand, why we need an explicit dependency declaration for components? If this should be used e.g. by some installer to deploy the components in the right order, isn't it required that the model must also include all other information to install these components, i.e. export/import data, deploy mechanism (helm,...)? Only modelling the component installation dependencies seems to be not sufficient for me and we still need another formalism to describe the complete installation information. |
Beta Was this translation helpful? Give feedback.
-
|
My thoughts are that in order the make the OCM a successfully open source project we should...
The top diagram describes the proposed relationship between component descriptors. The consumer of an application component descriptor would build an infrastructure component descriptor and an environment component descriptor which would contain a component reference to the infrastructure component descriptor and also a reference to the provider’s application component descriptor. When this consumer environment component descriptor is uploaded to the consumer’s OCI repository it will be deployed to the consumers deployment management cluster by Flux and processed by the OCM controllers. Each processing phase will follow the links to referenced component descriptors, applying those to the cluster for processing by the OCM controllers. The OCM processing flow will be implemented by a number of different controllers. The first step is to verify signatures. Only when the component descriptor has been verified with other steps progress. Optionally an OCM Synchronisation controller will apply the consumers policy relating to use of external resources. The consumer might be happy to consume artefacts directly from providers and other public repositories but they might want to copy all artefacts they plan to use into their own repository. Note that the synchronisation and pack/unpack steps could be nested, i.e. sync from provider to community repositories and then to repositories for specific consumers and even within a consumer to repositories within siloed environments within an organisation. The next step is to execute source and image scanning using tools determined by the consumer’s policy. Finally the OCM deployer will process the consumer’s application component descriptor and the component descriptors that it referenced, creating resource custom resources for processing by the appropriate resource deployer controller. Resource DeployerFor each type of resource a resource deployer will process the resources of that type. At this point dependencies will be considered, resource deployment will not commence until the status of the resources it depends on indicated success. Resource Deployer DiscoveryAny component descriptor that contains a resource of a given type should include a component reference to a component descriptor of the same name that will contain one or more deployers for that type. A component descriptor for each resource type will contain one or more resources that describe executables that can be used to deploy a resource of this type. These resources will have types and the component descriptor will contain a component reference to the component descriptor for that type’s deployer. The base deployer type will be platform specific deployer such as brew, yum, apt etc. A label inherited from a referring component descriptor will define the specific base deployer to use. In the absence of such a label local platform detection will be used to set the label. For example, a helm chart resource type can be described using HelmRelease and HelmRepository custom resources and deployed using Flux. Alternatively, it could be described using command line arguments and deployed using the helm client. In each case the resource type is a helm chart but the resource will be labelled accordingly to indicate the type of deployer it requires. The OCM processor will use the component reference to the helm chart deployer component descriptor and deploy the resources that match the label on the resource being deployed. BootstrapA consumer needs an OCM bootstrap process. We should provide a cli tool that can be used to deploy component descriptors on the local machine, other machines or cloud resources. We would also provide a component descriptor that described a Kubernetes controller based OCM deployer solution. For example the cli tool when deployed on a Debian server would be deployed using apt and thus would be able to rely on apt being available to deploy resources on that machine, it could specify some other prerequisites too if needed but my vision is that a user should be able to deploy the OCM command line tool and then use it to deploy a provider’s component, i.e. apt install ocm-cli We would also provide a Kubernetes OCM manager component descriptor that they could deploy. The default version of this would deploy a Kubernetes cluster using Kind on the local machine and deploy Flux, Landscaper and OCM controllers etc into it. We could provide versions of this for EKS, AKS etc. Alternatively the user could provide the url of their own component descriptor that defines the environment they want to build (or already have) and reference the Kubernetes OCM manager component descriptor as a component reference, effectively deploying it in an existing environment. From this point the user can deploy any component descriptor that defines resources that are deployed using Kubernetes controllers like Landscaper including any resource described using Terraform, via a Terraform controller. |
Beta Was this translation helpful? Give feedback.
-
|
I thought that the main purpose of OCM is to describe the artefacts of a component but not how to deploy them. Of course you could extend OCM with such information but this make it more complex. The alternative is to define a model to describe the deployment separately, decoupling these aspects. This is how the current approach with the landscaper looks like, where so called blueprints/installations define the deploy logic based on a component model quite similar to OCM. This allows everyone to use OCM as a basis and if required its own deployment model on top. @In-Ko told me that you already plan to discuss these alternatives during the next days. |
Beta Was this translation helpful? Give feedback.
-
|
My concern is that in its current form the OCM spec is just another SBOM, with targeted signing etc but not powerful enough to capture peoples imagination. Defining the deployment logic in a separate blueprint is not without merit but defining a framework for deployment in the spec makes the spec more powerful. |
Beta Was this translation helpful? Give feedback.
-
|
I agree, but do you think it makes a difference if we provide a deployment model in a separate a spec on top of OCM or that we include include it in the OCM spec itself? And what is preferable? I really don't know. |
Beta Was this translation helpful? Give feedback.
-
This thread is to consolidate all the things of the OCM proposal spec, which are problematic, unclear, need rephrasing, need examples, and so on ... Let's gather everything here, and then have a meeting to discuss all these points and agree on how to improve them.
Beta Was this translation helpful? Give feedback.
All reactions