Skip to content

Commit

Permalink
Remote Resolution Refactor
Browse files Browse the repository at this point in the history 
This PR implements an updated resolver framework with slight updates. This is to avoid backwards incompatibility while implementing [TEP-0154](tektoncd/community#1138).

The current framework only works with Params. e.g. The interface has ValidateParams and Resolve which takes in Params. Now that we also need to pass in a `URL`, we need to add new methods and change function signatures which leads to API incompatibility with existing custom resolvers. As a result, when users upgrade to new version of Tekton Pipelines, they will be forced to be compatible with the new format because of the interface changes.

This PR tries to make it future proof such that if we add new fields to the ResolutionSpec, it will be handled without the need to break users.
  • Loading branch information
@chitrangpatel
chitrangpatel committed May 14, 2024
1 parent b419b2c commit 0a88237
Show file tree
Hide file tree
Showing 74 changed files with 6,784 additions and 781 deletions.
15 changes: 8 additions & 7 deletions cmd/resolvers/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,13 @@ import (
"strings"

"github.com/tektoncd/pipeline/pkg/apis/resolution/v1alpha1"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/bundle"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/cluster"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/framework"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/git"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/http"
"github.com/tektoncd/pipeline/pkg/resolution/resolver/hub"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/bundle"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/cluster"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/framework"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/git"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/http"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/hub"
hubresolution "github.com/tektoncd/pipeline/pkg/resolution/resolver/hub"
filteredinformerfactory "knative.dev/pkg/client/injection/kube/informers/factory/filtered"
"knative.dev/pkg/injection/sharedmain"
"knative.dev/pkg/signals"
Expand All @@ -35,7 +36,7 @@ import (
func main() {
ctx := filteredinformerfactory.WithSelectors(signals.NewContext(), v1alpha1.ManagedByLabelKey)
tektonHubURL := buildHubURL(os.Getenv("TEKTON_HUB_API"), "")
artifactHubURL := buildHubURL(os.Getenv("ARTIFACT_HUB_API"), hub.DefaultArtifactHubURL)
artifactHubURL := buildHubURL(os.Getenv("ARTIFACT_HUB_API"), hubresolution.DefaultArtifactHubURL)

sharedmain.MainWithContext(ctx, "controller",
framework.NewController(ctx, &git.Resolver{}),
Expand Down
188 changes: 180 additions & 8 deletions docs/how-to-write-a-resolver.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,29 @@ a little bit of boilerplate.

Create `cmd/demoresolver/main.go` with the following setup code:

{{% tabs %}}

{{% tab "Latest Framework" %}}
```go
package main

import (
"context"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/framework"
"knative.dev/pkg/injection/sharedmain"
)

func main() {
sharedmain.Main("controller",
framework.NewController(context.Background(), &resolver{}),
)
}

type resolver struct {}
```
{{% /tab %}}

{{% tab "Previous Framework" %}}
```go
package main

Expand All @@ -115,6 +138,10 @@ func main() {
type resolver struct {}
```

{{% /tab %}}

{{% /tabs %}}

This won't compile yet but you can download the dependencies by running:

```bash
Expand Down Expand Up @@ -189,6 +216,24 @@ example resolver.

We'll also need to add another import for this package at the top:

{{% tabs %}}

{{% tab "Latest Framework" %}}
```go
import (
"context"

// Add this one; it defines LabelKeyResolverType we use in GetSelector
"github.com/tektoncd/pipeline/pkg/resolution/common"
"github.com/tektoncd/pipeline/pkg/remoteresolution/resolver/framework"
"knative.dev/pkg/injection/sharedmain"
pipelinev1 "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1"
)
```
{{% /tab %}}

{{% tab "Previous Framework" %}}

```go
import (
"context"
Expand All @@ -201,21 +246,48 @@ import (
)
```

## The `ValidateParams` method
{{% /tab %}}

{{% /tabs %}}

## The `Validate` method

The `ValidateParams` method checks that the params submitted as part of
The `Validate` method checks that the resolution-spec submitted as part of
a resolution request are valid. Our example resolver doesn't expect
any params so we'll simply ensure that the given map is empty.
any params in the spec so we'll simply ensure that the there are no params.
In the previous version, this was instead called `ValidateParams` method. See below
for the differences.

{{% tabs %}}

{{% tab "Latest Framework" %}}

```go
// ValidateParams ensures parameters from a request are as expected.
func (r *resolver) ValidateParams(ctx context.Context, params map[string]string) error {
if len(params) > 0 {
// Validate ensures that the resolution spec from a request is as expected.
func (r *resolver) Validate(ctx context.Context, req *v1beta1.ResolutionRequestSpec) error {
if len(req.Params) > 0 {
return errors.New("no params allowed")
}
return nil
}
```
{{% /tab %}}

{{% tab "Previous Framework" %}}

```go
// ValidateParams ensures that the params from a request are as expected.
func (r *resolver) ValidateParams(ctx context.Context, params []pipelinev1.Param) error {
if len(req.Params) > 0 {
return errors.New("no params allowed")
}
return nil
}
```

{{% /tab %}}

{{% /tabs %}}

You'll also need to add the `"errors"` package to your list of imports at
the top of the file.
Expand All @@ -228,13 +300,113 @@ going to return a hard-coded string of YAML. Since Tekton Pipelines
currently only supports fetching Pipeline resources via remote
resolution that's what we'll return.


The method signature we're implementing here has a
`framework.ResolvedResource` interface as one of its return values. This
is another type we have to implement but it has a small footprint:

{{% tabs %}}

{{% tab "Latest Framework" %}}


```go
// Resolve uses the given resolution spec to resolve the requested file or resource.
func (r *resolver) Resolve(ctx context.Context, req *v1beta1.ResolutionRequestSpec) (framework.ResolvedResource, error) {
return &myResolvedResource{}, nil
}

// our hard-coded resolved file to return
const pipeline = `
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: my-pipeline
spec:
tasks:
- name: hello-world
taskSpec:
steps:
- image: alpine:3.15.1
script: |
echo "hello world"
`

// myResolvedResource wraps the data we want to return to Pipelines
type myResolvedResource struct {}

// Data returns the bytes of our hard-coded Pipeline
func (*myResolvedResource) Data() []byte {
return []byte(pipeline)
}

// Annotations returns any metadata needed alongside the data. None atm.
func (*myResolvedResource) Annotations() map[string]string {
return nil
}

// RefSource is the source reference of the remote data that records where the remote
// file came from including the url, digest and the entrypoint. None atm.
func (*myResolvedResource) RefSource() *pipelinev1.RefSource {
return nil
}
```

{{% /tab %}}

{{% tab "Previous Framework" %}}


```go
// Resolve uses the given resolution spec to resolve the requested file or resource.
func (r *resolver) Resolve(ctx context.Context, params []pipelinev1.Param) (framework.ResolvedResource, error) {
return &myResolvedResource{}, nil
}

// our hard-coded resolved file to return
const pipeline = `
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: my-pipeline
spec:
tasks:
- name: hello-world
taskSpec:
steps:
- image: alpine:3.15.1
script: |
echo "hello world"
`

// myResolvedResource wraps the data we want to return to Pipelines
type myResolvedResource struct {}

// Data returns the bytes of our hard-coded Pipeline
func (*myResolvedResource) Data() []byte {
return []byte(pipeline)
}

// Annotations returns any metadata needed alongside the data. None atm.
func (*myResolvedResource) Annotations() map[string]string {
return nil
}

// RefSource is the source reference of the remote data that records where the remote
// file came from including the url, digest and the entrypoint. None atm.
func (*myResolvedResource) RefSource() *pipelinev1.RefSource {
return nil
}
```


{{% /tab %}}

{{% /tabs %}}

```go
// Resolve uses the given params to resolve the requested file or resource.
func (r *resolver) Resolve(ctx context.Context, params map[string]string) (framework.ResolvedResource, error) {
// Resolve uses the given resolution spec to resolve the requested file or resource.
func (r *resolver) Resolve(ctx context.Context, req *v1beta1.ResolutionRequestSpec) (framework.ResolvedResource, error) {
return &myResolvedResource{}, nil
}

Expand Down
26 changes: 23 additions & 3 deletions docs/resolver-reference.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,33 @@ a resolver](./how-to-write-a-resolver.md).
Implementing this interface is required. It provides just enough
configuration for the framework to get a resolver running.

{{% tabs %}}

{{% tab "Upgraded Framework" %}}

| Method to Implement | Description |
|----------------------|-------------|
| Initialize | Use this method to perform any setup required before the resolver starts receiving requests. |
| GetName | Use this method to return a name to refer to your Resolver by. e.g. `"Git"` |
| GetSelector | Use this method to specify the labels that a resolution request must have to be routed to your resolver. |
| Validate | Use this method to validate the resolution Spec given to your resolver. |
| Resolve | Use this method to perform get the resource based on the ResolutionRequestSpec as input and return it, along with any metadata about it in annotations |

{{% /tab %}}

{{% tab "Previous Framework" %}}

| Method to Implement | Description |
|----------------------|-------------|
| Initialize | Use this method to perform any setup required before the resolver starts receiving requests. |
| GetName | Use this method to return a name to refer to your Resolver by. e.g. `"Git"` |
| GetSelector | Use this method to specify the labels that a resolution request must have to be routed to your resolver. |
| ValidateParams | Use this method to validate the parameters given to your resolver. |
| Resolve | Use this method to perform get the resource and return it, along with any metadata about it in annotations |
| ValidateParams | Use this method to validate the params given to your resolver. |
| Resolve | Use this method to perform get the resource based on params as input and return it, along with any metadata about it in annotations |

{{% /tab %}}

{{% /tabs %}}

## The `ConfigWatcher` Interface

Expand All @@ -38,7 +58,7 @@ api endpoints or base urls, service account names to use, etc...

| Method to Implement | Description |
|---------------------|-------------|
| GetConfigName | Use this method to return the name of the configmap admins will use to configure this resolver. Once this interface is implemented your `ValidateParams` and `Resolve` methods will be able to access your latest resolver configuration by calling `framework.GetResolverConfigFromContext(ctx)`. Note that this configmap must exist when your resolver starts - put a default one in your resolver's `config/` directory. |
| GetConfigName | Use this method to return the name of the configmap admins will use to configure this resolver. Once this interface is implemented your `Validate` and `Resolve` methods will be able to access your latest resolver configuration by calling `framework.GetResolverConfigFromContext(ctx)`. Note that this configmap must exist when your resolver starts - put a default one in your resolver's `config/` directory. |

## The `TimedResolution` Interface

Expand Down
10 changes: 9 additions & 1 deletion docs/resolver-template/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,19 @@ You can use this as a template to quickly get a new Resolver up and
running with your own preferred storage backend.

To reuse the template, simply copy this entire subdirectory to a new
directory. The entire program is defined in
directory.

The entire program for the `latest` framework is defined in
[`./cmd/resolver/main.go`](./cmd/resolver/main.go) and provides stub
implementations of all the methods defined by the [`framework.Resolver`
interface](../../pkg/remoteresolution/resolver/framework/interface.go).

If you choose to use the previous framework (will be deprecated at some point) is defined in
[`./cmd/demoresolver/main.go`](./cmd/demoresolver/main.go) and provides stub
implementations of all the methods defined by the [`framework.Resolver`
interface](../../pkg/resolution/resolver/framework/interface.go).


Once copied you'll need to run `go mod init` and `go mod tidy` at the root
of your project. We don't need this in `tektoncd/resolution` because this
submodule relies on the `go.mod` and `go.sum` defined at the root of the repo.
Expand Down
Loading

0 comments on commit 0a88237

Please sign in to comment.