Skip to content

Commit

Permalink
Merge the content of design doc to the README
Browse files Browse the repository at this point in the history
This commit is to use the content of design doc to merge into the
README. The refined README is more like a 'user manual'.
  • Loading branch information
YolandaDu1997 committed Jul 30, 2020
1 parent 10b2db2 commit 178fef8
Showing 1 changed file with 67 additions and 22 deletions.
89 changes: 67 additions & 22 deletions generators/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,54 @@

# Tekton Generators
This project contains experimental code to create a tool for generating Tekton spec from simplified configs. This is important because right now it is not easy for users to use Tekton resources to bootstrap common workflows in a configurable way and users usually need to set up a couple of configuration files on their own. As a result, the objective of Tekton Generators is to help users create and run their pipelines more easily and efficiently.

Generators can use a simple spec input to automate the pipeline set up for users. Different ways of running the pipeline (eg. PipelineRun, Trigger) should also be generated at the same time. Users may need to use the resources on their cluster to help build the pipeline. A command line tool can be used to help interact with generators more easily.

See [tektoncd/pipeline/#2590](https://github.com/tektoncd/pipeline/issues/2590) for more information and background.

## Features
This experimental project has been broken down into the features as follows:
1. Parse the yaml file with io.Reader and store the result in the self-defined struct
2. Create tool that given an input spec with steps, generates the resulting Tekton resources for the particular type.
- Create binary that invokes another binary on the path based on the type.
- Read input steps and generate resulting pipeline config that mounts GitHub workspace and configures steps accordingly.
3. Add support for writing output to disk.
4. Add support for tasks and pipelines.
5. Add support for applying config to cluster.
6. Add support for deleting configs from cluster.

## CLI
Users can use the generators CLI to choose to generate the pipeline with PipelineRun or Triggers to run it.
The following commands help you understand and effectively use the generators CLI:
- `tkn-gen pipelinerun`: Manage the generated config with PipelineRun.
- `tkn-gen trigger`: Manage the generated config with Trigger.

In terms of the commands `tkn-gen pipelinerun` and `tkn-gen trigger`, their sub-commands are as follows:
- `show`: Print generated configuration.
- `write`: Write generated configuration to disk.
- `apply`: Apply generated configuration to Kubernetes (based on local k8s context).
- `delete`: Delete generated resources from the k8s cluster.

For example, assume we have the input file `test.yaml`, we can run the spec in the file with the PipelineRun on the cluster by:
```
tkn-gen pipelinerun apply -f test.yaml
```

For every `tkn-gen` command, you can use `-h` or `--help` flags to display specific help for that command.

This project contains experimental code to create a tool for generating Tekton spec from simplified configs. The goal is to help users bootstrap pipelines in a configurable way.

See [tektoncd/pipeline/#2590](https://github.com/tektoncd/pipeline/issues/2590) information and background.
## GitHub Generator

## GitHub token
GitHub is common for users to build their work. In order to make it easier to configure the Tekton resources, the goal is to build the GitHub type of Tekton Generator to help users work with their pipelines.

### Credentials
#### GitHub token
You will need to create a [GitHub Personal Access Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token#creating-a-token) and set it in the Kubernetes scecret `github` in the key `token` like this:
```
kubectl create secret generic github --from-literal token="YOUR_GITHUB_PERSONAL_ACCESS_TOKEN"
```
## Webhook secret

#### Webhook secret
You would expect to create a [Webhook secret token](https://developer.github.com/webhooks/securing/#setting-your-secret-token) and configure the GitHub webhook to use this value.
You can contain this value in the Kubernetes secret like this :
```yaml
Expand All @@ -26,7 +65,8 @@ Then you can create it on the command line with `kubectl` like this :
kubectl apply -f secret.yaml
```
Now it can be passed as a reference to the GitHub interceptor.
## Service Account
### Service Account
The [`serviceAccountName`](https://github.com/tektoncd/triggers/blob/master/docs/eventlisteners.md#serviceAccountName) is a required field for Tekton Triggers. You need to provide it in the input GitHub config settings. You can create the service account with the [webhook secret](#webhook-secret) that is used in the Kubernetes cluster like this :
```yaml
apiVersion: v1
Expand All @@ -38,26 +78,30 @@ secrets:
```
With the service account, you also need to create a role to follow certain roles and the role binding. See the [example](https://github.com/tektoncd/triggers/tree/master/examples/role-resources/triggerbinding-roles) to create your own.
Use `kubectl apply` to create these resources on the cluster as well.

**Please Note: If you use Kaniko in the task, you need to make the service account have enough credentials to push images. On GKE, it has been configured already. You can ignore this.**

## Dependent Tasks
The generated config would expect to use the tasks already on the cluster adding to the pipeline. The tasks include [`git-clone`](https://github.com/tektoncd/catalog/blob/v1beta1/git/git-clone.yaml) and [`github-set-status`](https://github.com/tektoncd/catalog/blob/v1beta1/github/set_status.yaml). The `git-clone` task is used to help clone a repo into the workspace. The `status-task` is used to help allow external services to mark GitHub commits with a state.
### Dependent Tasks

The generated config would expect to use the tasks already on the cluster adding to the pipeline. The tasks include [`git-clone`](https://github.com/tektoncd/catalog/blob/v1beta1/git/git-clone.yaml) and [`github-set-status`](https://github.com/tektoncd/catalog/blob/v1beta1/github/set_status.yaml). The `git-clone` task is used to help clone a repo into the workspace. The `status-task` is used to help allow external services to mark GitHub commits with a state.
**Please Note: this git-clone Task is only able to fetch code from the public repo for the time being.**
### Install the Task

#### Install the Task
You can install the tasks with the specified revision(commit SHA) on the command line like this:
```
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/<revision>/github/set_status.yaml
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/<revision>/git/git-clone.yaml
```

## Input configurations
Here is the example of the GitHub input config:
### Input configurations
The input provided by users is a simplified config file. Some useful identifiers like API version, Kind and Metadata are included in the input config. Users should be able to specify the runtime configuration however they want. A common case is to get started with users’ GitHub repo and build steps. This is to build the initial schema of the github generator. Here is the example of the GitHub input config:
```yaml
kind: GitHub
metadata:
name: github-build
spec:
url: "https://github.com/YolandaDu1997/hello-world"
revision: 6c6ed17cd60127f96da41f51224914b2e825f939
branch: "master"
storage: 1Gi
secretName: webhook-secret
Expand All @@ -73,8 +117,7 @@ spec:
- --destination=gcr.io/<use your project>/kaniko-test
- --verbosity=debug
```
### Fields
#### Fields
- **kind**: the kind of generators (*required*)
- **metadata**: the metadata that uniquely identifies the `GitHub` resource object. For example, a `name` (*required*)
- **spec**: the GitHub sepc
Expand All @@ -87,15 +130,17 @@ spec:
- **serviceAccountName**: the name of the service account used in triggers (*required* when generating Triggers)
- **steps**: the [Tekton steps](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md#defining-steps) to run in the pipeline (*required*)

## Features
With the input config, we can generate the Tekton task with the steps built by users, then the Pipeline using taskRef with the prepended git-clone task, generated task and finally task. The tasks are executed in a specific order within the Pipeline. Basically, the pipeline will do the following:
- Clone the given repo to the shared workspaces
- Build the steps that users want in the input config
- Set the commit status according to the execution result

This experimental project has been broken down into the features as follows:
### Run with PipelineRun
If you want to run Pipeline with PipelineRun, the fields `secretName`, `secretKey` and `serviceAccountName` are optional. You need to follow the document to install the dependent catalog tasks. Use the CLI to manage the PipelineRun with the specified parameters values, workspaces with specific disk storage and access modes in the input config. Then it will pass them to the Pipeline during execution.

1. Parse the yaml file with io.Reader and store the result in the self-defined struct
2. Create tool that given an input spec with steps, generates the resulting Tekton resources for the particular type.
- Create binary that invokes another binary on the path based on the type.
- Read input steps and generate resulting pipeline config that mounts GitHub workspace and configures steps accordingly.
3. Add support for writing output to disk.
4. Add support for tasks and pipelines.
5. Add support for applying config to cluster.
6. Add support for deleting configs from cluster.
### Run with Trigger
If you want to run Pipeline with Trigger, it is optional to set up the commit sha, because the generated Tekton EventListener will contain triggers to listen to both GitHub push and GitHub pull request events. We can capture fields like the commit sha from an event and store them as parameters in the TriggerBinding, which then can be passed to TriggerTemplate.

The resource template used in the TriggerTemplate is also the Tekton PipelineRun. The difference from the one described before is that the value of parameters in this PipelineRun is coming from TriggerTemplate. When generating the EventListener, the interceptors of processing GitHub push and pull request events are included in the triggers. The EventListener connects the TriggerBinding and TriggerTemplate. When the HTTP based events with JSON payloads comes, it creates the Template resources accordingly.

To run the pipeline with Triggers, except for the dependent catalog tasks, other dependencies like [webhook secret](#webhook-secret), [service account](#service-account), [role-based access control (RBAC)](https://github.com/tektoncd/triggers/blob/master/examples/role-resources/triggerbinding-roles/role.yaml) are also required.

0 comments on commit 178fef8

Please sign in to comment.