Skip to content

Commit

Permalink
TEP-0090: Matrix - Status
Browse files Browse the repository at this point in the history
[TEP-0090: Matrix][tep-0090] proposed executing a `PipelineTask` in
parallel `TaskRuns` and `Runs` with substitutions from combinations
of `Parameters` in a `Matrix`.

In this change, we describe how the status of a `PipelineRun` with
a matrixed `PipelineTask` would be in full and minimal modes.

Previously, we had included `matrixId` to the `PipelineRun` status
so that it can help with identifying `Results` from fanned out
`TaskRuns` or `Runs`. However, we ended up descoping that problem
so that it can be addressed when TEP-0075 and TEP-0076 are implemented.
See [`Results` from fanned-out `PipelineTasks`][results] for details.
In this change, we remove the `matrixId` in `PipelineRun` status so
that the problem can be addressed completely later.

[tep-0090]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md
[results]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md#results-from-fanned-out-pipelinetasks
  • Loading branch information
jerop committed Jun 10, 2022
1 parent ca370d5 commit b7defb6
Show file tree
Hide file tree
Showing 2 changed files with 45 additions and 31 deletions.
74 changes: 44 additions & 30 deletions teps/0090-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
status: implementable
title: Matrix
creation-date: '2021-10-13'
last-updated: '2022-02-14'
last-updated: '2022-06-06'
authors:
- '@jerop'
- '@pritidesai'
Expand Down Expand Up @@ -46,7 +46,6 @@ see-also:
- [Parameters](#parameters)
- [Substituting String Parameters in the Tasks](#substituting-string-parameters-in-the-tasks)
- [Substituting Array Parameters in the Tasks](#substituting-array-parameters-in-the-tasks)
- [Combinations of Parameters in the Matrix](#combinations-of-parameters-in-the-matrix)
- [Results](#results)
- [Specifying Results in the Matrix](#specifying-results-in-the-matrix)
- [Results from Fanned Out PipelineTasks](#results-from-fanned-out-pipelinetasks)
Expand All @@ -61,6 +60,7 @@ see-also:
- [When Expressions](#when-expressions)
- [Retries](#retries)
- [Timeouts](#timeouts)
- [Status](#status)
- [Design Evaluation](#design-evaluation)
- [API Conventions](#api-conventions)
- [Reusability](#reusability)
Expand Down Expand Up @@ -1021,34 +1021,6 @@ We plan to provide minimum feature set needed to meet the use cases, as describe
[simplicity][simplicity] design principle. However, this remains an option we can explore
later if we have use cases for it and already support `Arrays of Arrays` in Tekton Pipelines.

#### Combinations of Parameters in the Matrix

We need a way to identify a specific combination of `Parameters` in the `Matrix` so that
we can map it to a given `TaskRun` and its components, such as `Results`. We cannot rely
on the ordering of the `Parameters` in the `Matrix` to be maintained as specified in the
`Pipeline`. We propose that we add the generated combinations identifications to the
`PipelineRunStatusFields` in `PipelineRunStatus`. For example:

```yaml
taskRuns:
matrixed-pr-foo-0:
pipelineTaskName: foo
matrixId: 0
status:
...
matrixed-pr-foo-1:
pipelineTaskName: foo
matrixId: 1
status:
...
```

The `Parameters` were used for a given `TaskRun` or `Runs` would available in the `TaskRun`
status itself. The `matrixId` in the `PipelineRunStatus` is useful in identifying that the
`TaskRun` is part of a `Matrix`, especially if the `TaskRun` name was too long so got hashed.
This is also in line with providing the minimal information needed in the `PipelineRunStatus`
as discussed in [TEP-0100: Embedded TaskRuns and Runs Status in PipelineRuns][tep-0100].

### Results

#### Specifying Results in the Matrix
Expand Down Expand Up @@ -1340,6 +1312,48 @@ spec:
name: build-push
```

### Status

The status of `PipelineRuns` with fanned-out `PipelineTasks` will list all the `TaskRuns` and `Runs` created.

In [TEP-0100][tep-0100] we proposed changes to `PipelineRun` status to reduce the amount of information stored about
the status of `TaskRuns` and `Runs` to improve performance, reduce memory bloat and improve extensibility. Now that
those changes have been implemented, the `PipelineRun` status is set up to handle `Matrix` without exacerbating the
performance and storage issues that were there before.

We will populate `ChildReferences` for all fanned out `TaskRuns` and `Runs`, as shown below:

```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: matrixed-pr
...
spec:
...
status:
completionTime: "2020-05-04T02:19:14Z"
conditions:
- lastTransitionTime: "2020-05-04T02:19:14Z"
message: "Tasks Completed: 4, Skipped: 0"
reason: Succeeded
status: "True"
type: Succeeded
startTime: "2020-05-04T02:00:11Z"
childReferences:
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: matrixed-pr-foo-0
pipelineTaskName: foo
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: matrixed-pr-foo-1
pipelineTaskName: foo
```

The `ChildReferences` will be populated for matrixed `PipelineTasks` regardless of the embedded status flags
because that is the API behavior we're migrating to.

## Design Evaluation

### API Conventions
Expand Down
2 changes: 1 addition & 1 deletion teps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ This is the complete list of Tekton teps:
|[TEP-0086](0086-changing-the-way-result-parameters-are-stored.md) | Changing the way result parameters are stored | proposed | 2022-04-07 |
|[TEP-0088](0088-result-summaries.md) | Tekton Results - Record Summaries | proposed | 2021-10-01 |
|[TEP-0089](0089-nonfalsifiable-provenance-support.md) | Non-falsifiable provenance support | implementable | 2022-01-18 |
|[TEP-0090](0090-matrix.md) | Matrix | implementable | 2022-02-14 |
|[TEP-0090](0090-matrix.md) | Matrix | implementable | 2022-06-06 |
|[TEP-0092](0092-scheduling-timeout.md) | Scheduling Timeout | implementable | 2022-04-11 |
|[TEP-0094](0094-configuring-resources-at-runtime.md) | Configuring Resources at Runtime | implemented | 2022-03-11 |
|[TEP-0095](0095-common-repository-configuration.md) | Common Repository Configuration | proposed | 2021-11-29 |
Expand Down

0 comments on commit b7defb6

Please sign in to comment.