Skip to content

Commit

Permalink
Promote the provenance field in status
Browse files Browse the repository at this point in the history 
Fixes tektoncd#6309.

Prior, the `provenance` field in status was gated by the dedicated feature
flag named `enable-provenance-in-status`.

This PR moves the field out of the feature flag.

Signed-off-by: Chuang Wang <chuangw@google.com>
  • Loading branch information
@chuangw6
chuangw6 committed May 2, 2023
1 parent 09d422c commit d6aabf6
Show file tree
Hide file tree
Showing 14 changed files with 114 additions and 123 deletions.
9 changes: 4 additions & 5 deletions config/config-feature-flags.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,10 @@ data:
# If it is set to "warn", TaskRuns and PipelineRuns will run to completion if no matching policies are found, and an error will be logged.
# If it is set to "ignore", TaskRuns and PipelineRuns will run to completion if no matching policies are found, and no error will be logged.
trusted-resources-verification-no-match-policy: "ignore"
# Setting this flag to "true" enables populating the "provenance" field in TaskRun
# and PipelineRun status. This field contains metadata about resources used
# in the TaskRun/PipelineRun such as the source from where a remote Task/Pipeline
# definition was fetched.
enable-provenance-in-status: "false"
# Setting this flag will determine the version for custom tasks created by PipelineRuns.
# Acceptable values are "v1beta1" and "v1alpha1".
# The default is "v1beta1".
custom-task-version: "v1beta1"
# Setting this flag will determine how Tekton pipelines will handle non-falsifiable provenance.
# If set to "spire", then SPIRE will be used to ensure non-falsifiable provenance.
# If set to "none", then Tekton will not have non-falsifiable provenance.
Expand Down
10 changes: 5 additions & 5 deletions docs/additional-configs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -245,10 +245,11 @@ Defaults to "ignore".

- `results-from`: set this flag to "termination-message" to use the container's termination message to fetch results from. This is the default method of extracting results. Set it to "sidecar-logs" to enable use of a results sidecar logs to extract results instead of termination message.

- `enable-provenance-in-status`: set this flag to "true" to enable recording
the `provenance` field in `TaskRun` and `PipelineRun` status. The `provenance`
field contains metadata about resources used in the TaskRun/PipelineRun such as the
source from where a remote Task/Pipeline definition was fetched.
- `custom-task-version`: set this flag to "v1beta1" to have `PipelineRuns` create `CustomRuns`
from Custom Tasks. Set it to "v1alpha1" to have `PipelineRuns` create the legacy alpha
`Runs`. This may be needed if you are using legacy Custom Tasks that listen for `*v1alpha1.Run`
instead of `*v1beta1.CustomRun`. For more information, see [Runs](runs.md) and [CustomRuns](customruns.md).
Flag defaults to "v1beta1".

For example:

Expand Down Expand Up @@ -284,7 +285,6 @@ Features currently in "alpha" are:
| [Task-level Resource Requirements](compute-resources.md#task-level-compute-resources-configuration) | [TEP-0104](https://github.com/tektoncd/community/blob/main/teps/0104-tasklevel-resource-requirements.md) | [v0.39.0](https://github.com/tektoncd/pipeline/releases/tag/v0.39.0) | |
| [Object Params and Results](pipelineruns.md#specifying-parameters) | [TEP-0075](https://github.com/tektoncd/community/blob/main/teps/0075-object-param-and-result-types.md) | [v0.38.0](https://github.com/tektoncd/pipeline/releases/tag/v0.38.0) | | |
| [Trusted Resources](./trusted-resources.md) | [TEP-0091](https://github.com/tektoncd/community/blob/main/teps/0091-trusted-resources.md) | N/A | `trusted-resources-verification-no-match-policy` |
| [`Provenance` field in Status](pipeline-api.md#provenance) | [issue#5550](https://github.com/tektoncd/pipeline/issues/5550) | N/A | `enable-provenance-in-status` |
| [Larger Results via Sidecar Logs](#enabling-larger-results-using-sidecar-logs) | [TEP-0127](https://github.com/tektoncd/community/blob/main/teps/0127-larger-results-via-sidecar-logs.md) | [v0.43.0](https://github.com/tektoncd/pipeline/releases/tag/v0.43.0) | `results-from` |
| [Configure Default Resolver](./resolution.md#configuring-built-in-resolvers) | [TEP-0133](https://github.com/tektoncd/community/blob/main/teps/0133-configure-default-resolver.md) | N/A | |

Expand Down
7 changes: 0 additions & 7 deletions pkg/apis/config/feature_flags.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,6 @@ const (
DefaultEnforceNonfalsifiability = EnforceNonfalsifiabilityNone
// DefaultNoMatchPolicyConfig is the default value for "trusted-resources-verification-no-match-policy".
DefaultNoMatchPolicyConfig = IgnoreNoMatchPolicy
// DefaultEnableProvenanceInStatus is the default value for "enable-provenance-status".
DefaultEnableProvenanceInStatus = false
// DefaultResultExtractionMethod is the default value for ResultExtractionMethod
DefaultResultExtractionMethod = ResultExtractionMethodTerminationMessage
// DefaultMaxResultSize is the default value in bytes for the size of a result
Expand All @@ -87,7 +85,6 @@ const (
sendCloudEventsForRuns = "send-cloudevents-for-runs"
enforceNonfalsifiability = "enforce-nonfalsifiability"
verificationNoMatchPolicy = "trusted-resources-verification-no-match-policy"
enableProvenanceInStatus = "enable-provenance-in-status"
resultExtractionMethod = "results-from"
maxResultSize = "max-result-size"
)
Expand All @@ -113,7 +110,6 @@ type FeatureFlags struct {
// warn: skip trusted resources verification when no matching verification policies found and log a warning
// fail: fail the taskrun or pipelines run if no matching verification policies found
VerificationNoMatchPolicy string
EnableProvenanceInStatus bool
ResultExtractionMethod string
MaxResultSize int
}
Expand Down Expand Up @@ -167,9 +163,6 @@ func NewFeatureFlagsFromMap(cfgMap map[string]string) (*FeatureFlags, error) {
if err := setVerificationNoMatchPolicy(cfgMap, DefaultNoMatchPolicyConfig, &tc.VerificationNoMatchPolicy); err != nil {
return nil, err
}
if err := setFeature(enableProvenanceInStatus, DefaultEnableProvenanceInStatus, &tc.EnableProvenanceInStatus); err != nil {
return nil, err
}
if err := setResultExtractionMethod(cfgMap, DefaultResultExtractionMethod, &tc.ResultExtractionMethod); err != nil {
return nil, err
}
Expand Down
3 changes: 0 additions & 3 deletions pkg/apis/config/feature_flags_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ func TestNewFeatureFlagsFromConfigMap(t *testing.T) {
EnableAPIFields: config.DefaultEnableAPIFields,
SendCloudEventsForRuns: config.DefaultSendCloudEventsForRuns,
VerificationNoMatchPolicy: config.DefaultNoMatchPolicyConfig,
EnableProvenanceInStatus: config.DefaultEnableProvenanceInStatus,
ResultExtractionMethod: config.DefaultResultExtractionMethod,
MaxResultSize: config.DefaultMaxResultSize,
},
Expand All @@ -65,7 +64,6 @@ func TestNewFeatureFlagsFromConfigMap(t *testing.T) {
SendCloudEventsForRuns: true,
EnforceNonfalsifiability: "spire",
VerificationNoMatchPolicy: config.FailNoMatchPolicy,
EnableProvenanceInStatus: true,
ResultExtractionMethod: "termination-message",
MaxResultSize: 4096,
},
Expand Down Expand Up @@ -172,7 +170,6 @@ func TestNewFeatureFlagsFromEmptyConfigMap(t *testing.T) {
SendCloudEventsForRuns: config.DefaultSendCloudEventsForRuns,
EnforceNonfalsifiability: config.DefaultEnforceNonfalsifiability,
VerificationNoMatchPolicy: config.DefaultNoMatchPolicyConfig,
EnableProvenanceInStatus: config.DefaultEnableProvenanceInStatus,
ResultExtractionMethod: config.DefaultResultExtractionMethod,
MaxResultSize: config.DefaultMaxResultSize,
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/config/testdata/feature-flags-all-flags-set.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ data:
send-cloudevents-for-runs: "true"
enforce-nonfalsifiability: "spire"
trusted-resources-verification-no-match-policy: "fail"
enable-provenance-in-status: "true"
custom-task-version: "v1beta1"
11 changes: 3 additions & 8 deletions pkg/apis/pipeline/v1beta1/pipelinerun_conversion_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@ func TestPipelineRunConversionBadType(t *testing.T) {
}

func TestPipelineRunConversion(t *testing.T) {
defaultFeatureFlags, _ := config.NewFeatureFlagsFromMap(map[string]string{})

tests := []struct {
name string
in *v1beta1.PipelineRun
Expand Down Expand Up @@ -305,14 +307,7 @@ func TestPipelineRunConversion(t *testing.T) {
URI: "test-uri",
Digest: map[string]string{"sha256": "digest"},
},
FeatureFlags: &config.FeatureFlags{
RunningInEnvWithInjectedSidecars: config.DefaultRunningInEnvWithInjectedSidecars,
EnableAPIFields: config.DefaultEnableAPIFields,
AwaitSidecarReadiness: config.DefaultAwaitSidecarReadiness,
VerificationNoMatchPolicy: config.DefaultNoMatchPolicyConfig,
ResultExtractionMethod: config.DefaultResultExtractionMethod,
MaxResultSize: config.DefaultMaxResultSize,
},
FeatureFlags: defaultFeatureFlags,
},
},
},
Expand Down
11 changes: 3 additions & 8 deletions pkg/apis/pipeline/v1beta1/taskrun_conversion_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ func TestTaskRunConversionBadType(t *testing.T) {
}

func TestTaskRunConversion(t *testing.T) {
defaultFeatureFlags, _ := config.NewFeatureFlagsFromMap(map[string]string{})

tests := []struct {
name string
in *v1beta1.TaskRun
Expand Down Expand Up @@ -237,14 +239,7 @@ func TestTaskRunConversion(t *testing.T) {
URI: "test-uri",
Digest: map[string]string{"sha256": "digest"},
},
FeatureFlags: &config.FeatureFlags{
RunningInEnvWithInjectedSidecars: config.DefaultRunningInEnvWithInjectedSidecars,
EnableAPIFields: config.DefaultEnableAPIFields,
AwaitSidecarReadiness: config.DefaultAwaitSidecarReadiness,
VerificationNoMatchPolicy: config.DefaultNoMatchPolicyConfig,
ResultExtractionMethod: config.DefaultResultExtractionMethod,
MaxResultSize: config.DefaultMaxResultSize,
},
FeatureFlags: defaultFeatureFlags,
}},
},
},
Expand Down
22 changes: 10 additions & 12 deletions pkg/reconciler/pipelinerun/pipelinerun.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1224,19 +1224,17 @@ func storePipelineSpecAndMergeMeta(ctx context.Context, pr *v1beta1.PipelineRun,
// Propagate refSource from remote resolution to PipelineRun Status
// This lives outside of the status.spec check to avoid the case where only the spec is available in the first reconcile and source comes in next reconcile.
cfg := config.FromContextOrDefaults(ctx)
if cfg.FeatureFlags.EnableProvenanceInStatus {
if pr.Status.Provenance == nil {
pr.Status.Provenance = &v1beta1.Provenance{}
}
// Store FeatureFlags in the Provenance.
pr.Status.Provenance.FeatureFlags = cfg.FeatureFlags
if pr.Status.Provenance == nil {
pr.Status.Provenance = &v1beta1.Provenance{}
}
// Store FeatureFlags in the Provenance.
pr.Status.Provenance.FeatureFlags = cfg.FeatureFlags

if meta != nil && meta.RefSource != nil && pr.Status.Provenance.RefSource == nil {
pr.Status.Provenance.RefSource = meta.RefSource
}
if meta != nil && meta.RefSource != nil && pr.Status.Provenance.ConfigSource == nil {
pr.Status.Provenance.ConfigSource = (*v1beta1.ConfigSource)(meta.RefSource)
}
if meta != nil && meta.RefSource != nil && pr.Status.Provenance.RefSource == nil {
pr.Status.Provenance.RefSource = meta.RefSource
}
if meta != nil && meta.RefSource != nil && pr.Status.Provenance.ConfigSource == nil {
pr.Status.Provenance.ConfigSource = (*v1beta1.ConfigSource)(meta.RefSource)
}

return nil
Expand Down
51 changes: 39 additions & 12 deletions pkg/reconciler/pipelinerun/pipelinerun_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ var (

now = time.Date(2022, time.January, 1, 0, 0, 0, 0, time.UTC)
testClock = clock.NewFakePassiveClock(now)

defaultFeatureFlags, _ = config.NewFeatureFlagsFromMap(map[string]string{})
alphaFeatureFlags, _ = config.NewFeatureFlagsFromMap(map[string]string{
"enable-api-fields": "alpha",
})
)

const (
Expand Down Expand Up @@ -4591,6 +4596,9 @@ status:
`)

expectedPr := expectedPrStatus
expectedPr.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: defaultFeatureFlags,
}

if d := cmp.Diff(expectedPr, reconciledRun, ignoreResourceVersion, ignoreLastTransitionTime, ignoreCompletionTime, ignoreStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected to see pipeline run results created. Diff %s", diff.PrintWantGot(d))
Expand Down Expand Up @@ -4739,6 +4747,9 @@ status:
`)

expectedPr := expectedPrStatus
expectedPr.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: defaultFeatureFlags,
}

if d := cmp.Diff(expectedPr, reconciledRun, ignoreResourceVersion, ignoreLastTransitionTime, ignoreCompletionTime, ignoreStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected to see pipeline run results created. Diff %s", diff.PrintWantGot(d))
Expand Down Expand Up @@ -4898,15 +4909,7 @@ metadata:
Provenance: &v1beta1.Provenance{
RefSource: refSource.DeepCopy(),
ConfigSource: (*v1beta1.ConfigSource)(refSource.DeepCopy()),
FeatureFlags: &config.FeatureFlags{
RunningInEnvWithInjectedSidecars: config.DefaultRunningInEnvWithInjectedSidecars,
EnableAPIFields: config.DefaultEnableAPIFields,
AwaitSidecarReadiness: config.DefaultAwaitSidecarReadiness,
VerificationNoMatchPolicy: config.DefaultNoMatchPolicyConfig,
EnableProvenanceInStatus: true,
ResultExtractionMethod: config.DefaultResultExtractionMethod,
MaxResultSize: config.DefaultMaxResultSize,
},
FeatureFlags: defaultFeatureFlags,
},
},
}
Expand Down Expand Up @@ -4957,17 +4960,16 @@ metadata:
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
ctx := ttesting.EnableFeatureFlagField(context.Background(), t, "enable-provenance-in-status")
// mock first reconcile
if err := storePipelineSpecAndMergeMeta(ctx, pr, tc.reconcile1Args.pipelineSpec, tc.reconcile1Args.resolvedObjectMeta); err != nil {
if err := storePipelineSpecAndMergeMeta(context.Background(), pr, tc.reconcile1Args.pipelineSpec, tc.reconcile1Args.resolvedObjectMeta); err != nil {
t.Errorf("storePipelineSpec() error = %v", err)
}
if d := cmp.Diff(pr, tc.wantPipelineRun); d != "" {
t.Fatalf(diff.PrintWantGot(d))
}

// mock second reconcile
if err := storePipelineSpecAndMergeMeta(ctx, pr, tc.reconcile2Args.pipelineSpec, tc.reconcile2Args.resolvedObjectMeta); err != nil {
if err := storePipelineSpecAndMergeMeta(context.Background(), pr, tc.reconcile2Args.pipelineSpec, tc.reconcile2Args.resolvedObjectMeta); err != nil {
t.Errorf("storePipelineSpec() error = %v", err)
}
if d := cmp.Diff(pr, tc.wantPipelineRun); d != "" {
Expand Down Expand Up @@ -8085,6 +8087,9 @@ spec:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -8332,6 +8337,10 @@ labels:
t.Errorf("expected to see TaskRun %v created. Diff %s", expectedTaskRuns[i].Name, diff.PrintWantGot(d))
}
}

tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("found PipelineRun does not match expected PipelineRun. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -8870,6 +8879,9 @@ spec:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -9092,6 +9104,9 @@ spec:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -9685,6 +9700,9 @@ spec:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -10033,6 +10051,9 @@ spec:
}
}

tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -10477,6 +10498,9 @@ status:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, cmpopts.SortSlices(lessChildReferences), cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down Expand Up @@ -10997,6 +11021,9 @@ spec:
if err != nil {
t.Fatalf("Got an error getting reconciled run out of fake client: %s", err)
}
tt.expectedPipelineRun.Status.Provenance = &v1beta1.Provenance{
FeatureFlags: alphaFeatureFlags,
}
if d := cmp.Diff(tt.expectedPipelineRun, pipelineRun, ignoreResourceVersion, ignoreTypeMeta, ignoreLastTransitionTime, ignoreStartTime, ignoreFinallyStartTime, cmpopts.EquateEmpty()); d != "" {
t.Errorf("expected PipelineRun was not created. Diff %s", diff.PrintWantGot(d))
}
Expand Down
26 changes: 12 additions & 14 deletions pkg/reconciler/taskrun/taskrun.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -881,20 +881,18 @@ func storeTaskSpecAndMergeMeta(ctx context.Context, tr *v1beta1.TaskRun, ts *v1b
}

cfg := config.FromContextOrDefaults(ctx)
if cfg.FeatureFlags.EnableProvenanceInStatus {
if tr.Status.Provenance == nil {
tr.Status.Provenance = &v1beta1.Provenance{}
}
// Store FeatureFlags in the Provenance.
tr.Status.Provenance.FeatureFlags = cfg.FeatureFlags
// Propagate RefSource from remote resolution to TaskRun Status
// This lives outside of the status.spec check to avoid the case where only the spec is available in the first reconcile and refSource comes in next reconcile.
if meta != nil && meta.RefSource != nil && tr.Status.Provenance.RefSource == nil {
tr.Status.Provenance.RefSource = meta.RefSource
}
if meta != nil && meta.RefSource != nil && tr.Status.Provenance.ConfigSource == nil {
tr.Status.Provenance.ConfigSource = (*v1beta1.ConfigSource)(meta.RefSource)
}
if tr.Status.Provenance == nil {
tr.Status.Provenance = &v1beta1.Provenance{}
}
// Store FeatureFlags in the Provenance.
tr.Status.Provenance.FeatureFlags = cfg.FeatureFlags
// Propagate RefSource from remote resolution to TaskRun Status
// This lives outside of the status.spec check to avoid the case where only the spec is available in the first reconcile and refSource comes in next reconcile.
if meta != nil && meta.RefSource != nil && tr.Status.Provenance.RefSource == nil {
tr.Status.Provenance.RefSource = meta.RefSource
}
if meta != nil && meta.RefSource != nil && tr.Status.Provenance.ConfigSource == nil {
tr.Status.Provenance.ConfigSource = (*v1beta1.ConfigSource)(meta.RefSource)
}

return nil
Expand Down
Loading

0 comments on commit d6aabf6

Please sign in to comment.