Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-v0.44.x] Fix spammy logs #6780

Conversation

tekton-robot
Copy link
Collaborator

This is an automated cherry-pick of #6051

/assign lbernick

NONE

A couple of lines are causing a lot of spam in the controller logs:
- logging the content of steps
- logging that the cache client cannot be find, which is expected
  since the cache is only available in the Run/CustomRun controller
  and not for TaskRun and PipelineRun controllers

Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>
@tekton-robot tekton-robot added the release-note-none Denotes a PR that doesnt merit a release note. label Jun 6, 2023
@tekton-robot tekton-robot mentioned this pull request Jun 6, 2023
7 tasks
@tekton-robot tekton-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jun 6, 2023
@tekton-robot
Copy link
Collaborator Author

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lbernick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 6, 2023
@tekton-robot
Copy link
Collaborator Author

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 84.9% 85.2% 0.3
pkg/reconciler/pipelinerun/pipelinerun.go 90.5% 86.6% -3.9
pkg/reconciler/taskrun/taskrun.go 84.7% 84.5% -0.2

@tekton-robot
Copy link
Collaborator Author

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 84.9% 85.2% 0.3
pkg/reconciler/pipelinerun/pipelinerun.go 90.5% 86.6% -3.9
pkg/reconciler/taskrun/taskrun.go 84.7% 84.5% -0.2

@Yongxuanzhang
Copy link
Member

/retest
All of the integration tests failed on TestExamples/v1beta1/taskruns/gcs-resource
logs:
https://prow.tekton.dev/view/gs/tekton-prow/pr-logs/pull/tektoncd_pipeline/6780/pull-tekton-pipeline-alpha-integration-tests/1666131122163027968
https://prow.tekton.dev/view/gs/tekton-prow/pr-logs/pull/tektoncd_pipeline/6780/pull-tekton-pipeline-beta-integration-tests/1666131122246914048
https://prow.tekton.dev/view/gs/tekton-prow/pr-logs/pull/tektoncd_pipeline/6780/pull-tekton-pipeline-integration-tests/1666131122104307712
Something like:

 build_logs.go:37: build logs 
        >>> Container step-create-dir-source-9bhvk:
        
        >>> Container step-fetch-source-l2chz:
        BucketNotFoundException: 404 gs://build-crd-tests bucket does not exist.
        

@Yongxuanzhang
Copy link
Member

/kind misc

@tekton-robot tekton-robot added the kind/misc Categorizes issue or PR as a miscellaneuous one. label Jun 6, 2023
@Yongxuanzhang
Copy link
Member

I guess we need to manually fix and patch this?

@vdemeester
Copy link
Member

@Yongxuanzhang I think we need to do something so that we don't get unrelated failure when we do these cherry-picks. All the jobs are not versioned per branch, so changes to their definition to accommodate the main branch may break supported release branch.

@lbernick
Copy link
Member

lbernick commented Jun 7, 2023

This failure is happening in #6787 as well but I unfortunately wasn't able to reproduce it locally. Do any of you know whether these tests are actually fetching from a real GCS bucket gs://build-crd-tests (which doesn't exist) or whether there's a local gcs (?) of some sort, similar to how we use in-cluster image registries?
I looked at our prow config and the only changes since the last commits were merged on this release branch were bumping the test-runner image, which I'm not sure explains this failure.

@Yongxuanzhang
Copy link
Member

Yongxuanzhang commented Jun 7, 2023

This failure is happening in #6787 as well but I unfortunately wasn't able to reproduce it locally. Do any of you know whether these tests are actually fetching from a real GCS bucket gs://build-crd-tests (which doesn't exist) or whether there's a local gcs (?) of some sort, similar to how we use in-cluster image registries? I looked at our prow config and the only changes since the last commits were merged on this release branch were bumping the test-runner image, which I'm not sure explains this failure.

The example has been removed, could it be the corresponding resource was removed as well? So we cannot pass it now

@lbernick
Copy link
Member

lbernick commented Jun 7, 2023

This failure is happening in #6787 as well but I unfortunately wasn't able to reproduce it locally. Do any of you know whether these tests are actually fetching from a real GCS bucket gs://build-crd-tests (which doesn't exist) or whether there's a local gcs (?) of some sort, similar to how we use in-cluster image registries? I looked at our prow config and the only changes since the last commits were merged on this release branch were bumping the test-runner image, which I'm not sure explains this failure.

The example has been removed, could it be the corresponding resource was removed as well? So we cannot pass it now

The example and functionality were removed in #6150, which made it into the v0.46 release. I'm not sure why this functionality would fail on the v0.44.x branch.
I'm fairly sure pipelineresources are operating since the pod that gets created has some containers named "step-create-dir-source-wgbt6" and "step-fetch-source-gnv5g" which come from pipelineresource functionality. I'm not sure why there aren't any pipelineresources dumped in the output though

@Yongxuanzhang
Copy link
Member

I think I reproduced this issue when I apply the v0.41 pipeline and then create the taskrun.
describe the deployment:

"step-fetch-source-6q6tz" exited with code 1 (image: "gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:27b2c22bf259d9bc1a291e99c63791ba0c27a04d2db0a43241ba0f1f20f4067f"); for logs run: kubectl -n default logs gcs-resource-l9mrx-pod -c step-fetch-source-6q6tz

log:

 File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/credentials_lib.py", line 226, in _GceMetadataRequest
    response = opener.open(request)
  File "/usr/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

@vdemeester
Copy link
Member

metallb-system 27m Warning Failed pod/speaker-v9hbl Failed to pull image "metallb/speaker:v0.9.3": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/metallb/speaker:v0.9.3": failed to resolve reference "docker.io/metallb/speaker:v0.9.3": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

There is also this 🤔

@vdemeester
Copy link
Member

/retest

@vdemeester
Copy link
Member

I think we can "just" move this example test into "no-ci" for now 🐼

Signed-off-by: Vincent Demeester <vdemeest@redhat.com>
@tekton-robot
Copy link
Collaborator Author

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 84.9% 85.2% 0.3
pkg/reconciler/pipelinerun/pipelinerun.go 90.7% 86.6% -4.1
pkg/reconciler/taskrun/taskrun.go 84.7% 84.5% -0.2

@tekton-robot
Copy link
Collaborator Author

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/events/cloudevent/cloud_event_controller.go 84.9% 85.2% 0.3
pkg/reconciler/pipelinerun/pipelinerun.go 90.7% 86.6% -4.1
pkg/reconciler/taskrun/taskrun.go 84.7% 84.5% -0.2

@vdemeester
Copy link
Member

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 9, 2023
@tekton-robot tekton-robot merged commit f278c34 into tektoncd:release-v0.44.x Jun 9, 2023
@lbernick
Copy link
Member

lbernick commented Jun 9, 2023

:o I did not know you could push commits to others' PRs

@Yongxuanzhang
Copy link
Member

+1, how did you do that :O

@vdemeester
Copy link
Member

@lbernick @Yongxuanzhang this is a github feature. As maintainers, if the author of the PR allows us to do it (which is by default), it's relatively easy to do.

$ gh checkout pr 6780
# […] do your thing
$ git commit -sS -m ""
$ git push

And it just works 😝

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/misc Categorizes issue or PR as a miscellaneuous one. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesnt merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants