Broken Pipe error

[gopivemula1]

jenkins_build_error.log
Environment: Cent Os

• Jib version: 3.1.1
• Build tool: maven
• OS: CentOS

Description of the issue:
Execute a build with Jenking using command "clean package com.google.cloud.tools:jib-maven-plugin:3.1.1:build -Dimage=gcr.io/careful-record-316411/config-ui"
It sends error while pushing image to GCP Container registry
[ERROR] I/O error for image [gcr.io/careful-record-316411/config-ui]:
[ERROR] java.net.SocketException
[ERROR] Broken pipe (Write failed)
[ERROR] broken pipe: the server shut down the connection. Check the server log if possible. This could also be a proxy issue. For example, a proxy may prevent sending packets that are too large.

Expected behavior:
Image should be created in GCP Container Registry.

Steps to reproduce:

1. Take any Spring Boot Project
2. Execute maven package with JIB to push image to GCP COntainer Registry

Log output: Attached

Additional Information:

jenkins_build_error.log

[dzou]

Thanks for the report.

I wasn't able to reproduce this problem following your instructions. I took the Jib Sample Spring Boot Project and ran the command:

./mvnw compile jib:build -Dimage=gcr.io/my-project/jib-sample

I was able to successfully upload the image to GCR.

Note that broken pipe means an error on server-side and may be a transient error. Could you try again and see if it works? If not, please provide a reproducer that we can try.

[gopivemula1]

Hi @dzou

Thanks for the reply. Issue is when using same command from Jenkins.
I did set roles/cloudbuild.builds.builder to service account in GCP, i think JIB does not use "Google Service Account from private key" configuration of Jenkins.

Error log:

[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.1.1:build (default-cli) on project liverpool-configuration: Build image failed, perhaps you should make sure you have permissions for gcr.io/careful-record-316411/config-ui and set correct credentials. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-forbidden-or-denied for help: Unauthorized for gcr.io/careful-record-316411/config-ui: 403 Forbidden
[ERROR] PUT https://gcr.io/v2/careful-record-316411/config-ui/manifests/latest
[ERROR] {"errors":[{"code":"DENIED","message":"Operation on project 'careful-record-316411' failed. Caller does not have permission 'storage.objects.create'. To configure permissions, follow instructions at: https://cloud.google.com/container-registry/docs/access-control"}]}
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[dzou]

Interesting. So I think the error message means that your service account is getting picked up, however you have not configured the correct permissions for it to access Google Cloud Storage.

Can you confirm that your service account does indeed have storage permissions? Maybe try storage.admin first, then reducing to the minimal working scope. Follow instructions here:
https://cloud.google.com/container-registry/docs/access-control

[gopivemula1]

Hi @dzou

I set following permissions.

Cloud Build Service Account
Compute Admin
Compute Network Admin
Compute Security Admin
Security Admin
Service Account Key Admin
Project IAM Admin
Storage Admin
Storage Object Viewer

Issue is jib in Jenkins does not take credentials that are configured as "Google Service Account from private key"

[dzou]

Issue is jib in Jenkins does not take credentials that are configured as "Google Service Account from private key"

Jib supported service account key authentication since version 1.5.0, so the issue is that something in your configuration is not quite correct.

Could you share how you are setting the service account in Jenkins?

According to your Jenkins logs, it looks like you already have some default credentials that is different from your service account key on Jenkins:

[DEBUG] Google ADC found
[INFO] Using Google Application Default Credentials for gcr.io/careful-record-316411/config-ui
[DEBUG] attempting bearer auth for gcr.io/careful-record-316411/config-ui...

You seem to be able to log in, but you do not have storage permissions on this GCP account. So you either have to look in your Jenkins script, figure out what account is being used, and then add the storage permissions on that

Or, in the Jenkins script, make sure you are actually setting the service account:

$ GOOGLE_APPLICATION_CREDENTIALS=my-service-account-key.json
$ mvn -X jib:build

[chanseokoh]

I'll assume the issue is resolved. What's certain is that Jib picked up some Application Default Credentials and used them to successfully authenticate with gcr.io. The authentication was successful, and it's not that there's anything Jib is intervening or has to intervene in this auth flow. What really happened is that the ADC isn't authorized (i.e., doesn't have the permissions) to store image bits on GCS. Either the ADC wasn't the one you intended to use or the ADC doesn't actually have the permissions. It could also be that you have to re-generate a key?

For completeness sake, I'll document here that setting GOOGLE_APPLICATION_CREDENTIALS=my-service-account-key.json is not the only way to use a service account key (although this might be the simplest way in your case).