Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Kill hanged docker build process to avoid build timeout issue. #13597

Merged
merged 1 commit into from
Feb 22, 2023

Conversation

liushilongbuaa
Copy link
Contributor

@liushilongbuaa liushilongbuaa commented Feb 2, 2023

Why I did it

Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.

How I did it

Start a daemon process to scan running time more than 1 hours, and kill the process.

How to verify it

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

@liushilongbuaa liushilongbuaa marked this pull request as ready for review February 3, 2023 03:28
@liushilongbuaa
Copy link
Contributor Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liushilongbuaa
Copy link
Contributor Author

draft

@liushilongbuaa liushilongbuaa changed the title kill [ci] Kill hanged docker build process to avoid build timeout issue. Feb 3, 2023
…onic-net#13726)

Why I did it
Docker build has a low rate of hanging up.
It hangs on different steps. So, it looks like a bug in docker daemon.

How I did it
Start a daemon process to scan running time more than 1 hours, and kill the process.

How to verify it
@@ -1,5 +1,11 @@
steps:
- script: |
set -x
# kill daemon process
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it cherry pick from master? Looks like missing the logic to check if the file /tmp/azp_daemon_kill_docker_pid exists and the pid exists.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is cherry pick from master.
I didn't 'set -e'. If there is not this file. Just go ahead.

@xumia xumia merged commit 9d20dda into sonic-net:202111 Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants