Parallel building of sonic dockers using native dockerd(dood). #10352
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Kalimuthu-Velappan
requested review from
qiluo-msft,
xumia and
lguohan
as code owners
xumia
reviewed
Mar 25, 2022
|
/azpw run Azure.sonic-buildimage |
|
/AzurePipelines run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Kalimuthu-Velappan
force-pushed
the
DOOD
branch
from
9e70050 to
e7b696a
Compare
Currently, the build dockers are created as a user dockers(docker-base-stretch-<user>, etc) that are
specific to each user. But the sonic dockers (docker-database, docker-swss, etc) are
created with a fixed docker name and common to all the users.
docker-database:latest
docker-swss:latest
When multiple builds are triggered on the same build server that creates parallel building issue because
all the build jobs are trying to create the same docker with latest tag.
This happens only when sonic dockers are built using native host dockerd for sonic docker image creation.
This patch creates all sonic dockers as user sonic dockers and then, while
saving and loading the user sonic dockers, it rename the user sonic
dockers into correct sonic dockers with tag as latest.
docker-database:latest <== SAVE/LOAD ==> docker-database-<user>:tag
The user sonic docker names are derived from 'DOCKER_USERNAME and DOCKER_USERTAG' make env
variable and using Jinja template, it replaces the FROM docker name with correct user sonic docker name for
loading and saving the docker image.
Kalimuthu-Velappan
force-pushed
the
DOOD
branch
from
e7b696a to
23d9627
Compare
xumia
approved these changes
Apr 21, 2022
|
@liushilongbuaa , could you please help review it? Is it any impact on the reproducible build of docker image? |
It looks good for me |
liushilongbuaa
approved these changes
Apr 22, 2022
liushilongbuaa
pushed a commit
to liushilongbuaa/sonic-buildimage
that referenced
this pull request
Jun 20, 2022
…-net#10352) Currently, the build dockers are created as a user dockers(docker-base-stretch-<user>, etc) that are specific to each user. But the sonic dockers (docker-database, docker-swss, etc) are created with a fixed docker name and common to all the users. docker-database:latest docker-swss:latest When multiple builds are triggered on the same build server that creates parallel building issue because all the build jobs are trying to create the same docker with latest tag. This happens only when sonic dockers are built using native host dockerd for sonic docker image creation. This patch creates all sonic dockers as user sonic dockers and then, while saving and loading the user sonic dockers, it rename the user sonic dockers into correct sonic dockers with tag as latest. docker-database:latest <== SAVE/LOAD ==> docker-database-<user>:tag The user sonic docker names are derived from 'DOCKER_USERNAME and DOCKER_USERTAG' make env variable and using Jinja template, it replaces the FROM docker name with correct user sonic docker name for loading and saving the docker image.
zhangyanzhao
added
Request for 202211 Branch
and removed
Request for 202211 Branch
labels
|
@Kalimuthu-Velappan PR conflicts with 202211 branch |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the build dockers are created as user dockers(docker-base-stretch-, etc) that are
specific to each user. But the sonic dockers (docker-database, docker-swss, etc) are
created with a fixed docker name and are common to all the users.
When multiple builds are triggered on the same build server that creates parallel building issues because
all the build jobs are trying to create the same docker with the latest tag.
This happens only when sonic dockers are built using native host dockerd for sonic docker image creation.
This patch creates all sonic dockers as user sonic dockers and then, while
saving and loading the user sonic dockers, it renames the user sonic
dockers into correct sonic dockers with the tag as latest.
The user sonic docker names are derived from 'DOCKER_USERNAME and DOCKER_USERTAG' make env variable and using Jinja template, it replaces the FROM docker name with correct user sonic docker name for loading and saving the docker image.
Why I did it
How I did it
How to verify it
Which release branch to backport (provide reason below if selected)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)