Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Docker BuildX config #266

Conversation

amarburg
Copy link
Collaborator

@amarburg amarburg commented Aug 20, 2024

Changes Made

Adds a .docker/docker-bake.hcl buildx bake configuration file. The default behavior is to pulls images from ghcr.io/robotic-decision-making-lab/blue to prime the cache, but only push to local cache.

Also updates the Github action to use buildx-bake. In CI it overrides the default config to cache to/from images in the Github package registry.

All docker images ('ci', 'robot', 'desktop' and 'desktop-nvidia') can be built with:

cd .docker && docker buildx bake

This is a standalone PR with the buildx config; very similar files are commited in the more complex #241. If merged, this will lead to a conflict in that PR which will need to be managed by hand.

Associated Issues

Related to #241

Testing

Above docker buildx bake call completes successfully. Resulting images for rolling are functional to complete joystick teleop demo.

Copy link
Collaborator

@evan-palmer evan-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm excited to see this finished. This is a big improvement over the current pipeline configuration.

If merged, this will lead to a conflict in that PR which will need to be managed by hand.

Are you hoping to have #241 merged before this?

.github/workflows/docker.yaml Outdated Show resolved Hide resolved
.github/workflows/docker.yaml Show resolved Hide resolved
@amarburg
Copy link
Collaborator Author

Are you hoping to have #241 merged before this?

No, I think this PR is more atomic. More a note not to be surprised when merging this breaks #241

@evan-palmer
Copy link
Collaborator

Let me know when this is ready for review and I'll do a review/run some tests

@amarburg
Copy link
Collaborator Author

I believe this is stable now. As pull requests don't use the GHA caching, or do arm64 compilation, that's harder to test.

Copy link
Collaborator

@evan-palmer evan-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of comments about the workflow. Feel free to trigger the build from your branch so that you can do some testing with the cache too. The previous images are stored for a while and can be retrieved if need-be.

.github/workflows/docker.yaml Outdated Show resolved Hide resolved
.github/workflows/docker.yaml Show resolved Hide resolved
.github/workflows/docker.yaml Outdated Show resolved Hide resolved
.github/workflows/docker.yaml Outdated Show resolved Hide resolved
Copy link
Collaborator

@evan-palmer evan-palmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last stylistic comment then we can finish up

.github/workflows/docker.yaml Outdated Show resolved Hide resolved
@evan-palmer evan-palmer merged commit a8681d0 into Robotic-Decision-Making-Lab:main Aug 22, 2024
4 checks passed
@evan-palmer
Copy link
Collaborator

Thanks for your work on this @amarburg!

@evan-palmer evan-palmer added backport-iron Backport this PR to iron backport-humble Backport this PR to humble backport-jazzy Backport this PR to jazzy labels Aug 22, 2024
mergify bot pushed a commit that referenced this pull request Aug 22, 2024
* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml
mergify bot pushed a commit that referenced this pull request Aug 22, 2024
* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml
mergify bot pushed a commit that referenced this pull request Aug 22, 2024
* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml
amarburg added a commit that referenced this pull request Aug 22, 2024
amarburg added a commit that referenced this pull request Aug 22, 2024
amarburg added a commit that referenced this pull request Aug 22, 2024
evan-palmer pushed a commit that referenced this pull request Aug 22, 2024
* Add Docker BuildX config (#266)

* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml

* Resolve conflict while merging #266

---------

Co-authored-by: Aaron Marburg <amarburg@uw.edu>
evan-palmer pushed a commit that referenced this pull request Aug 22, 2024
* Add Docker BuildX config (#266)

* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml

* Resolve conflicts while merging #266

---------

Co-authored-by: Aaron Marburg <amarburg@uw.edu>
evan-palmer pushed a commit that referenced this pull request Aug 22, 2024
* Add Docker BuildX config (#266)

* Allow pushing to apl-ocean-engineering

* Install cppzmq-dev for Gazebo

* Allow pushing to apl-ocean-engineering

* Reverse change in docker.yaml action.

* Preliminary version of docker-bake.hcl and updated Docker workflow.

* Updated comments in Dockerfile

* Attempt to allow manual triggering of builds

* Nope, that's not it.

* Add preliminary label to all Docker images.

* Removed redundant libcppzmq-dev

* Hack to lowercase repo name

* Re-activate the metadata-action, go back to stages in matrix.

* Add comment on lowercasing.

* Place files config at correct level.

* Re-activate the metadata-action, go back to stages in matrix.

* Place files config at correct level.

* Switch to registry cache for CI

* Removed stage from matrix

* Update repo URL, fix list formatting.

(cherry picked from commit a8681d0)

# Conflicts:
#	.github/workflows/docker.yaml

* Resolve conflicts while merging #266

---------

Co-authored-by: Aaron Marburg <amarburg@uw.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-humble Backport this PR to humble backport-iron Backport this PR to iron backport-jazzy Backport this PR to jazzy
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants