Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build multi-architecture docker images using docker buildx #482

Closed
jay-m-dev opened this issue Dec 15, 2022 · 3 comments
Closed

Build multi-architecture docker images using docker buildx #482

jay-m-dev opened this issue Dec 15, 2022 · 3 comments
Assignees

Comments

@jay-m-dev
Copy link
Contributor

jay-m-dev commented Dec 15, 2022

We need to build all docker images: dbmongo, machine, lab, unit_tester, int_tester using buildx, which allows targeting multiple architectures from a single machine (instead of having to build on the target system)

So far, all python packages, mongo db and nodejs are being updated to latest versions (see issue #481). scikit-learn, pandas, numpy and surprise v1.1.1.1 will still need to be updated to versions that have arm64 available from PyPi or piwheels.

A workaround for the missing wheels is to have the compiled wheels in the repo until we can break from completely from this dependency.

@jay-m-dev jay-m-dev self-assigned this Dec 15, 2022
@jay-m-dev jay-m-dev linked a pull request Dec 22, 2022 that will close this issue
@jay-m-dev
Copy link
Contributor Author

All packages have been updated to compatible versions for both x64 and arm64. Building docker images on arm64 takes a long time, since all requirements' wheels are built during this process. To speed up the docker build process we will need to prebuild the updated versions of wheels for all requirements.
As of right now, x64 and arm64 docker images need to be built on systems of the corresponding architecture. But we are very close to making this work with docker buildx.

@jay-m-dev jay-m-dev removed a link to a pull request Dec 22, 2022
@jay-m-dev
Copy link
Contributor Author

Reopening issue, it was closed by branch merge. Removed branch link. We still need to setup and make docker buildx work.

@jay-m-dev
Copy link
Contributor Author

Docker buildx is now being used to build multi-architecture machines. This is part of the release process, images are pushed directly to docker hub for release. Docker-compose can still be used as usual for local development.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant