Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Next Generation - Per language container strategy #7474

Merged
merged 6 commits into from
Sep 26, 2023

Conversation

heliocastro
Copy link
Contributor

@heliocastro heliocastro commented Sep 7, 2023

Docker now builds independent language images, and the final image can be assembled with ORT main binary + languages.
All images are deployed under ghcr.io registry by default.

Default runtime image contains Java, Python, Rust, Golang, NodeJS and Ruby components, and other components can be be added using the new custom-docker.sh script.

Extra components available are:
Android, Swift, Dotnet, PHP, Haskell

This closes original external PR:
Closes #7447

@heliocastro heliocastro self-assigned this Sep 7, 2023
@heliocastro heliocastro requested a review from a team as a code owner September 7, 2023 10:32
@heliocastro heliocastro marked this pull request as draft September 7, 2023 10:32
@heliocastro heliocastro added enhancement Issues that are considered to be enhancements new feature Issues that are considered to be new features docker About Docker topics build Issues with building ORT labels Sep 7, 2023
@codecov
Copy link

codecov bot commented Sep 7, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (4df3094) 68.03% compared to head (0e19bbc) 68.03%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #7474   +/-   ##
=========================================
  Coverage     68.03%   68.03%           
  Complexity     2023     2023           
=========================================
  Files           344      344           
  Lines         16727    16727           
  Branches       2371     2371           
=========================================
  Hits          11381    11381           
  Misses         4363     4363           
  Partials        983      983           
Flag Coverage Δ
funTest-docker 69.40% <ø> (ø)
funTest-non-docker 36.51% <ø> (ø)
test 35.52% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 3 times, most recently from 912e03d to 7b4bdfa Compare September 7, 2023 14:33
@heliocastro heliocastro marked this pull request as ready for review September 7, 2023 16:15
@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 13 times, most recently from e5ad366 to df598bc Compare September 12, 2023 07:43
@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 5 times, most recently from 5773f42 to 38ed35d Compare September 16, 2023 15:50
@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 2 times, most recently from 9a74410 to 9991b54 Compare September 17, 2023 08:36
@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 7 times, most recently from 0aa4b51 to 1931b80 Compare September 25, 2023 14:16
@tsteenbe tsteenbe self-requested a review September 26, 2023 14:10
@heliocastro heliocastro force-pushed the heliocastro/docker_nextgen branch 4 times, most recently from 97c2df0 to 7091d83 Compare September 26, 2023 15:56
ORT language components are built now in independent images.
This enables the possibility to create custom tailored containers
with only the required language components, reducing final size
and improving build process.

Main runtime contains Python, NodeJS, Rust, Ruby and Golang components.
Other language components have their own images built, and can be
included on custom containers.

Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
Workflow dependencies relies on a single workflow,
which make build pipelines for multiple docker containers
complex and most of the time erratic.

Move to single jobs dependency based with a internal action
to check and build images if needed improve the logic of the
process and allow to visually see the running pipeline.

The pipeline will run on following cases:
- On pull_request if .versions file is changed, but not
  publishing images on the registry.
- On push or workflow_dispatch, publishing to defined registry.

Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
Language images lives under the ort namespace to avoid
clash on main organization image naming,
e.g. ghcr.io/oss-review-toolkit/python versus
ghcr.io/oss-review-toolkit/ort/python.

Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
Signed-off-by: Helio Chissini de Castro <heliocastro@gmail.com>
@tsteenbe tsteenbe merged commit 029f1ca into main Sep 26, 2023
36 checks passed
@tsteenbe tsteenbe deleted the heliocastro/docker_nextgen branch September 26, 2023 18:41
@mnonnenmacher
Copy link
Member

Too late for a review, so just some general remarks from me:

  • Indentation in workflow files is wrong, we always indent with two spaces.
  • Some commits probably should have been squashed, why introduce so many new workflow files only to remove them again within the same PR?
  • Last commit should (at least mostly) have been integrated into the previous commits.
  • Some commit messages are fine, but for example the 2nd and 3rd commit would have needed an explanation to understand the motivation.
  • I don't like the name "Mega image", it's not very telling.

@sschuberth
Copy link
Member

Too late for a review, so just some general remarks from me:

Thanks for these remarks. In sum, very surprising that this was approved as-is by two (!) reviewers then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues with building ORT docker About Docker topics enhancement Issues that are considered to be enhancements new feature Issues that are considered to be new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants