Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(docker): reduce size of image by about half #381

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

SuperVK
Copy link
Member

@SuperVK SuperVK commented Oct 28, 2024

During the Ubuntu Summit I was lead down a rabbit hole of inspecting docker images. I noticed that the sudosos-backend could use some improvement.

Description

I noticed that the npm ci still installed dev dependencies in the runtime container, so that was fixed. Furthermore when installing npm packages, it also adds to the cache. Meaning it used to have 2x the node_modules installed in the image. By installing in the build container and copying to the runtime container, we circumvent this issue.

image
The build from 6 days ago is the current develop docker image, the docker image from 9 minutes ago is this the one that has the improvement.

Here you can see the cache being added which is about the same size as the node_modules themself:
image

My improved image is currently running on develop (28-10-2024 10:53), it seems to work, only the pdf is broken, but pretty sure that has always been broken on develop?

Related issues/external references

Types of changes

  • CI/CD (Changes to the CI/CD configuration)

@SuperVK SuperVK requested a review from Yoronex October 28, 2024 09:57
@SuperVK
Copy link
Member Author

SuperVK commented Oct 28, 2024

It would be ideal if some build tool could minify the dependencies and only include what is needed, like vite does for the frontend. But that's not super needed, and I think requires quite a lot more changes.

Copy link

github-actions bot commented Oct 28, 2024

SudoSOS Coverage Report

Commit: 73d4385
Base: develop@dd9ecf5

Type Base This PR
Total Statements Coverage  85.21%  85.21% (+0%)
Total Branches Coverage  81.42%  81.42% (+0%)
Total Functions Coverage  87.44%  87.44% (+0%)
Total Lines Coverage  85.55%  85.55% (+0%)
Details (changed files)
FileStatementsBranchesFunctionsLines
Details (all files)
FileStatementsBranchesFunctionsLines
/src/index.ts 88.57% 20% 50% 90.44%
/src/authentication/token-handler.ts 100% 100% 100% 100%
/src/controller/authentication-controller.ts 86.62% 100% 96.15% 86.47%
/src/controller/authentication-secure-controller.ts 88.23% 100% 100% 88.23%
/src/controller/balance-controller.ts 78.04% 50% 100% 78.04%
/src/controller/banner-controller.ts 82.52% 94.11% 100% 82.52%
/src/controller/base-controller.ts 100% 100% 100% 100%
/src/controller/container-controller.ts 85.96% 78.57% 100% 85.45%
/src/controller/debtor-controller.ts 82.96% 53.33% 100% 86.71%
/src/controller/event-controller.ts 82.66% 90% 100% 82.66%
/src/controller/event-shift-controller.ts 75.82% 90% 100% 75.82%
/src/controller/invoice-controller.ts 78.33% 75.86% 92.3% 78.08%
/src/controller/payout-request-controller.ts 69.36% 73.68% 84.61% 69.36%
/src/controller/point-of-sale-controller.ts 83.01% 78.94% 100% 83%
/src/controller/product-category-controller.ts 82.45% 100% 100% 82.45%
/src/controller/product-controller.ts 85.6% 85.71% 100% 86.77%
/src/controller/rbac-controller.ts 87.58% 87.5% 100% 87.32%
/src/controller/root-controller.ts 86.2% 100% 100% 86.2%
/src/controller/seller-payout-controller.ts 84.55% 84.61% 100% 84.44%
/src/controller/server-settings-controller.ts 100% 100% 100% 100%
/src/controller/simple-file-controller.ts 11.62% 0% 0% 11.62%
/src/controller/stripe-controller.ts 92.59% 100% 100% 92.59%
/src/controller/stripe-webhook-controller.ts 97.56% 100% 87.5% 97.56%
/src/controller/test-controller.ts 33.33% 100% 0% 33.33%
/src/controller/transaction-controller.ts 85.14% 96% 100% 84.69%
/src/controller/transfer-controller.ts 84.21% 93.33% 100% 83.63%
/src/controller/user-controller.ts 84.74% 92.85% 100% 84.61%
/src/controller/vat-group-controller.ts 87.8% 100% 100% 87.8%
/src/controller/voucher-group-controller.ts 79.41% 71.42% 100% 79.41%
/src/controller/write-off-controller.ts 85.45% 100% 100% 85.45%
/src/controller/request/file-request.ts 0% 0% 0% 0%
/src/controller/request/validators/container-request-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/general-validators.ts 86.53% 66.66% 77.77% 84.61%
/src/controller/request/validators/invoice-request-spec.ts 96.15% 80% 100% 100%
/src/controller/request/validators/point-of-sale-request-spec.ts 95% 100% 100% 94.11%
/src/controller/request/validators/product-request-spec.ts 90.32% 57.14% 100% 90%
/src/controller/request/validators/rbac-request-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/string-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/update-local-request-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/update-nfc-request-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/update-pin-request-spec.ts 100% 100% 100% 100%
/src/controller/request/validators/user-request-spec.ts 82.6% 25% 85.71% 83.33%
/src/controller/request/validators/validation-errors.ts 87.8% 100% 62.96% 100%
/src/controller/response/dinero.ts 0% 0% 0% 0%
/src/database/database.ts 95.5% 50% 33.33% 97.7%
/src/database/schema.ts 92% 100% 50% 95.83%
/src/database/with-manager.ts 100% 100% 100% 100%
/src/entity/banner.ts 100% 100% 100% 100%
/src/entity/base-entity-without-id.ts 100% 100% 100% 100%
/src/entity/base-entity.ts 100% 100% 100% 100%
/src/entity/server-setting.ts 84.61% 0% 100% 100%
/src/entity/vat-group.ts 100% 100% 100% 100%
/src/entity/authenticator/authentication-method.ts 100% 100% 100% 100%
/src/entity/authenticator/ean-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/hash-based-authentication-method.ts 100% 100% 100% 100%
/src/entity/authenticator/key-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/ldap-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/local-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/member-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/nfc-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/pin-authenticator.ts 100% 100% 100% 100%
/src/entity/authenticator/reset-token.ts 100% 100% 100% 100%
/src/entity/container/container-revision.ts 95.23% 100% 83.33% 94.11%
/src/entity/container/container.ts 100% 100% 100% 100%
/src/entity/event/event-shift-answer.ts 100% 100% 100% 100%
/src/entity/event/event-shift.ts 100% 100% 100% 100%
/src/entity/event/event.ts 100% 100% 100% 100%
/src/entity/file/banner-image.ts 100% 100% 100% 100%
/src/entity/file/base-file.ts 100% 100% 100% 100%
/src/entity/file/invoice-pdf.ts 100% 100% 100% 100%
/src/entity/file/payout-request-pdf.ts 100% 100% 100% 100%
/src/entity/file/pdf-able.ts 88.88% 83.33% 88.88% 93.75%
/src/entity/file/pdf-file.ts 100% 100% 100% 100%
/src/entity/file/product-image.ts 100% 100% 100% 100%
/src/entity/file/seller-payout-pdf.ts 100% 100% 100% 100%
/src/entity/fine/fine.ts 100% 100% 100% 100%
/src/entity/fine/fineHandoutEvent.ts 100% 100% 100% 100%
/src/entity/fine/userFineGroup.ts 100% 100% 100% 100%
/src/entity/invoices/invoice-status.ts 100% 100% 100% 100%
/src/entity/invoices/invoice.ts 100% 100% 100% 100%
/src/entity/point-of-sale/point-of-sale-revision.ts 94.44% 100% 75% 93.33%
/src/entity/point-of-sale/point-of-sale.ts 100% 100% 100% 100%
/src/entity/point-of-sale/product-ordering.ts 100% 100% 100% 100%
/src/entity/product/product-category.ts 100% 100% 100% 100%
/src/entity/product/product-revision.ts 96.55% 100% 83.33% 96.15%
/src/entity/product/product.ts 100% 100% 100% 100%
/src/entity/rbac/assigned-role.ts 100% 100% 100% 100%
/src/entity/rbac/permission.ts 100% 100% 100% 100%
/src/entity/rbac/role-user-type.ts 100% 100% 100% 100%
/src/entity/rbac/role.ts 100% 100% 100% 100%
/src/entity/report/fine-report.ts 100% 100% 100% 100%
/src/entity/report/report.ts 100% 100% 100% 100%
/src/entity/stripe/stripe-deposit.ts 100% 100% 100% 100%
/src/entity/stripe/stripe-payment-intent-status.ts 100% 100% 100% 100%
/src/entity/stripe/stripe-payment-intent.ts 100% 100% 100% 100%
/src/entity/transactions/balance.ts 100% 100% 100% 100%
/src/entity/transactions/sub-transaction-row.ts 100% 100% 100% 100%
/src/entity/transactions/sub-transaction.ts 100% 100% 100% 100%
/src/entity/transactions/transaction.ts 100% 100% 100% 100%
/src/entity/transactions/transfer.ts 100% 100% 100% 100%
/src/entity/transactions/write-off.ts 100% 100% 100% 100%
/src/entity/transactions/payout/base-payout.ts 100% 100% 100% 100%
/src/entity/transactions/payout/payout-request-status.ts 100% 100% 100% 100%
/src/entity/transactions/payout/payout-request.ts 100% 100% 100% 100%
/src/entity/transactions/payout/seller-payout.ts 100% 100% 100% 100%
/src/entity/transformer/dinero-transformer.ts 100% 100% 100% 100%
/src/entity/user/invoice-user.ts 100% 100% 100% 100%
/src/entity/user/local-user.ts 100% 100% 100% 100%
/src/entity/user/user-voucher-group.ts 100% 100% 100% 100%
/src/entity/user/user.ts 97.82% 83.33% 100% 100%
/src/entity/user/voucher-group.ts 100% 100% 100% 100%
/src/errors/index.ts 100% 100% 50% 100%
/src/errors/not-implemented-error.ts 20% 0% 0% 20%
/src/errors/pdf-error.ts 100% 100% 100% 100%
/src/files/initialize.ts 100% 100% 100% 100%
/src/files/response.ts 28.57% 100% 0% 28.57%
/src/files/storage/disk-storage.ts 82.14% 66.66% 60% 82.14%
/src/files/storage/file-storage.ts 100% 100% 100% 100%
/src/files/storage/index.ts 100% 100% 50% 100%
/src/files/storage/locations.ts 100% 100% 100% 100%
/src/gewis/gewis.ts 77.14% 28.57% 71.42% 81.81%
/src/gewis/controller/gewis-authentication-controller.ts 87.5% 66.66% 100% 87.5%
/src/gewis/database/seed.ts 100% 100% 100% 100%
/src/gewis/entity/gewis-user.ts 100% 100% 100% 100%
/src/gewis/helpers/gewis-helper.ts 100% 83.33% 100% 100%
/src/gewis/service/gewisdb-service.ts 87.67% 86.66% 80% 89.39%
/src/helpers/ad.ts 90% 100% 100% 90%
/src/helpers/bindings.ts 100% 100% 100% 100%
/src/helpers/database.ts 100% 100% 100% 100%
/src/helpers/express-pdf.ts 100% 50% 100% 100%
/src/helpers/hash.ts 100% 100% 100% 100%
/src/helpers/ordering.ts 90.9% 75% 100% 90%
/src/helpers/pagination.ts 100% 87.87% 100% 100%
/src/helpers/pdf.ts 93.93% 63.63% 100% 96.66%
/src/helpers/query-filter.ts 95.08% 93.75% 94.44% 96.07%
/src/helpers/raw-body.ts 100% 100% 100% 100%
/src/helpers/revision-to-response.ts 91.3% 64.28% 88.88% 95.23%
/src/helpers/specification-validation.ts 94.59% 87.5% 88.88% 96.87%
/src/helpers/timestamps.ts 58.82% 60% 66.66% 56.25%
/src/helpers/token-helper.ts 100% 100% 100% 100%
/src/helpers/transaction-mapper.ts 90% 100% 76.19% 91.07%
/src/helpers/validators.ts 94.54% 88.88% 100% 97.67%
/src/mailer/index.ts 100% 100% 100% 100%
/src/mailer/mail-body-generator.ts 91.66% 83.33% 100% 91.66%
/src/mailer/mail-message.ts 100% 100% 100% 100%
/src/mailer/mailer.ts 100% 100% 100% 100%
/src/mailer/transporter.ts 100% 100% 100% 100%
/src/mailer/messages/changed-pin.ts 85.71% 100% 0% 85.71%
/src/mailer/messages/forgot-event-planning.ts 76.92% 100% 57.14% 76.92%
/src/mailer/messages/hello-world.ts 100% 100% 100% 100%
/src/mailer/messages/index.ts 100% 100% 100% 100%
/src/mailer/messages/mail-content-builder.ts 81.81% 53.33% 100% 81.81%
/src/mailer/messages/membership-expiry-notification.ts 86.66% 50% 66.66% 86.66%
/src/mailer/messages/password-reset.ts 33.33% 0% 0% 33.33%
/src/mailer/messages/user-debt-notification.ts 85.71% 100% 60% 85.71%
/src/mailer/messages/user-got-fined.ts 76.92% 100% 57.14% 76.92%
/src/mailer/messages/user-will-get-fined.ts 70.37% 50% 60% 70.37%
/src/mailer/messages/welcome-to-sudosos.ts 85.71% 100% 60% 85.71%
/src/mailer/messages/welcome-with-reset.ts 85.71% 100% 60% 85.71%
/src/middleware/policy-middleware.ts 100% 100% 100% 100%
/src/middleware/request-validator-middleware.ts 92.3% 66.66% 100% 100%
/src/middleware/restriction-middleware.ts 96.15% 93.33% 75% 100%
/src/middleware/token-middleware.ts 100% 100% 100% 100%
/src/migrations/1707251162194-invoice-refactor.ts 5.71% 0% 0% 6.06%
/src/migrations/1720608140757-soft-deletes.ts 25% 100% 0% 25%
/src/migrations/1720610649657-payout-request-pdf.ts 12.5% 100% 0% 14.28%
/src/migrations/1720624912260-database-rbac.ts 13.72% 0% 4.76% 14.43%
/src/migrations/1721916495084-transfers-vat.ts 16.66% 0% 0% 18.18%
/src/migrations/1722004753128-write-offs.ts 23.8% 0% 0% 26.31%
/src/migrations/1722022351000-pos-cashiers.ts 33.33% 100% 0% 33.33%
/src/migrations/1722083254200-server-settings.ts 50% 100% 0% 50%
/src/migrations/1722084520361-pos-users.ts 21.05% 100% 0% 22.22%
/src/migrations/1722118077157-invoice-rework.ts 13.33% 0% 0% 14.28%
/src/migrations/1722517212441-nested-product-categories.ts 15.38% 100% 0% 16.66%
/src/migrations/1722869409448-stripe-payment-intents.ts 11.11% 100% 16.66% 12.12%
/src/migrations/1724506999318-invoice-as-topups.ts 11.39% 0% 0% 12.5%
/src/migrations/1724855153990-seller-payouts.ts 13.55% 0% 5.55% 15.23%
/src/migrations/1725196803203-user-type-enums.ts 15% 0% 0% 15%
/src/migrations/1725388477226-custom-invoice-entries.ts 15.38% 0% 0% 16.66%
/src/migrations/1726066600389-seller-payout-pdf.ts 23.07% 100% 25% 25%
/src/migrations/1726689003147-ldap-objectguid.ts 12% 0% 0% 13.63%
/src/rbac/default-roles.ts 100% 100% 100% 100%
/src/rbac/role-manager.ts 94% 72.72% 100% 97.61%
/src/server-settings/server-settings-store.ts 98.24% 88.88% 100% 100%
/src/server-settings/setting-defaults.ts 100% 100% 100% 100%
/src/service/ad-service.ts 94.38% 50% 100% 98.55%
/src/service/authentication-service.ts 93.33% 76.92% 100% 92.92%
/src/service/balance-service.ts 98.27% 87.27% 100% 98.05%
/src/service/banner-service.ts 95.23% 88.88% 100% 95.12%
/src/service/container-service.ts 97.02% 85.18% 100% 100%
/src/service/debtor-service.ts 98.41% 85% 100% 100%
/src/service/event-service.ts 100% 100% 100% 100%
/src/service/file-service.ts 97.22% 72.22% 100% 97.18%
/src/service/invoice-service.ts 93.75% 57.14% 95.55% 97.9%
/src/service/payout-request-service.ts 100% 97.36% 100% 100%
/src/service/point-of-sale-service.ts 97.26% 88.88% 100% 100%
/src/service/product-category-service.ts 100% 100% 100% 100%
/src/service/product-service.ts 94.44% 78.26% 95.45% 94.87%
/src/service/rbac-service.ts 96.34% 86.95% 100% 100%
/src/service/report-service.ts 99.01% 90% 100% 100%
/src/service/seller-payout-service.ts 100% 100% 100% 100%
/src/service/stripe-service.ts 98.5% 92.85% 100% 98.41%
/src/service/transaction-service.ts 96.96% 83.75% 100% 96.68%
/src/service/transfer-service.ts 100% 97.43% 100% 100%
/src/service/user-service.ts 90.16% 54.71% 84.61% 93.57%
/src/service/vat-group-service.ts 97.36% 76.47% 93.75% 96.96%
/src/service/voucher-group-service.ts 100% 100% 100% 100%
/src/service/write-off-service.ts 100% 85.71% 100% 100%
/src/service/pdf/invoice-pdf-service.ts 97.05% 87.5% 100% 96.66%
/src/service/pdf/payout-request-pdf-service.ts 100% 100% 100% 100%
/src/service/pdf/pdf-service.ts 100% 75% 100% 100%
/src/service/pdf/report-pdf-service.ts 100% 100% 100% 100%
/src/service/pdf/seller-payout-pdf-service.ts 50% 100% 20% 57.14%
/src/service/pdf/user-report-pdf-service.ts 95% 50% 100% 100%
/src/start/swagger.ts 82.5% 53.33% 66.66% 82.05%
/src/subscriber/index.ts 100% 100% 100% 100%
/src/subscriber/transaction-subscriber.ts 91.17% 88.88% 66.66% 92.85%
/src/subscriber/transfer-subscriber.ts 100% 100% 100% 100%

@JustSamuel
Copy link
Contributor

@SuperVK we could switch to vite?

@JustSamuel JustSamuel self-requested a review October 28, 2024 13:00
@Yoronex
Copy link
Member

Yoronex commented Oct 31, 2024

I don't really follow why this reduces the size of the image, probably because you do npm ci --production. Why can't we just do run that command in the target image? Why do we have to run this command in the build image and then copy it to the target? This does not make sense to me, and for performance reasons, I would argue you would do the former, as that allows both npm ci commands to be run in parallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants