Dockerfile optimization proposal

[benbrummer]

docker image seems to be quite big, so I try to identify ways to shrink it

Ideas

• apt-get --no-install-recommends
• Review apt packages:
  • snappdf requirements: https://github.com/beganovich/snappdf?tab=readme-ov-file#headless-chrome-doesnt-launch-on-unix
• Review php modules
  • https://invoiceninja.github.io/en/self-host-installation/
• https://docs.docker.com/build/building/multi-stage/
• php 8.3

apt packages

required:

• curl => google-chrome-stable
• gnupg2 => google-chrome-stable
• gosu => init.sh
• default-mysql-client => initial user creation

obsolete (?):

• git
• unzip
• wget
• zip
• ...

php modules

[PHP Modules]
bcmath
Core
ctype
curl
date
dom
exif
fileinfo
filter
gd
gmp
hash
iconv
imagick
json
libxml
mbstring
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
random
readline
redis
Reflection
saxonc
session
SimpleXML
soap
sodium
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
Zend OPcache
zip
zlib
[Zend Modules]
Zend OPcache

Misc

• dockerfiles/debian/Dockerfile

  Line 149 in a6d8f66

  RUN sed -i "s/user = www-data/user = www-data/g" /usr/local/etc/php-fpm.d/www.conf \

  seems to do nothing
• Exact versions for php modlues e.g saxon vs no version for other modules?

[turbo124]

@benbrummer we can certainly start with the obsolete ones like zip/unzip

[turbo124]

@benbrummer

related #663 I had to roll back the architecture if/fi blocks as they were preventing required dependencies from installing.

[benbrummer]

Apart of the default-mysql-client, did you face any other issues?

Most of them will be installed as dependencies of google-chrome-stable, so we could move them to arm64 section. The other ones are not clear to me.

    fonts-liberation \ google-chrome-stable
    fonts-noto-cjk \ ?
    fonts-noto-cjk-extra \ ?
    fonts-wqy-microhei \ ?
    fonts-wqy-zenhei \ ?
    libasound2 \ google-chrome-stable
    libatk-bridge2.0-0 \ google-chrome-stable
    libatk1.0-0 \ google-chrome-stable
    libatspi2.0-0 \ google-chrome-stable
    libcups2 \ google-chrome-stable
    libdbus-1-3 \ google-chrome-stable
    libdrm2 \ google-chrome-stable
    libgbm1 \ google-chrome-stable
    libgtk-3-0 \ google-chrome-stable
    libnspr4 \ google-chrome-stable
    libnss3 \ google-chrome-stable
    libonig-dev \ ?
    libpng-dev \ ? => google-chrome-stable (libpng16-16)
    libwayland-client0 \ google-chrome-stable
    libxcomposite1 \ google-chrome-stable
    libxdamage1 \ google-chrome-stable
    libxfixes3 \ google-chrome-stable
    libxkbcommon0 \ google-chrome-stable
    libxml2-dev \ ?
    libxrandr2 \ google-chrome-stable
    xdg-utils \ google-chrome-stable
    xfonts-wqy \ ?

[turbo124]

The fonts are definitely needed to provide unicode support for PDF generation is regions like China.

The others, if they are dependencies of chrome won't hurt to have in place I would have thought.

[benbrummer]

@turbo124

Are these apt packages needed?
I assume the *-dev can be ommited and xfonts-wgy is propably required for pdf?

    libonig-dev \
    libpng-dev \
    libxml2-dev \
    xfonts-wqy \

[turbo124]

I'm not sure if any of these are subdependencies of anything

[benbrummer]

If I compare the php list https://invoiceninja.github.io/en/self-host-installation/#server-requirements with the modules, which are installed in the image, the intl module is missing.

Additional modules in the image installed with install-php-extensions

• exif 
  

• opcache => Needed for caching/jit
  

• pcntl
  

• pdo_mysql
  

• redis => Needed for redis
  

• saxon-12.5.0
  

@turbo124 can you tell me what are the required ones. E.g pdo_mysql vs mysqli?

Update:
After disabling some of the modules i get

[turbo124]

mysqli is fine for use,.
exif would be to extract metadata from certain filetypes,
gd is also for image processing
pcntl I think is required for Laravel
saxon definitely needed for xslt transformations
redis - for cache access

[benbrummer]

mysqli is fine for use,. exif would be to extract metadata from certain filetypes, gd is also for image processing pcntl I think is required for Laravel saxon definitely needed for xslt transformations redis - for cache access

I reduced it for a test to just

RUN ( curl -sSLf https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - || echo 'return 1' ) | sh -s \
    bcmath \
    # exif \
    gd \
    # gmp \
    # imagick \
    # intl \
    # mysqli \
    # opcache \
    # pcntl \
    pdo_mysql \
    # redis \
    # saxon-${saxon} \
    # soap \
    zip \
    @composer

Everything seems to work. Even without opcache and redis it is quick and responsive. pdo_mysql is required, otherwise the database connection fails, mysqli seems to be unnecessary. All Laravel requirements are installed https://laravel.com/docs/11.x/deployment#server-requirements.

Predis => https://laracasts.com/discuss/channels/laravel/what-is-difference-between-phpredis-and-predis-in-laravel => phpredis is not needed

[turbo124]

predis is more than sufficient here, i agree.

[benbrummer]

@turbo124 I'm not able to find a usage for these modules. Do you have an example, where these modules are used in invoiceninja. Can I somehow produce an error somewhere?

Modules, which seem to be unused

exif
gmp
imagick
intl
mysqli
pcntl
saxon-12.5.0
soap

Required/used modules

RUN ( curl -sSLf https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions -o - || echo 'return 1' ) | sh -s \
    bcmath \
    gd \
    opcache \
    pdo_mysql \
    zip \
    @composer

[turbo124]

Soap needed for vat number verification
Saxon for xslt
Imagick for image compression
Intl also required the exact reason currently escapes me, but I believe around some mb4 support

Pcntl, I believe is needed by Laravel
GMP or bcmath needed
Some form of php MySQL extension required.

[benbrummer]

I'm not able to find any evidenz for the packages.

[benbrummer]

composer suggest --all

awobaz/compoships suggests:
 - awobaz/blade-active: Blade directives for the Laravel 'Active' package
 - awobaz/eloquent-auto-append: Automatically append accessors to model serialization
 - awobaz/eloquent-mutators: Reusable mutators (getters/setters) for Laravel 5's Eloquent
 - awobaz/syntactic: Syntactic sugar for named and indexed parameters call.

aws/aws-crt-php suggests:
 - ext-awscrt: Make sure you install awscrt native extension to use any of the functionality.

aws/aws-sdk-php suggests:
 - aws/aws-php-sns-message-validator: To validate incoming SNS notifications
 - doctrine/cache: To use the DoctrineCacheAdapter
 - ext-sockets: To use client-side monitoring

barryvdh/reflection-docblock suggests:
 - dflydev/markdown: ~1.0
 - erusev/parsedown: ~1.0

dompdf/dompdf suggests:
 - ext-gmagick: Improves image processing performance

endroid/qr-code suggests:
 - khanamiryan/qrcode-detector-decoder: Enables you to use the image validator
 - roave/security-advisories: Makes sure package versions with known security issues are not installed

fakerphp/faker suggests:
 - doctrine/orm: Required to use Faker\ORM\Doctrine

filp/whoops suggests:
 - whoops/soap: Formats errors as SOAP responses

google/apiclient suggests:
 - cache/filesystem-adapter: For caching certs and tokens (using Google\Client::setCache)

guzzlehttp/psr7 suggests:
 - laminas/laminas-httphandlerrunner: Emit PSR-7 responses

intervention/image suggests:
 - intervention/imagecache: Caching extension for the Intervention Image library

jms/serializer suggests:
 - doctrine/collections: Required if you like to use doctrine collection types as ArrayCollection.
 - symfony/cache: Required if you like to use cache functionality.

larastan/larastan suggests:
 - orchestra/testbench: Using Larastan for analysing a package needs Testbench

laravel/framework suggests:
 - ably/ably-php: Required to use the Ably broadcast driver (^1.0).
 - ext-apcu: Required to use the APC cache driver.
 - ext-ftp: Required to use the Flysystem FTP driver.
 - ext-memcached: Required to use the memcache cache driver.
 - ext-redis: Required to use the Redis cache and queue drivers (^4.0|^5.0|^6.0).
 - league/flysystem-ftp: Required to use the Flysystem FTP driver (^3.25.1).
 - league/flysystem-path-prefixing: Required to use the scoped driver (^3.25.1).
 - league/flysystem-read-only: Required to use read-only disks (^3.25.1)
 - league/flysystem-sftp-v3: Required to use the Flysystem SFTP driver (^3.25.1).
 - pda/pheanstalk: Required to use the beanstalk queue driver (^5.0).
 - resend/resend-php: Required to enable support for the Resend mail transport (^0.10.0).
 - symfony/cache: Required to PSR-6 cache bridge (^7.0).

laravel/scout suggests:
 - algolia/algoliasearch-client-php: Required to use the Algolia engine (^3.2).
 - meilisearch/meilisearch-php: Required to use the Meilisearch engine (^1.0).
 - typesense/typesense-php: Required to use the Typesense engine (^4.9).

league/fractal suggests:
 - pagerfanta/pagerfanta: Pagerfanta Paginator
 - zendframework/zend-paginator: Zend Framework Paginator

maximebf/debugbar suggests:
 - kriswallsmith/assetic: The best way to manage assets

mindee/mindee suggests:
 - ghostscript/ghostscript: Required for PDF rasterization features

mollie/mollie-api-php suggests:
 - mollie/oauth2-mollie-php: Use OAuth to authenticate with the Mollie API. This is needed for some endpoints. Visit https://docs.mollie.com/ for more information.

monolog/monolog suggests:
 - doctrine/couchdb: Allow sending log messages to a CouchDB server
 - ext-amqp: Allow sending log messages to an AMQP server (1.0+ required)
 - ext-mongodb: Allow sending log messages to a MongoDB server (via driver)
 - ext-sockets: Allow sending log messages to a Syslog server (via UDP driver)
 - mongodb/mongodb: Allow sending log messages to a MongoDB server (via library)
 - php-amqplib/php-amqplib: Allow sending log messages to an AMQP server using php-amqplib
 - rollbar/rollbar: Allow sending log messages to Rollbar
 - ruflin/elastica: Allow sending log messages to an Elastic Search server

nelexa/zip suggests:
 - ext-bz2: Needed to support BZIP2 compression

open-telemetry/context suggests:
 - ext-ffi: To allow context switching in Fibers

paragonie/random_compat suggests:
 - ext-libsodium: Provides a modern crypto API that can be used to generate random bytes.

php-http/client-common suggests:
 - php-http/cache-plugin: PSR-6 Cache plugin
 - php-http/logger-plugin: PSR-3 Logger plugin
 - php-http/stopwatch-plugin: Symfony Stopwatch plugin

php-http/message suggests:
 - laminas/laminas-diactoros: Used with Diactoros Factories
 - slim/slim: Used with Slim Framework PSR-7 implementation

phpmyadmin/sql-parser suggests:
 - phpmyadmin/motranslator: Translate messages to your favorite locale

phpoffice/phpspreadsheet suggests:
 - mitoteam/jpgraph: Option for rendering charts, or including charts with PDF or HTML Writers
 - tecnickcom/tcpdf: Option for rendering PDF with PDF Writer

phpseclib/phpseclib suggests:
 - ext-libsodium: SSH2/SFTP can make use of some algorithms provided by the libsodium-php extension.
 - ext-mcrypt: Install the Mcrypt extension in order to speed up a few other cryptographic operations.

phpunit/php-code-coverage suggests:
 - ext-pcov: PHP extension that provides line coverage
 - ext-xdebug: PHP extension that provides line coverage as well as branch and path coverage

predis/predis suggests:
 - ext-relay: Faster connection with in-memory caching (>=0.6.2)

psy/psysh suggests:
 - ext-pdo-sqlite: The doc command requires SQLite to work.

ramsey/uuid suggests:
 - paragonie/random-lib: Provides RandomLib for use with the RandomLibAdapter
 - ramsey/uuid-doctrine: Allows the use of Ramsey\Uuid\Uuid as Doctrine field type.

rmccue/requests suggests:
 - art4/requests-psr18-adapter: For using Requests as a PSR-18 HTTP Client

socialiteproviders/apple suggests:
 - ahilmurugesan/socialite-apple-helper: Automatic Apple client key generation and management.

spatie/error-solutions suggests:
 - openai-php/client: Require get solutions from OpenAI
 - simple-cache-implementation: To cache solutions from OpenAI

spatie/ignition suggests:
 - openai-php/client: Require get solutions from OpenAI
 - simple-cache-implementation: To cache solutions from OpenAI

spatie/laravel-ignition suggests:
 - openai-php/client: Require get solutions from OpenAI

spaze/phpstan-stripe suggests:
 - phpstan/extension-installer: Allows automatic requirement of extension.neon

sprain/swiss-qr-bill suggests:
 - fpdf/fpdf: Needed to create pdfs with FpdfOutput
 - tecnickcom/tcpdf: Needed to create pdfs with TcPdfOutput

Findings:

imagick:
There is one interesting suggestion for dompdf => gmagick, but from the docs, imagick is supported, too

 dompdf/dompdf suggests:
 - ext-gmagick: Improves image processing performance

soup:
only mentioned here, but not the ext-soap

filp/whoops suggests:
 - whoops/soap: Formats errors as SOAP responses

no match for exif, gmp, imagick, intl, mysqli, pcntl, saxon

I think the dependency confusion, discrepancy of docs and composer feels like an issue, which should be handled in https://github.com/invoiceninja/invoiceninja.

[turbo124]

whilst there may be no composer related dependencies, the app directly engages the ones I have mentioned above.

[benbrummer]

whilst there may be no composer related dependencies, the app directly engages the ones I have mentioned above.

So than they should be listed here? https://github.com/invoiceninja/invoiceninja/blob/7b02668d4bd1de6462717a98a88e6ebea706eaa5/composer.json#L33

[turbo124]

Not necessarily. The app can certainly work without some of these. We do not enforce all possible extensions.

The app is installed on a wide range of services including shared hosting which can present challenges for meeting some dependencies.

In environments where we can control the underlying extensions, then it makes sense to use them.

[benbrummer]

Understand :-)

Is there a documentation for all the possible extensions and their purpuose?

https://invoiceninja.github.io/en/self-host-installation/#server-requirements should list pdo_mysql, database connection does not work (But it works without mysqli). Maybe a list of mandatory modules (or a link to the require section in composer.json) can be added, and a list of optional modules.

pcntl:
seems to be not required by laravel: https://laravel.com/docs/11.x/deployment#server-requirements

[turbo124]

"suggest": {
        "ext-pcntl": "Required for process control (CLI only)",
        "ext-gd": "Required for image processing",
        "ext-mbstring": "Required for multibyte string handling"
    }

We can certainly add the list of non-required extensions into a suggest block.

In regards to pdo_mysql, I believe this is automatically included (at least when using apt) when running apt-get install php-mysql

pnctl is part of PHP core, not really an extension, my guess here is upstream they check for this flag and replace with a binary that has pnctl enabled or disabled.

given pnctl's role in process control, and laravel does use SIG* commands for process controls, I would assume this is very much needed.

[benbrummer]

@turbo124 I tried to to my best and combined all Information (composer.json, laravel docs, invoiceninja docs, the base dockerimages 8.2-8.4-fpm)

"require": {
    // php 8.4 works with some notices like PHP Deprecated:  App\Models\CompanyGateway::driver(): Implicitly marking parameter $client as nullable is deprecated, the explicit nullable type must be used instead in /var/www/html/app/Models/CompanyGateway.php on line 206
    "php": ">=8.2",
    // https://github.com/invoiceninja/dockerfiles/issues/660#issuecomment-2511321698
    "ext-gd": "*, insall needed",
    "ext-curl": "*",    
    "ext-zip": "*, install needed", 
    "ext-openssl": "*",
    "ext-mbstring": "*",
    "ext-dom": "*",
    "ext-xml": "*",
    "ext-bcmath": "*, install needed",
    "ext-iconv": "*",
    // Additional mandatory module
    "pdo_mysql": "*, Connection.php... could not find driver..., install needed"
    }

"suggest": {
    "ext-simplexml": "*, ?, was listed in 'require'",
    // All of these are not preinstalled in php:8.2/8.3-fpm
    "ext-exif": "*, ",
    "ext-gmp": "*, ",
    "ext-imagick": "*, suggest for DomPDF suggested",
    "ext-intl": "*, Was missing in previous invoiceninja-debian images, twig/intl requires symfony/intl, but none of them require/suggest ext_intl",
    "ext-mysqli": "*, ?",
    "ext-opcache": "*, caching",
    "ext-pcntl": "*, Required for process control (CLI only), php needs to be compiled for it. Not listed in requirements for Laravel https://laravel.com/docs/11.x/deployment#server-requirements ",
    "ext-saxonc": ">=12.5.0, xslt processing during",
    "ext-soap": "*, VAT validation"        
    }

Additional modules listed in documentation https://invoiceninja.github.io/en/self-host-installation/#server-requirements, which are not already listed above

"suggest?": {
    "ctype": "*, ?",
    "tokenizer": "*, ?",
    "gmp": "*, ?, install needed",
}

php-common will be installed anyway, when any php-* package is installed, and it is no php modul itself, so I think it is not needed to list in the docs.