Elasticsearch 7.6.0 cannot be used as service in GitLab pipeline anymore

[dawi]

Starting with version 7.6.0 Elasticsearch cannot be started as service in a Gitlab pipeline anymore.

Gitlab sets a CI_SERVER_TLS_CA_FILE variable which contains a path to a file.
It seems that this variable is now expanded to contain the content of the file.

Service container logs:
2020-02-19T06:52:56.510497268Z ERROR: File -----BEGIN CERTIFICATE-----
2020-02-19T06:52:56.510521348Z ....
2020-02-19T06:52:56.510608065Z -----END CERTIFICATE-----
2020-02-19T06:52:56.510610718Z  from CI_SERVER_TLS_CA_FILE does not exist

I am not completely sure if this is an Elasticsearch issue or not, but I figured out that Elasticsearch does some environment variable processing and it could very well be the cause of this issue:

Also extract the support for _FILE env vars into a separate script, so
that it can be called from more than once place (the behaviour is
idempotent).

https://github.com/elastic/elasticsearch/blob/v7.6.0/distribution/src/bin/elasticsearch-env-from-file

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

[jasontedor]

@pugnascotia Can you take a look?

[pugnascotia]

This is caused by #49182. We implemented a generic mechanism for sourcing secrets from files instead of environment variables, by settings SECRET_VAR_FILE (holding a file name) to populate SECRET_VAR.

However, as far as I can tell there are only really two variables where we might want this behaviour. We could apply this mechanism only for recognised variables. @jasontedor what do you think? I think that would be better than skipping processing of _FILE variables that don't point to files.