Quarkus 3.18.1 does not allow overwriting lists from application.yaml with lists supplied in environment variables

[snophey]

Describe the bug

We use the quarkus-config-yaml extension to provide a default baseline configuration for our application in YAML format:

my:
  list:
  - hello
  - world

We inject this into the application as follows:

@ConfigProperty(name = "my.list")
List<String> myList;

When running in production, the list from application.yaml may need to be overwritten with other values. We do this by providing a comma separated list as the value of the MY_LIST environment variable (e.g. export MY_LIST=hello,bob).

After switching from the current LTS version (3.15.3) to the most recent Quarkus version (3.18.1). This behavior no longer works. The value of MY_LIST is ignored. It seems that only YAML lists are affected by this, as overwriting String-typed config values works fine.

Expected behavior

If MY_LIST=hello,bob then we expect myList to contain exactly the values "hello" and "bob".

Actual behavior

The value injected into myList contains the values "hello' and "world" although MY_LIST=hello,bob.

How to Reproduce?

https://github.com/snophey/quarkus-reproduce-list-bug (follow the steps in the README)

Output of uname -a or ver

Linux codespaces-5565d0 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

OpenJDK 64-Bit Server VM Corretto-21.0.5.11.1 (build 21.0.5+11-LTS, mixed mode, sharing)

Quarkus version or git rev

3.18.1

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)

Additional information

No response

[quarkus-bot]

/cc @radcortez (config)

[radcortez]

I'll have a look.

[radcortez]

A collection configuration value can be represented as list=foo,bar or list[0]=foo,list[1]=bar. While the first version is convenient, it only really works for simple type elements and not for composed types. The API became a bit inconsistent, so we tried to make things a bit more consistent by prioritizing indexed names when populating collections. (BTW, the indexed and the APIs for collections are a SmallRye Config exclusive feature; MicroProfile Config only deals with simple collection elements and the comma-separated values configuration).

This usually is not a problem because users use only one of the styles, but the YAML source is tricky. The ConfigSource API forces us to translate a YAML file content to property-based content. In reality, the correct representation of a collection in YAML is the indexed form name[i]. The YAMLConfigSource predates the support of indexed properties, so we indeed flattened and transformed such cases to the comma-separated values (which I wanted to remove in smallrye/smallrye-config#1203).

All the SmallRyeConfig APIs can properly use both styles and override as expected, but the code that handles injection by @ConfigProperty is a bit custom, so it will always give priority to the indexed format, which, of course, is a bug.

While we fix this, my recommendation is to use a @ConfigMapping instead, which delegates all the config lookups to the SmallRyeConfig APIs, and handles this case as expected: https://smallrye.io/smallrye-config/3.11.1/config/mappings/

[radcortez]

Alternatively, you can also use MY_LIST_0_, etc.

[jasmdk]

I think the same issue is present in application.properties too

[radcortez]

Did you try it with Quarkus 3.18.2?