-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inconsistent env var parsing from docker-compose.yml #2854
Comments
I think this is working as expected. In the docker example above, bash is interpreting the quotes for you, so the application never gets In the compose example using map, yaml is interpreting the quotes for you, so Compose never sees the quotes. In the list of strings example: I don't think compose should alter the expected behaviour of yaml, and remove quotes from the string. |
@dnephin I agree with your break down, the problem is that the resulting behaviour is as if compose is setting If this really is the intended behaviour, it should be documented. |
Also, should note that because the list syntax It seems like compose must be doing some internal parsing, splitting the var name and value, then recomposing them in a different format. |
True, There are plenty of other cases where this is also the case: Try sending it to the API directly:
Or from an env_file with docker-cli:
In both cases the double quotes are still there. Edit: My point is that the quotes are not actually part of the API definition. They're read and handled by bash, so I don't think you can use them outside of bash and expect them to be correct. It just so happens that yaml strings support the same character, so it will work if you define a string value, but you can't add them in other places and expect them to get stripped. |
hmmm, it would at least be good to document. This problem came to light because I was getting complaints about |
At first, my opinion was to allow To illustrate: # Resulting value is given in comments
# Using dicts
environment:
PLAIN_STRING: test # test
QUOTED_STRING: "test" # test
ESCAPED_STRING: "tes\u0074" # test
WITH_COLON: "test: abc" # test: abc
# Using lists
environment:
- PLAIN_STRING=test # test
- QUOTED_STRING="test" # "test"
- ESCAPED_STRING="tes\u0074" # "tes\u0074" # Note the uninterpreted escapes
- WITH_COLON="test: abc" # CRASHES (Yaml Spec says: Plain scalars must never contain the “: ” and “ #” character combinations) My suggestion would therefore be to output a warning, like it is done for |
This is also unintuitive behavior in env_file. Suppose you want to set
In shell, this would be taken as export |
This has caused me grief as well... |
so, what's the solution? |
My use case was to pass custom parameters to a command, e.g. A stopgap solution might be something like https://stackoverflow.com/questions/9733338/shell-script-remove-first-and-last-quote-from-a-variable where you have to explicitly do some processing when using some variables... |
See docker/compose#2854 for related issue.
So, whats the proper way if I wanna env file which I can just mount the file inside the container and run |
I'm probably beating a dead horse here, but this has really caused some major confusion in my scenario as well. I've created an example to demonstrate all the possible ways you can screw up environment variables in Here's my
and my
Let's start the betting to see which tests come out with no quotes at all in the values themselves 🎲 🎲 Results:
Test 1 and 8 are controls, so they don't count. That leaves only test 2 and 6 from the array syntax, and 9 and 11 from the map syntax:
Particularly troubling was test 5, which ended up with a quoted word inside a quoted sentence, effectively breaking out of the quotes, ala SQL injection. My biggest complaint is that |
If you would support changing the behavior to clean quotes from values in I'm not a native Python speaker, but this should do the trick:
right here: compose/compose/config/environment.py Line 21 in 7ae632a
|
This fixes docker-compose issues with inconsistent .env parsing. See: docker/compose#2854 However, this will likely break variables such as passwords with spaces in them.
* docker / docker-compose .env files do not play well with quoted values (unlike any other dotenv implementation it seems) * see docker/compose#2854 * added an option to output in a "docker-friendly" dotenv format
it appears i'm having problems due to docker-compose keeping quotes as part of the value when using the per the comments in the thread i can understand (even if i don't agree with) the rationale for not "altering the expected behavior of yaml" , however since we have both this issue has been hanging open for a few years, so if this isn't going to happen, maybe it should get an official response and be closed. |
Obviously, that's a biased view, but it feels cleaner then asking for |
dotenv is a specific format used by more than just docker-compose. The dotenv format allows quoted values. The expected behavior of dotenv files is bash-like. So in my opinion, docker-compose should make one of two possible changes: The current behavior seems totally incorrect and surprising, considering what people expect. I don't think the expectations are unreasonable either, since the |
Any updates on this? |
Too bad the fix didn't make it into the cli. Same issue using |
I am still experiencing this same issue in 1.27. Had to put back the hack of compiling separate docker env file. |
I came here because of this issue. Thought I was seeing a Docker Compose problem, but I guess Compose is actually ahead of the curve on this one. Now the problem is inconsistency. If I start my container using docker-compose |
Hello everyone. I'm really sorry to hear that you are having problems/doubts with Also it is very hard for us to know if all of you refer to the same issue. Given that, I invite you to update Thank you all for the feedback! |
I'm still seeing this issue in docker-compose 1.29.2 ENVs set like
produce an ENV in the container set as
instead of the expected
Like normal Unix behavior for decades. |
This is still an issue with Docker version 25.0.3, build 4debf41 |
Still have the problem Files:
TEST_VAR='{"key": "value"}'
FROM ubuntu
CMD echo "$TEST_VAR"
services:
service:
build:
dockerfile: test.Dockerfile
env_file: .env.test
image: test-image Execution:
We need unified results, otherwise, apps may not work correctly Docker version 27.1.1, build 6312585 |
If I declare my env vars in bash, I get the following:
However, if I create the following files:
Dockerfile:
test.sh:
docker-compose.yml
I get the following output:
Changing the environment variable to
TESTVAR=test
produces:For the record, docker itself appears to handle the quotes correctly (
test:latest
is the same Dockerfile above being run directly):Also, compose behaves correctly when using the map syntax, i.e.
TESTVAR: "test"
andTESTVAR: test
behave the same way, and consistently with bash (no quotes
is printed).The text was updated successfully, but these errors were encountered: