Use env variable for docker image entry point config

[CareF]

With current setting, it would be hard to modify the dir and http_address with k8s deployment.

I was supposing

command: ["/app/bazel-remote-base-arm64.binary"]
args: ["--dir=/bazel-cache", "--http_address=:8080", "--profile_address=:6060", "--max_size=450"]

in k8s yaml should do the work but I'm getting

Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/app/bazel-remote-base-arm64.binary": stat /app/bazel-remote-base-arm64.binary: no such file or directory: unknown

And the way bazel is making the image, it is hard to debug what was happening :(

[mostynb]

Hi, I don't understand what this change is trying to do. Have you tried the example kubernetes config in examples/kubernetes.yml ?

[CareF]

Hi, I don't understand what this change is trying to do. Have you tried the example kubernetes config in examples/kubernetes.yml ?

@mostynb Yes I did. The error message comes from a modified version of examples/kubernetes.yml where I'm trying to make a change to BAZEL_REMOTE_DIR. I found that since --dir is hardcoded in the container image definition, the env BAZEL_REMOTE_DIR does not work. However, if I try to override the entry point behavior in the k8s yaml with what I mentioned in the PR description

command: ["/app/bazel-remote-base-arm64.binary"]
args: ["--dir=/bazel-cache", "--http_address=:8080", "--profile_address=:6060", "--max_size=450"]

I'm getting the error as described above.

[CareF]

So what I'm trying to do is to allow me and future users to customize the data directory (--dir) and the --http_address when using the image in a k8s cluster, without having to re-package the image ourselves.

[CareF]

@mostynb Did my explaination make sense to you, or do I need to explain anything in more detail?

[zaka634]

Just saw this and this is the same issue that I raised here : #736., there are a bunch of ways to do, we can also hard code the dir value and http_address value in golang code, but hard code the value in container runtime will ignore the value setting in k8s file, because for the code here : https://github.com/buchgr/bazel-remote/blob/master/config/config.go#L502-L505, it will pull the http_address value directly from the entry point instead of read the value set via BAZEL_REMOTE_HTTP_ADDRESS since the env variable values are just a default used for fall back.

[CareF]

Yes. In my use case it is exactly the same. I would like to be able to override the BAZEL_REMOTE_DIR config in my k8s yaml file, but the corrent image definition hard-coded --dir entrypoint option, will will disable the env setting.

[zaka634]

@mostynb , any update on this ?
echo back on my previous comment, here is the flag setting :
https://github.com/buchgr/bazel-remote/blob/master/utils/flags/flags.go#L56-L60

it's using urfave cli v2 library code, based on the precedence order : https://github.com/urfave/cli/blob/main/docs/v2/examples/flags.md#precedence

for the value set from entry point from here :
https://github.com/buchgr/bazel-remote/blob/master/BUILD.bazel#L144-L146

it will be considered as the command line flag value from user, hence, it won't take the ENV var value set from here :
https://github.com/buchgr/bazel-remote/blob/master/examples/kubernetes.yml#L46-L51

because they are considered as the env values, which has a lower precedence compared to entry_point value. Hence, the kubernetes.yml example for setting bazel_remote_max_size and dir config never works.