[Bug]: Configuration behavior is unpredictable

[khushvind]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

When I change the model, and api_key in config.toml, OpenDevin still runs with the previous model, and api_key specified in the last session. To use the new model and key, I also need to specify them again in the UI.

Specifying it once should be enough.

Current OpenDevin version

0.8.2

Installation and Configuration

make build; make run

Model and Agent

Model: claude-3-5-sonnet
Agent: CodeActAgent

Operating System

No response

Reproduction Steps

No response

Logs, Errors, Screenshots, and Additional Context

No response

[neubig]

We had a discussion on slack: https://opendevin.slack.com/archives/C06U8UTKSAD/p1722599155702189

Based on the discussion some of my personal suggestions for how we could improve the behavior now:

First, make consistent behavior of configuration:

1. Read from the defaults specified in config.py in the OpenDevin repo
2. Read from config.toml and override settings
3. (optional) Read from environment variables and/or command line and override settings

Here, 3. increases complexity, but it also could be handy because it allows overriding settings without writing to a file (e.g. for sensitive API keys).

Second, make UI-changed settings overwrite config.toml. This would allow the settings to be saved to and read from a consistent place.

[HenrikBach1]

Plz consider that your environment settings when using them in Dockerfile configuration use case, will be invalid as they are as of today in #3196.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.