PyprojectTomlConfigSettingsSource overwrite toml_file Path from model_config

[sbrugman]

To use pydantic for handling pyproject.toml configuration, I would like to be able to pass a custom path to the PyprojectTomlConfigSettingsSource. This is possible for the TomlConfigSettingsSource source, but that lacks the pyproject.toml support (very minor layer to get the tool.my-tool section of the Toml file. For PyprojectTomlConfigSettingsSource, the settings_cls.model_config.get('toml_file') lookup is missing.

toml_file is ignored:

class Settings(BaseSettings):
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (PyprojectTomlConfigSettingsSource(settings_cls),)

class MySettings(BaseSettings):
    model_config = SettingsConfigDict(toml_file=Path("config.toml").resolve(), pyproject_toml_table_header=("tool", "my-tool"))

toml_file is respected:

class Settings(BaseSettings):
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (TomlConfigSettingsSource(settings_cls),)

class MySettings(BaseSettings):
    model_config = SettingsConfigDict(toml_file=Path("config.toml").resolve())

It looks like a fairly simple and straight-forward enhancement in sources.py.

My current workaround is to create a custom class with elements of both classes:

class CustomPyprojectTomlConfigSettingsSource(TomlConfigSettingsSource):
    def __init__(
        self,
        settings_cls: type[BaseSettings],
        toml_file: PathType | None = DEFAULT_PATH,
    ):
        self.toml_file_path = (
            toml_file
            if toml_file != DEFAULT_PATH
            else settings_cls.model_config.get(
                "toml_file",
                PyprojectTomlConfigSettingsSource._pick_pyproject_toml_file(
                    toml_file, settings_cls.model_config.get("pyproject_toml_depth", 0)
                ),
            )
        )
        self.toml_data = self._read_files(self.toml_file_path)
        self.toml_table_header: tuple[str, ...] = settings_cls.model_config.get(
            "pyproject_toml_table_header", ("tool", "pydantic-settings")
        )
        for key in self.toml_table_header:
            self.toml_data = self.toml_data.get(key, {})
        super(TomlConfigSettingsSource, self).__init__(settings_cls, self.toml_data)

Would be great if this functionality can come out-of-the-box.

[hramezani]

Thanks @sbrugman for reporting this issue. Yeah, the PyprojectTomlConfigSettingsSource doesn't consider the toml_file in the config. actually, it doesn't have any config.

BTW, I would suggest defining a new config for PyprojectTomlConfigSettingsSource because toml_file accepts list of files but PyprojectTomlConfigSettingsSource accepts one file. also, if we enable both PyprojectTomlConfigSettingsSource and TomlConfigSettingsSource sources together, it will be confusing.

I would suggest defining pyproject_toml_file for this source.

Would you like to work on it?

[sbrugman]

Thanks for the swift reply. Indeed, pyproject_toml_file is a better naming choice for the configuration.

I'm not able to contribute the feature soon, so if anyone is interested in working on it that would be much appreciated :)

[braindevices]

I did not get the idea of configurable pyproject_toml_file.

• pyproject.toml has fixed name, you should not change the name, if one want to use different toml why not just use TomlConfigSettingsSource? After all, the PyprojectTomlConfigSettingsSource simply provides a easy way to scan for the pyproject.toml
• if you want to search the pyproject.toml in different place rather than current dir, then one should use pyproject_toml_depth=<int>
• use An explicit file path according to the doc. This way one can also change the file name (for me this does not really make sense except debuging)

  An explicit file path can be provided to the source when it is instantiated (e.g. PyprojectTomlConfigSettingsSource(settings_cls, Path('~/.config').resolve() / 'pyproject.toml')). If a file path is provided this way, it will be treated as absolute (no other locations are checked).

@hramezani I think this is existing feature and also is elaborated carefully in the doc already.

[hramezani]

Thanks @braindevices

I agree this exists already.