support regex to load config that falls under the same roof

[MarcBresson]

Hello ! It's me again :)

l'm using fsspec to implement a general interface, and I would like our end users to be able to use any storage they want and supply the required connection parameters through env variables.

Because I don't really want to specify all the different models for all the different storage solutions, I thought it would be quite cool to have one BaseSettings model that would capture any env variable that matches a regex. In my case, it would give something like:

from pydantic_settings import BaseSettings, SettingsConfigDict

class A(BaseSettings):
    model_config = SettingsConfigDict(match="STORAGE_*")

    name: str

of course it means that this model is also configured with extra="allow"

[MarcBresson]

maybe it's too similar to env_nested_delimiter option, i'll let you be the hudge of that :)

The workaround is to have one nested model like the following:

from pydantic import BaseModel, ConfigDict
from pydantic_settings import BaseSettings, SettingsConfigDict

class StorageParameters(BaseModel):
    model_config = ConfigDict(extra="allow")


class StorageConfig(BaseSettings):
    model_config = SettingsConfigDict(env_nested_delimiter="__")

    storage_name: str
    storage: StorageParameters

import os

os.environ["STORAGE_NAME"] = "s3"
os.environ["STORAGE__USERNAME"] = "michel"
os.environ["STORAGE__PASSWORD"] = "1234"

StorageConfig()
#> StorageConfig(storage_name='s3', storage=StorageParameters(username='michel', password='1234'))

this is not it, but it is close enough