Raise an exception if ProfileConfig is instantiated with both profile_mapping and profiles_yml_filepath

[tatiana]

As of Cosmos 1.2.2, it does not complain if users set both profile_mapping and profiles_yml_filepath in a ProfileConfig instance. Cosmos should raise an error in this scenario.

This issue was noticed in the Slack thread:
https://apache-airflow.slack.com/archives/C059CC42E9W/p1699365889141509

[dosubot]

🤖

Hello @tatiana,

Nice to meet you! I'm Dosu, a bot here to assist you with the Cosmos repository. I'm here to help troubleshoot bugs, answer questions, and guide you on your journey as a contributor while we wait for a human maintainer. Let's work together on this!

Thank you for bringing this to our attention. You're correct that the current behavior of the ProfileConfig class in Cosmos v1.2.2 does not raise an exception if both profile_mapping and profiles_yml_filepath are set. Instead, it prioritizes the user-supplied profiles.yml file specified by profiles_yml_filepath and only uses profile_mapping if profiles_yml_filepath is not set or does not exist.

To change this behavior and raise an exception if both profile_mapping and profiles_yml_filepath are set, you can add a validation in the __post_init__ method of the ProfileConfig class. Here's an example of how you can do this:

def __post_init__(self) -> None:
    "Validates that we have enough information to render a profile."
    # if using a user-supplied profiles.yml, validate that it exists
    if self.profiles_yml_filepath and not Path(self.profiles_yml_filepath).exists():
        raise CosmosValueError(f"The file {self.profiles_yml_filepath} does not exist.")
    
    # raise an exception if both profiles_yml_filepath and profile_mapping are set
    if self.profiles_yml_filepath and self.profile_mapping:
        raise CosmosValueError("Both profiles_yml_filepath and profile_mapping cannot be set at the same time.")

This code will raise a CosmosValueError if both profiles_yml_filepath and profile_mapping are set when a ProfileConfig instance is created. The CosmosValueError is a custom exception class in the Cosmos codebase that is used to handle invalid configuration values.

Please note that this change will need to be tested thoroughly to ensure it doesn't introduce any unexpected behavior or break any existing functionality. If you need further assistance, feel free to ask.

Sources

• cosmos/config.py
• cosmos/exceptions.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.