Configuration parameter dag_bundle_config_list is generated without indentation

[zachliu]

Apache Airflow version

main (development)

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

I've been using airflow config list --include-descriptions --include-examples to generate new configurations. but i noticed the indentations were removed from the dag_bundle_config_list. I have to add them back manually to avoid a configparser.ParsingError error.

What you think should happen instead?

Either we include the indentation or make the configparser accept the parameter without indentation. My guess is that the former should be much easier.

How to reproduce

Use the cli airflow config list --include-descriptions --include-examples to generate a new set of configuration parameters.

Operating System

LM 21.3 with docker python:3.12-slim-bookworm

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[nailo2c]

Hi, I just opened PR #57388, which fixes this by adding INI-style indentation. The configparser.ParsingError is gone with this change, does this approach look OK to you?

[zachliu]

@nailo2c i noticed the self.get_default_value() was able to return dag_bundle_config_list with the correct indentations

airflow/airflow-core/src/airflow/configuration.py

Line 580 in b32603f

default_value = self.get_default_value(section_to_write, option, raw=True)

it is the self.get() removes them

airflow/airflow-core/src/airflow/configuration.py

Line 584 in b32603f

value = self.get(section_to_write, option, fallback=default_value, raw=True)

do you know why?

[nailo2c]

Hi @zachliu, I found out self.get() is inherited from Python's built-in configparser.ConfigParser(), so we can't control its indentation.

How about using airflow config list --include-descriptions --include-examples --defaults? It produces a commented-out template, so you'd just need to remove the comments yourself.

[zachliu]

@nailo2c well, i thought about it. but even the almighty AI can't remove all the comments accurately for a ~2000-row file 😂

i noticed the --defaults superseded many other options:

Show only defaults - do not include local configuration, sources, includes descriptions, examples, variables. Comment out everything.

how about this: let's make the Comment out everything optional while using --defaults?

[zachliu]

wait... aren't both get from Python's built-in configparser.ConfigParser()?

airflow/airflow-core/src/airflow/configuration.py

Line 2036 in b32603f

def create_default_config_parser(configuration_description: dict[str, dict[str, Any]]) -> ConfigParser:

[nailo2c]

I added a --defaults-no-comments flag to handle this issue. Let's wait for the reviewers' feedback.

#58115