diff --git a/airflow-core/docs/faq.rst b/airflow-core/docs/faq.rst index 8727deac08e71..9eb5fa79ad007 100644 --- a/airflow-core/docs/faq.rst +++ b/airflow-core/docs/faq.rst @@ -206,6 +206,37 @@ until ``min_file_process_interval`` is reached since DAG Parser will look for mo return dag +What to do if you see disappearing DAGs on UI? +---------------------------------------------- +There are several reasons why DAGs might disappear from the UI. Common causes include: + +* **Total parsing of all DAGs is too long** - If parsing takes longer than :ref:`config:core__dagbag_import_timeout`, + files may not be processed completely. This often occurs when DAGs don't follow + :ref:`DAG writing best practices` like: + + * Excessive top-level code execution + * External system calls during parsing + * Complex dynamic DAG generation + +* **Inconsistent dynamic DAG generation** - DAGs created through + :doc:`dynamic generation ` must produce stable DAG IDs across parses. + Verify consistency by running ``python your_dag_file.py`` repeatedly. + +* **File processing configuration issues** - A certain combination of parameters may lead to scenarios which certain DAGs are less likely to be processed at each loop. Check these parameters: + + * :ref:`config:dag_processor__file_parsing_sort_mode` - Ensure sorting method matches your sync strategy + * :ref:`config:dag_processor__parsing_processes` - Number of parallel parsers + * :ref:`config:scheduler__parsing_cleanup_interval` - Controls stale DAG cleanup frequency + * :ref:`config:scheduler__dag_stale_not_seen_duration` - Time threshold for marking DAGs as stale + +* **File synchronization problems** - Common with git-sync setups: + + * Symbolic link swapping delays + * Permission changes during sync + * `mtime` preservation issues + +* **Time synchronization issues** - Ensure all nodes (database, schedulers, workers) use NTP with <1s clock drift. + DAG construction ^^^^^^^^^^^^^^^^