Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions airflow-core/docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,37 @@ until ``min_file_process_interval`` is reached since DAG Parser will look for mo

return dag

What to do if you see disappearing DAGs on UI?
----------------------------------------------
There are several reasons why DAGs might disappear from the UI. Common causes include:

* **Total parsing of all DAGs is too long** - If parsing takes longer than :ref:`config:core__dagbag_import_timeout`,
files may not be processed completely. This often occurs when DAGs don't follow
:ref:`DAG writing best practices<best_practice:writing_a_dag>` like:

* Excessive top-level code execution
* External system calls during parsing
* Complex dynamic DAG generation

* **Inconsistent dynamic DAG generation** - DAGs created through
:doc:`dynamic generation </howto/dynamic-dag-generation>` must produce stable DAG IDs across parses.
Verify consistency by running ``python your_dag_file.py`` repeatedly.

* **File processing configuration issues** - A certain combination of parameters may lead to scenarios which certain DAGs are less likely to be processed at each loop. Check these parameters:

* :ref:`config:dag_processor__file_parsing_sort_mode` - Ensure sorting method matches your sync strategy
* :ref:`config:dag_processor__parsing_processes` - Number of parallel parsers
* :ref:`config:scheduler__parsing_cleanup_interval` - Controls stale DAG cleanup frequency
* :ref:`config:scheduler__dag_stale_not_seen_duration` - Time threshold for marking DAGs as stale

* **File synchronization problems** - Common with git-sync setups:

* Symbolic link swapping delays
* Permission changes during sync
* `mtime` preservation issues

* **Time synchronization issues** - Ensure all nodes (database, schedulers, workers) use NTP with <1s clock drift.


DAG construction
^^^^^^^^^^^^^^^^
Expand Down
Loading