Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli | airflow dags unpause - fail cause not founding dags #38834

Closed
1 of 2 tasks
raphaelauv opened this issue Apr 8, 2024 · 1 comment · Fixed by #40345
Closed
1 of 2 tasks

cli | airflow dags unpause - fail cause not founding dags #38834

raphaelauv opened this issue Apr 8, 2024 · 1 comment · Fixed by #40345

Comments

@raphaelauv
Copy link
Contributor

raphaelauv commented Apr 8, 2024

Apache Airflow version

2.9.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

in 2.8.4

running airflow dags unpause XXX in a container not having access to the airflow dags files was working

in 2.9.0 it fail with

Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/__main__.py", line 58, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/cli_config.py", line 49, in command
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/cli.py", line 114, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/commands/dag_command.py", line 219, in dag_unpause
    set_is_paused(False, args)
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/commands/dag_command.py", line 228, in set_is_paused
    for dag in get_dags(args.subdir, dag_id=args.dag_id, use_regex=args.treat_dag_id_as_regex)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/cli.py", line 256, in get_dags
    return [get_dag(subdir, dag_id)]
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/cli.py", line 245, in get_dag
    raise AirflowException(
airflow.exceptions.AirflowException: Dag 'docker_dag' could not be found; either it does not exist or it failed to parse.

so I need to run the cmd from a container having access to the dag files

is that normal ?

What you think should happen instead?

No response

How to reproduce

run airflow dags unpause from a container not having access to the airlfow dags files

Operating System

ubuntu 22.04

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@raphaelauv raphaelauv added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Apr 8, 2024
@potiuk potiuk added this to the Airflow 2.9.1 milestone Apr 8, 2024
@potiuk potiuk removed the needs-triage label for new issues that we didn't triage yet label Apr 8, 2024
@eladkal eladkal removed this from the Airflow 2.9.3 milestone Jun 9, 2024
@boushphong
Copy link
Contributor

I could work on this!

ephraimbuddy pushed a commit that referenced this issue Jul 1, 2024
…sing (#40345)

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
utkarsharma2 pushed a commit that referenced this issue Jul 2, 2024
…sing (#40345)

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
(cherry picked from commit e3d62c3)
ephraimbuddy pushed a commit that referenced this issue Jul 2, 2024
…sing (#40345)

closes: #38834

#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
(cherry picked from commit e3d62c3)
romsharon98 pushed a commit to romsharon98/airflow that referenced this issue Jul 26, 2024
…sing (apache#40345)

closes: apache#38834

apache#38265 added bulk pause and resume of DAGs. However, this PR seems to reuse the cli util method that collects DAGs from the default dag folder but not from the metadata DB. Hence, this would cause the unpause command to fail on environments where the dag folder is missing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants