Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-3865] Added api to get python code of a given Dag id #4687

Merged
merged 1 commit into from
Feb 21, 2019

Conversation

ramandumcs
Copy link
Contributor

@ramandumcs ramandumcs commented Feb 11, 2019

Make sure you have checked all steps below.

Jira

Description

  • Here are some details about my PR, including screenshots of any UI changes:
    It exposes an Api to get code of a given DagId

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:
    PR added a UT to test get_dag_code Api

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
    • All the public functions and the classes in the PR contain docstrings that explain what it does

Code Quality

  • Passes flake8


def get_code(dag_id):
"""Return python code of a given dag_id."""
dagbag = DagBag()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a potentially expensive operation (as it will load all dags from disk!)

Could you instead do something like in #4407 where we load the dag model from the DB instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ashb . Incorporated it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashb could you please review it and provide your feedback/comments

@ramandumcs ramandumcs force-pushed the AIRFLOW-3865 branch 2 times, most recently from 33a9f80 to 4451835 Compare February 12, 2019 05:58
@codecov-io
Copy link

Codecov Report

Merging #4687 into master will decrease coverage by 0.03%.
The diff coverage is 26.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4687      +/-   ##
==========================================
- Coverage   74.35%   74.31%   -0.04%     
==========================================
  Files         430      431       +1     
  Lines       27964    27988      +24     
==========================================
+ Hits        20792    20800       +8     
- Misses       7172     7188      +16
Impacted Files Coverage Δ
airflow/api/common/experimental/get_code.py 25% <25%> (ø)
airflow/www/api/experimental/endpoints.py 85.2% <30%> (-2.97%) ⬇️
airflow/models/__init__.py 92.74% <0%> (-0.06%) ⬇️
airflow/contrib/hooks/databricks_hook.py 95.57% <0%> (ø) ⬆️
airflow/executors/celery_executor.py 76.92% <0%> (ø) ⬆️
airflow/contrib/hooks/bigquery_hook.py 58.06% <0%> (ø) ⬆️
airflow/contrib/hooks/spark_submit_hook.py 80.08% <0%> (ø) ⬆️
airflow/contrib/utils/mlengine_operator_utils.py
...rflow/contrib/utils/mlengine_prediction_summary.py
...rflow/contrib/operators/mlengine_operator_utils.py 95.23% <0%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a738e5...706c7b2. Read the comment docs.

@shistoff
Copy link

@ashb ashb merged commit f4277cb into apache:master Feb 21, 2019
antonimaciej pushed a commit to PolideaInternal/airflow that referenced this pull request Feb 26, 2019
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants