Skip to content

Add back missing [sources] link in generated documentation's includes #49978

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
potiuk merged 2 commits into from
Apr 30, 2025
Merged

Add back missing [sources] link in generated documentation's includes #49978

potiuk merged 2 commits into from
Apr 30, 2025

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Apr 29, 2025

The exampleinclude of ours used to have generated "sources" link to link to sources of the examples. This was however gone after recent doc build refactoring.

The problem was the example directive include import and lack of tests folder available on the python path. Both issues fixed in this PR.

Fixes: #49893

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added area:production-image Production image improvements and fixes kind:documentation labels Apr 29, 2025
@potiuk
Copy link
Member Author

potiuk commented Apr 29, 2025

I will have to regenerate the docs for airflow and providers released since the doc restructuring once this one is merged :)

@potiuk potiuk added this to the Airflow 3.0.1 milestone Apr 29, 2025
@potiuk potiuk added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Apr 29, 2025
@potiuk
Copy link
Member Author

potiuk commented Apr 29, 2025

Hah. Turns out a numbef of examples will need to be moved

@gopidesupavan
Copy link
Member

gopidesupavan commented Apr 29, 2025

Hah. Turns out a numbef of examples will need to be moved

oh.. can we patch some paths?

@potiuk potiuk force-pushed the fix-source-linx-in-generated-docs branch from 10f2014 to 58f879e Compare April 29, 2025 19:09
@potiuk potiuk requested a review from ashb as a code owner April 29, 2025 19:09
@potiuk
Copy link
Member Author

potiuk commented Apr 29, 2025

No.. I really had to move a number of examples to "standar" provider - because they belonged there

@potiuk potiuk force-pushed the fix-source-linx-in-generated-docs branch from 58f879e to 0aade65 Compare April 29, 2025 20:07
@potiuk potiuk requested a review from vincbeck as a code owner April 29, 2025 20:07
@potiuk
Copy link
Member Author

potiuk commented Apr 30, 2025

Question. That change will lead to less number of example dags visible when Airflow starts with "load example dags"- becuase most of the example dags belong to "standard" provider now.

But I think it's not necessary a bad thing - because IMHO we had far too many of those there. I'd say we should maybe (separately) make a small change when loading example dags will only load a subset of those, and maybe even load some from the standard provider? WDYT? This would be by far cleanest solution I think.

@kaxil
Copy link
Member

kaxil commented Apr 30, 2025

But I think it's not necessary a bad thing - because IMHO we had far too many of those there.

Agreed

@potiuk potiuk force-pushed the fix-source-linx-in-generated-docs branch from 0aade65 to d2a3ab5 Compare April 30, 2025 10:05
@potiuk potiuk force-pushed the fix-source-linx-in-generated-docs branch from f9f7f14 to 1d0e0f3 Compare April 30, 2025 13:30
@potiuk potiuk force-pushed the fix-source-linx-in-generated-docs branch 3 times, most recently from 7ea66b0 to d58d66a Compare April 30, 2025 15:16
@kaxil kaxil mentioned this pull request Apr 30, 2025
Merged
@github-actions github-actions bot mentioned this pull request May 21, 2025
Merged
potiuk added a commit that referenced this pull request May 21, 2025
@github-actions @potiuk
[v3-0-test] Fix reading number of dags from direct system providers p…
38225c6 
…ath (#50843) (#50877)

The original PR #49978 that fixed generation of proper documentation for
standard provider examples added retrieval of the path by checking
import location for the standard provider. Instead we are now using
direct path to the provider.
(cherry picked from commit 13c6020)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request May 21, 2025
@potiuk @nailo2c
Fix reading number of dags from direct system providers path (apache#…
db0a861 
…50843)

The original PR apache#49978 that fixed generation of proper documentation for
standard provider examples added retrieval of the path by checking
import location for the standard provider. Instead we are now using
direct path to the provider.
potiuk added a commit to potiuk/airflow that referenced this pull request May 21, 2025
@potiuk
Make test_dagbag test work locally, not only in Breeze
33917ba 
There were two issues with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
@potiuk potiuk mentioned this pull request May 21, 2025
Merged
potiuk added a commit to potiuk/airflow that referenced this pull request May 21, 2025
@potiuk
Make test_scheduler_job work locally after moving examples
02c0000 
There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
@potiuk potiuk mentioned this pull request May 21, 2025
Merged
potiuk added a commit that referenced this pull request May 21, 2025
@potiuk
Make test_dagbag test work locally, not only in Breeze (#50916)
396810d 
There were two issues with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
github-actions bot pushed a commit that referenced this pull request May 21, 2025
@potiuk
[v3-0-test] Make test_dagbag test work locally, not only in Breeze (#…
10e9606 
…50916)

There were two issues with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
(cherry picked from commit 396810d)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@github-actions github-actions bot mentioned this pull request May 21, 2025
Merged
potiuk added a commit to potiuk/airflow that referenced this pull request May 21, 2025
@potiuk
Make test_scheduler_job work locally after moving examples
ed8c97b 
There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
potiuk added a commit that referenced this pull request May 22, 2025
@github-actions @potiuk
[v3-0-test] Make test_dagbag test work locally, not only in Breeze (#…
ea7e160 
…50916) (#50924)

There were two issues with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
(cherry picked from commit 396810d)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
potiuk added a commit to potiuk/airflow that referenced this pull request May 22, 2025
@potiuk
Make test_scheduler_job work locally after moving examples
5fd46ff 
There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
potiuk added a commit to potiuk/airflow that referenced this pull request May 22, 2025
@potiuk
Make test_scheduler_job work locally after moving examples
b838702 
There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
dadonnelly316 pushed a commit to dadonnelly316/airflow that referenced this pull request May 26, 2025
@potiuk @dadonnelly316
Fix reading number of dags from direct system providers path (apache#…
1c59a3f 
…50843)

The original PR apache#49978 that fixed generation of proper documentation for
standard provider examples added retrieval of the path by checking
import location for the standard provider. Instead we are now using
direct path to the provider.
dadonnelly316 pushed a commit to dadonnelly316/airflow that referenced this pull request May 26, 2025
@potiuk @dadonnelly316
Make test_dagbag test work locally, not only in Breeze (apache#50916)
4eddf21 
There were two issues with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
@kneepdee kneepdee mentioned this pull request May 27, 2025
Closed
2 tasks
potiuk added a commit that referenced this pull request May 29, 2025
@potiuk
Make test_scheduler_job work locally after moving examples (#50917)
21f4a40 
There was an issue with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env.
github-actions bot pushed a commit that referenced this pull request May 29, 2025
@potiuk
[v3-0-test] Make test_scheduler_job work locally after moving examples (
bddd7b0 
#50917)

There was an issue with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env.
(cherry picked from commit 21f4a40)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@github-actions github-actions bot mentioned this pull request May 29, 2025
Merged
potiuk added a commit that referenced this pull request Jun 2, 2025
@potiuk
[v3-0-test] Make test_scheduler_job work locally after moving examples (
1553b63 
#50917)

There was an issue with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env.
(cherry picked from commit 21f4a40)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
potiuk added a commit that referenced this pull request Jun 2, 2025
@github-actions @potiuk
[v3-0-test] Make test_scheduler_job work locally after moving examples (
eb677fa 
#50917) (#51221)

There was an issue with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env.
(cherry picked from commit 21f4a40)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
kaxil pushed a commit that referenced this pull request Jun 3, 2025
@github-actions @potiuk @kaxil
[v3-0-test] Fix reading number of dags from direct system providers p…
0f7a15d 
…ath (#50843) (#50877)

The original PR #49978 that fixed generation of proper documentation for
standard provider examples added retrieval of the path by checking
import location for the standard provider. Instead we are now using
direct path to the provider.
(cherry picked from commit 13c6020)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
kaxil pushed a commit that referenced this pull request Jun 3, 2025
@github-actions @potiuk @kaxil
[v3-0-test] Make test_dagbag test work locally, not only in Breeze (#…
66d4532 
…50916) (#50924)

There were two issues with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
(cherry picked from commit 396810d)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
kaxil pushed a commit that referenced this pull request Jun 3, 2025
@github-actions @potiuk @kaxil
[v3-0-test] Make test_scheduler_job work locally after moving examples (
464d6b5 
#50917) (#51221)

There was an issue with the test after migrating standard provider
examples to ... standard provider (#49978) that made the test work only
in Breeze and not in local env.
(cherry picked from commit 21f4a40)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@kaxil kaxil mentioned this pull request Jun 3, 2025
Closed
sanederchik pushed a commit to sanederchik/airflow that referenced this pull request Jun 7, 2025
@potiuk @sanederchik
Fix reading number of dags from direct system providers path (apache#…
b01b27f 
…50843)

The original PR apache#49978 that fixed generation of proper documentation for
standard provider examples added retrieval of the path by checking
import location for the standard provider. Instead we are now using
direct path to the provider.
sanederchik pushed a commit to sanederchik/airflow that referenced this pull request Jun 7, 2025
@potiuk @sanederchik
Make test_dagbag test work locally, not only in Breeze (apache#50916)
7fba8a3 
There were two issues with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env:

* bad import of AIRFLOW_ROOT_PATH from scripts
* reading system.standard import during parsing the pytest file

Both issuses fixed here.
sanederchik pushed a commit to sanederchik/airflow that referenced this pull request Jun 7, 2025
@potiuk @sanederchik
Make test_scheduler_job work locally after moving examples (apache#50917
0f88055 
)

There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
jose-lehmkuhl pushed a commit to jose-lehmkuhl/airflow that referenced this pull request Jul 11, 2025
@potiuk @jose-lehmkuhl
Make test_scheduler_job work locally after moving examples (apache#50917
6c2ccca 
)

There was an issue with the test after migrating standard provider
examples to ... standard provider (apache#49978) that made the test work only
in Breeze and not in local env.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ashb ashb ashb left review comments

@gopidesupavan gopidesupavan gopidesupavan approved these changes

@eladkal eladkal eladkal approved these changes

@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner

@amoghrajesh amoghrajesh Awaiting requested review from amoghrajesh amoghrajesh is a code owner

@vincbeck vincbeck Awaiting requested review from vincbeck vincbeck is a code owner

@jason810496 jason810496 Awaiting requested review from jason810496 jason810496 is a code owner

@XD-DENG XD-DENG Awaiting requested review from XD-DENG XD-DENG is a code owner

@jedcunningham jedcunningham Awaiting requested review from jedcunningham jedcunningham is a code owner

@ephraimbuddy ephraimbuddy Awaiting requested review from ephraimbuddy ephraimbuddy is a code owner

+1 more reviewer

@aritra24 aritra24 aritra24 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area:production-image Production image improvements and fixes backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch kind:documentation

Projects

None yet

Milestone

Airflow 3.0.1

Development

Successfully merging this pull request may close these issues.

Example Operator usage does not include import statements

6 participants

@potiuk @gopidesupavan @kaxil @ashb @aritra24 @eladkal