Move tests_common package to devel-common project #47281
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
potiuk
requested review from
Fokko,
ashb,
bolkedebruin,
eladkal,
hussein-awala,
jedcunningham,
josh-fell,
jscheffl,
mik-laj,
mobuchowski and
o-nikolas
as code owners
boring-cyborg
bot
added
area:dev-tools
area:production-image
jscheffl
reviewed
Mar 3, 2025
jscheffl
reviewed
Mar 3, 2025
Contributor
jscheffl
left a comment
jscheffl
left a comment
There was a problem hiding this comment.
Reviewed code, looks good! Just pipeline needs to turn green :-D
potiuk
commented
Mar 3, 2025
Member
Author
On it :) |
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
from
0bd4054 to
22326a9
Compare
o-nikolas
approved these changes
Mar 3, 2025
Contributor
o-nikolas
left a comment
o-nikolas
left a comment
There was a problem hiding this comment.
A couple of doc suggestions and a question. The code changes look mostly mechanical and overall LGTM
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
3 times, most recently
from
f320253 to
6fd248e
Compare
Member
Author
|
While fixing the remaining issues, I have found that it will be easier, when I incorporate all "devel" dependencies into "tests-common" distribution - this way I could (@ashb -> I think you will be super-happy) of ALL This might be followed by similar |
Member
Author
|
That actualy makes me think if we should not name it |
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
2 times, most recently
from
5973e1f to
b8b14ec
Compare
potiuk
changed the title
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
from
b8b14ec to
087add7
Compare
Member
Author
|
Renamed the project to |
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
6 times, most recently
from
fa0b97d to
6e23f42
Compare
Member
Author
|
All right, I had to do a bit of custom installation of devel dependencies for the "compatibility" tests - because we are basically removing sources for airflow and task_sdk from those builds and only use the "devel-common" and "tests" and the rest is installed from packages, but I think I finally nailed it :) |
This PR moves all commo tests to a "devel-common" sub-project in Airlfow repo. This project is now part of the airflow's workspace, which means that it is installed by default when `uv sync` is run. The following changes have been implemented: * removed pytest options from providers and moved to common pyproject.toml * devel-common now keeps all the test dependencies that are used by other packages - they are automatically installed when `uv sync` is run. We do not need devel-tests extra any more * providers_src_folder fixture was effectively not used and replaced by __file__ deriving the source of where the appropriate package is imported from. * pytest init options are removed from provider's pyproject.toml because they overwrote the options defined in the main pyproject.toml. Instead --ignore-glob command line is passed to addopts * devel dependencies from task_sdk are removed and devel-common package is added as required dependency as well as standard and common.sql package - both needed to run task_sdk tests. This allows to run tests in task_sdk tests from withing task_sdk package. * all dev dependency group settings for all providers contain the dependent sources thaat allow to treat each provider separately. * all devel and bundle dependencies and deprecated dependencies have been removed complely - the devel dependencies have been incorporated into "devel-common" distribution - so you can install all development dependencies (except docs) by installing `devel-common`. Doc deps will be later extracted to a separate distriubution.
potiuk
force-pushed
the
move_tests_common_to_tests-common
branch
from
6e23f42 to
0638e15
Compare
jscheffl
reviewed
Mar 5, 2025
jscheffl
approved these changes
Mar 5, 2025
Contributor
jscheffl
left a comment
jscheffl
left a comment
There was a problem hiding this comment.
Besides the one question - code looks good.
Member
Author
|
wooohoo! |
Member
|
Woohooo :) |
Contributor
|
Huge 😮 🎉 |
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Mar 6, 2025
The change apache#47281 introduced ``devel-common`` and simplified hatch_build.py with removal of unneeded bundle extras, but it removed ``all-core`` extra that was actually used in constraint generation. This is a follow-up for it, fixing the problem and restoring ``all-core`` - also it synchronizes documentation and adds newsfragment describing the removal and explaining how users who used ``all`` extra before (for example in their ``CI``) with ``uv pip install --all-extras``.
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Mar 6, 2025
The change apache#47281 introduced ``devel-common`` and simplified hatch_build.py with removal of unneeded bundle extras, but it removed ``all-core`` extra that was actually used in constraint generation. This is a follow-up for it, fixing the problem and restoring ``all-core`` - also it synchronizes documentation and adds newsfragment describing the removal and explaining how users who used ``all`` extra before (for example in their ``CI``) with ``uv pip install --all-extras``.
potiuk
added a commit
that referenced
this pull request
Mar 6, 2025
The change #47281 introduced ``devel-common`` and simplified hatch_build.py with removal of unneeded bundle extras, but it removed ``all-core`` extra that was actually used in constraint generation. This is a follow-up for it, fixing the problem and restoring ``all-core`` - also it synchronizes documentation and adds newsfragment describing the removal and explaining how users who used ``all`` extra before (for example in their ``CI``) with ``uv pip install --all-extras``.
67 tasks
nailo2c
pushed a commit
to nailo2c/airflow
that referenced
this pull request
Apr 4, 2025
This PR moves all commo tests to a "devel-common" sub-project in Airlfow repo. This project is now part of the airflow's workspace, which means that it is installed by default when `uv sync` is run. The following changes have been implemented: * removed pytest options from providers and moved to common pyproject.toml * devel-common now keeps all the test dependencies that are used by other packages - they are automatically installed when `uv sync` is run. We do not need devel-tests extra any more * providers_src_folder fixture was effectively not used and replaced by __file__ deriving the source of where the appropriate package is imported from. * pytest init options are removed from provider's pyproject.toml because they overwrote the options defined in the main pyproject.toml. Instead --ignore-glob command line is passed to addopts * devel dependencies from task_sdk are removed and devel-common package is added as required dependency as well as standard and common.sql package - both needed to run task_sdk tests. This allows to run tests in task_sdk tests from withing task_sdk package. * all dev dependency group settings for all providers contain the dependent sources thaat allow to treat each provider separately. * all devel and bundle dependencies and deprecated dependencies have been removed complely - the devel dependencies have been incorporated into "devel-common" distribution - so you can install all development dependencies (except docs) by installing `devel-common`. Doc deps will be later extracted to a separate distriubution.
nailo2c
pushed a commit
to nailo2c/airflow
that referenced
this pull request
Apr 4, 2025
…7441) The change apache#47281 introduced ``devel-common`` and simplified hatch_build.py with removal of unneeded bundle extras, but it removed ``all-core`` extra that was actually used in constraint generation. This is a follow-up for it, fixing the problem and restoring ``all-core`` - also it synchronizes documentation and adds newsfragment describing the removal and explaining how users who used ``all`` extra before (for example in their ``CI``) with ``uv pip install --all-extras``.
This was referenced Apr 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR moves all commo tests to a "devel-common" sub-project in Airlfow repo.
This project is now part of the airflow's workspace, which means that it is installed by default when
uv syncis run.The following changes have been implemented:
removed pytest options from providers and moved to common pyproject.toml
devel-common now keeps all the test dependencies that are used by other packages - they are automatically installed when
uv syncis run. We do not need devel-tests extra any moreproviders_src_folder fixture was effectively not used and replaced by file deriving the source of where the appropriate package is imported from.
pytest init options are removed from provider's pyproject.toml because they overwrote the options defined in the main pyproject.toml. Instead --ignore-glob command line is passed to addopts
devel dependencies from task_sdk are removed and devel-common package is added as required dependency as well as standard and common.sql package - both needed to run task_sdk tests. This allows to run tests in task_sdk tests from withing task_sdk package.
all dev dependency group settings for all providers contain the dependent sources thaat allow to treat each provider separately.
all devel and bundle dependencies and deprecated dependencies have
been removed complely - the devel dependencies have been incorporated
into "devel-common" distribution - so you can install all development
dependencies (except docs) by installing
devel-common. Doc depswill be later extracted to a separate distriubution.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.