Adding ArangoDB Provider #22548
Merged
Adding ArangoDB Provider #22548
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eladkal
reviewed
Mar 27, 2022
| :param arangodb_conn_id: Reference to :ref:`ArangoDB connection id <howto/connection:arangodb>`. | ||
| """ | ||
|
|
||
| template_fields: Sequence[str] = ('sql',) |
There was a problem hiding this comment.
Should we add also template_ext?
There was a problem hiding this comment.
yeah, make sense
added
| :param arangodb_db: Target ArangoDB name. | ||
| """ | ||
|
|
||
| template_fields: Sequence[str] = ('sql',) |
There was a problem hiding this comment.
Should we add also template_ext?
|
Can you fix static checks please. |
|
|
|
|
|
@potiuk can you take a look at the test failure? I don't recall that when adding a new provider we need to edit the CI script |
Not everything in providers has to be me :) - this test was added by @mik-laj actually: 621d17b It looks like for some reason prodcution image produced in this build contains many more providers than it should |
|
Yeah: seems that for some reason it contains all providers: |
|
This is VERY strange as it seems that when the image was built, it actually used only a small subset (as expected): |
|
Let me rebase and see it happening again :) |
|
I don't recall we had such issue when GitHub provider was added (and it was after 621d17b ) |
Me neither. It basicallly SHOUD NOT happen :D. Yet it seems it did again |
|
OK. I know what causes it but I do not know why it happens yet. When PROD build image is prepared we prepare "airflow" package so that it can be installed there from latest sources. But for SOME reason, it contains "all" providers as well. not only airflow. I do not know where it came from yet. But It proves the tests from @mik-laj are useful to catch it. |
|
I actually think it could come from the new setuptools release https://pypi.org/project/setuptools/61.2.0/ |
|
Still puzzled :) but I am getting closer to solve it |
|
thanks @potiuk. |
|
voila 🥳, |
potiuk
approved these changes
Apr 1, 2022
github-actions
bot
added
the
full tests needed
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
|
I'll take a look later today |
eladkal
approved these changes
Apr 3, 2022
| @@ -256,6 +256,8 @@ Those are extras that add dependencies needed for integration with other softwar | |||
| +---------------------+-----------------------------------------------------+-------------------------------------------+ | |||
| | trino | ``pip install 'apache-airflow[trino]'`` | All Trino related operators & hooks | | |||
| +---------------------+-----------------------------------------------------+-------------------------------------------+ | |||
| | arangodb | ``pip install 'apache-airflow[arangodb]'`` | ArangoDB operators, sensors and hook | | |||
There was a problem hiding this comment.
I think this list is sorted alphabetically?
|
🎉 🎉 🎉 🎉 🎉 🎉 🎉 |
ephraimbuddy
added
the
changelog:skip
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes: #17778
Description
Adding ArangoDB provider based on Python SDK https://github.com/ArangoDB-Community/python-arango
Users can create their own custom operators leveraging the ArangoDBHook directly
or building their operator on AQLOperator by providing result_processor method,
Sensor can be implemented by SQL