-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
openlineage: migrate OpenLineage provider to V2 facets. #39530
openlineage: migrate OpenLineage provider to V2 facets. #39530
Conversation
2e86fb5
to
9947104
Compare
9947104
to
fc72488
Compare
0479252
to
e43e266
Compare
I ran |
e43e266
to
74bc580
Compare
#39747 should fix the docker test in "Latest Botocore". |
74bc580
to
feaa46c
Compare
@JDarDagran we probably need to bump cross-provider version dependencies. If someone has older OL provider version, and bumps some of the other provider versions, the openlineage methods will fail. |
As discussed with @mobuchowski -> we need to add back-compatibility |
feaa46c
to
366b3ed
Compare
I've added imports with weird-looking structure at first glance: if TYPE_CHECKING:
# v2 imports
else:
try:
# v2 imports
except ImportError:
# v1 imports This satisfies mypy and works with previous OL provider versions. |
366b3ed
to
be583ac
Compare
Should we re-start discussion about common provider for that one? I have an idea.. |
3d9a6af
to
5ca57f8
Compare
d128fb6
to
7233d74
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a huge amount of work done, thanks @JDarDagran 🚀 I left some nit comments
7233d74
to
2f876b6
Compare
2f876b6
to
5383a9c
Compare
d92a54c
to
0cb2346
Compare
@potiuk looks like I'm on green path finally consistently 🟢 Would love to get this merged soon as new features arrive and I still need to rebase and apply new imports :) |
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
Signed-off-by: Jakub Dardzinski <kuba0221@gmail.com>
0cb2346
to
1bfad60
Compare
In OpenLineage/OpenLineage#2520 released in
openlineage-python==1.13.1
there were introduced V2 classes for facets and events.