-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the communityprovider:airbyteprovider:alibabaprovider:cncf-kubernetesKubernetes (k8s) provider related issuesKubernetes (k8s) provider related issuesprovider:databricksprovider:hashicorpHashicorp provider related issuesHashicorp provider related issuesprovider:microsoft-azureAzure-related issuesAzure-related issuesprovider:mongoprovider:mysqlprovider:openlineageAIP-53AIP-53provider:sendgridprovider:snowflakeIssues related to Snowflake providerIssues related to Snowflake providerprovider:trinoprovider:weaviatetesting statusStatus of testing releasesStatus of testing releases
Description
Body
I have a kind request for all the contributors to the latest provider distributions release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider airbyte: 5.2.0rc1
Provider alibaba: 3.2.0rc1
- Add MaxComputeSQLOperator, MaxComputeHook and AlibabaBaseHook to Alibaba Provider (#50178): @jsjasonseba
- Fix MaxComputeSQLOperator Argument (#51513): @jsjasonseba
- Fix inconsistent conn_name_attr on Alibaba Hooks (#51112): @jsjasonseba
Provider amazon: 9.9.0rc1
- Add
MessageDeduplicationIdsupport toSnsPublishOperator(#51383): @seanghaeli
Linked issues: - Add support for RequestPay=requester option in AWS (#51098): @valentinDruzhinin
- AWS Lambda Executor (#50516): @o-nikolas
- Removed unnecessary aws_conn_id param from operators constructors (#51236): @dominikhei
Linked issues: - Fix ECS reattach (#51412): @o-nikolas
Linked issues: - Fix EKS token generation (#51333): @o-nikolas
- Fix
EksPodOperatorin deferrable mode (#51255): @vincbeck - Rds Operator pass custom conn_id to superclass (#51196): @dominikhei
Linked issues: - Fix remote logging CloudWatch handler initialization and stream name assignment (#51022): @gopidesupavan
Linked issues: - Fix aws_conn_id defaulting after dag.test was updated to use TaskSDK. (#50515): @ferruzzi
- ECS Executor run tasks in bridge network mode (#50713): @bmyers-ozette
- Remove unused entries from
DagAccessEntity(#51174): @vincbeck - Update Redshift cluster operator and sensor to inherit AwsBaseOperator (#51129): @ellisms
- Remove Airflow 2 code path in executors (#51009): @kaxil
- Move AWS auth dependencies to python3-saml extra (#50449): @eladkal
- Update comment in CloudWatchRemoteLogIO (#51092): @gopidesupavan
Provider apache.hdfs: 4.10.0rc1
- Feature/support hdfs cookies and headers (#50955): @romsharon98
Linked issues: - Fix SDK compatibility for WebHDFSHook by replacing get_password() with password (#50759): @roach231428
Linked issues:
Provider celery: 3.12.0rc1
- Enhance Celery CLI with Worker and Queue Management Features (#51257): @dheerajturaga
- Fix log streaming when EXECUTE_TASKS_NEW_PYTHON_INTERPRETER is enabled in Celery executor (#50682): @vatsrahul1001
- Add worker_umask to celery provider.yaml (#51218): @dheerajturaga
- Remove Airflow 2 code path in executors (#51009): @kaxil
- Prevent legacy hybrid executors to be running in Airflow 3 (#51733): @jscheffl
Linked issues: - Update doc to highlight the statically coded hybrid executors are no longer supported in Airflow 3.0.0+ (#51715): @XD-DENG
Provider cncf.kubernetes: 10.6.0rc1
- [KubernetesPodOperator] pushes xcom after failed pod (#51475): @AutomationDev85
- [KubernetesPodOperator] Reads Kubernetes events and writes them into log (#50192): @AutomationDev85
- [KubernetesPodOperator] Added detection of terminated pod during reattach (#51158): @AutomationDev85
- [KubernetesPodOperator] Add xcom sidecar terminated detection (#51144): @AutomationDev85
- Fix
EksPodOperatorin deferrable mode (#51255): @vincbeck - Fix a bug where Kube config "worker_pod_pending_fatal_container_state_reasons" is parsed wrongly (#50931): @XD-DENG
- Relax Kubernetes client version support to <33.0.0 (#51195): @eladkal
- Remove Airflow 2 code path in executors (#51009): @kaxil
- Port
ti.runto Task SDK execution path (#50141): @kaxil - Prevent legacy hybrid executors to be running in Airflow 3 (#51733): @jscheffl
Linked issues: - Update doc to highlight the statically coded hybrid executors are no longer supported in Airflow 3.0.0+ (#51715): @XD-DENG
Provider cohere: 1.5.1rc1
Provider common.sql: 1.27.2rc1
- fix: Use source hook instead of destination hook when reading records in non-paginated mode in GenericTransfer (#50598): @dabla
- Port
ti.runto Task SDK execution path (#50141): @kaxil
Provider databricks: 7.5.0rc1
- feat: Add OpenLineage support for DatabricksSQLStatementsOperator (#50891): @kacpermuda
- feat: Add OpenLineage support for DatabricksCopyIntoOperator (#50821): @kacpermuda
Linked issues: - Adding
DatabricksSQLStatementSensorSensor with Deferrability (#49516): @jroachgolf84
Linked issues:
Provider edge3: 1.1.1rc1
- Fix Edge Worker Remove when in unknown state (#50717): @jscheffl
Linked issues: - Remove Airflow 2 code path in executors (#51009): @kaxil
- Refactor Edge Worker CLI for smaller module (#50738): @jscheffl
- Edge list worker cli command to list active job metrics (#51720): @dheerajturaga
- Extend command column in the edge_job table to accomodate more chars (#51716): @dheerajturaga
Provider fab: 2.2.1rc1
- Fix: allow users with specific DAG permissions to access DAGs when no pecific DAG is requested (#51462): @willyyang
Linked issues: - Fix fab asset compilation hashing script (#51446): @potiuk
- Add back security api in FAB auth manager (#51578): @vincbeck
Linked issues:
Provider git: 0.0.3rc1
- add user_name to http git (#51256): @codenamelxl
Linked issues:
Provider google: 16.0.0rc1
- Allow creation of empty files on GCS (#51669): @kiran2706
- Add HttpToGCSOperator for transferring data from HTTP to GCS (#49625): @josuegen
- Add project_id to some of the BigQuery Operators (#51006): @Crowiant
- refactor: allow to edit the file listings before copy (#49766): @AlexisBRENON
- feat: implementation of BaseOperatorLink for cloud run jobs to expose the GCP cloud logging url (#46911): @ramonvermeulen
- Add
columnNameCharacterMapoption toGCSToBigQueryOperator(#49458): @guan404ming
Linked issues: - Add new DataflowJobStateCompleteTrigger (#47993): @olegkachur-e
- Create operators for working with Ray for VertexAI (#47340): @MaksYermak
- Remove direct access to DB for safe_to_cancel() method for Dataproc and BigQuery triggers (#49711): @MaksYermak
- Update the default value of
PubSubPullOperator.deferrabletoconf.getboolean("operators", "default_deferrable", fallback=False)(#50935): @le - Fix CloudRunExecuteJobOperator job_name param description (#50628): @jbandoro
- Make gcs remote log args optional when using connection id (#50590): @gopidesupavan
- Replace
converted_rows[0].keys()withfields(#50442): @nailo2c
Linked issues: - Fix gcp remote log module import in airflow local settings (#49788): @gopidesupavan
- feat: add
polling_period_secondsandtimeout_secondsto the default template fields of theCloudRunExecuteJobOperator(#49704): @ramonverme - Add extra fields and hide unnecessary fields for LevelDB (#51343): @nailo2c
- Remove sqlalchemy-spanner exclusion after it's been yanked. (#51433): @potiuk
- Exclude
sqlalchemy-spanner1.12.0 (#51379): @vincbeck
Linked issues: - Preemptively add proper apache license for wordcount.go (#50968): @amoghrajesh
- Add GoogleAdsHook connection types and documentation (#50755): @nailo2c
- Issue #50206: add more template_fields to RunPipelineJobOperator (#50220): @ferenc-hechler
Linked issues:- Linked Issue #50206: @ferenc-hechler
- Make match_glob parameter visible in
Rendered templatesfield on UI by adding it to templated_fields (#50393): @VladaZakharova - Migrate
BigQueryHookto useget_df(#50341): @guan404ming
Linked issues: - add root parent information to OpenLineage events (#49237): @mobuchowski
- Adding dependency on
google-cloud-bigquery-storagetemporarily for google (#49807): @amoghrajesh
Linked issues: - Remove on_finish_action from template fields (#49637): @MaksYermak
- Add Looker connection types and LevelDB documentation (#51058): @nailo2c
Provider hashicorp: 4.3.0rc1
- Support cert and proxies in Vault client session setup (#49906): @vuonghoangbntt
Provider microsoft.azure: 12.4.1rc1
Provider mongo: 5.2.0rc1
- feat: add create_collection function to MongoHook (#50518): @nailo2c
Linked issues:- Linked Issue #33076: @gemunet
Provider mysql: 6.3.1rc1
Provider neo4j: 3.9.1rc1
Provider openlineage: 2.4.0rc1
- feat: Add NominalTimeRunFacet to all OL events (#51404): @kacpermuda
- feat: Add TagsJobFacet to DAGRun OpenLineage events (#51303): @kacpermuda
- feat: Add Airflow-specific OL system tests validation (#51084): @kacpermuda
- feat: merge TimeDeltaSensorAsync to TimeDeltaSensor (#51133): @jameshyphen
Linked issues: - expose OpenLineage's lineage_root_* macros in plugin (#50532): @mobuchowski
- fix: Use task owner for TASK level Ownership facet (#51305): @kacpermuda
- Fix OpenLineage macro _get_logical_date (#51210): @astro-anand
- Fix simple grammar mistakes in doc (#51138): @XD-DENG
- Implementing
log_urlon RuntimeTaskInstance in task sdk (#50376): @ayush3singh
Linked issues:- Linked Issue #50330: @jnstockley
- nit: task-level facets should not overwrite integration-level facets (#51690): @kacpermuda
Linked issues: - Make duration in 'List Dag Run' page sortable (#51495): @valentinDruzhinin
- Import MappedOperator from airflow.sdk.definitions.mappedoperator in openlineage (#51492): @sunank200
Provider oracle: 4.1.1rc1
Provider pinecone: 2.3.1rc1
Provider sendgrid: 4.1.1rc1
- Add lower bound to Sendgrid after #50221 (#51278): @jscheffl
Linked issues: - Bump min version of sendgrid sdk (#50221): @eladkal
Provider sftp: 5.3.1rc1
- Updating SFTPSensor to properly handle scenario where file is missing (#51167): @jroachgolf84
Linked issues:- Linked Issue #51151: @ITC00kie
Provider slack: 9.1.1rc1
- [Providers][Slack] Add automatic 429‑retry to channel‑lookup logic (#51265): @nailo2c
Linked issues:- Linked Issue #47225: @zachliu
Provider snowflake: 6.4.0rc1
- Extend SnowflakeHook OAuth implementation to support external IDPs and client_credentials grant (#51620): @pgagnon
- fix: make query_ids in SnowflakeSqlApiOperator in deferrable mode consistent (#51542): @kacpermuda
- fix: Duplicate region in Snowflake URI no longer breaks OpenLineage (#50831): @kacpermuda
- Do not allow semicolons in
CopyFromExternalStageToSnowflakeOperatorfields (#51734): @amoghrajesh - Port
ti.runto Task SDK execution path (#50141): @kaxil
Provider standard: 1.3.0rc1
- feat: merge TimeDeltaSensorAsync to TimeDeltaSensor (#51133): @jameshyphen
Linked issues: - Add timezone support for date calculation in TimeSensor (#51043): @SML0127
- Merging
TimeSensorAsyncwithTimeSensor(#50864): @jroachgolf84
Linked issues: - Fix Airflow V2 incompatibility in ExternalTaskSensor (#51479): @fuatcakici
- bug fix: DateTimeSensor can't render jinja template if use native obj (#50744): @nailo2c
Linked issues:- Linked Issue #38336: @dwreeves
- Fix backward compatibility for timeout in defer() with Airflow 2.11 (#50869): @amoghrajesh
- Port
ti.runto Task SDK execution path (#50141): @kaxil
Provider trino: 6.3.0rc1
- Add extra_credentials and roles to TrinoHook (#51298): @zach-overflow
Provider weaviate: 3.2.0rc1
- Add delete_by_property method in weaviate hook (#50735): @sjyangkevin
Linked issues:- Linked Issue #42565: @harjeevanmaan
- Add batch_create_links method in weaviate hook (#50750): @sjyangkevin
Linked issues:- Linked Issue #42568: @harjeevanmaan
All users involved in the PRs:
@dominikhei @mobuchowski @codenamelxl @fuatcakici @jsjasonseba @olegkachur-e @eladkal @roach231428 @Crowiant @ellisms @zach-overflow @valentinDruzhinin @willyyang @vuonghoangbntt @romsharon98 @jscheffl
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Metadata
Metadata
Assignees
Labels
area:providerskind:metaHigh-level information important to the communityHigh-level information important to the communityprovider:airbyteprovider:alibabaprovider:cncf-kubernetesKubernetes (k8s) provider related issuesKubernetes (k8s) provider related issuesprovider:databricksprovider:hashicorpHashicorp provider related issuesHashicorp provider related issuesprovider:microsoft-azureAzure-related issuesAzure-related issuesprovider:mongoprovider:mysqlprovider:openlineageAIP-53AIP-53provider:sendgridprovider:snowflakeIssues related to Snowflake providerIssues related to Snowflake providerprovider:trinoprovider:weaviatetesting statusStatus of testing releasesStatus of testing releases