Apply security and compatibility updates for PyAirbyte dependencies (Sep 2024)

[aaronsteers]

There are a number of dependencies of PyAirbyte which are stale relative to security updates. We'd also like to build a formal process to stay on top of relevant security and dependency updates.

This issue will track a number of action items:

Tasks

Beta Give feedback

1. Enable GitHub security scanning
2. Enable dependabot in GitHub
3. Handle initial of dependabot recommendations
4. Update SQLAlchemy to 2.x #382
   +0
5. Chore: Bump CDK dependency, DuckDB, Postgres, Pandas, and others #383
   +0
6. Make issues for any other remaining long-term action items
7. Chore: Bump to Sqlalchemy 2.0 #396
   +0
8. Add support for Pendulum 3.0 #397
   +0
9. Add support for latest version of jsonschema library #398
   +0

[aaronsteers]

As noted in task list above, this week we enabled dependabot updates and GitHub security scanning.

We've also merged 9 dependabot updates this week, which will release shortly.

The largest blocker right now for the other libraries is the SQLAlchemy 2.x bump. That was previously blocked by Snowflake and BigQuery drivers not supporting the 2.x, which both have now gotten caught up on. I have this draft PR as a WIP, but it may need to be broken into smaller updates.

[aaronsteers]

Just released:

• Patch release with low-hanging fruit dependabot updates:
  • https://github.com/airbytehq/PyAirbyte/releases/tag/v0.17.8

[aaronsteers]

The latest version of PyAirbyte,, released today, includes support for SQLalchemy 2.0 along with the latest version of the Airbyte CDK, along with several other dependency updates.

https://github.com/airbytehq/PyAirbyte/releases/tag/v0.17.9

Closing this issue as resolved for our high pri updates. New issues have been created to track Pendulum and jsonschema.

Any other dependency updates that are needed may be requested by creating a new GitHub issue here in this repo.