Source Github: Convert to airbyte-lib

[flash1293]

Make the connector ready to be consumed by airbyte-lib

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Jan 27, 2024 6:29am

[github-actions]

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

• PR name follows PR naming conventions
• Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
• Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
• You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
• Secrets in the connector's spec are annotated with airbyte_secret
• All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
• Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
• Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
• If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

1. Check for hidden checklists in your PR description

2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

[lazebnyi]

CAT failed on expected records, so PR can be approved

[aaronsteers]

@flash1293 - fyi, I was testing this in a new Colab notebook:

https://colab.research.google.com/drive/1sphTQAZAkt8sXpAkO2ANxZMX_Hg4NemV#scrollTo=qQVRO69c2DoA

In the process, I found that the package_data in setup.py does not include the "events" subdirectory.

AirbyteConnectorReadError: AirbyteConnectorReadError: Error when reading from the connector..

 Log output: FileNotFoundError: [Errno 2] No such file or directory: '/content/.venv-source-github/lib/python3.10/site-packages/source_github//schemas/shared/events/comment.json'

For this connector and others, it is probably safer to use "**/*.json". It looks like the docker-based delivery mechanism was bundling the whole directory regardless of package_data, so we may want to double-check this - or just expect some debugging. 🤷

For this issue, I will contribute a commit to try to fix, and will retest in the notebook.

Update: Double-asterisks ("**/...") do not work in this context, unfortunately. We can accomplish something similar by including n levels of wildcards, up to 4 or 5 levels. My latest commit takes this approach, and I've confirmed that it now runs successfully in the Colab notebook above. Since some connectors store spec as a yaml file, I thought it was a good general precedent to also include ".yml" and ".yaml" files at the base of the package directory.

[flash1293]

Thanks for the callout @aaronsteers - I don't plan to merge this PR anymore, but I rolled out this change to all connectors on the big refactoring PR: #34552