🎉 New Source: Captain Data [Configuration based CDK]

[elliottrabac]

What

Describe what the change is solving
It helps to add screenshots if it affects the frontend.

Implement a new Source for Captain Data.

How

Describe the solution

Developed using (Configuration Based Source) low-code CDK.

Recommended reading order

1. manifest.yaml
2. schemas/*

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user?

For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.

If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

None. New Source only.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

• Community member? Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Connector version is set to 0.0.1
  • Dockerfile has version 0.0.1
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • docs/integrations/<source or destination>/<name>.md including changelog with an entry for the initial version. See changelog example
  • docs/integrations/README.md
  • airbyte-integrations/builds.md
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• If new credentials are required for use in CI, add them to GSM. Instructions.
• /test connector=connectors/<name> command is passing
• New Connector version released on Dockerhub by running the /publish command described here
• After the connector is published, connector added to connector index as described here
• Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Connector Generator

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed

[marcosmarxm]

Left some comments @elliottrabac thanks for the contribution

[elliottrabac]

@marcosmarxm thanks for the review. I updated the PR 👍

[elliottrabac]

bumping this @marcosmarxm 🙏

[elliottrabac]

@marcosmarxm 🙏 Any chance to have a quick review here?

[marcosmarxm]

@elliottrabac sure! we're going to review this week!

[sajarin]

@sajarin what I am supposed to add for definitionId in metadata.yml?

definitionId should be a UUID, you can generate one here: https://www.uuidgenerator.net/version4

[elliottrabac]

@sajarin I did the suggested changes

I struggle with the gradle setup :/ You can create an account easily here -> https://app.captaindata.co/signup

Also, we are running this connector in production internally as a custom connector (and some of our clients too).

python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json is working fine:

As soon as the connector is live, we'll do the migration with some of our clients and will monitor everything.

[elliottrabac]

@sajarin let me know if that's okay :)

[elliottrabac]

@sajarin any news? Captain Data would love to implement Airbyte Cloud and this connector for their customer as soon as possible :)

[elliottrabac]

@sajarin @marcosmarxm sorry for pinging you again

Captain Data would like to suggest their customers to use Airbyte Cloud with this beta connector. Do you have an ETA for this review?

[marcosmarxm]

@sajarin @marcosmarxm sorry for pinging you again

Captain Data would like to suggest their customers to use Airbyte Cloud with this beta connector. Do you have an ETA for this review?

Sorry the delay and missing reply @elliottrabac. I'll return to your contribution in this week.

[marcosmarxm]

Thanks @elliottrabac the implementation looks correct. I'm going to release the connector during the week, right now our CI is broken.

[elliottrabac]

Thanks @elliottrabac the implementation looks correct. I'm going to release the connector during the week, right now our CI is broken.

Thanks a lot!

[marcosmarxm]

/approve-and-merge reason="The implementation looks good and correct, right now Airbyte doesn't have test credentials to run integration tests (it costs more 200 euros/month)."

[octavia-approvington]

You did it!

[elliottrabac]

@marcosmarxm is the connector available on Airbyte Cloud?