Generate connector builder api client #18274
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lmossman
changed the title
github-actions
bot
added
area/platform
lmossman
commented
Oct 20, 2022
Comment on lines
+114
to
+116
| config: | ||
| type: object | ||
| description: The config blob containing the user inputs for testing |
There was a problem hiding this comment.
I realized that this was missing from the /stream/read endpoint, so I added it here
lmossman
commented
Oct 20, 2022
| request: | ||
| $ref: "#/components/schemas/HttpRequest" | ||
| response: | ||
| $ref: "#/components/schemas/HttpResponse" | ||
| StreamReadRequestBody: | ||
| type: object | ||
| required: | ||
| - definition | ||
| - manifest |
There was a problem hiding this comment.
There are several changes in this spec to rename "connector definition" to "manifest", following this decision made in slack: https://airbytehq-team.slack.com/archives/C03TP6W8081/p1666214768391589
|
@girarda @brianjlai added you both for context since I made some changes to the connector builder openAPI spec in this PR, if you could take a look at those changes specifically |
lmossman
commented
Oct 21, 2022
| @@ -61,53 +63,71 @@ components: | |||
| StreamRead: | |||
There was a problem hiding this comment.
@girarda @brianjlai @sherifnada
I've made a few changes to the StreamRead object here, if you could take a look. The main thing is that I've split out airbyteMessages into separate fields for logs, records, and state.
The reasoning behind this is mainly coming from not being able to pull in the AirbyteProtocol struct into this openAPI spec, so we will need the connector-builder-server to filter out the various types of AirbyteMessages into separate fields like this so that the frontend knows what to do with them.
I pulled logs up to the top level of this response, since I feel like users will just want to view all of the logs for the entire read operation (and that also matches the "logs tray" design in the prototype), rather than seeing them split up by page. But let me know if you disagree.
I also pulled state up to the slice-level of this response, since we expect that there should be one state message for each slice.
There was a problem hiding this comment.
I agree with moving the state to the slice-level.
No strong opinion on whether the the logs should be at the page-level or global. I think global is fine
There was a problem hiding this comment.
also seems fine to me. We sometimes have to deal w/ consecutive state messages (or the case where slices have no records) so maybe this approach works better anyway so the underlying python server can obfuscate the complexity of formatting what some of these outputs might look like
edmundito
approved these changes
Oct 24, 2022
| @@ -5,6 +5,7 @@ const defaultConfig: Config = { | |||
| healthCheckInterval: 20000, | |||
| version: "dev", | |||
| apiUrl: `${window.location.protocol}//${window.location.hostname}:8001/api`, | |||
| connectorBuilderApiUrl: `${window.location.protocol}//${window.location.hostname}:8080/`, | |||
There was a problem hiding this comment.
8080 seems like a very common port
There was a problem hiding this comment.
Happy to change it to whatever port we think is appropriate
There was a problem hiding this comment.
We could use 8003, since it seems like that is not currently used by anything else in OSS or Cloud (8001 is used by airbyte-server, and 8002 is used by the airyte-cloud-server in Cloud)
There was a problem hiding this comment.
Either way, I'm just going to commit this as is for now, since we can easily change this later
jhammarstedt
pushed a commit
to jhammarstedt/airbyte
that referenced
this pull request
Oct 31, 2022
* update openapi for connector builder to work with orval for now * add orval configuration for generating the connector builder client * create ConnectorBuilderApiService with methods to call the generated connector builder API client code * rename connector definition to manifest * refactor builder api service to match existing patterns * fix name of pages folder * improve comment * modify structure of StreamRead * fix path
natalyjazzviolin
pushed a commit
that referenced
this pull request
Nov 3, 2022
* update openapi for connector builder to work with orval for now * add orval configuration for generating the connector builder client * create ConnectorBuilderApiService with methods to call the generated connector builder API client code * rename connector definition to manifest * refactor builder api service to match existing patterns * fix name of pages folder * improve comment * modify structure of StreamRead * fix path
am6010
added a commit
to rudderlabs/airbyte
that referenced
this pull request
Nov 10, 2022
* Fix broken link to config in .env comment (#18287) * Add message migration to discover schema (#18205) * upgrade commons-text to 1.10.0 (#18275) * test * fix * fix other modules * version bumps + changelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * [low-code] Propagate options to InterpolatedRequestInputProvider (#18050) * properly propagate options * cleanup * turn into dataclass * rename * no need for deepcopy * fix test * bump * cleaner * Add message translation to GetSpec (#18130) * Update SpecActivityImpl to build a VersionedStreamFactory * Enable Protocol Version detection from Stream for SPEC * Print log before the action for better debugging * Fix buffer size for protocol detection * Improve detectVersion error handling * extract constan * Rename attribute for clarity * only rm file if it exists (#18310) * remove extra check (#10554) * remove extra check * simplify logic * Google ads: Upgrade CDK to incorporate config error handling (#18309) * Bump version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * :tada: Source Linkedin Ads - for adDirectSponsoredContents stream skip accounts which are not part of organization (#18111) * for adDirectSponsoredContents stream skip accounts which are part of organization * updated PR number * updated log message, added unit tests * additional formatting * updated connector version in source_definitions.yaml * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Add SchemaChange to WebBackendConnectionRead object (#17969) * add schemaChange field to WebBackendConnectionRead and breakingChange to ConnectionRead * Destinations BigQuery and GCS: update docs about bucket encryption (#18315) * Bump Airbyte version from 0.40.16 to 0.40.17 (#18316) Co-authored-by: edgao <edgao@users.noreply.github.com> * Updates `getPrivateKeyPair` to throw exception when private key pair does not exist (#18263) * Changes method from returning null to throwing Exception to match method structure * Added javadoc explaining that KeyPair should not be returning null * Moved ConnectionErrorException to commons package * Adds error handling for SshWrappedDestinations and markdown to include changes * Bumps version numbers for Postgres, MySQL, MSSQL * auto-bump connector version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: xkcd [python cdk] (#18049) * Add new source: xkcd * Add documentation for xckd connector * Fix unit test test_parse_response * Fix xkcd changelog * Improve iteration method * Update tests test_next_page_token and test_parse_response * add timeout for full read * add seed config and eof * correct source def id * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source pinterest: Fix type of `start_date` (#18285) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * requester = MagicMock(use_cache=False) (#18289) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * fix Connector Base build (#18351) * SAT: new `test_strictness_level` field in config (#18218) * 🪟 🐛 Fixed static <ConditionSection /> width (#18233) * fixed ConditionSection width * fixed relative scss colors import * 🎉 Source Shopify: update API version to `2022-10` (#18298) * Update airbyte-db reference (#18368) * 🪟 🐛 Load global CSS in storybook (#18353) * Bump helm chart version reference to 0.40.33 (#18371) Co-authored-by: xpuska513 <xpuska513@users.noreply.github.com> Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com> * enable support for publishing datadog metrics from a local environment (#18282) * local datadog docker-compose * add datadog docker-compose * update comments * Simplify the OSS documentation deploy system (#2670) (#18377) + Adds a placeholder for 'Deploy Docs' Github Workflow. * fix: add task to list all dependencies (#18304) * fix: add task to list all dependencies * fix: remove newline * remove useless json_size metric (#18247) * 🎉Destination-elasticsearch: added custom sertificate support (#18177) * [11356] Destination-elasticsearch: added custom certificate support * update spotbugs to latest (#18208) * Source Jira - Pagination fix on jira users stream (#16944) * pagination fix on jira users stream * Update spec.json * Update spec.json * Update streams.py * bump docker version and changelog * Update spec.json * Update streams.py * update jsons and source * update source * Update source.py * auto-bump connector version Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 Source Google Ads: Add fields to the campaign stream 🚨 (#18069) * feat: add new fields to campaign stream * docs: updated docs and bumped Dockerfile version * Update campaigns.json * Update Dockerfile * auto-bump connector version Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * fixed check-style (#18390) * Readded xkcd to source def yaml (#18386) * readded xkcd to source def yaml * remove changes in klavyio * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Trace errors in orchestrator workers (#18381) * Trace errors in orchestrator workers * Trace cancel method of workers * Formatting * Restore final modifier * Source Shopify: add unit tests (#17944) * Export python type information for airbyte-cdk (#18387) Fixes #18384 * Fix error highlight of select boxes (#18396) * Source Hubspot: update `contacts` scope in docs (#18380) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * remove extra ' in version (#18400) - rm extra `'` in version string; `"2.1.1'"` -> `"2.1.1"` * Improve listSourcesForWorkspace route (#18207) * 🪟 🐛 Fix additional buttons for connection creation (#18372) * make button enabled regardless credit balance * remove FeatureItem.AllowCreateConnection at all, since we don't need it anymore * update tests * Replace "AllowCreateConnection" feature flag with "AllowDBTCloudIntegration" in tests * Change port for acceptance test destination db (#18391) * Bmoric/update connection list with breaking (#18125) * add schemaChange * merge conflict * frontend tests * tests * l * fix source catalog id * test * formatting * move schema change to build backend web connection * check if actor catalog id is different * fix * tests and fixes * remove extra var * remove logging * continue to pass back new catalog id * api updates * fix mockdata * tests * add schemaChange * merge conflict * frontend tests * tests * l * fix source catalog id * test * formatting * move schema change to build backend web connection * check if actor catalog id is different * fix * tests and fixes * remove extra var * remove logging * continue to pass back new catalog id * api updates * fix mockdata * tests * tests * optional of nullable * Tmp * For diff * Add test * More test * Fix test and add some * Fix merge and test * Fix PMD * Fix test * Rm dead code * Fix pmd * Address PR comments * RM unused column Co-authored-by: alovew <anne@airbyte.io> * Generate connector builder api client (#18274) * update openapi for connector builder to work with orval for now * add orval configuration for generating the connector builder client * create ConnectorBuilderApiService with methods to call the generated connector builder API client code * rename connector definition to manifest * refactor builder api service to match existing patterns * fix name of pages folder * improve comment * modify structure of StreamRead * fix path * :window: :wrench: Generify connector documentation layout into resizable panels component (#18272) * generify connector documentation layout into resizable panels component * add intl string for connector builder expand message * update name of formatted download yaml message * fix stylelint * Update airbyte-webapp/src/components/ui/ResizablePanels/ResizablePanels.module.scss Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * move splitter styles into scss module * simplify ResizablePanels component * reorder * use variables * remove units on 0 * Update airbyte-webapp/src/views/Connector/ConnectorDocumentationLayout/ConnectorDocumentationLayout.module.scss Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * Update airbyte-webapp/src/components/ui/ResizablePanels/ResizablePanels.module.scss Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * Update airbyte-webapp/src/components/ui/ResizablePanels/ResizablePanels.module.scss Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * SAT: declare `bypass_reason` in `acceptance-test-config.yml` (#18364) * Source Iterable - better processing of 401 and 429 errors (#18292) * #829 source iterable - better processing of 401 and 429 errors * source iterable: upd changelog * #829 source iterable: fix unit tests * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 Source Asana: fix request-cache test (#18375) * Source Asana: fix request-cache test * Source Asana: Bump version * 🐛 Source Asana: revert docker label; update CDK * 🐛 Source Freshdesk: fix test; update CDK (#18389) * 🐛 Source Freshdesk: fix test; update CDK * 🐛 Source FreshDesk: revert docker label; update CDK * Update cdk-speedrun.md (#18416) * Update README.md * Update Facebook-marketing.md with warning about ongoing setup issue (#18422) There is an ongoing issue blocking anyone with the most recent API update from being able to setup a Facebook Marketing Source Connector. Put this warning at the top of the setup page until it is fixed. * Source Greenhouse: extend users stream schema (#18154) * #17506 source greenhouse: extend users stream schema * #17506 source greenhouse: upd changelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 Source Hubspot: update CDK; fix expected test result according to 0.2.2 update (#18424) * Additional worker APM tracing (#18427) * Publish new CDK version with typehints exported (#18398) * Extract the AttemptApi out of the ConfigurationApi (#18406) * Tmp * Extract the Attempt API from the V1 API * Add comments * format * Rename to Controller * 🐛 Source Salesforce: update CDK; fix test supported_sync_modes is required for AirbyteStream (#18429) * :window: :wrench: Introduce useAuthentication hook (#17965) * Introduce useAuthentication hook * Work for both auth mechanism * Finalize implementation * Add documentation * Fix unit tests * Add unit tests * Remove getValues call * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/views/Connector/ServiceForm/useAuthentication.test.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Change variable name Co-authored-by: Lake Mossman <lake@airbyte.io> * Update: aws ec2 deployment guide (#18360) * Updated Deploy on GCP Doc (#18286) * Updated Deploy on GCP Doc * Added formatting changes * Updated deploying Airbyte on Azure section (#18331) * feat: deploy airbyte on kubernetes doc (#18339) * New Source: Zoom [low-code CDK] (#18179) * feat: implement meetings & users API in zoom connector * feat: add support fot all zoom GET APIs * fix: unhandled error cases by adding default handler and minor refactor * feat: sync the catalog with the latest zoom API responses * docs: add the new zoom connector built with low-code CDK * chore: remove unnecessary files, and rename connector in definitions * fix: forgot to add source_definitions.yaml earlier 😅 * remove some empty streams * add eof * correct spec yaml * docs: update docs/integrations and replace zoom with low code CDK zoom * fix: definition generation err * udpate seed file and remove zoom-singer * remove zoom singer spec * add icon Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> * 🐛 Source SurveyMonkey: fix expected test result according to 0.2.2 update (#18431) * Update postgres.md (#18432) * Update README.md * SAT: make all tests mandatory when `test_strictness_level` == `high` (#18414) * Refactor SourceHandler:buildSourceRead function (#18430) This function takes as input a `sourceId` and performs a database call to retrieve the full source information. This refactoring ensures that call sites that already have the full object can bypass that database call. This is especially important for methods that fetch and return a list of sources. * Make YouTube login button Google styled (#18443) * Edited Notion Doc (#18405) * edited doc * some wording changes * edited step * SAT: enforce `bypass_reason` declaration on `empty_streams` when `test_strictness_level == high` (#18425) * Fix the tag in openAPI (#18445) * Fix the tag * remove unused * use api to do jobpersistence query (#18308) * use api to do jobpersistence query * renaming some variables * fix test * RefreshSchema Activity (#18388) * Add RefreshSchema Activity * Follow-up fix for SSHExceptions (#18383) * Follow-up fix for SSHExceptions * Catch RuntimeException instead of augmenting connection error messaging for now * Update tests * Bump versions + docs * auto-bump connector version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Refactor the buildDestinationRead function (#18446) Create a new buildDestinationRead function, that function can be called by top-level functions in the handler that already have a DestinationConnection object. * 🐛 Fix S3 destination integration tests to use bucket path (#18031) * Fix S3 destination tests * Add Changelog * CR Fixes * add unit test * version bump * fix dockerfile * auto-bump connector version Co-authored-by: Edward Gao <edward.gao@airbyte.io> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * fix formatting + imports (#18457) * RefreshSchema method on RefreshSchema activity (#18460) * Add refreshSchema method to RefreshSchema activity * 🐛 Source Tiktok Marketing: fix test timeout (#18433) * Postgres Source : updated docs for CDC setup (#18291) * Postgres Source updated docs for CDC setup * fixed typo * fixed typo * rephrased * rephrased * Source postgres: disable allow and prefer ssl modes in CDC mode (#18256) * Source postgres: disable ssl_mode allow and prefer in CDC mode * Source postgres: code review fixes * Source postgres: code review fixes * Source postgres: improve ssl mode handling * Source postgres: bump version * auto-bump connector version * Source postgres: bump version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Twilio: implement slicing (#18423) * #804 source twilio: implement slicing * #804 source twilio - format code * #804 source twilio - implement slicing * #804 source twilio: configure incremental streams * #804 fix expected records * #804 extend Calls stream schema * #804 source twilio: allow configurable slice for SATs * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Fix broken table in the docs (#18466) * 🎉 New Connector: Clockify [python cdk] (#17767) * new connector source-clockify * feat: enable caching for streams (users, projects) * chore: pep8 changes at unit_tests * chore: pep8 changes at integration_tests * chore: update schema * Adds python formatting, removes unused import. * Makes the task duration field nullable to pass integration test. * fix: add second type to null values to the schema files * Adds a null fallback value to task duration. * Updates airbyte-cdk dependency. * Adds UUID in source definitions. * auto-bump connector version * Requested changes. * add clockify to source def seed * correct spec.json add titles * add icon * run format * remove source spec * correct spec * add eof gitignore * auto-bump connector version Co-authored-by: nataly <nataly@airbyte.io> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Nataly Merezhuk <65251165+natalyjazzviolin@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> * Optimize listDestinationsForWorkspace route (#18456) Add a function to fetch all destinations in a workspace * Add new streams table components (#18262) * Add new streams table components * Update CatalogTree and Catalog section to render new components based on env-based flag * Add basic Bulk edit panel, stream panel, table row, table headers * Split the overlay out of the modal and make it its own component. Use in StreamPanel * Add Fields to en.json * StreamPanel -> StreamDetailsPanel * Add new stream fields table components * Use new stream fiels table props iface on Stream details panel * add basic auth for Lever (#17996) * add basic auth for Lever * Update lever-hiring.md * Update spec.json * Update source.py * Revert "Update spec.json" This reverts commit 77b10a653b1056c959664416a6c89eafb6f49e07. * Revert "Update source.py" This reverts commit 3d6d83c6c63597d0b8ed46038d17989b38ba9712. * Update spec.json * update spec.json, replace sample config, change check to auth_type * unit tests with parametrize * gradlew formatting python + fixing bugs to ensure it works * modifs as suggested by reviewer * Update test_source.py * Update test_source.py * fixing specs in order for integration tests to pass * fix acceptance test TestSpec.test_oauth_flow_parameters * bump the docker image version to 0.1.3 * auto-bump connector version Co-authored-by: Yiyang Li <yiyangli2010@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 Source File: fix check method (#18481) * 🐛 Source File: fix check method * 🐛 Source File: bump version * 🐛 Source File: test for wrong format * 🐛 Source File: docs update * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Make script both Unix/Linux and OSX compatible (#18493) * Update to latest Temporal SDK release (#18492) * Trace launcher worker run method (#18441) * Bmoric/extract connection api (#18409) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * Fix PMD errors * Add explicit path to the controller * 🎉 New Source: Auth0 [python cdk] (#18338) * 🎉 New Source: Auth0 [python cdk] - Full import and incremental are supported - Only users stream is added - Added unit tests and ensure all acceptance tests are passed locally - It's free to create a new account in Auth0, so it's free to setup an integration account. * fix: remove unused import in source.py * auto-bump connector version Co-authored-by: sajarin <sajarindider@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Destination Redshift: handle empty s3 bucket_path (#18434) * make bucket_path required * check for nonempty path * not required * handle null/empty path * version bump + changelog * fix comment * auto-bump connector version * format Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Use Google business mails by default for login (#18462) * support custom error messaging for error response + retryable errors (#18204) * support custom error messaging for error response + retryable errors * remove changed backoff i was using for testing * refactor filter to construct response status internally * pr feedback * bump version and update changelog * Alternate way to view temp/CSV files on Windows. (#18482) Document alternate method, using Docker intrinsic functionality, to copy temp files locally in the situation that cannot be browsed through WSL2's file explorer. * docs: fix link reference (#18503) * 🐛Destination-elasticsearch: enforce ssl connection on cloud (#18341) * [16992] Destination-elasticsearch: enforce ssl connection on cloud * Bmoric/extract destination api (#18502) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * Fix PMD errors * Add explicit path to the controller * Extract destiantion API * bump to v15.0 (#18467) * bump to v15.0 * disable backwards compatible test for 0.2.69 * fix spec * skip comprehensive * bump * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Persist geography updates (#18501) * persist geography in connection update * use ApiPojoConverters * add test coverage for persisting geography update * persist geography column for workspace * use US instead of AUTO in mock data so that tests don't pass due to the database-level default column value * use ApiPojoConverters instead of Enum.convertTo in workspace handler * format * Add SSH tunnel for MongoDB Destination (#18280) * Add SSH tunnel for MongoDB Destination * Fixed host/port resolution issues with Docker containers * Formatting changes * Bump version number and adds documentation for SSH tunnel * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Improve performance of the operation that retrieve all connector for a given definition (#18499) * 🎉 New Source: News API (#18301) * Initial commit * Add streams * Simplify using options in request_parameters * Address comments * Add examples to query * Add docs * Change authenticator to ApiKeyAuthenticator * add seed config * remove icon * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 Source Hubspot: acceptance fix (#18498) * 🎉 New Source: Workable (low-code SDK) (#18033) * Generate initial Workable connector * Implement jobs route * Make acceptance tests pass * Implement pagination * Try to fix pagination (not yet working) * Make pagination work * Add created_after parameter * Add Workable docs * Move unused test * Add candidates stream * Fix acceptance tests * Remove unused files * Add stages stream * Add recruiters stream * Remove sample_state file * Add Workable to docs/integrations/README.md * Update changelog * Change Sentry page size back * Rename created_after_date to start_date * Remove data type map * Add start_date to sample_config.json * Add streams documentation to yaml * add title to params inspec * format files * fix: added source to source_definitions Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> * source-airtable: improve discovery logic (#18491) * 🎉 Destination Meilisearch: migrating to python and fix integration (#18036) * Add meilisearch python destionation * add destination * first working version * Writting tests * writer more resilient * Adding unit test * correct integration test stream param * update doc * remove unused variables * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 Source File: add retry logic for `Connection reset by peer - 104` error (#18428) * 🪟 🐛 Split Heading and Text components (#17992) * split Text and Heading components * replace Text with Heading where appropriate * 🐛 Source Facebook Marketing: increase timeout (#18536) * Bmoric/extract db migration api (#18459) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * tmp * Fix PMD errors * Extract DB migrator * Add something that I forgot * restore destination factory initialization * format * 🪟 Apply default sync mode logic to new streams (#18451) * brute force solution * cleanup * testing and comment * review cleanup * Bmoric/extract destination definition api (#18515) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * tmp * Fix PMD errors * Extract DB migrator * Add something that I forgot * extract destination definition api * restore destination factory initialization * format * format * explicit path * 🪟 🎨 Add `PillSelect` UI component (#18376) * Add PillButton UI component Update Text component to support xs size * Update PillButton prop names * Remove outline and tint from PillButton until it's resolved with design * Add PillButton example to popout * PillButton -> PillSelect Remove story from Popout Create PillSelect component that manages select logic Make PillButton part of PillSelect * Support text overflow in pill button * Show tooltip on pill select multiple items * Use border-thin var in PillButton style * Update gap in PillButton to use spacing var * extract destination definition specification api (#18521) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * tmp * Fix PMD errors * Extract DB migrator * Add something that I forgot * extract destination definition api * restore destination factory initialization * extract destination definition specification api * format * format * format * explicit path * Restore docs build (#18551) + Corrects two errors, unblocks docs build * add authentication info to local development guide (#18509) * add authentication info to local development guide * rephrase * 🪟 🐛 Fix UI issues in connection streams table (#18544) * Fix PathPopout styling * Ensure that text overflow still works * Only show tooltip when there is more than 1 * Use caret instad of sort icon so that it's centered * Fix option content for SyncSettignsDropdown so that it fits * Fix @forward on StreamHeader scss * Rollback tooltip updates * Simplify the OSS documentation deploy system (#2670) (#18436) + Updates tools/bin/deploy_docusaurus to work on Github Runners + Adds 'Deploy docs.airbyte.com' Github workflow * Simplify the OSS documentation deploy system (#2670) (#18448) + Updates Airbyte Docs: informs about automated Docs site build and deployment. * Source Iterable: distinguish 401 from empty stream (#18537) * #829 source iterable: distinguish 401 from empty stream * #829 source iterable: upd changelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Databricks destination: introduce personal staging location (#18557) * Update s3 parquet writer - Rename gcsFileLocation. - Merge local variables. * Implement staging location getter * Add integration test and fix post request * Add comment and format code * Ensure that restart delay is set before any other activity is executed (#18547) * Ensure that restart delay is set before any other activity is executed * Add version check * Formatting * 🎉 New Source: ActiveCampaign (#18335) * Init source ActiveCampaign * Added documentation files * Added contacts and campaigns streams * Added deals stream * added segments stream * Added lists stream * Formatted schemas * Added references to API docs * added forms stream * Added pagination * Updated documentation * Formatted schemas * Removed unnecessary files * Added changelog * Updated base URL * Added sample config * add source seed * run format * generate uuid for activecampaign * auto-bump connector version Co-authored-by: Rytis Mickevičius <rytis.mickevicius1@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * normalization: Fix failing mypy checks (#18534) * Use Protocol Versions in Syncs (#18324) * Pass protocol version into IntegrationLauncherConfig * Use VersionedStreamStreamFactory in AirbyteSource/Destination * Add AirbyteMessageBufferedWriter * Use VersionedBufferedWriter * 🪟 🎉 Client-side validation of cron expressions <= 1 hr for cloud (#18044) * add feature for cron expression format * add new validation function for sub one hr cron * add feature-dependent validation of cronExpression * add test coverage for create & edit connection forms * enable sub one hour cron expressions on cloud temporarily * Lowcode connectors: Fix few issues with stream slicers (#18529) * Fix a few bugs * bump * Protocol change: Define a set of well-known data types (#17486) * add WellKnownTypes.yaml * rename to snakecase + put in airbyte-protocol * add examples * more descriptoins * descriptions, more restrictions, better regex * update documentation * explicitly call out BC support * 🎉 New Source: Coin API (#18302) * Initial commit * Add streams * Correct acceptance test config * Make secret * Update docsurl * Remove catalog.json * Add second stream * Add default to spec * Change to start_date and end_date * Add links to CoinAPI docs in spec * Change to start_date and end_date * Add docs * add coin api to source seed file * remove icon * generate source spec for coin api Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> * 🎉 New Source: TVMaze Schedule (#18333) * Initial commit * Add streams * Remove file and update country code description * Add docs * format * correct schema * add tvmaze schedule to source spec seed * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * 🎉 New Source: MailerLite (#18336) * Init source MailerLite * Added documentation * Removed unnecessary files and a comment * Added changelog * Added sample config * Updated base URL * format * add mailerlite to source def seed * update docker image tag version * auto-bump connector version Co-authored-by: Dainius Salkauskas <dainiuxazz@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: Nasa APOD [python cdk] (#18394) * NASA APOD connector first implementation * Fix Acceptance and Unit tests * Fix DecrecationWarning on NoAuth * Update changelog with PR id * `date` input removal in favor of `start_date` + Minor fixes * Connector renaming to support different Nasa streams * Update docs and source definitions with connector rename * Support incremental updates * Fix typo on abnormal_state.json, making acceptance tests to fail * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Add missing generated python files (#18580) * 🎉 New Source - Lokalise [low-code SDK] (#18522) * Add keys stream * Fix keys stream for now * Start to add languages stream * Add docs * Start to add pagination * Remove paginator again * Revert "Remove paginator again" This reverts commit 3f8daace174473be767ef78d0fa852be81925203. * Pagination seems to work * Clean up yaml * Increase max page size * Docs improvements * Add comments stream * Add contributors stream * Fix contributors stream * Add translations source * Add new acceptance test format * Add abnormal state * Add PR number * run format * simplify selector * add eof * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: Oura (#18224) * Initial commit * Add streams * Final tidying * Make secret * Remove start_date and end_date parameters * Add docs * update cdk version * run format + add source def for oura * run config process manually Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> * 🎉 New Source: Ashby (#18334) * create boilerplate * configure connector * add ashby documentation * update airbyte docs * configure tests * Revert "update airbyte docs" This reverts commit 7dcf2b2dad1a77007b64aa8453aeb1c27e74ae58. * update airbyte documentation * update catalogs * add start date for specific streams * update config files * add ashby to source def seed * update doc * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source - Sonar Cloud (#18475) * init commit * add docs * add docs * fix comments * change string to array * 🎉 New source: SFTP Bulk [python cdk] (#17691) * [INIT] setup source-ftp Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [ADD] add logic Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [ADD] unit, integration, acceptance tests Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] update docs Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] clean up crew Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] update host to localhost Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] rename from ftp to sftp bulk Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [INIT] setup source-ftp Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [ADD] add logic Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [ADD] unit, integration, acceptance tests Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] update docs Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] clean up crew Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] update host to localhost Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] rename from ftp to sftp bulk Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * update branch * [FIX] integration tests Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [FIX] acceptance test fixture Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * [UPD] change ftp port Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * add uuid to image build * add source def * [FIX] generate ssh keys Signed-off-by: Henri Blancke <blanckehenri@gmail.com> * auto-bump connector version Signed-off-by: Henri Blancke <blanckehenri@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: Mailjet Mail API (#18332) * init commit * add streams * add schemas * add docs * add docs * fix comments * restore vscode config * restore vscode config * clean * lint * lint * lint * lint * fix comments Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * Sonar Cloud: add to source spec and publish connector (#18584) * solve conflict * solve conflict spec.yaml * add sonar cloud to seed + format * add eof * generate config source spec * Source Mailjet Mail: publish connector and additional format (#18586) * init commit * add streams * add schemas * solve conflict * add docs * fix comments * restore vscode config * restore vscode config * clean * lint * lint * lint * lint * fix comments * run format * update acceptance test * update schema * update acceptance test * auto-bump connector version Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Implement Connector Builder testing panel (first iteration) (#18449) * [SPIKE] add Builder page with editor * add YamlEditor component * export colors from CSS * move template into dedicated file * fix initial load from local storage * move download button into separate component * fix stylelint * remove console log * add todo * make monaco background transparent and apply gradient to parent div background * clarify comment * remove unnecessary 180deg * lock the builder UI behind a feature * use rgb instead of hex to fix stylelint * use _colors.scss and disable hex length stylelint rule * disable rule in file * generify ConnectorDocumentationLayout into TwoPanelLayout * use TwoPanelLayout for BuilderPage * add darker gradient color to _colors.scss and use in gradient * move /builder to /connector-builder * rename TwoPanelLayout to ResizablePanels * put gradient on parent container instead * make lightOverlay actually overlay instead of un-rendering children * add testing panel component and stream selector * fix rotated text * fix more styling * remove unnecessary comment * add starting flex to right panel * replace react-select dropdown with headless-ui listbox * add test controls and fix button icon styling * add Configure Test Input button and modal * add connector builder client * move connector builder state into dedicated context * fix bold prop in Text component to not throw error * pass parsed yaml->json into query * store config in connector builder state context * add result display * fix styles of result display * move config menu into testing panel * capitalize stream names * undo changes to Text component, since that is being addressed in a separate PR * fix more merge issues and fix generated schema usage * capitalize header * remove console log * rename data to streamRead * fix stylelint and use variables * use another variable for spacing * use stream name in state data object * move ConnectorBuilderStateProvider into ConnectorBuilderPage * fix sizing of modal * fix styling of active stream selector option * Update airbyte-webapp/src/components/StreamTestingPanel/ResultDisplay.tsx Co-authored-by: Tim Roes <tim@airbyte.io> * Update airbyte-webapp/src/components/StreamTestingPanel/ResultDisplay.tsx Co-authored-by: Tim Roes <tim@airbyte.io> * hide result display panel if no data * remove button specificity styling hacks, use size prop instead * Update airbyte-webapp/src/components/StreamTestingPanel/StreamSelector.tsx Co-authored-by: Tim Roes <tim@airbyte.io> * change logs to errors Co-authored-by: Tim Roes <tim@airbyte.io> * Source: Google Search Console: validate custom_reports.dimension (#18568) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * SAT: make `expect_records` mandatory in `high` `test_strictness_level` (#18497) * 🎉 New destination: Yugabytedb (#18039) * add yugabytedb destination connector * add dest def + add changelog * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * fix worker volumeMounts (#18604) * 🎉 New Source: RD Station Marketing [python cdk] (#18348) * Add new source: RD Station Marketing * Update catalog with all streams * Update changelog with the rigth PR reference number * Minor fixes to RD Station docs * Change field replication_start_date to start_date * Fix unit tests after changing replication_start_date name * Fix typo * Refact next_page_token logic * Fix unit tests * Remove print function * Update airbyte-cdk version to 0.2 * Update airbyte-integrations/connectors/source-rd-station-marketing/source_rd_station_marketing/streams.py Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * Change from yield to yield from * Apply format * add rd station to source def * update doc changelog * auto-bump connector version Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Simplify the OSS documentation deploy system (#2670) (#18598) + Adds Slack notification about failed Docs builds (slack notifications fall into "#oss-master-build-failure" for now) + (while here) Unbreaks docs build * improved the digitalocean deployment guide (#18566) * fixed failed build (#18610) * 🎉 New Source: ConvertKit (#18455) * Init source omnisend * Removed unnecessary files * Init source convertkit * Added forms, sequences streams * Added tags stream * Added subscribers, broadcasts streams * Added documentation * Removed unnecessary files * Updated pull request information in documentation * Added sample config * Updated invalid config * Formatting, removed abnormal state, uncommented full_refresh acceptance test * Added pagination for subscribers stream * fix: add source definition for convertkit * auto-bump connector version Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉New Source: Google Webfonts [low-code cdk] (#18496) * New source: Google webfonts * chore: Add Docs * chore: update changelog * chore: resolved given comments for PR * chore: unwanted files removed * fix: generate and add source definitions Co-authored-by: sajarin <sajarindider@gmail.com> * Clean up build.gradle. (#18555) Upstream has fixed this bug so we no longer need to host it internally. Remove this to clean up build.gradle. * 🎉 New Destination: Typesense (#18349) * Initial boilerplate * 🎉 New Destination: Typesense * remove .java-version * fix doc * add typesense to dest def * add release stage * add requirement to main * auto-bump connector version * add changelog Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: Waiteraid [low-code cdk] (#18165) * added source waiteraid * 🎉 New Source: Waiteraid * add searchBookings stream * add P/R number * add SUMMARY entry * add docs/integrations/README.md entry * add builds.md entry * add docs to each endpoint * fix schema ~ * Update airbyte-integrations/builds.md added web address Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * Delete catalog.json * Delete TODO.md * Update spec.yaml * add waiteraid to source def seed * auto-bump connector version Co-authored-by: Sebastian Brickel <sebastianbrickel@Sebastians-MacBook-Air.local> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Replace `recipesLink` link with `tutorialsLink` (#18616) * Protocol Change: `AirbyteControlMessage.ConnectorConfig` (#17907) * Protocol Change: AirbyteConfigMessage * update PR link in docs * Lint * Update python files * Update docs/understanding-airbyte/airbyte-protocol.md Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update docs/understanding-airbyte/airbyte-protocol.md Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * `AirbyteConfigMessage` -> `AirbyteConnectorConfigMessage` * AirbyteOrchestratorMessage * Update docs * `AirbyteControlConnectorConfigMessage` Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Remove workflow version check (#18613) * Improved the Oracle cloud deployment guide (#18615) * Update OCI VM deployment guide * Update OCI deployment guide * Update on oci-vm.md * 🐛 Lowcode: ListStreamSlicer and SubstreamSlicer should get the stream_slice from the arguments (#18574) * fix list stream slicer * get the stream slice from the parameters * delete unused code * Add comments * missing test * newline * with pytest.raise * reset to master * newline * Update __init__.py * Update __init__.py * 🎉 New Source: ConvertKit (#18455) * Init source omnisend * Removed unnecessary files * Init source convertkit * Added forms, sequences streams * Added tags stream * Added subscribers, broadcasts streams * Added documentation * Removed unnecessary files * Updated pull request information in documentation * Added sample config * Updated invalid config * Formatting, removed abnormal state, uncommented full_refresh acceptance test * Added pagination for subscribers stream * fix: add source definition for convertkit * auto-bump connector version Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉New Source: Google Webfonts [low-code cdk] (#18496) * New source: Google webfonts * chore: Add Docs * chore: update changelog * chore: resolved given comments for PR * chore: unwanted files removed * fix: generate and add source definitions Co-authored-by: sajarin <sajarindider@gmail.com> * Clean up build.gradle. (#18555) Upstream has fixed this bug so we no longer need to host it internally. Remove this to clean up build.gradle. * 🎉 New Destination: Typesense (#18349) * Initial boilerplate * 🎉 New Destination: Typesense * remove .java-version * fix doc * add typesense to dest def * add release stage * add requirement to main * auto-bump connector version * add changelog Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Source: Waiteraid [low-code cdk] (#18165) * added source waiteraid * 🎉 New Source: Waiteraid * add searchBookings stream * add P/R number * add SUMMARY entry * add docs/integrations/README.md entry * add builds.md entry * add docs to each endpoint * fix schema ~ * Update airbyte-integrations/builds.md added web address Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * Delete catalog.json * Delete TODO.md * Update spec.yaml * add waiteraid to source def seed * auto-bump connector version Co-authored-by: Sebastian Brickel <sebastianbrickel@Sebastians-MacBook-Air.local> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Replace `recipesLink` link with `tutorialsLink` (#18616) * Protocol Change: `AirbyteControlMessage.ConnectorConfig` (#17907) * Protocol Change: AirbyteConfigMessage * update PR link in docs * Lint * Update python files * Update docs/understanding-airbyte/airbyte-protocol.md Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update docs/understanding-airbyte/airbyte-protocol.md Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * `AirbyteConfigMessage` -> `AirbyteConnectorConfigMessage` * AirbyteOrchestratorMessage * Update docs * `AirbyteControlConnectorConfigMessage` Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Remove workflow version check (#18613) * Improved the Oracle cloud deployment guide (#18615) * Update OCI VM deployment guide * Update OCI deployment guide * Update on oci-vm.md * bump Co-authored-by: Dainius Salkauskas <dainiuxazz@gmail.com> Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Balasubramanian T K <btk.codedev@gmail.com> Co-authored-by: Davin Chia <davinchia@gmail.com> Co-authored-by: Cirdes <cirdes@linkana.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: sbrickel-nimble <113671803+sbrickel-nimble@users.noreply.github.com> Co-authored-by: Sebastian Brickel <sebastianbrickel@Sebastians-MacBook-Air.local> Co-authored-by: Evan Tahler <evan@airbyte.io> Co-authored-by: Sherif A. Nada <snadalive@gmail.com> Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io> Co-authored-by: Victor Ikechukwu <vickyikechukwu13@gmail.com> * Add versioning logging (#18618) * Bmoric/extract healt api (#18523) * Tmp * Extract the Attempt API from the V1 API * Add comments * Move Connection API out of configuration API * format * format * Rename to Controller * Rename to Controller * Add values to the factory * Change the constructor to use hadler instead of objects needed by the handler * Update with new tags. * tmp * Fix PMD errors * Extract DB migrator * Add something that I forgot * extract destination definition api * restore destination factory initialization * extract destination definition specification api * format * format * format * extract health check api * fix test * format * Fix ConfiguredCatalog for Resets (#18625) We should only force (FULL_REFRESH,OVERWRITE) for the streams to reset. For the other streams, we only want to replace OVERWRITE with APPEND to avoid having destination clear the stream. The other cases should be left as is. * :tada: New Source: Mailjet SMS (#18345) * init commit * add doc * add doc * change authentication methode * fix doc * fix doc * add docs * fix comments * use request_options_provider * lint * fix comments Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> * More comprehensive temporal error message (#18608) * To be remove * Remove the signal for waiting after a failed activity and ensure we are waiting the expected time * Revert "To be remove" This reverts commit 3a5f7b4f729724e6943450af15b324ca05ac6c4d. * Remove unused and move failure reason to the helper * Avoid repetitive new Set() * :tada: New Connector: Zendesk Sell [python cdk] (#17888) * first working connector - overwrite only * edit readme and source definitions * Update source_definitions.yaml * changing catalog and adding all available streams * remove custom fields * remove documents * manage incremental syncs * add unit tests ; add start date to specs * try catch for next page token method * Create source_to_test.py * Update source.py * Update source_to_test.py * working source * Delete TODO.md * Update source.py * Update spec.yaml * Update test_streams.py * give up on incremental and have something that works * removing start date necessity in spec and sample configs * add connector to source def * run format * update airbyte cdk version * correct unit test * run format * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Mailjet SMS: publish PR and add source def (#18620) * init commit * add docs * fix comments * use request_options_provider * solve conflict * solve doc conflict * Source Mailjet Mail: publish connector and additional format (#18586) * init commit * add streams * add schemas * solve conflict * add docs * fix comments * restore vscode config * restore vscode config * clean * lint * lint * lint * lint * fix comments * run format * update acceptance test * update schema * update acceptance test * auto-bump connector version Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * add source def + correct tests * update schema Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Add endpoint to retrieve manifest template (#18578) * add endpoint to retrieve manifest template * remove 400 error from endpoint * Source S3: use AirbyteTracedException (#18602) * #750 # 837 #904 Source S3: use AirbyteTracedException * source s3: upd changelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Fix source spec diff (#18645) * Rename maxCursor and add comment (#18570) * Rename maxCursor to make the code easier to understand * Rename and update comment * ami-0d648081937c75a73 -> ami-06cf12549e3d9c522 (#18667) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * 🐛 Source Google Analytics (Universal Analytics): add `Custom Reports` schema validation on `Check Connection` (#18670) * 🪟 🐛 Fix trailing whitespace and ending with a forward slash transformation GitHub URL issue (#18299) * add regex matcher for custom transformation GitHub url * move validationSchema to separate file * add tests for validationSchema * 🎉 New Source: Omnisend (#18577) * Init source omnisend * Removed unnecessary files * added campaigns, carts, orders, products streams * Fix acceptance tests * Added documentation * Added pagination * Removed abnormal state, uncommented acceptance tests * Added docs references * Added reference to PR * Added stop condition for pagination * Added title to API Key in spec.yaml * add omnisend source to def seed * run format * auto-bump connector version Co-authored-by: Rytis Mickevičius <rytis.mickevicius1@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Added a note about chained dbt transformations (#18678) * Source metabase: allow using only https urls (#18607) * #16519 source metabase: allow using only https urls * source metabase: upd changelog * #16519 fix url regexp * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🪟 🎉 Add sync mode, primary key, cursor select components to new streams table (#18627) * Add SyncModeSelect component Add button type to pill button and stop propagation Stop propagation on popoout outside click listener and migrate to scss module * Update DropDown option to handle click event directly and stop propagation * Add StreamPathSelect component to select the primary key and cursor paths Update PillSelect to handle nil values * Add SyncMode, Primary Key, and Cursor selects to CatalogTreeTableRow * Replace popup click catcher with Overlay component Add Overlay color variant and onClick handler option Fix Overlay import * Source Intercom: change airbyte-cdk version to 0.2 (#18681) * Source Intercom: change airbyte-cdk version to 0.2 * Source Intercom: update chagelog * Source Intercom: bump version and update chagelog * Source Intercom: Fix PR id in chagelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * feat: Add airbyte-cron Helm chart (#18542) * feat: Add airbyte-cron Helm chart * New chart resides in `charts/airbyte-cron` * Updated the parent chart (`airbyte`) `values.yaml` to include config key for `cron` * Add airbyte-cron to release workflow * chore: remove TEMPORAL_* vars Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com> * fix: Typo in semver format for chart (#18686) This looks to have been a mistaken change that was made to the workflow. Reverting back to the original value. * 🐛 Source Amplitude: fix empty `series` for `AverageSessionLength` stream (#18684) * Destination Redis: TLS support (#18358) * 16249 Destination Redis TLS support * Refactoring based on pr review * Remove duplicate paragraph (#18662) Removes a duplicate of the paragraph that begins "Note that we're also setting the `stream_cursor_field` in the stream's `$options`..." which followed an outdated schema. * Simplify the job persistence method. (#18698) Method interface: - has generic type that isn't needed. Confusing because no generic is happening here. - has two existing parameters that are data fields of one of the already existing parameters. Confusing since the two additional parameters aren't actually supposed to be passed in separately from the existing parameter. Remove both. * chore: Add airbyte-cron chart to .bumpversion.cfg (#18688) * fix helm release workflow (#18716) * AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * Remove unused test script. (#18715) * ci: replace set-output with new github actions syntax (#18691) * Sources postgres, mysql, alloydb: encode database name (#18538) * Source postgres: encode database name * Source postgres, mysql: move encoding in util class, apply for mysql * Source postgres, mysql: make var final * Source postgres, mysql: bump version * Source postgres, mysql: format code * auto-bump connector version * Source mysql: bump version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Prestashop: prevent users from using http url (#18599) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * add nonBreakingChangePreference and notifySchemaChanges (#18636) * add nonBreakingChangePreference and notifySchemaChanges * Changes docs deploy failures to show up in #docs (#18738) * Green house docs (#18314) * corrected the documentation for FreshDesk Connector * corrected the documentation for Greenhouse Connector * Resurfaces hidden cloud connectors within UI (#18748) * Update helpers functions in subcharts (#18757) * Update helpers functions in subcharts; Refactor component.imageTag function call * fix typo in _helpers.tpl * Update chart.yaml to reference right subchart (#18718) * Update chart.yaml to reference right subchart * Update airbyte-cron chart name * 🎉 New Destination: Heap Analytics [python cdk] (#18530) * 🎉 New Desination: Heap Analytics [python cdk] - implement a heap client to load data via the server-side API: https://developers.heap.io/reference/server-side-apis-overview - the connector supports a generic data source, and the api_type determines the output schema. The output schema is dynamic. - users pick the columns that will be loaded to the destination - Consequently, each configured catalog only includes one stream * add a bootstrap to illustrate the connector * add destination dest def * run format all files * correct unit test * auto-bump connector version Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Salesforce: add get_error_display_message for ConnectionError (#18753) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * fix helm release workflow (#18776) * Update to latest Micronaut release (#18774) * fix cron resources spec (#18778) * ci: use oldgithub actions notation for output in publish-helm-charts worklow (#18780) * improve connector dependency github action (#18480) * Use more specific variable name * Write affected sources and destinations to files * Use Markdown template to format results * Update workflow to edit existing comment instead of making a new one * Check whether changed_files.txt is empty before proceeding * move template to .github/ * Use different quotes for true * Move logic about continuing back to after dependency report because of ignores * Use correct filename in conditional * Use if to avoid early exit * Create folder for comment templates and use constant for file path * Add handling for non-source non-destination info, pull out markdown listing * 🐛 Add a drop table hook to drop scd tables in case of overwrite sync (#18015) * Add a drop table hook to drop scd tables in case of overwrite sync * Add an integration test for dropping SCD table on overwrite * skip new test for Oracle and TiDB * Add normalization run after initial reset * Bump normalization version * Move components to `components/common` (#18728) * Move ApiErrorboundary to components/common and cleanup * Move PageViewContainer to…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Resolves #17812
This PR adds the necessary configuration to generate the connector builder API client code from the open API spec.
How
This PR does the following:
orval.config.tsto auto-generate Connector Builder API client code, and ignores the generated file in gitignore (similar to AirbyteClient)connectorBuilderApiUrlentry todefaultConfig.tsand its corresponding type. This is used in the Api service code to create the service, but won't actually be used yet since responses are currently being mockedConnectorBuilderApiService, similar to our other AirbyteRequestServices, which implements the react-query calls to the generated api client code. However, this file currently does not actually call the real API and instead just mocks out the response data, because the connector builder server has not yet been implemented. This class will be used in a follow-up PR, but I wanted to keep this PR smaller in scope first.