Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Authentication Error handling #21087

Merged
merged 179 commits into from
Jan 23, 2023
Merged
Changes from 1 commit
Commits
Show all changes
179 commits
Select commit Hold shift + click to select a range
2bc0f2c
refactor checks
jbfbell Jan 5, 2023
2de7385
resolve conflicts
jbfbell Jan 6, 2023
f003856
Add Auth Error Checks
jbfbell Jan 12, 2023
8130a40
rebase from master
jbfbell Jan 12, 2023
66a8201
some weird versioning issue
jbfbell Jan 12, 2023
d684daf
🪟🎉 Connector builder: Substream slicer and cartesian slicer (#20861)
Jan 5, 2023
fc66209
🪟🔧 Connector builder: Performance improvements (#20620)
Jan 5, 2023
7beef18
Remove workspace helper from fetchConfigActivity (#21048)
alovew Jan 5, 2023
ca10a01
Publish new version of destination-redshift (#21083)
grishick Jan 5, 2023
6f8e578
Progress Bar Read APIs (#20937)
davinchia Jan 6, 2023
88c3518
Rm temporal version (#21045)
benmoriceau Jan 6, 2023
de1a587
🪟🎉 Connector builder: Available inputs dropdown (#20983)
Jan 6, 2023
3387d09
Fixed appstore docs link (#21098)
Amruta-Ranade Jan 6, 2023
e0fd67b
fill in all default values on switch (#21059)
Jan 6, 2023
fb60113
🐛Source Freshdesk: Fix schema types (#20416)
lazebnyi Jan 6, 2023
4437101
CDK: Add schema inferrer class (#20941)
Jan 6, 2023
9bb61bb
Source Facebook Marketing: Docs update (#21105)
artem1205 Jan 6, 2023
23ef1c1
Source Amazon Ads: fix bug with handling: "Report date is too far in …
grubberr Jan 6, 2023
a7a0a8f
Revert "Progress Bar Read APIs (#20937)" (#21115)
supertopher Jan 6, 2023
10c67fd
Remove unneeded margin top and bottom (#21111)
krishnaglick Jan 6, 2023
15e7e76
feat(Platform): update actor configuration when receiving control mes…
pedroslopez Jan 6, 2023
e6306bc
🐛Source Looker: Fix schema transformation issue (#20182)
lazebnyi Jan 6, 2023
8b6e557
[ISSUE #20322] add datetime_granularity logic to DatetimeStreamSlicer…
maxi297 Jan 6, 2023
4fd4537
Source Slack: update schema; data from openapi spec (#20767)
artem1205 Jan 6, 2023
06788f0
make sure name is rendered in first position (#21101)
Jan 6, 2023
6403184
Source Github: Raise Error if no organizations or repos are available…
artem1205 Jan 6, 2023
c071fc5
Convert the server to micronaut (#19194)
benmoriceau Jan 6, 2023
dd9b783
🪟 🐛 🎨 Fix stream table icon checkboxes and icons (#21108)
edmundito Jan 6, 2023
18ea9ec
Use explicit configuration for Micronaut endpoints (#20759)
jdpgrailsdev Jan 6, 2023
f76ba9b
process config control messages during `check` and `discover` (#20894)
pedroslopez Jan 6, 2023
7c2ff54
🪟🧹 Connector builder: Fast fields performance improvements (#20957)
Jan 6, 2023
98042f0
In Cloud, out of date connectors call always returns 0 (#21126)
krishnaglick Jan 6, 2023
7132633
Move hadoop-lzo to test dependency (#21085)
grishick Jan 6, 2023
320ed3f
Update Connector Builder docs to reflect configuration UI additions (…
lmossman Jan 6, 2023
b7e2948
Revert "Convert the server to micronaut (#19194)" (#21132)
benmoriceau Jan 6, 2023
44ebed3
🐛 Source Google Ads: publish missing `0.2.6` (#21103)
bazarnov Jan 6, 2023
836e2e5
Remerge Progress Bar Read API. (#21124)
davinchia Jan 6, 2023
0fc73d8
Bump Airbyte version from 0.40.26 to 0.40.27 (#21135)
octavia-squidington-iii Jan 6, 2023
8b61e77
add import rule changes for Kotlin code (#21136)
colesnodgrass Jan 6, 2023
0ab8432
Source Okta: OAuth2.0 support - disabled (#20877)
lazebnyi Jan 6, 2023
4f14b79
Remove the quarantine status (#21088)
benmoriceau Jan 6, 2023
972794a
normalization: delete supportsDbt and supportsNormalization from Dest…
alafanechere Jan 7, 2023
c779c49
Allow for custom requesters to be defined in low-code manifests (#21001)
brianjlai Jan 7, 2023
0cc751e
adjust readme (#20945)
Jan 9, 2023
961aae0
🪟🎨 Connector form: Improve logs look and feel (#20951)
Jan 9, 2023
44daeb5
Adjust GitHub flow for FE chapter (#20813)
timroes Jan 9, 2023
ad140b5
Source: Google Analytics 4 (GA4) - improve config validation and SAT …
grubberr Jan 9, 2023
6becd86
Simplify messaging for sync canceled jobs (#20999)
edmundito Jan 9, 2023
43baa05
Remove deprecated getter (#21089)
benmoriceau Jan 9, 2023
98da7ae
Import error broke master (#21165)
krishnaglick Jan 9, 2023
00780e9
allow the cursors and primary keys to be deselected when the sync mod…
mfsiega-airbyte Jan 9, 2023
9a9ca55
Connector builder server: Add inferred schema to read API response (#…
Jan 9, 2023
ecdd732
New destination: databend (community PR #19815) (#20909)
grishick Jan 9, 2023
252333c
Update postgres.md (#21170)
evantahler Jan 9, 2023
5360239
Source Facebook Marketing: Update schema ad_account (#21149)
artem1205 Jan 9, 2023
c83b293
Revert "Revert Convert the server to micronaut" (#21133)
benmoriceau Jan 9, 2023
afb63cc
Amazon Seller Partner: Validate connections without orders data (#20896)
animer3009 Jan 9, 2023
3673aab
Source Harvest: Skip 403 FORBIDDEN for all streams (#21151)
artem1205 Jan 9, 2023
de8a177
test for behavior when a column is removed in an incremental sync (#2…
davinchia Jan 9, 2023
4c60cb5
Source Mailchimp - fix the way request params are built (#20765)
davydov-d Jan 9, 2023
07c4fbf
🐛 🎉 Source Airtable: migrate to the `Metadata API` for dynamic schema…
bazarnov Jan 9, 2023
cd00b48
Extend connection list filtering (#21094)
gosusnp Jan 9, 2023
77f463d
test cleanup (#21178)
mfsiega-airbyte Jan 9, 2023
81947aa
removed error (#20612)
sophia-wiley Jan 10, 2023
a93ac03
Adjust connector icons (#20547)
timroes Jan 10, 2023
fa548e9
Use jest directly as test runner (#21174)
timroes Jan 10, 2023
85c042c
🪟 🎨 Refining BulkEditPanel component (#20474)
YatsukBogdan1 Jan 10, 2023
2b11350
🪟 🔧 Add tests for BulkEditService (#20820)
YatsukBogdan1 Jan 10, 2023
14b618f
Connector builder: E2e tests (#21122)
Jan 10, 2023
edbde3b
🎉 Source Airtable: update `releaseStage` to `beta` (#20935)
bazarnov Jan 10, 2023
d780fe9
🪟 🎉 Disable deselection of cursor field/primary key in the UI (#20844)
josephkmh Jan 10, 2023
c3d842f
🪟 🔧 Update frontend typing of cloud api response (#21086)
josephkmh Jan 10, 2023
b68d130
🪟 🔧 Use CSS Custom Properties for colors (#19344)
josephkmh Jan 10, 2023
7e0b526
ci-connector-ops: change required reviewers logic (#21158)
alafanechere Jan 10, 2023
af5faad
[ISSUE-20322] updating tutorial documentation (#21163)
maxi297 Jan 10, 2023
2ae3d51
Connector builder server: Fix unit tests (#21107)
Jan 10, 2023
c83ffff
Destination connector for teradata vantage (rebased community PR) (#2…
grishick Jan 10, 2023
53557c3
🎉 Updates for edit password field in connector (#20723)
natalyjazzviolin Jan 10, 2023
1a25de2
convert EnvVariableFeatureFlag to an injected dependency (#21171)
colesnodgrass Jan 10, 2023
e42355b
Updated ISO certification date (#21181)
sophia-wiley Jan 10, 2023
6a336fe
Source google ads: add more logs (#20755)
davydov-d Jan 10, 2023
4a2a5eb
🎉 Destination Local CSV: add custom delimiter (#17998)
natalyjazzviolin Jan 10, 2023
de748b8
Source Airtable: fix field names (#21215)
artem1205 Jan 10, 2023
31e1860
Source S3: update block size for json (#21210)
artem1205 Jan 10, 2023
29e2a30
🪟 🧹 Custom connectors in Cloud UI updates (#21034)
teallarson Jan 10, 2023
66a475d
lowcode docs: add link to YAML reference (#21231)
sherifnada Jan 10, 2023
167e528
Update platform-workflow-labels.yml (#21225)
colesnodgrass Jan 10, 2023
18c1776
Update README.md (#21236)
evantahler Jan 11, 2023
a8a10bb
Enforce HTTPS (#21182)
grishick Jan 11, 2023
26a556d
🪟🐛 Connector builder UI: Fix datetime stream slicer (#21161)
Jan 11, 2023
9918386
[ISSUE #19981] testing version bump (#21106)
maxi297 Jan 11, 2023
b911717
[Low-Code CDK] Handle forward references in manifest (#20893)
clnoll Jan 11, 2023
be86bd5
Update form field labels in connection form to match design (#21036)
Jan 11, 2023
4821113
Hide no credits banner for no billing accounts (#21218)
timroes Jan 11, 2023
3437500
Fix auto detect schema change backdrop color CSS (#21246)
edmundito Jan 11, 2023
e15bb43
Source notion: fix schema (#20639)
davydov-d Jan 11, 2023
68f3858
[Connector-builder server] Allow client to specify record limit and e…
clnoll Jan 11, 2023
43e3a24
Upgrade to Micronaut 3.8.0 and related dependencies (#21077)
jdpgrailsdev Jan 11, 2023
0084339
Source Google Analytics Data API: slicer updated, unit tests added (#…
grubberr Jan 11, 2023
94021fa
Bmoric/restore old interface (#21235)
benmoriceau Jan 11, 2023
77b08a4
Switch CI to use npm ci (#21259)
timroes Jan 11, 2023
76585a9
postgres-source-cdc: handle null values for array data types (#21003)
subodh1810 Jan 11, 2023
307c499
🪟 🐛 Fix broken switch UI state - when checked state provided as a "va…
dizel852 Jan 11, 2023
b578675
🪟 🔧 Use experiment for new streams table design (#21230)
edmundito Jan 11, 2023
9df3e0a
Source Snapchat Marketing: fix error response (#21267)
artem1205 Jan 11, 2023
a2bc9e4
[Low-Code CDK] Parse incoming manifest objects into Pydantic models (…
brianjlai Jan 11, 2023
b280df2
[Low Code CDK] Add type annotations and cleanup usage of `visited` se…
clnoll Jan 12, 2023
9c32277
feat: mention user on failed master branch build (#21201)
cuyk Jan 12, 2023
64702d7
🪟 🔧 Fixed passed props to `<CheckBoxControl />` which cause error in …
dizel852 Jan 12, 2023
2de2bb6
Connector builder server: Fail on failing tests (#21198)
Jan 12, 2023
5bdc39c
Source Google Ads: Update docs (#21308)
artem1205 Jan 12, 2023
c97685c
Migrate airbyte-bootloader to Micronaut (#21073)
jdpgrailsdev Jan 12, 2023
45c2f20
🪟 🧹 Reorganize connections pages (#20845)
edmundito Jan 12, 2023
cf876b1
Add bootloader/server to published images for Cloud (#21325)
jdpgrailsdev Jan 12, 2023
9ba1fce
🪟 🚦 E2E: Fix duplicated database object warning (#20926)
dizel852 Jan 12, 2023
6eaec8b
Run Airbyte CI on pull_request (#21266)
Jan 12, 2023
026e857
🎉 Source bigcommerce: add brands and categories streams (#20518)
PhilipCorr Jan 12, 2023
3a4ac04
🎉 New Destination: Weaviate (#20094)
samos123 Jan 12, 2023
a7089da
Make schema field in source-snowflake mean a subset of the specified …
rodireich Jan 12, 2023
a558b8d
Ensure Local Secret Persistence is Initialized (#21329)
jdpgrailsdev Jan 12, 2023
b336d6c
December release notes docs (#21270)
sophia-wiley Jan 12, 2023
7d3d837
Add dependencies to docker-compose (#19321)
gosusnp Jan 12, 2023
f715310
store env-var response (#21327)
colesnodgrass Jan 12, 2023
021b715
Better escape the select query for a case a schema name starts with a…
rodireich Jan 12, 2023
db25e3c
manually bump source definition version (#21349)
rodireich Jan 12, 2023
507a90a
:bug: Source Jira: `check_connection` fails if no projects are define…
philippeboyd Jan 12, 2023
8063c35
airbyte-common-workers: Collect trace message on failed connection_st…
alafanechere Jan 12, 2023
1024e3f
🪟 🎉 Verify auth status on tab focus (#21175)
teallarson Jan 12, 2023
8b943a7
Make documentation_url optional in a declarative connector spec (#21347)
lmossman Jan 12, 2023
a79fe8c
Avoid parsing a message if this message is too big (#21090)
benmoriceau Jan 13, 2023
bcad152
JDBC Destinations: improve error message for conflicting streams (#21…
edgao Jan 13, 2023
1f0c569
Bump Airbyte version from 0.40.27 to 0.40.28 (#21359)
octavia-squidington-iii Jan 13, 2023
4d5ed05
Add records committed to job stats (#21364)
gosusnp Jan 13, 2023
a060797
[Low-Code CDK] Construct declarative components from Pydantic models …
brianjlai Jan 13, 2023
3ac31da
update destination-python generator (#21298)
alafanechere Jan 13, 2023
8b65da1
fixed current_value in usage_triggers stream (#21304)
darynaishchenko Jan 13, 2023
1441218
🪟 🔧 Add stylelint plugin & no-color-variables-in-rgba rule (#21269)
josephkmh Jan 13, 2023
974915e
fix e2e tests (#21380)
Jan 13, 2023
f956265
🪟🎉 Connector builder: Schema inferrer UI (#21154)
Jan 13, 2023
c1f1175
🎉 New Source: Dremio (#19912)
arsenlosenko Jan 13, 2023
f467803
Remove `supportsNormalization` `supportsDbt` from non GA destination …
alafanechere Jan 13, 2023
3396fce
SAT: check `future_state` only for incremental streams. (#21248)
grubberr Jan 13, 2023
c8f0e1c
:tada: Typeform - beta certification (#21145)
midavadim Jan 13, 2023
16c2f2c
Update databend.md (#21179)
hantmac Jan 13, 2023
30d49c0
<Callout /> component (#21273)
teallarson Jan 13, 2023
7900e2c
Update to Micronaut 3.8.1 (#21388)
jdpgrailsdev Jan 13, 2023
43109da
Source Slack: retry timeout error (#21321)
artem1205 Jan 13, 2023
e91dc6c
Source Facebook Marketing: Videos stream remove filtering (#21385)
artem1205 Jan 13, 2023
bced390
Source Klaviyo: enable SAT high strictness (#21374)
arsenlosenko Jan 13, 2023
2196c7a
Source File: enable high strictness level (#21243)
roman-yermilov-gl Jan 13, 2023
e68b907
introduce code coverage report step (#21376)
mfsiega-airbyte Jan 13, 2023
fc1bdc8
added edits (#21346)
sophia-wiley Jan 13, 2023
c3a2e63
🪟 🐛 Fix jest tests to fail build correctly (#21408)
timroes Jan 13, 2023
218b42a
Allow the OSS server only on OSS (#21416)
benmoriceau Jan 13, 2023
2256412
Update create connection page to full width, and fix trivial UI issue…
edmundito Jan 13, 2023
6aa54ac
Improvements to edge cases of CheckStream (#21404)
erohmensing Jan 13, 2023
d174dd0
Add AllowedHosts to Actor Definitions and Config Database (#21363)
evantahler Jan 13, 2023
beae36c
Update on-gcp-compute-engine.md (#21429)
gosusnp Jan 13, 2023
eb07473
refactor checks
jbfbell Jan 5, 2023
3646cda
rebase from master
jbfbell Jan 12, 2023
6f3fcc8
PR Feedback
jbfbell Jan 13, 2023
c9e254f
PR feedback
jbfbell Jan 13, 2023
6783d58
conflicts
jbfbell Jan 14, 2023
18bf629
revert unintended version downgrade
jbfbell Jan 14, 2023
0e4a808
merge conflict resolution
jbfbell Jan 18, 2023
50b38ba
add version updates and changelog
jbfbell Jan 18, 2023
046f3fb
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 19, 2023
79e0245
auto-bump connector version
octavia-squidington-iii Jan 19, 2023
2f91349
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 20, 2023
c4d9892
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 20, 2023
4302b67
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 20, 2023
5c6f5b4
auto-bump connector version
octavia-squidington-iii Jan 20, 2023
159ca81
formatting
jbfbell Jan 20, 2023
5c70686
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 20, 2023
680f211
Merge branch 'master' into joseph.bell/20092/config-exception-for-auth
jbfbell Jan 20, 2023
5177665
bump snowflake destination version
jbfbell Jan 20, 2023
9244363
manual version bump for s3
jbfbell Jan 20, 2023
a22e2a3
auto-bump connector version
octavia-squidington-iii Jan 21, 2023
fd1537e
auto-bump connector version
octavia-squidington-iii Jan 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Allow for custom requesters to be defined in low-code manifests (#21001)
* Allow for custom requesters to be defined in low-code manifests

* add test for custom requester component

* bump versions and changelog
  • Loading branch information
brianjlai authored and jbfbell committed Jan 13, 2023
commit c779c49e9b7ca7fe9b88caa7b2faac0f38f4d9d4
3 changes: 3 additions & 0 deletions airbyte-cdk/python/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Changelog

## 0.18.1
Allow for CustomRequester to be defined within declarative manifests

## 0.18.0
Adding `cursor_granularity` to the declarative API of DatetimeStreamSlicer

Original file line number Diff line number Diff line change
@@ -273,6 +273,23 @@ definitions:
$options:
type: object
additionalProperties: true
CustomRequester:
description: Requester component whose behavior is derived from a custom code implementation of the connector
type: object
additionalProperties: true
required:
- type
- class_name
properties:
type:
type: string
enum: [CustomRequester]
class_name:
type: string
additionalProperties: true
$options:
type: object
additionalProperties: true
CustomRetriever:
description: Retriever component whose behavior is derived from a custom code implementation of the connector
type: object
@@ -963,7 +980,9 @@ definitions:
record_selector:
"$ref": "#/definitions/RecordSelector"
requester:
"$ref": "#/definitions/HttpRequester"
anyOf:
- "$ref": "#/definitions/CustomRequester"
- "$ref": "#/definitions/HttpRequester"
name:
type: string
default: ""
Original file line number Diff line number Diff line change
@@ -99,6 +99,15 @@ class Config:
_options: Optional[Dict[str, Any]] = Field(None, alias="$options")


class CustomRequester(BaseModel):
class Config:
extra = Extra.allow

type: Literal["CustomRequester"]
class_name: str
_options: Optional[Dict[str, Any]] = Field(None, alias="$options")


class CustomRetriever(BaseModel):
class Config:
extra = Extra.allow
@@ -461,7 +470,7 @@ class ParentStreamConfig(BaseModel):
class SimpleRetriever(BaseModel):
type: Literal["SimpleRetriever"]
record_selector: RecordSelector
requester: HttpRequester
requester: Union[CustomRequester, HttpRequester]
name: Optional[str] = ""
paginator: Optional[Union[DefaultPaginator, NoPagination]] = None
primary_key: Optional[PrimaryKey] = None
2 changes: 1 addition & 1 deletion airbyte-cdk/python/setup.py
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@

setup(
name="airbyte-cdk",
version="0.18.0",
version="0.18.1",
description="A framework for writing Airbyte Connectors.",
long_description=README,
long_description_content_type="text/markdown",
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

from airbyte_cdk.sources.declarative.requesters import HttpRequester


class SampleCustomComponent(HttpRequester):
"""
A test class used to validate manifests that rely on custom defined Python components
"""

pass
Original file line number Diff line number Diff line change
@@ -94,7 +94,31 @@ def test_valid_manifest(self):
},
"record_selector": {"extractor": {"field_pointer": ["result"]}},
},
}
},
{
"type": "DeclarativeStream",
"$options": {"name": "stream_with_custom_requester", "primary_key": "id", "url_base": "https://api.sendgrid.com"},
"schema_loader": {
"name": "{{ options.stream_name }}",
"file_path": "./source_sendgrid/schemas/{{ options.name }}.yaml",
},
"retriever": {
"paginator": {
"type": "DefaultPaginator",
"page_size": 10,
"page_size_option": {"inject_into": "request_parameter", "field_name": "page_size"},
"page_token_option": {"inject_into": "path"},
"pagination_strategy": {"type": "CursorPagination", "cursor_value": "{{ response._metadata.next }}"},
},
"requester": {
"type": "CustomRequester",
"class_name": "unit_tests.sources.declarative.external_component.SampleCustomComponent",
"path": "/v3/marketing/lists",
"custom_request_parameters": {"page_size": 10},
},
"record_selector": {"extractor": {"field_pointer": ["result"]}},
},
},
],
"check": {"type": "CheckStream", "stream_names": ["lists"]},
}