-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Connector Builder] Add paginator (#20698)
* move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly Co-authored-by: Joe Reuter <joe@airbyte.io>
- Loading branch information
Showing
14 changed files
with
420 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,4 +18,5 @@ | |
| .form { | ||
| flex: 1; | ||
| padding: variables.$spacing-xl; | ||
| overflow: auto; | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
49 changes: 49 additions & 0 deletions
49
airbyte-webapp/src/components/connectorBuilder/Builder/InjectRequestOptionFields.tsx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| import { useField } from "formik"; | ||
|
|
||
| import { RequestOption } from "core/request/ConnectorManifest"; | ||
|
|
||
| import { injectIntoValues } from "../types"; | ||
| import { BuilderField } from "./BuilderField"; | ||
|
|
||
| interface InjectRequestOptionFieldsProps { | ||
| path: string; | ||
| descriptor: string; | ||
| excludeInjectIntoValues?: string[]; | ||
| } | ||
|
|
||
| export const InjectRequestOptionFields: React.FC<InjectRequestOptionFieldsProps> = ({ | ||
| path, | ||
| descriptor, | ||
| excludeInjectIntoValues, | ||
| }) => { | ||
| const [field, , helpers] = useField<RequestOption>(path); | ||
|
|
||
| return ( | ||
| <> | ||
| <BuilderField | ||
| type="enum" | ||
| path={`${path}.inject_into`} | ||
| options={ | ||
| excludeInjectIntoValues | ||
| ? injectIntoValues.filter((val) => !excludeInjectIntoValues.includes(val)) | ||
| : injectIntoValues | ||
| } | ||
| onChange={(newValue) => { | ||
| if (newValue === "path") { | ||
| helpers.setValue({ inject_into: newValue, field_name: undefined }); | ||
| } | ||
| }} | ||
| label="Inject into" | ||
| tooltip={`Configures where the ${descriptor} should be set on the HTTP requests`} | ||
| /> | ||
| {field.value.inject_into !== "path" && ( | ||
| <BuilderField | ||
| type="string" | ||
| path={`${path}.field_name`} | ||
| label="Field name" | ||
| tooltip={`Configures which key should be used in the location that the ${descriptor} is being injected into`} | ||
| /> | ||
| )} | ||
| </> | ||
| ); | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
164 changes: 164 additions & 0 deletions
164
airbyte-webapp/src/components/connectorBuilder/Builder/PaginationSection.tsx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| import { useField } from "formik"; | ||
|
|
||
| import GroupControls from "components/GroupControls"; | ||
| import { ControlLabels } from "components/LabeledControl"; | ||
|
|
||
| import { BuilderCard } from "./BuilderCard"; | ||
| import { BuilderField } from "./BuilderField"; | ||
| import { BuilderOneOf } from "./BuilderOneOf"; | ||
| import { InjectRequestOptionFields } from "./InjectRequestOptionFields"; | ||
| import { ToggleGroupField } from "./ToggleGroupField"; | ||
|
|
||
| interface PaginationSectionProps { | ||
| streamFieldPath: (fieldPath: string) => string; | ||
| } | ||
|
|
||
| export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFieldPath }) => { | ||
| const [field, , helpers] = useField(streamFieldPath("paginator")); | ||
| const [pageSizeField] = useField(streamFieldPath("paginator.strategy.page_size")); | ||
| const [, , pageSizeOptionHelpers] = useField(streamFieldPath("paginator.pageSizeOption")); | ||
|
|
||
| const handleToggle = (newToggleValue: boolean) => { | ||
| if (newToggleValue) { | ||
| helpers.setValue({ | ||
| strategy: { | ||
| type: "OffsetIncrement", | ||
| }, | ||
| pageTokenOption: { | ||
| inject_into: "request_parameter", | ||
| }, | ||
| }); | ||
| } else { | ||
| helpers.setValue(undefined); | ||
| } | ||
| }; | ||
| const toggledOn = field.value !== undefined; | ||
|
|
||
| const pageTokenOption = ( | ||
| <GroupControls | ||
| label={ | ||
| <ControlLabels | ||
| label="Page token option" | ||
| infoTooltipContent="Configures how the page token will be sent in requests to the source API" | ||
| /> | ||
| } | ||
| > | ||
| <InjectRequestOptionFields path={streamFieldPath("paginator.pageTokenOption")} descriptor="page token" /> | ||
| </GroupControls> | ||
| ); | ||
|
|
||
| const pageSizeOption = ( | ||
| <ToggleGroupField | ||
| label="Page size option" | ||
| tooltip="Configures how the page size will be sent in requests to the source API" | ||
| fieldPath={streamFieldPath("paginator.pageSizeOption")} | ||
| initialValues={{ | ||
| inject_into: "request_parameter", | ||
| field_name: "", | ||
| }} | ||
| > | ||
| <InjectRequestOptionFields | ||
| path={streamFieldPath("paginator.pageSizeOption")} | ||
| descriptor="page size" | ||
| excludeInjectIntoValues={["path"]} | ||
| /> | ||
| </ToggleGroupField> | ||
| ); | ||
|
|
||
| return ( | ||
| <BuilderCard | ||
| toggleConfig={{ | ||
| label: ( | ||
| <ControlLabels | ||
| label="Pagination" | ||
| infoTooltipContent="Configure how pagination is handled by your connector" | ||
| /> | ||
| ), | ||
| toggledOn, | ||
| onToggle: handleToggle, | ||
| }} | ||
| > | ||
| <BuilderOneOf | ||
| path={streamFieldPath("paginator.strategy")} | ||
| label="Mode" | ||
| tooltip="Pagination method to use for requests sent to the API" | ||
| options={[ | ||
| { | ||
| label: "Offset Increment", | ||
| typeValue: "OffsetIncrement", | ||
| children: ( | ||
| <> | ||
| <BuilderField | ||
| type="number" | ||
| path={streamFieldPath("paginator.strategy.page_size")} | ||
| label="Page size" | ||
| tooltip="Set the size of each page" | ||
| /> | ||
| {pageSizeOption} | ||
| {pageTokenOption} | ||
| </> | ||
| ), | ||
| }, | ||
| { | ||
| label: "Page Increment", | ||
| typeValue: "PageIncrement", | ||
| children: ( | ||
| <> | ||
| <BuilderField | ||
| type="number" | ||
| path={streamFieldPath("paginator.strategy.page_size")} | ||
| label="Page size" | ||
| tooltip="Set the size of each page" | ||
| /> | ||
| <BuilderField | ||
| type="number" | ||
| path={streamFieldPath("paginator.strategy.start_from_page")} | ||
| label="Start from page" | ||
| tooltip="Page number to start requesting pages from" | ||
| optional | ||
| /> | ||
| {pageSizeOption} | ||
| {pageTokenOption} | ||
| </> | ||
| ), | ||
| }, | ||
| { | ||
| label: "Cursor Pagination", | ||
| typeValue: "CursorPagination", | ||
| children: ( | ||
| <> | ||
| <BuilderField | ||
| type="string" | ||
| path={streamFieldPath("paginator.strategy.cursor_value")} | ||
| label="Cursor value" | ||
| tooltip="Value of the cursor to send in requests to the API" | ||
| /> | ||
| <BuilderField | ||
| type="string" | ||
| path={streamFieldPath("paginator.strategy.stop_condition")} | ||
| label="Stop condition" | ||
| tooltip="Condition that determines when to stop requesting further pages" | ||
| optional | ||
| /> | ||
| <BuilderField | ||
| type="number" | ||
| path={streamFieldPath("paginator.strategy.page_size")} | ||
| onChange={(newValue) => { | ||
| if (newValue === undefined || newValue === "") { | ||
| pageSizeOptionHelpers.setValue(undefined); | ||
| } | ||
| }} | ||
| label="Page size" | ||
| tooltip="Set the size of each page" | ||
| optional | ||
| /> | ||
| {pageSizeField.value && pageSizeField.value !== "" && pageSizeOption} | ||
| {pageTokenOption} | ||
| </> | ||
| ), | ||
| }, | ||
| ]} | ||
| /> | ||
| </BuilderCard> | ||
| ); | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
12 changes: 12 additions & 0 deletions
12
airbyte-webapp/src/components/connectorBuilder/Builder/ToggleGroupField.module.scss
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| @use "scss/variables"; | ||
|
|
||
| .label { | ||
| display: flex; | ||
| align-items: center; | ||
| gap: variables.$spacing-md; | ||
| height: 34px; | ||
|
|
||
| label { | ||
| padding-bottom: 0; | ||
| } | ||
| } |
Oops, something went wrong.