Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪟🎉 Connector builder: Allow defining inputs #20431

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
699470e
move connector builder components into the same shared components/con…
lmossman Dec 1, 2022
827532b
move diff over from poc branch
lmossman Dec 1, 2022
4899673
save current progress
lmossman Dec 3, 2022
a1b62f1
add modal for adding streams
lmossman Dec 3, 2022
39e8ba8
focus stream after adding and reset button style
lmossman Dec 5, 2022
2352ef9
add reset confirm modal and select view on add
lmossman Dec 6, 2022
df492fa
style global config and streams buttons
lmossman Dec 6, 2022
5a99c7a
styling improvements
lmossman Dec 6, 2022
62fc08c
handle long stream names better
lmossman Dec 6, 2022
7b26b78
pull in connector manifest schema directly
lmossman Dec 6, 2022
71a88f1
add box shadows to resizable panels
lmossman Dec 6, 2022
d7bee54
upgrade orval and use connector manifest schema directly
lmossman Dec 6, 2022
162ecf2
remove airbyte protocol from connector builder api spec
lmossman Dec 6, 2022
def60b0
Merge branch 'master' into lmossman/use-connector-manifest-schema-dir…
lmossman Dec 6, 2022
ff9aa76
generate python models from openapi change
lmossman Dec 6, 2022
3678956
merge with lmossman/use-connector-manifest-schema-directly
lmossman Dec 6, 2022
b16d4a8
fix position of yaml toggle
lmossman Dec 6, 2022
67fb9dd
handle no stream case with better looking message
lmossman Dec 7, 2022
bb42007
group global fields into single object and fix console error
lmossman Dec 7, 2022
9bb1812
confirmation modal on toggling dirty form + cleanup
lmossman Dec 7, 2022
587ca2f
merge with master
lmossman Dec 7, 2022
81cf108
fix connector name display
lmossman Dec 7, 2022
094a045
undo change to manifest schema
lmossman Dec 7, 2022
d1c8c80
remove commented code
lmossman Dec 7, 2022
aebac20
remove unnecessary change
lmossman Dec 7, 2022
1e39be4
fix spacing
lmossman Dec 7, 2022
09cf875
use shadow mixin for connector img
lmossman Dec 7, 2022
ec65d67
add comment about connector img
lmossman Dec 7, 2022
f9a3fa1
Merge remote-tracking branch 'origin/master' into lmossman/connector-…
Dec 8, 2022
603820a
change onSubmit to no-op
lmossman Dec 8, 2022
cb2f299
remove console log
lmossman Dec 8, 2022
d1672a0
clean up styling
lmossman Dec 8, 2022
7255784
simplify sidebar to remove StreamSelectButton component
lmossman Dec 8, 2022
65cd42b
swap colors of toggle
lmossman Dec 8, 2022
10d7893
move FormikPatch to src/core/form
lmossman Dec 8, 2022
fc89001
move types up to connectorBuilder/ level
lmossman Dec 8, 2022
24eaaff
use grid display for ui yaml toggle button
lmossman Dec 8, 2022
de604f6
use spread instead of setting array index directly
lmossman Dec 8, 2022
e696e57
add intl in missing places
lmossman Dec 8, 2022
5952f70
merge with master
lmossman Dec 8, 2022
b7c08f6
pull connector manifest schema in through separate openapi spec
lmossman Dec 8, 2022
dd04e61
erge branch 'lmossman/connector-builder-config-ui' of github.com:airb…
Dec 9, 2022
d6f10b4
use correct intl string id
lmossman Dec 9, 2022
5ffe20e
throttle setting json manifest in yaml editor
lmossman Dec 9, 2022
b82ba2a
use button prop instead of manually styling
lmossman Dec 9, 2022
b9580b4
consolidate AddStreamButton styles
lmossman Dec 9, 2022
1587243
fix sidebar flex styles
lmossman Dec 9, 2022
62a60e5
use specific flex properties instead of flex
lmossman Dec 9, 2022
092821d
clean up download and reset button styles
lmossman Dec 10, 2022
7b8dec8
use row-reverse for yaml editor download button
lmossman Dec 10, 2022
6d68db9
fix stream selector styles to remove margins
lmossman Dec 10, 2022
91bc357
give connector setup guide panel same corner and shadow styles
lmossman Dec 10, 2022
4ffb969
remove blur from page display
lmossman Dec 10, 2022
70afdf8
set view to stream when selected in test panel
lmossman Dec 10, 2022
f656506
add placeholder when stream name is empty
lmossman Dec 10, 2022
c437d30
switch to index-based stream selection to preserve testing panel sele…
lmossman Dec 10, 2022
3cdbc70
handle empty name in stream selector
lmossman Dec 10, 2022
831f574
Merge branch 'lmossman/connector-builder-config-ui' of github.com:air…
Dec 12, 2022
e2730fe
Merge branch 'lmossman/connector-builder-config-ui' into flash1293/in…
Dec 12, 2022
269ae82
make connector form work in connector builder
Dec 12, 2022
901fc63
wip
Dec 12, 2022
4944d3f
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 13, 2022
3255298
fix small stuff
Dec 13, 2022
333fb0b
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 13, 2022
cec206f
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 13, 2022
ba28014
add basic input UI
Dec 13, 2022
53ad55f
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 14, 2022
5300933
Merge remote-tracking branch 'origin/master' into flash1293/integrate…
Dec 14, 2022
935188b
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 14, 2022
b694dc5
user inputs
Dec 14, 2022
e7d81ea
make most of inputs configuration work
Dec 14, 2022
3de021a
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 15, 2022
552fc65
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 15, 2022
ace1299
fix a bunch of stuff
Dec 15, 2022
7fa1752
handle unknown config types
Dec 15, 2022
7196409
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 16, 2022
4a37a67
add warning label
Dec 16, 2022
10fa73e
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 16, 2022
7e48ba5
fix label
Dec 16, 2022
75db697
fix some styling
Dec 16, 2022
b340ef4
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 16, 2022
98c146c
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 16, 2022
9835a5c
review comments
Dec 18, 2022
c352df0
Merge branch 'feature/connector-builder' into flash1293/integrate-con…
Dec 19, 2022
ed43eab
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 19, 2022
7958ef1
improve state management and error handling
Dec 19, 2022
beb4968
Merge branch 'feature/connector-builder' into flash1293/integrate-con…
Dec 19, 2022
b3cfb8f
Merge branch 'flash1293/integrate-connector-form' into flash1293/allo…
Dec 19, 2022
49f55a1
Merge branch 'flash1293/allow-configuring-inputs' of github.com:airby…
Dec 19, 2022
9ffa34e
handle stored form values that don't contain new fields properly
lmossman Dec 19, 2022
c0350d3
Merge branch 'flash1293/allow-configuring-inputs' of github.com:airby…
Dec 20, 2022
e98ebc1
Merge branch 'feature/connector-builder' into flash1293/allow-configu…
Dec 20, 2022
7cce350
Update airbyte-webapp/src/locales/en.json
Dec 21, 2022
20e8044
Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsV…
Dec 21, 2022
50bddcb
inputs editing weirdness
Dec 21, 2022
0ca37ad
input form reset
Dec 21, 2022
8095fda
using the Label component
Dec 21, 2022
2bc3541
🪟🎉 Connector builder authentication (#20645)
Dec 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,13 @@ export const AddStreamButton: React.FC<AddStreamButtonProps> = ({ onAddStream, b
<ModalBody className={styles.body}>
<BuilderField
path="streamName"
type="text"
type="string"
label={formatMessage({ id: "connectorBuilder.addStreamModal.streamNameLabel" })}
tooltip={formatMessage({ id: "connectorBuilder.addStreamModal.streamNameTooltip" })}
/>
<BuilderField
path="urlPath"
type="text"
type="string"
label={formatMessage({ id: "connectorBuilder.addStreamModal.urlPathLabel" })}
tooltip={formatMessage({ id: "connectorBuilder.addStreamModal.urlPathTooltip" })}
/>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
import { BuilderCard } from "./BuilderCard";
import { BuilderField } from "./BuilderField";
import { BuilderOneOf } from "./BuilderOneOf";
import { BuilderOptional } from "./BuilderOptional";
import { KeyValueListField } from "./KeyValueListField";
import { UserInputField } from "./UserInputField";

export const AuthenticationSection: React.FC = () => {
return (
<BuilderCard>
<BuilderOneOf
path="global.authenticator"
label="Authentication"
tooltip="Authentication method to use for requests sent to the API"
options={[
{ label: "No Auth", typeValue: "NoAuth" },
{
label: "API Key",
typeValue: "ApiKeyAuthenticator",
default: {
api_token: "{{ config['api_key'] }}",
header: "",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.header"
label="Header"
tooltip="HTTP header which should be set to the API Key"
/>
<UserInputField
label="API Key"
tooltip="The API key issued by the service. Fill it in in the user inputs"
/>
</>
),
},
{
label: "Bearer",
typeValue: "BearerAuthenticator",
default: {
api_token: "{{ config['api_key'] }}",
},
children: (
<UserInputField
label="API Key"
tooltip="The API key issued by the service. Fill it in in the user inputs"
/>
),
},
{
label: "Basic HTTP",
typeValue: "BasicHttpAuthenticator",
default: {
username: "{{ config['username'] }}",
password: "{{ config['password'] }}",
},
children: (
<>
<UserInputField label="Username" tooltip="The username for the login. Fill it in in the user inputs" />
<UserInputField label="Password" tooltip="The password for the login. Fill it in in the user inputs" />
</>
),
},
{
label: "OAuth",
typeValue: "OAuthAuthenticator",
default: {
client_id: "{{ config['client_id'] }}",
client_secret: "{{ config['client_secret'] }}",
refresh_token: "{{ config['client_refresh_token'] }}",
refresh_request_body: [],
token_refresh_endpoint: "",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.token_refresh_endpoint"
label="Token refresh endpoint"
tooltip="The URL to call to obtain a new access token"
/>
<UserInputField label="Client ID" tooltip="The OAuth client ID" />
<UserInputField label="Client secret" tooltip="The OAuth client secret" />
<UserInputField label="Refresh token" tooltip="The OAuth refresh token" />
<BuilderOptional>
<BuilderField
type="array"
path="global.authenticator.scopes"
optional
label="Scopes"
tooltip="Scopes to request"
/>
<BuilderField
type="array"
path="global.authenticator.token_expiry_date_format"
optional
label="Token expiry date format"
tooltip="The format of the expiry date of the access token as obtained from the refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.expires_in_name"
optional
label="Token expiry property name"
tooltip="The name of the property which contains the token exipiry date in the response from the token refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.access_token_name"
optional
label="Access token property name"
tooltip="The name of the property which contains the access token in the response from the token refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.grant_type"
optional
label="Grant type"
tooltip="The grant type to request for access_token"
/>
<KeyValueListField
path="global.authenticator.refresh_request_body"
label="Request Parameters"
tooltip="The request body to send in the refresh request"
/>
</BuilderOptional>
</>
),
},
{
label: "Session token",
typeValue: "SessionTokenAuthenticator",
default: {
username: "{{ config['username'] }}",
password: "{{ config['password'] }}",
session_token: "{{ config['session_token'] }}",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.header"
label="Header"
tooltip="Specific HTTP header of source API for providing session token"
/>
<BuilderField
type="string"
path="global.authenticator.session_token_response_key"
label="Session token response key"
tooltip="Key for retrieving session token from api response"
/>
<BuilderField
type="string"
path="global.authenticator.login_url"
label="Login url"
tooltip="Url for getting a specific session token"
/>
<BuilderField
type="string"
path="global.authenticator.validate_session_url"
label="Validate session url"
tooltip="Url to validate passed session token"
/>
<UserInputField label="Username" tooltip="The username" />
<UserInputField label="Password" tooltip="The password" />
<UserInputField
label="Session token"
tooltip="Session token generated by user (if provided username and password are not required)"
/>
</>
),
},
]}
/>
</BuilderCard>
);
};
Original file line number Diff line number Diff line change
@@ -1,31 +1,41 @@
import { Form } from "formik";
import { useEffect } from "react";

import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { BuilderView, useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { BuilderFormValues } from "../types";
import { builderFormValidationSchema, BuilderFormValues } from "../types";
import styles from "./Builder.module.scss";
import { BuilderSidebar } from "./BuilderSidebar";
import { GlobalConfigView } from "./GlobalConfigView";
import { InputsView } from "./InputsView";
import { StreamConfigView } from "./StreamConfigView";

interface BuilderProps {
values: BuilderFormValues;
toggleYamlEditor: () => void;
}

function getView(selectedView: BuilderView) {
switch (selectedView) {
case "global":
return <GlobalConfigView />;
case "inputs":
return <InputsView />;
default:
return <StreamConfigView streamNum={selectedView} />;
}
}

export const Builder: React.FC<BuilderProps> = ({ values, toggleYamlEditor }) => {
const { setBuilderFormValues, selectedView } = useConnectorBuilderState();
useEffect(() => {
setBuilderFormValues(values);
setBuilderFormValues(values, builderFormValidationSchema.isValidSync(values));
}, [values, setBuilderFormValues]);

return (
<div className={styles.container}>
<BuilderSidebar className={styles.sidebar} toggleYamlEditor={toggleYamlEditor} />
<Form className={styles.form}>
{selectedView === "global" ? <GlobalConfigView /> : <StreamConfigView streamNum={selectedView} />}
</Form>
<Form className={styles.form}>{getView(selectedView)}</Form>
</div>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@ import { useField } from "formik";
import { FormattedMessage } from "react-intl";

import { ControlLabels } from "components/LabeledControl";
import { LabeledSwitch } from "components/LabeledSwitch";
import { DropDown } from "components/ui/DropDown";
import { Input } from "components/ui/Input";
import { TagInput } from "components/ui/TagInput";
import { Text } from "components/ui/Text";
import { InfoTooltip } from "components/ui/Tooltip/InfoTooltip";

import styles from "./BuilderField.module.scss";

Expand All @@ -28,10 +30,17 @@ interface BaseFieldProps {
path: string;
label: string;
tooltip?: string;
readOnly?: boolean;
optional?: boolean;
}

type BuilderFieldProps = BaseFieldProps & ({ type: "text" | "array" } | { type: "enum"; options: string[] });
type BuilderFieldProps = BaseFieldProps &
(
| { type: "string" | "number" | "integer"; onChange?: (newValue: string) => void }
| { type: "boolean"; onChange?: (newValue: boolean) => void }
| { type: "array"; onChange?: (newValue: string[]) => void }
| { type: "enum"; onChange?: (newValue: string) => void; options: string[] }
);

const EnumField: React.FC<EnumFieldProps> = ({ options, value, setValue, error, ...props }) => {
return (
Expand All @@ -51,23 +60,56 @@ const ArrayField: React.FC<ArrayFieldProps> = ({ name, value, setValue, error })
return <TagInput name={name} fieldValue={value} onChange={(value) => setValue(value)} error={error} />;
};

export const BuilderField: React.FC<BuilderFieldProps> = ({ path, label, tooltip, optional = false, ...props }) => {
export const BuilderField: React.FC<BuilderFieldProps> = ({
path,
label,
tooltip,
optional = false,
readOnly,
...props
}) => {
const [field, meta, helpers] = useField(path);
const hasError = !!meta.error && meta.touched;

if (props.type === "boolean") {
return (
<LabeledSwitch
{...field}
checked={field.value}
label={
<>
{label} {tooltip && <InfoTooltip placement="top-start">{tooltip}</InfoTooltip>}
</>
}
/>
);
}

const setValue = (newValue: unknown) => {
props.onChange?.(newValue as string & string[]);
helpers.setValue(newValue);
};

return (
<ControlLabels className={styles.container} label={label} infoTooltipContent={tooltip} optional={optional}>
{props.type === "text" && <Input {...field} type={props.type} value={field.value ?? ""} error={hasError} />}
{(props.type === "number" || props.type === "string" || props.type === "integer") && (
<Input
{...field}
onChange={(e) => {
field.onChange(e);
props.onChange?.(e.target.value);
}}
type={props.type}
value={field.value ?? ""}
error={hasError}
readOnly={readOnly}
/>
)}
{props.type === "array" && (
<ArrayField name={path} value={field.value ?? []} setValue={helpers.setValue} error={hasError} />
<ArrayField name={path} value={field.value ?? []} setValue={setValue} error={hasError} />
)}
{props.type === "enum" && (
<EnumField
options={props.options}
value={field.value ?? props.options[0]}
setValue={helpers.setValue}
error={hasError}
/>
<EnumField options={props.options} value={field.value} setValue={setValue} error={hasError} />
)}
{hasError && (
<Text className={styles.error}>
Expand Down
Loading