Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪟🎉 Connector builder: Add manual schema #20862

Merged
merged 113 commits into from
Jan 5, 2023
Merged
Show file tree
Hide file tree
Changes from 101 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
d2ed762
improve some types
Dec 5, 2022
f474a1c
Merge remote-tracking branch 'upstream/master' into chore-improve-con…
Dec 5, 2022
5b0521f
improve further
Dec 5, 2022
af659fd
Merge remote-tracking branch 'upstream/master' into chore-improve-con…
Dec 6, 2022
1cd3bbb
clean up a bit more
Dec 6, 2022
807c436
refactor loading state
Dec 6, 2022
6c76e0f
Merge remote-tracking branch 'upstream/master' into flash1293/connect…
Dec 6, 2022
05c3107
move loading state up
Dec 6, 2022
eee4303
remove isLoading references
Dec 6, 2022
7d6f977
remove unused props and make fetch connector error work
Dec 6, 2022
0fe11f0
remove special component for name
Dec 6, 2022
34d4814
remove top level state for unifinished flows
Dec 6, 2022
67d62c5
Merge remote-tracking branch 'upstream/master' into flash1293/connect…
Dec 6, 2022
afea882
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 6, 2022
c7e7475
Merge branch 'master' into flash1293/connector-form-loading-state
Dec 6, 2022
cc7f65f
Merge branch 'chore-improve-connector-form-types' into flash1293/conn…
Dec 6, 2022
a1c6cbd
start removing uiwidget
Dec 6, 2022
9be8337
Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard…
Dec 6, 2022
37cee8f
Merge remote-tracking branch 'upstream/master' into flash1293/connect…
Dec 6, 2022
5686241
Merge branch 'flash1293/connector-form-loading-state' of github.com:a…
Dec 6, 2022
7b73364
remove undefined option for selected id
Dec 6, 2022
7058d26
Merge branch 'flash1293/connector-form-loading-state' into flash1293/…
Dec 6, 2022
dd7e82a
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 6, 2022
6d4c7d3
Merge remote-tracking branch 'upstream/master' into flash1293/connect…
Dec 7, 2022
d624792
Merge branch 'flash1293/connector-form-loading-state' into flash1293/…
Dec 7, 2022
36cf941
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 7, 2022
b7a6322
Merge remote-tracking branch 'upstream/master' into flash1293/connect…
Dec 7, 2022
72b602a
Merge branch 'flash1293/connector-form-loading-state' into flash1293/…
Dec 7, 2022
5e47a4a
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 7, 2022
6c71028
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 8, 2022
809625f
remove unused prop
Dec 8, 2022
10446eb
Merge branch 'flash1293/connector-form-loading-state' into flash1293/…
Dec 8, 2022
4a7a885
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 8, 2022
2024b60
Merge branch 'flash1293/connector-form-unifinished-flows' into flash1…
Dec 8, 2022
ee30092
fix types
Dec 8, 2022
d91df38
remove uiwidget state
Dec 8, 2022
93fe558
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 9, 2022
cbc023a
Merge branch 'flash1293/connector-form-name-override' into flash1293/…
Dec 9, 2022
ec54d38
Merge branch 'flash1293/connector-form-unifinished-flows' into flash1…
Dec 9, 2022
47152ea
clean up
Dec 9, 2022
6ec0e10
adjust comment
Dec 9, 2022
1aa7829
handle errors in a nice way
Dec 9, 2022
7b90088
do not respect default on oneOf fields
Dec 9, 2022
fbefbdd
rename to formblock
Dec 9, 2022
6b3e5e2
reduce re-renders
Dec 9, 2022
9aa3f80
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 12, 2022
c765003
Merge branch 'flash1293/connector-form-unifinished-flows' into flash1…
Dec 12, 2022
5cd77ec
pass error to secure inputs
Dec 12, 2022
5dfbc22
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 13, 2022
a6aa741
simplify and improve styling
Dec 13, 2022
95717d2
align top
Dec 13, 2022
40d162d
Merge branch 'flash1293/connector-form-unifinished-flows' into flash1…
Dec 13, 2022
b9384ee
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 13, 2022
582a5f2
code review
Dec 13, 2022
18d6ce8
remove comment
Dec 13, 2022
60c1f1c
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Dec 14, 2022
9cb595d
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 15, 2022
fe74a45
review comments
Dec 15, 2022
5c00ad0
rename file
Dec 15, 2022
bd0dffc
be strict about boolean values
Dec 15, 2022
1ff8c13
add example
Dec 15, 2022
c6ee526
track form error in error boundary
Dec 15, 2022
05bbd6f
Merge branch 'flash1293/connector-form-remove-ui-widget-state' of git…
Dec 15, 2022
fff4703
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 16, 2022
94ba866
review comments
Dec 16, 2022
97bb0a7
handle unexpected cases better
Dec 16, 2022
633b9b2
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Dec 16, 2022
e47517f
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Dec 16, 2022
e931956
Merge remote-tracking branch 'origin/master' into feature/connector-b…
Dec 16, 2022
6889f66
enrich error with connector id
Dec 19, 2022
7b60406
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 19, 2022
d5e56eb
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Dec 19, 2022
35a74ba
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 19, 2022
02c8096
🪟🎉 Add copy stream button (#20577)
Dec 19, 2022
526adfc
rename prop
Dec 20, 2022
7d49233
🪟🎉 Connector builder: Integrate connector form for test input (#20385)
Dec 20, 2022
c5da764
Merge remote-tracking branch 'origin/master' into flash1293/connector…
Dec 20, 2022
6c0bf9d
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 20, 2022
477977f
use request_body_json instead of request_body_data
lmossman Dec 20, 2022
200cfe7
:window: :art: Move `Add` button into the line of Connector Builder k…
lmossman Dec 21, 2022
c7fd310
🪟🎉 Connector builder: Allow defining inputs (#20431)
Dec 22, 2022
e0e3796
fix merge conflict with dropdown prop being renamed to control
Dec 22, 2022
e23a6d8
[Connector Builder] Add paginator (#20698)
lmossman Dec 22, 2022
610e668
[Connector Builder] Add stream slicer (#20748)
lmossman Dec 22, 2022
e68ff49
debounce form builder values update to reduce load
Dec 22, 2022
9ccf294
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Dec 22, 2022
ae9a229
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Dec 22, 2022
a9e1d09
🪟🔧 Connector builder: use new lowcode manifest (#20715)
Dec 22, 2022
1965efe
add manual schema
Dec 23, 2022
c8f3920
make buttons not submit the form accidentally
Dec 23, 2022
e4f7bba
debounce validation as well
Dec 27, 2022
16c71ae
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Jan 2, 2023
54c070d
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Jan 2, 2023
a8b9aae
Merge branch 'feature/connector-builder' into flash1293/add-manual-sc…
Jan 2, 2023
2e20fa6
akways show stream test button in error state if there are errors
Jan 2, 2023
65199d0
fix type of oauth input
Jan 2, 2023
92722e0
Merge branch 'master' into flash1293/connector-form-remove-ui-widget-…
Jan 3, 2023
718c233
Merge branch 'flash1293/connector-form-remove-ui-widget-state' into f…
Jan 3, 2023
c303a5d
Merge branch 'feature/connector-builder' into flash1293/add-manual-sc…
Jan 3, 2023
6a8be0b
Merge remote-tracking branch 'origin/master' into feature/connector-b…
Jan 3, 2023
753241d
Merge branch 'feature/connector-builder' into flash1293/add-manual-sc…
Jan 3, 2023
16ce351
add validation schema for add stream form
lmossman Jan 3, 2023
25d6586
validate all views on test click
lmossman Jan 3, 2023
64a5eb9
add type to prevent console warning
lmossman Jan 4, 2023
1db62dd
Merge branch 'feature/connector-builder' into flash1293/add-manual-sc…
Jan 4, 2023
2a3776f
Merge remote-tracking branch 'origin/master' into flash1293/add-manua…
Jan 4, 2023
abb9c0b
improve error indicator
Jan 4, 2023
08e6d99
make schema editor take up full height (#21000)
lmossman Jan 4, 2023
12db741
Merge branch 'flash1293/add-manual-schema' of github.com:airbytehq/ai…
Jan 4, 2023
7088b2c
prevent resizing card when error message is shown or hidden
lmossman Jan 4, 2023
5423192
Merge remote-tracking branch 'origin/master' into flash1293/add-manua…
Jan 5, 2023
daae429
Revert "improve error indicator"
Jan 5, 2023
4a7ffc6
Merge remote-tracking branch 'origin/master' into flash1293/add-manua…
Jan 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,10 @@ $border-width: variables.$border-thick;
white-space: nowrap;
}

.dropdown {
.control {
margin-left: auto;
padding: 0 variables.$spacing-xs;
background-color: colors.$white;
min-width: calc(50% - 100px);
}

.content {
Expand Down
14 changes: 11 additions & 3 deletions airbyte-webapp/src/components/GroupControls/GroupControls.tsx
Original file line number Diff line number Diff line change
@@ -1,21 +1,29 @@
import classNames from "classnames";
import React from "react";

import styles from "./GroupControls.module.scss";

interface GroupControlsProps {
label: React.ReactNode;
dropdown?: React.ReactNode;
control?: React.ReactNode;
controlClassName?: string;
name?: string;
}

const GroupControls: React.FC<React.PropsWithChildren<GroupControlsProps>> = ({ label, dropdown, children, name }) => {
const GroupControls: React.FC<React.PropsWithChildren<GroupControlsProps>> = ({
label,
control,
children,
name,
controlClassName,
}) => {
return (
// This outer div is necessary for .content > :first-child padding to be properly applied in the case of nested GroupControls
<div>
<div className={styles.container}>
<div className={styles.title}>
<div className={styles.label}>{label}</div>
<div className={styles.dropdown}>{dropdown}</div>
<div className={classNames(styles.control, controlClassName)}>{control}</div>
</div>
<div className={styles.content} data-testid={name}>
{children}
Expand Down
19 changes: 19 additions & 0 deletions airbyte-webapp/src/components/GroupControls/index.stories.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { ComponentStory, ComponentMeta } from "@storybook/react";

import { Button } from "components/ui/Button";
import { Card } from "components/ui/Card";

import { FormBlock, FormConditionItem } from "core/form/types";
Expand Down Expand Up @@ -73,3 +74,21 @@ WithContent.args = {
</>
),
};

export const EmptyWithControl = Template.bind({});
EmptyWithControl.args = {
label,
control: <Button variant="secondary">Control</Button>,
};

export const ControlAndContent = Template.bind({});
ControlAndContent.args = {
label,
control: <Button variant="secondary">Control</Button>,
children: (
<>
<SectionContainer>Content part 1</SectionContainer>
<SectionContainer>Content part 2</SectionContainer>
</>
),
};
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { Form, Formik, useField } from "formik";
import merge from "lodash/merge";
import { useState } from "react";
import React from "react";
import { FormattedMessage, useIntl } from "react-intl";

import { Button } from "components/ui/Button";
Expand All @@ -8,7 +10,7 @@ import { Modal, ModalBody, ModalFooter } from "components/ui/Modal";
import { FormikPatch } from "core/form/FormikPatch";

import { ReactComponent as PlusIcon } from "../../connection/ConnectionOnboarding/plusIcon.svg";
import { BuilderStream } from "../types";
import { BuilderStream, DEFAULT_BUILDER_STREAM_VALUES } from "../types";
import styles from "./AddStreamButton.module.scss";
import { BuilderField } from "./BuilderField";

Expand All @@ -19,40 +21,40 @@ interface AddStreamValues {

interface AddStreamButtonProps {
onAddStream: (addedStreamNum: number) => void;
button?: React.ReactElement;
initialValues?: Partial<BuilderStream>;
}

export const AddStreamButton: React.FC<AddStreamButtonProps> = ({ onAddStream }) => {
export const AddStreamButton: React.FC<AddStreamButtonProps> = ({ onAddStream, button, initialValues }) => {
const { formatMessage } = useIntl();
const [isOpen, setIsOpen] = useState(false);
const [streamsField, , helpers] = useField<BuilderStream[]>("streams");
const numStreams = streamsField.value.length;

const buttonClickHandler = () => {
setIsOpen(true);
};

return (
<>
<Button
className={styles.addButton}
onClick={() => {
setIsOpen(true);
}}
icon={<PlusIcon />}
/>
{button ? (
React.cloneElement(button, {
onClick: buttonClickHandler,
})
) : (
<Button className={styles.addButton} onClick={buttonClickHandler} icon={<PlusIcon />} />
)}
{isOpen && (
<Formik
initialValues={{ streamName: "", urlPath: "" }}
onSubmit={(values: AddStreamValues) => {
helpers.setValue([
...streamsField.value,
{
merge({}, DEFAULT_BUILDER_STREAM_VALUES, {
...initialValues,
name: values.streamName,
urlPath: values.urlPath,
fieldPointer: [],
httpMethod: "GET",
requestOptions: {
requestParameters: [],
requestHeaders: [],
requestBody: [],
},
},
}),
]);
setIsOpen(false);
onAddStream(numStreams);
Expand All @@ -71,13 +73,13 @@ export const AddStreamButton: React.FC<AddStreamButtonProps> = ({ onAddStream })
<ModalBody className={styles.body}>
<BuilderField
path="streamName"
type="text"
type="string"
label={formatMessage({ id: "connectorBuilder.addStreamModal.streamNameLabel" })}
tooltip={formatMessage({ id: "connectorBuilder.addStreamModal.streamNameTooltip" })}
/>
<BuilderField
path="urlPath"
type="text"
type="string"
label={formatMessage({ id: "connectorBuilder.addStreamModal.urlPathLabel" })}
tooltip={formatMessage({ id: "connectorBuilder.addStreamModal.urlPathTooltip" })}
/>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
import { BuilderCard } from "./BuilderCard";
import { BuilderField } from "./BuilderField";
import { BuilderOneOf } from "./BuilderOneOf";
import { BuilderOptional } from "./BuilderOptional";
import { KeyValueListField } from "./KeyValueListField";
import { UserInputField } from "./UserInputField";

export const AuthenticationSection: React.FC = () => {
return (
<BuilderCard>
<BuilderOneOf
path="global.authenticator"
label="Authentication"
tooltip="Authentication method to use for requests sent to the API"
options={[
{ label: "No Auth", typeValue: "NoAuth" },
{
label: "API Key",
typeValue: "ApiKeyAuthenticator",
default: {
api_token: "{{ config['api_key'] }}",
header: "",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.header"
label="Header"
tooltip="HTTP header which should be set to the API Key"
/>
<UserInputField
label="API Key"
tooltip="The API key issued by the service. Fill it in in the user inputs"
/>
</>
),
},
{
label: "Bearer",
typeValue: "BearerAuthenticator",
default: {
api_token: "{{ config['api_key'] }}",
},
children: (
<UserInputField
label="API Key"
tooltip="The API key issued by the service. Fill it in in the user inputs"
/>
),
},
{
label: "Basic HTTP",
typeValue: "BasicHttpAuthenticator",
default: {
username: "{{ config['username'] }}",
password: "{{ config['password'] }}",
},
children: (
<>
<UserInputField label="Username" tooltip="The username for the login. Fill it in in the user inputs" />
<UserInputField label="Password" tooltip="The password for the login. Fill it in in the user inputs" />
</>
),
},
{
label: "OAuth",
typeValue: "OAuthAuthenticator",
default: {
client_id: "{{ config['client_id'] }}",
client_secret: "{{ config['client_secret'] }}",
refresh_token: "{{ config['client_refresh_token'] }}",
refresh_request_body: [],
token_refresh_endpoint: "",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.token_refresh_endpoint"
label="Token refresh endpoint"
tooltip="The URL to call to obtain a new access token"
/>
<UserInputField label="Client ID" tooltip="The OAuth client ID" />
<UserInputField label="Client secret" tooltip="The OAuth client secret" />
<UserInputField label="Refresh token" tooltip="The OAuth refresh token" />
<BuilderOptional>
<BuilderField
type="array"
path="global.authenticator.scopes"
optional
label="Scopes"
tooltip="Scopes to request"
/>
<BuilderField
type="string"
path="global.authenticator.token_expiry_date_format"
optional
label="Token expiry date format"
tooltip="The format of the expiry date of the access token as obtained from the refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.expires_in_name"
optional
label="Token expiry property name"
tooltip="The name of the property which contains the token exipiry date in the response from the token refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.access_token_name"
optional
label="Access token property name"
tooltip="The name of the property which contains the access token in the response from the token refresh endpoint"
/>
<BuilderField
type="string"
path="global.authenticator.grant_type"
optional
label="Grant type"
tooltip="The grant type to request for access_token"
/>
<KeyValueListField
path="global.authenticator.refresh_request_body"
label="Request Parameters"
tooltip="The request body to send in the refresh request"
/>
</BuilderOptional>
</>
),
},
{
label: "Session token",
typeValue: "SessionTokenAuthenticator",
default: {
username: "{{ config['username'] }}",
password: "{{ config['password'] }}",
session_token: "{{ config['session_token'] }}",
},
children: (
<>
<BuilderField
type="string"
path="global.authenticator.header"
label="Header"
tooltip="Specific HTTP header of source API for providing session token"
/>
<BuilderField
type="string"
path="global.authenticator.session_token_response_key"
label="Session token response key"
tooltip="Key for retrieving session token from api response"
/>
<BuilderField
type="string"
path="global.authenticator.login_url"
label="Login url"
tooltip="Url for getting a specific session token"
/>
<BuilderField
type="string"
path="global.authenticator.validate_session_url"
label="Validate session url"
tooltip="Url to validate passed session token"
/>
<UserInputField label="Username" tooltip="The username" />
<UserInputField label="Password" tooltip="The password" />
<UserInputField
label="Session token"
tooltip="Session token generated by user (if provided username and password are not required)"
/>
</>
),
},
]}
/>
</BuilderCard>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,5 @@
.form {
flex: 1;
padding: variables.$spacing-xl;
overflow: auto;
}
Loading