Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📝 Docs: update Stripe docs page #29254

Merged
merged 6 commits into from
Aug 9, 2023
Merged

📝 Docs: update Stripe docs page #29254

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a55df39
add API key setup instructions
ChristoGrab Aug 7, 2023
6af724b
re-order bullet points
ChristoGrab Aug 7, 2023
ff48ad4
complete main docs update
ChristoGrab Aug 7, 2023
a8fc55e
update lookback window explanation
ChristoGrab Aug 7, 2023
fdb9394
small fix to lookback window example
ChristoGrab Aug 7, 2023
1138f4d
update data request description
ChristoGrab Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 24 additions & 13 deletions docs/integrations/sources/stripe.inapp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,35 @@
## Prerequisites

- Your [Stripe `Account ID`](https://dashboard.stripe.com/settings/account)
- Your [Stripe `Secret Key`](https://dashboard.stripe.com/apikeys)
- Access to the Stripe account containing the data to replicate

## Setup Guide

1. Enter a name for your source.
2. For **Account ID**, enter your [Stripe `Account ID`](https://dashboard.stripe.com/settings/account).
3. For **Secret Key**, enter your [Stripe `Secret Key`](https://dashboard.stripe.com/apikeys)
:::note
To authenticate the Stripe connector, you need an API key with **Read** privileges for the data to replicate. For steps on obtaining and setting permissions for this key, refer to our [full Stripe documentation](https://docs.airbyte.com/integrations/sources/stripe#setup-guide).
:::

We recommend creating a secret key specifically for Airbyte to control which resources Airbyte can access. For ease of use, we recommend granting read permission to all resources and configuring which resource to replicate in the Airbyte UI. You can also use the API keys for the [test mode](https://stripe.com/docs/keys#obtain-api-keys) to try out the Stripe integration with Airbyte.
1. For **Source name**, enter a name to help you identify this source.
2. For **Account ID**, enter your Stripe Account ID. This ID begins with `acct_`, and can be found in the top-right corner of your Stripe [account settings page](https://dashboard.stripe.com/settings/account).
3. For **Secret Key**, enter your Stripe API key, which can be found at your Stripe [API keys page](https://dashboard.stripe.com/apikeys).
4. For **Replication Start Date**, use the provided datepicker or enter a UTC date and time programmatically in the format `YYYY-MM-DDTHH:mm:ssZ`. The data added on and after this date will be replicated.
5. (Optional) For **Lookback Window**, you may specify a number of days from the present day to reread data. This allows the connector to retrieve data that might have been updated after its initial creation, and is useful for handling any post-transaction adjustments (such as tips, refunds, chargebacks, etc).

4. For **Replication start date**, enter the date in `YYYY-MM-DDTHH:mm:ssZ` format. Data created on and after this date will be replicated.
5. (Optional) For **Lookback Window in days**, select the number of days the value in days prior to the start date that you to sync your data with. If your data is updated after setting up this connector, you can use the this option to reload data from the past N days. Example: If the Replication start date is set to `2021-01-01T00:00:00Z`, then:
- If you leave the Lookback Window in days parameter to its the default value of 0, Airbyte will sync data from the Replication start date `2021-01-01T00:00:00Z`
- If the Lookback Window in days value is set to 1, Airbyte will consider the Replication start date to be `2020-12-31T00:00:00Z`
- If the Lookback Window in days value is set to 7, Airbyte will sync data from `2020-12-25T00:00:00Z`
6. Click **Set up source**.
- Leaving the **Lookback Window** at its default value of 0 means Airbyte will not re-export data after it has been synced.
- Setting the **Lookback Window** to 1 means Airbyte will re-export and capture any data changes within the last day.
- Setting the **Lookback Window** to 7 means Airbyte will re-export and capture any data changes within the last week.

6. (Optional) For **Data Request Window**, you may specify the time window in days used by the connector when requesting data from the Stripe API. This window defines the span of time covered in each request, with larger values encompassing more days in a single request. Generally speaking, the lack of overhead from making fewer requests means a larger window is faster to sync. However, this also means the state of the sync will persist less frequently. If an issue occurs or the sync is interrupted, a larger window means more data will need to be resynced, potentially causing a delay in the overall process.

For example, if you are replicating three years worth of data:

- A **Data Request Window** of 365 days means Airbyte makes 3 requests, each for a year. This is generally faster but risks needing to resync up to a year's data if the sync is interrupted.
- A **Data Request Window** of 30 days means 36 requests, each for a month. This may be slower but minimizes the amount of data that needs to be resynced if an issue occurs.

If you are unsure of which value to use, we recommend leaving this setting at its default value of 365 days.
7. Click **Set up source** and wait for the tests to complete.

### Stripe API limitations
- When syncing `events` data from Stripe, data is only [returned for the last 30 days](https://stripe.com/docs/api/events). Using the full-refresh-overwrite sync from Airbyte will delete the events data older than 30 days from your target destination. Use an append sync mode to ensure historical data is retained.

- When syncing `events` data from Stripe, data is only [returned for the last 30 days](https://stripe.com/docs/api/events). Using the Full Refresh (Overwrite) sync from Airbyte will delete the events data older than 30 days from your target destination. Use an Append sync mode to ensure historical data is retained.

For detailed information on supported sync modes, supported streams, performance considerations, refer to the full documentation for [Stripe](https://docs.airbyte.com/integrations/sources/stripe).
92 changes: 58 additions & 34 deletions docs/integrations/sources/stripe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,50 @@
# Stripe

:::warning
Stripe API Restriction: Access to the events endpoint is [guaranteed only for the last 30 days](https://stripe.com/docs/api/events). Using the full-refresh-overwrite sync from Airbyte will delete the events data older than 30 days from your target destination.
:::

This page guides you through the process of setting up the Stripe source connector.
This page contains the setup guide and reference information for the Stripe source connector.

## Prerequisites

- Your [Stripe `Account ID`](https://dashboard.stripe.com/settings/account)
- Your [Stripe `Secret Key`](https://dashboard.stripe.com/apikeys)
- Access to the Stripe account containing the data you wish to replicate

## Setup Guide

To authenticate the Stripe connector, you need to use a Stripe API key. Although you may use an existing key, we recommend that you create a new restricted key specifically for Airbyte and grant it **Read** privileges only. We also recommend granting **Read** privileges to all available permissions, and configuring the specific data you would like to replicate within Airbyte.

### Create a Stripe Secret Key

1. Log in to your [Stripe account](https://dashboard.stripe.com/login).
2. In the top navigation bar, click **Developers**.
3. In the top-left corner, click **API keys**.
4. Click **+ Create restricted key**.
5. Choose a **Key name**, and select **Read** for all available permissions.
6. Click **Create key**. You may be prompted to enter a confirmation code sent to your email address.

For more information on Stripe API Keys, see the [Stripe documentation](https://stripe.com/docs/keys).

### Set up the Stripe source connector in Airbyte

1. Log in to your [Airbyte Cloud](https://cloud.airbyte.com/workspaces) or Airbyte Open Source account.
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+ New source**.
3. Find and select **Stripe** from the list of available sources.
4. For **Source name**, enter a name to help you identify this source.
5. For **Account ID**, enter your Stripe Account ID. This ID begins with `acct_`, and can be found in the top-right corner of your Stripe [account settings page](https://dashboard.stripe.com/settings/account).
6. For **Secret Key**, enter the restricted key you created for the connection.
7. For **Replication Start Date**, use the provided datepicker or enter a UTC date and time programmatically in the format `YYYY-MM-DDTHH:mm:ssZ`. The data added on and after this date will be replicated.
8. (Optional) For **Lookback Window**, you may specify a number of days from the present day to reread data. This allows the connector to retrieve data that might have been updated after its initial creation, and is useful for handling any post-transaction adjustments (such as tips, refunds, chargebacks, etc).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great context!


## Set up the Stripe source connector
- Leaving the **Lookback Window** at its default value of 0 means Airbyte will not re-export data after it has been synced.
- Setting the **Lookback Window** to 1 means Airbyte will re-export data from the past day, capturing any changes made in the last 24 hours.
- Setting the **Lookback Window** to 7 means Airbyte will re-export and capture any data changes within the last week.

1. Log into your [Airbyte Cloud](https://cloud.airbyte.com/workspaces) or Airbyte Open Source account.
2. Click **Sources** and then click **+ New source**.
3. On the Set up the source page, select **Stripe** from the Source type dropdown.
4. Enter a name for your source.
5. For **Account ID**, enter your [Stripe `Account ID`](https://dashboard.stripe.com/settings/account).
6. For **Secret Key**, enter your [Stripe `Secret Key`](https://dashboard.stripe.com/apikeys)
9. (Optional) For **Data Request Window**, you may specify the time window in days used by the connector when requesting data from the Stripe API. This window defines the span of time covered in each request, with larger values encompassing more days in a single request. Generally speaking, the lack of overhead from making fewer requests means a larger window is faster to sync. However, this also means the state of the sync will persist less frequently. If an issue occurs or the sync is interrupted, a larger window means more data will need to be resynced, potentially causing a delay in the overall process.

We recommend creating a secret key specifically for Airbyte to control which resources Airbyte can access. For ease of use, we recommend granting read permission to all resources and configuring which resource to replicate in the Airbyte UI. You can also use the API keys for the [test mode](https://stripe.com/docs/keys#obtain-api-keys) to try out the Stripe integration with Airbyte.
For example, if you are replicating three years worth of data:

7. For **Replication start date**, enter the date in `YYYY-MM-DDTHH:mm:ssZ` format. The data added on and after this date will be replicated.
8. For **Lookback Window in days (Optional)**, select the number of days the value in days prior to the start date that you to sync your data with. If your data is updated after setting up this connector, you can use the this option to reload data from the past N days. Example: If the Replication start date is set to `2021-01-01T00:00:00Z`, then:
- If you leave the Lookback Window in days parameter to its the default value of 0, Airbyte will sync data from the Replication start date `2021-01-01T00:00:00Z`
- If the Lookback Window in days value is set to 1, Airbyte will consider the Replication start date to be `2020-12-31T00:00:00Z`
- If the Lookback Window in days value is set to 7, Airbyte will sync data from `2020-12-25T00:00:00Z`
9. Click **Set up source**.
- A **Data Request Window** of 365 days means Airbyte makes 3 requests, each for a year. This is generally faster but risks needing to resync up to a year's data if the sync is interrupted.
- A **Data Request Window** of 30 days means 36 requests, each for a month. This may be slower but minimizes the amount of data that needs to be resynced if an issue occurs.

If you are unsure of which value to use, we recommend leaving this setting at its default value of 365 days.
10. Click **Set up source** and wait for the tests to complete.

## Supported sync modes

Expand All @@ -40,10 +57,11 @@ The Stripe source connector supports the following [sync modes](https://docs.air
Since the Stripe API does not allow querying objects which were updated since the last sync, the Stripe connector uses the `created` field to query for new data in your Stripe account.
:::

## Supported Streams
## Supported streams

The Stripe source connector supports the following streams:

- [Accounts](https://stripe.com/docs/api/accounts/list) \(Incremental\)
- [Application Fees](https://stripe.com/docs/api/application_fees) \(Incremental\)
- [Application Fee Refunds](https://stripe.com/docs/api/fee_refunds/list)
- [Authorizations](https://stripe.com/docs/api/issuing/authorizations/list) \(Incremental\)
Expand All @@ -52,18 +70,23 @@ The Stripe source connector supports the following streams:
- [Cardholders](https://stripe.com/docs/api/issuing/cardholders/list) \(Incremental\)
- [Cards](https://stripe.com/docs/api/issuing/cards/list) \(Incremental\)
- [Charges](https://stripe.com/docs/api/charges/list) \(Incremental\)
- The `amount` column defaults to the smallest currency unit. (See [charge object](https://stripe.com/docs/api/charges/object) for more details)
:::note
The `amount` column defaults to the smallest currency unit. Check [the Stripe docs](https://stripe.com/docs/api/charges/object) for more details.
:::
- [Checkout Sessions](https://stripe.com/docs/api/checkout/sessions/list)
- [Checkout Sessions Line Items](https://stripe.com/docs/api/checkout/sessions/line_items)
- [Coupons](https://stripe.com/docs/api/coupons/list) \(Incremental\)
- [CreditNotes](https://stripe.com/docs/api/credit_notes/list) \(Full Refresh\)
- [Credit Notes](https://stripe.com/docs/api/credit_notes/list) \(Full Refresh\)
- [Customer Balance Transactions](https://stripe.com/docs/api/customer_balance_transactions/list)
- [Customers](https://stripe.com/docs/api/customers/list) \(Incremental\)
- This endpoint does not include deleted customers
:::note
This endpoint does _not_ include deleted customers
:::
- [Disputes](https://stripe.com/docs/api/disputes/list) \(Incremental\)
- [Early Fraud Warnings](https://stripe.com/docs/api/radar/early_fraud_warnings/list) \(Incremental\)
- [Events](https://stripe.com/docs/api/events/list) \(Incremental\)
- The Stripe API does not guarantee access to events older than 30 days, so this stream will only pull events created from the 30 days prior to the initial sync and not from the Replication start date.
- [File Links](https://stripe.com/docs/api/file_links/list) \(Incremental\)
- [Files](https://stripe.com/docs/api/files/list) \(Incremental\)
- [Invoice Items](https://stripe.com/docs/api/invoiceitems/list) \(Incremental\)
- [Invoice Line Items](https://stripe.com/docs/api/invoices/invoice_lines)
- [Invoices](https://stripe.com/docs/api/invoices/list) \(Incremental\)
Expand All @@ -77,24 +100,25 @@ The Stripe source connector supports the following streams:
- [Products](https://stripe.com/docs/api/products/list) \(Incremental\)
- [Refunds](https://stripe.com/docs/api/refunds/list) \(Incremental\)
- [Reviews](https://stripe.com/docs/api/radar/reviews/list) \(Incremental\)
- [SetupIntents](https://stripe.com/docs/api/setup_intents/list) \(Incremental\)
- [ShippingRates](https://stripe.com/docs/api/shipping_rates/list) \(Incremental\)
- [Setup Attempts](https://stripe.com/docs/api/setup_attempts/list) \(Incremental\)
- [Setup Intents](https://stripe.com/docs/api/setup_intents/list) \(Incremental\)
- [Shipping Rates](https://stripe.com/docs/api/shipping_rates/list) \(Incremental\)
- [Subscription Items](https://stripe.com/docs/api/subscription_items/list)
- [Subscription Schedule](https://stripe.com/docs/api/subscription_schedules) \(Incremental\)
- [Subscriptions](https://stripe.com/docs/api/subscriptions/list) \(Incremental\)
- [Top Ups](https://stripe.com/docs/api/topups/list) \(Incremental\)
- [Transactions](https://stripe.com/docs/api/transfers/list) \(Incremental\)
- [Transfers](https://stripe.com/docs/api/transfers/list) \(Incremental\)
- [Transfer Reversals](https://stripe.com/docs/api/transfer_reversals/list)
- [Accounts](https://stripe.com/docs/api/accounts/list) \(Incremental\)
- [Setup Attempts](https://stripe.com/docs/api/setup_attempts/list) \(Incremental\)
- [Usage Records](https://stripe.com/docs/api/usage_records/subscription_item_summary_list)
- [TopUps](https://stripe.com/docs/api/topups/list) \(Incremental\)
- [Files](https://stripe.com/docs/api/files/list) \(Incremental\)
- [FileLinks](https://stripe.com/docs/api/file_links/list) \(Incremental\)

:::warning
**Stripe API Restriction on Events Data**: Access to the events endpoint is [guaranteed only for the last 30 days](https://stripe.com/docs/api/events) by Stripe. If you use the Full Refresh Overwrite sync, be aware that any events data older than 30 days will be **deleted** from your target destination and replaced with the data from the last 30 days only. Use an Append sync mode to ensure historical data is retained.
:::

### Data type mapping

The [Stripe API](https://stripe.com/docs/api) uses the same [JSONSchema](https://json-schema.org/understanding-json-schema/reference/index.html) types that Airbyte uses internally \(`string`, `date-time`, `object`, `array`, `boolean`, `integer`, and `number`\), so no type conversions are performed for the Stripe connector.
The [Stripe API](https://stripe.com/docs/api) uses the same [JSON Schema](https://json-schema.org/understanding-json-schema/reference/index.html) types that Airbyte uses internally \(`string`, `date-time`, `object`, `array`, `boolean`, `integer`, and `number`\), so no type conversions are performed for the Stripe connector.

### Performance considerations

Expand Down