Skip to content

Commit

Permalink
šŸ› Source Shopify: Minor specification re-order, fixed links in documeā€¦
Browse files Browse the repository at this point in the history
ā€¦ntation (#10794)
  • Loading branch information
bazarnov authored Mar 4, 2022
1 parent ed19bcc commit 21a8844
Show file tree
Hide file tree
Showing 6 changed files with 87 additions and 82 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,7 @@
- name: Shopify
sourceDefinitionId: 9da77001-af33-4bcd-be46-6252bf9342b9
dockerRepository: airbyte/source-shopify
dockerImageTag: 0.1.33
dockerImageTag: 0.1.34
documentationUrl: https://docs.airbyte.io/integrations/sources/shopify
icon: shopify.svg
sourceType: api
Expand Down
25 changes: 14 additions & 11 deletions airbyte-config/init/src/main/resources/seed/source_specs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7306,7 +7306,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-shopify:0.1.33"
- dockerImage: "airbyte/source-shopify:0.1.34"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/shopify"
connectionSpecification:
Expand All @@ -7321,19 +7321,13 @@
shop:
type: "string"
title: "Shopify Store"
description: "The name of the shopify store. For https://EXAMPLE.myshopify.com,\
\ the shop name is 'EXAMPLE'."
start_date:
type: "string"
title: "Start Date"
description: "The date you would like to replicate data from. Format: YYYY-MM-DD.\
\ Any data before this date will not be replicated."
examples:
- "2021-01-01"
pattern: "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"
description: "The name of your Shopify store found in the URL. For example,\
\ if your URL was https://NAME.myshopify.com, then the name would be 'NAME'."
order: 1
credentials:
title: "Shopify Authorization Method"
type: "object"
order: 2
oneOf:
- type: "object"
title: "OAuth2.0"
Expand Down Expand Up @@ -7381,6 +7375,15 @@
description: "The API Password for your private application in the\
\ `Shopify` store."
airbyte_secret: true
start_date:
type: "string"
title: "Start Date"
description: "The date you would like to replicate data from. Format: YYYY-MM-DD.\
\ Any data before this date will not be replicated."
examples:
- "2021-01-01"
pattern: "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"
order: 3
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-shopify/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@ COPY source_shopify ./source_shopify
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.33
LABEL io.airbyte.version=0.1.34
LABEL io.airbyte.name=airbyte/source-shopify
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"type": "object",
"type": ["null", "object"],
"properties": {
"note_attributes": {
"type": ["null", "array"],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,13 @@
"shop": {
"type": "string",
"title": "Shopify Store",
"description": "The name of the shopify store. For https://EXAMPLE.myshopify.com, the shop name is 'EXAMPLE'."
},
"start_date": {
"type": "string",
"title": "Start Date",
"description": "The date you would like to replicate data from. Format: YYYY-MM-DD. Any data before this date will not be replicated.",
"examples": ["2021-01-01"],
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"
"description": "The name of your Shopify store found in the URL. For example, if your URL was https://NAME.myshopify.com, then the name would be 'NAME'.",
"order": 1
},
"credentials": {
"title": "Shopify Authorization Method",
"type": "object",
"order": 2,
"oneOf": [
{
"type": "object",
Expand Down Expand Up @@ -76,6 +71,14 @@
}
}
]
},
"start_date": {
"type": "string",
"title": "Start Date",
"description": "The date you would like to replicate data from. Format: YYYY-MM-DD. Any data before this date will not be replicated.",
"examples": ["2021-01-01"],
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$",
"order": 3
}
}
},
Expand Down
119 changes: 59 additions & 60 deletions docs/integrations/sources/shopify.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,71 +13,24 @@ This source can sync data for the [Shopify API](https://help.shopify.com/en/api/

This Source Connector is based on a [Airbyte CDK](https://docs.airbyte.io/connector-development/cdk-python).

## Troubleshooting

Check out common troubleshooting issues for the BigQuery destination connector on our Discourse [here](https://discuss.airbyte.io/tags/c/connector/11/source-shopify).

### Output schema

This Source is capable of syncing the following core Streams:

* [Abandoned Checkouts](https://help.shopify.com/en/api/reference/orders/abandoned_checkouts)
* [Collects](https://help.shopify.com/en/api/reference/products/collect)
* [Custom Collections](https://help.shopify.com/en/api/reference/products/customcollection)
* [Customers](https://help.shopify.com/en/api/reference/customers)
* [Draft Orders](https://help.shopify.com/en/api/reference/orders/draftorder)
* [Discount Codes](https://shopify.dev/docs/admin-api/rest/reference/discounts/discountcode)
* [Metafields](https://help.shopify.com/en/api/reference/metafield)
* [Orders](https://help.shopify.com/en/api/reference/orders)
* [Orders Refunds](https://shopify.dev/api/admin/rest/reference/orders/refund)
* [Orders Risks](https://shopify.dev/api/admin/rest/reference/orders/order-risk)
* [Products](https://help.shopify.com/en/api/reference/products)
* [Transactions](https://help.shopify.com/en/api/reference/orders/transaction)
* [Tender Transactions](https://shopify.dev/api/admin-rest/2022-01/resources/tendertransaction))
* [Pages](https://help.shopify.com/en/api/reference/online-store/page)
* [Price Rules](https://help.shopify.com/en/api/reference/discounts/pricerule)
* [Locations](https://shopify.dev/api/admin-rest/2021-10/resources/location)
* [InventoryItems](https://shopify.dev/api/admin-rest/2021-10/resources/inventoryItem)
* [InventoryLevels](https://shopify.dev/api/admin-rest/2021-10/resources/inventorylevel)
* [Fulfillment Orders](https://shopify.dev/api/admin-rest/2021-07/resources/fulfillmentorder)
* [Fulfillments](https://shopify.dev/api/admin-rest/2021-07/resources/fulfillment)
* [Shop](https://shopify.dev/api/admin-rest/2021-07/resources/shop)

#### NOTE:

For better experience with `Incremental Refresh` the following is recommended:

* `Order Refunds`, `Order Risks`, `Transactions` should be synced along with `Orders` stream.
* `Discount Codes` should be synced along with `Price Rules` stream.

If child streams are synced alone from the parent stream - the full sync will take place, and the records are filtered out afterwards.

### Data type mapping

| Integration Type | Airbyte Type | Notes |
| :--- | :--- | :--- |
| `string` | `string` | |
| `number` | `number` | |
| `array` | `array` | |
| `object` | `object` | |
| Integration Type | Airbyte Type |
| :--- | :--- |
| `string` | `string` |
| `number` | `number` |
| `array` | `array` |
| `object` | `object` |
| `boolean` | `boolean` |

### Features

| Feature | Supported?\(Yes/No\) | Notes |
| :--- | :--- | :--- |
| Full Refresh Sync | Yes | |
| Incremental - Append Sync | Yes | |
| Namespaces | No | |

### Performance considerations

Shopify has some [rate limit restrictions](https://shopify.dev/concepts/about-apis/rate-limits). Typically, there should not be issues with throttling or exceeding the rate limits but in some edge cases, user can receive the warning message as follows:
| Feature | Supported?\(Yes/No\) |
| :--- | :--- |
| Full Refresh Sync | Yes |
| Incremental - Append Sync | Yes |
| Namespaces | No |

```text
"Caught retryable error '<some_error> or null' after <some_number> tries. Waiting <some_number> seconds then retrying..."
```

This is expected when the connector hits the 429 - Rate Limit Exceeded HTTP Error. With given error message the sync operation is still goes on, but will require more time to finish.

## Getting started

Expand All @@ -98,11 +51,57 @@ This connector support both: `OAuth 2.0` and `API PASSWORD` (for private applica
2. Proceed the authentication using your credentials for your Shopify account.


### Output Streams Schemas

This Source is capable of syncing the following core Streams:

* [Abandoned Checkouts](https://shopify.dev/api/admin-rest/2022-01/resources/abandoned-checkouts#top)
* [Collects](https://shopify.dev/api/admin-rest/2022-01/resources/collect#top)
* [Custom Collections](https://shopify.dev/api/admin-rest/2022-01/resources/customcollection#top)
* [Customers](https://shopify.dev/api/admin-rest/2022-01/resources/customer#top)
* [Draft Orders](https://shopify.dev/api/admin-rest/2022-01/resources/draftorder#top)
* [Discount Codes](https://shopify.dev/api/admin-rest/2022-01/resources/discountcode#top)
* [Metafields](https://shopify.dev/api/admin-rest/2022-01/resources/metafield#top)
* [Orders](https://shopify.dev/api/admin-rest/2022-01/resources/order#top)
* [Orders Refunds](https://shopify.dev/api/admin-rest/2022-01/resources/refund#top)
* [Orders Risks](https://shopify.dev/api/admin-rest/2022-01/resources/order-risk#top)
* [Products](https://shopify.dev/api/admin-rest/2022-01/resources/product#top)
* [Transactions](https://shopify.dev/api/admin-rest/2022-01/resources/transaction#top)
* [Tender Transactions](https://shopify.dev/api/admin-rest/2022-01/resources/tendertransaction)
* [Pages](https://shopify.dev/api/admin-rest/2022-01/resources/page#top)
* [Price Rules](https://shopify.dev/api/admin-rest/2022-01/resources/pricerule#top)
* [Locations](https://shopify.dev/api/admin-rest/2022-01/resources/location)
* [InventoryItems](https://shopify.dev/api/admin-rest/2022-01/resources/inventoryItem)
* [InventoryLevels](https://shopify.dev/api/admin-rest/2021-01/resources/inventorylevel)
* [Fulfillment Orders](https://shopify.dev/api/admin-rest/2022-01/resources/fulfillmentorder)
* [Fulfillments](https://shopify.dev/api/admin-rest/2022-01/resources/fulfillment)
* [Shop](https://shopify.dev/api/admin-rest/2022-01/resources/shop)

#### Notes:

For better experience with `Incremental Refresh` the following is recommended:

* `Order Refunds`, `Order Risks`, `Transactions` should be synced along with `Orders` stream.
* `Discount Codes` should be synced along with `Price Rules` stream.

If child streams are synced alone from the parent stream - the full sync will take place, and the records are filtered out afterwards.

### Performance considerations

Shopify has some [rate limit restrictions](https://shopify.dev/concepts/about-apis/rate-limits). Typically, there should not be issues with throttling or exceeding the rate limits but in some edge cases, user can receive the warning message as follows:

```text
"Caught retryable error '<some_error> or null' after <some_number> tries. Waiting <some_number> seconds then retrying..."
```

This is expected when the connector hits the 429 - Rate Limit Exceeded HTTP Error. With given error message the sync operation is still goes on, but will require more time to finish.

## Changelog

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.33 | 2022-02-17 | [10419](https://github.com/airbytehq/airbyte/pull/10419) | Fixed wrong field type for tax_exemptions |
| 0.1.34 | 2022-03-02 | [10794](https://github.com/airbytehq/airbyte/pull/10794) | Minor specification re-order, fixed links in documentation |
| 0.1.33 | 2022-02-17 | [10419](https://github.com/airbytehq/airbyte/pull/10419) | Fixed wrong field type for tax_exemptions for `Abandoned_checkouts` stream |
| 0.1.32 | 2022-02-18 | [10449](https://github.com/airbytehq/airbyte/pull/10449) | Added `tender_transactions` stream |
| 0.1.31 | 2022-02-08 | [10175](https://github.com/airbytehq/airbyte/pull/10175) | Fixed compatibility issues for legacy user config |
| 0.1.30 | 2022-01-24 | [9648](https://github.com/airbytehq/airbyte/pull/9648) | Added permission validation before sync |
Expand Down

0 comments on commit 21a8844

Please sign in to comment.