tap-square

This tap is in development.

This is a Singer tap that produces JSON-formatted data following the Singer spec.

Description

This tap:

• Downloads data from Square API

• Extracts from the following sources to produce streams. Below is a list of all the streams available. See the streams file for a list of classes where each one has a constant indiciating if the stream's replication_method is INCREMENTAL or FULL_TABLE and what the replication_key is, usually updated_at field if it's incremental.

  • Items
  • Categories
  • Discounts
  • Taxes
  • TeamMembers
  • Locations
  • BankAccounts
  • Refunds
  • Payments
  • Payouts
  • ModifierLists
  • Inventories
  • Orders
  • Roles
  • Shifts
  • CashDrawerShifts
  • Customers

• Includes a schema for each resource reflecting most recent tested data retrieved using the api. See the schema folder for details.

• Some streams incrementally pull data based on the previously saved state. See the bookmarking strategy section for more details.

Bookmarking Strategy

The Square API for some objects supports a begin_time parameter that limits the query to only return objects with a created_at after the begin_time. For an example see Square's list payments api documentation. Others support searching and sorting by custom fields and this tap bookmarks using the updated_at field in those cases. For example, see Square's search orders api documentation.

Finally, some apis do not support incremental search, but to limit data volume output by the tap the tap queries for all and then filters out by the updated_at field. The shifts stream is one such example.

Some apis allow sorting based on the updated_at value while others do not. If sorting is not supported one can only use the maximum updated_at value if and only if the stream has completly synced. To bookmark progress during the sync of a stream the cursor value is saved and used to paginate through the API (https://developer.squareup.com/docs/working-with-apis/pagination).

Authentication

Authentication is handled with oauth v2. In the tap configuration the following fields are required for authentication to work correctly:

• client_id
• client_secret
• refresh_token

These values are all obtained from the oauth steps documented on square's documentation page. The url for oauth credentials once logged in should be something like https://developer.squareup.com/apps/<app_id>/oauth. The application id and secret refer to the client_id and client_secret values above. The refresh token is obtained by selecting Authorize an Account and then selecting the account.

Quick Start

1. Install

   Clone this repository, and then install using setup.py. We recommend using a virtualenv:

   $ virtualenv -p python3 venv
   $ source venv/bin/activate
   $ pip install -e .

2. Create your tap's config.json file. The tap config file for this tap should include these entries:

   • start_date - the default value to use if no bookmark exists for an endpoint (rfc3339 date string)
   • user_agent (string, optional): Process and email for API logging purposes. Example: tap-square <api_user_email@your_company.com>
   • sandbox (string, optional): Whether to communication with square's sandbox or prod account for this application. If you're not sure leave out. Defaults to false.

   And the other values mentioned in the authentication section above.

   {
   	"client_id": "<app_id>",
   	"start_date": "2020-08-21T00:00:00Z",
   	"refresh_token": "<refresh_token>",
   	"client_secret": "<app_secret>",
   	"sandbox": "<true|false>",
   	"user_agent": "Stitch Tap (+support@stitchdata.com)"
   }

3. Run the Tap in Discovery Mode This creates a catalog.json for selecting objects/fields to integrate:

   tap-square --config config.json --discover > catalog.json

   See the Singer docs on discovery mode here.

4. Run the Tap in Sync Mode (with catalog) and write out to state file

   For Sync mode:

   $ tap-square --config tap_config.json --catalog catalog.json >> state.json
   $ tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

   To load to json files to verify outputs:

   $ tap-square --config tap_config.json --catalog catalog.json | target-json >> state.json
   $ tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

   To pseudo-load to Stitch Import API with dry run:

   $ tap-square --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run >> state.json
   $ tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

---

Copyright © 2020 Stitch