tap-closeio

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

• Pulls raw data from Close.io's REST API
• Extracts the following resources from Close.io:
  • Activities
  • Leads
• Outputs the schema for each resource
• Incrementally pulls data based on the input state

Quick start

1. Install

   > pip install tap-closeio

2. Get your Close.io API Key

   Login to your Close.io account, navigate to your account settings and "Your API Keys". Generate a New API Key, you'll need it for the next step.

3. Create the config file

   Create a JSON file called config.json containing the api key you just generated

   {
       "start_date": "2010-01-01",
       "api_key": "your-api-token",
       "date_window": 10
   }

   The start_date is the date at which the tap will begin syncing data. Ie. if there is data in your Close.io account older than start_date, it will not be synced.

   The date_window is the window in the which API calls will take place. The default window size is 15 days.

4. Run the tap in discovery mode

   tap-closeio --config config.json --discover

   See the Singer docs on discovery mode here.

5. Run the tap in sync mode

   tap-closeio --config config.json --properties catalog.json

Event Logs

The event log endpoint returns two fields that are troublesome in terms of describing them with a JSON schema and fitting them into tabular structures, like PostgreSQL or Redshift. They are the data and previous_data fields. These fields vary depending on the type of the event, meaning an event for a lead will have a vastly different structure than an event for a task. Due to the varying nature of these fields, the tap JSON-encodes the fields during sync.

Activities

The activities endpoint does not provide a way to filter data based on when an activity was updated. Because of this, there is no way to have the tap sync changes to previously-synced activities without syncing the entire data set during every run. As an alternative, your configuration file can contain the key activities_window_seconds. When provided, any activity which was created activities_window_seconds seconds before the bookmark in the state.json file will be synced. For example, if your configuration file includes

{
    ...,
    "activities_window_seconds": 3600
}

and the previous run of tap synced activities up until 10am today, the next sync will start syncing activities that were created at 9am today.

---

Copyright © 2017 Stitch