Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Posthog: increase batch size #25603

Merged
merged 7 commits into from
Apr 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21171,7 +21171,7 @@
"sourceDefinitionId": "af6d50ee-dddf-4126-a8ee-7faee990774f",
"name": "PostHog",
"dockerRepository": "airbyte/source-posthog",
"dockerImageTag": "0.1.9",
"dockerImageTag": "0.1.10",
"documentationUrl": "https://docs.airbyte.com/integrations/sources/posthog",
"icon": "posthog.svg",
"sourceType": "api",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1607,7 +1607,7 @@
- name: PostHog
sourceDefinitionId: af6d50ee-dddf-4126-a8ee-7faee990774f
dockerRepository: airbyte/source-posthog
dockerImageTag: 0.1.9
dockerImageTag: 0.1.10
documentationUrl: https://docs.airbyte.com/integrations/sources/posthog
icon: posthog.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12118,7 +12118,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-posthog:0.1.9"
- dockerImage: "airbyte/source-posthog:0.1.10"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/posthog"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-posthog/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.9
LABEL io.airbyte.version=0.1.10
LABEL io.airbyte.name=airbyte/source-posthog
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ definitions:
pagination_strategy:
type: "CursorPagination"
cursor_value: "{{ response['next'] }}"
page_size: 100
page_size: 10000
page_token_option:
type: RequestPath
$parameters:
Expand Down
2 changes: 1 addition & 1 deletion connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@
| **Pocket** | <img alt="Pocket icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/pocket.svg" height="30" height="30"/> | Source | airbyte/source-pocket:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/pocket) | [connectors/source/pocket](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/pocket) | [source-pocket](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-pocket) | <small>`b0dd65f1-081f-4731-9c51-38e9e6aa0ebf`</small> |
| **PokeAPI** | <img alt="PokeAPI icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/pokeapi.svg" height="30" height="30"/> | Source | airbyte/source-pokeapi:0.1.5 | alpha | [docs](https://docs.airbyte.com/integrations/sources/pokeapi) | [connectors/source/pokeapi](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/pokeapi) | [source-pokeapi](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-pokeapi) | <small>`6371b14b-bc68-4236-bfbd-468e8df8e968`</small> |
| **Polygon Stock API** | <img alt="Polygon Stock API icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/polygon.svg" height="30" height="30"/> | Source | airbyte/source-polygon-stock-api:0.1.1 | alpha | [docs](https://docs.airbyte.com/integrations/sources/polygon-stock-api) | [connectors/source/polygon-stock-api](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/polygon-stock-api) | [source-polygon-stock-api](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-polygon-stock-api) | <small>`5807d72f-0abc-49f9-8fa5-ae820007032b`</small> |
| **PostHog** | <img alt="PostHog icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/posthog.svg" height="30" height="30"/> | Source | airbyte/source-posthog:0.1.9 | beta | [docs](https://docs.airbyte.com/integrations/sources/posthog) | [connectors/source/posthog](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/posthog) | [source-posthog](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-posthog) | <small>`af6d50ee-dddf-4126-a8ee-7faee990774f`</small> |
| **PostHog** | <img alt="PostHog icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/posthog.svg" height="30" height="30"/> | Source | airbyte/source-posthog:0.1.10 | beta | [docs](https://docs.airbyte.com/integrations/sources/posthog) | [connectors/source/posthog](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/posthog) | [source-posthog](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-posthog) | <small>`af6d50ee-dddf-4126-a8ee-7faee990774f`</small> |
| **Postgres** | <img alt="Postgres icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/postgresql.svg" height="30" height="30"/> | Source | airbyte/source-postgres:2.0.27 | generally_available | [docs](https://docs.airbyte.com/integrations/sources/postgres) | [connectors/source/postgres](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/postgres) | [source-postgres](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-postgres) | <small>`decd338e-5647-4c0b-adf4-da0e75f5a750`</small> |
| **Postmark App** | <img alt="Postmark App icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/postmark.svg" height="30" height="30"/> | Source | airbyte/source-postmarkapp:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/postmarkapp) | [connectors/source/postmarkapp](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/postmarkapp) | [source-postmarkapp](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-postmarkapp) | <small>`cde75ca1-1e28-4a0f-85bb-90c546de9f1f`</small> |
| **PrestaShop** | <img alt="PrestaShop icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/prestashop.svg" height="30" height="30"/> | Source | airbyte/source-prestashop:0.3.1 | beta | [docs](https://docs.airbyte.com/integrations/sources/prestashop) | [connectors/source/prestashop](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/prestashop) | [source-prestashop](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-prestashop) | <small>`d60a46d4-709f-4092-a6b7-2457f7d455f5`</small> |
Expand Down
48 changes: 25 additions & 23 deletions docs/integrations/sources/posthog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@ This page contains the setup guide and reference information for the PostHog sou

## Prerequisites

* api_key - obtain Private API Key for your account following these [steps](https://posthog.com/docs/api/overview#how-to-obtain-a-personal-api-key)
* base_url - 'https://app.posthog.com' by default, but it can be different if self-hosted posthog instances is used
- api_key - obtain Private API Key for your account following these [steps](https://posthog.com/docs/api/overview#how-to-obtain-a-personal-api-key)
- base_url - 'https://app.posthog.com' by default, but it can be different if self-hosted posthog instances is used

## Setup guide

### Step 1: Set up PostHog

* PostHog Account
- PostHog Account

## Step 2: Set up the PostHog connector in Airbyte

Expand All @@ -20,28 +21,28 @@ This page contains the setup guide and reference information for the PostHog sou
2. In the left navigation bar, click **Sources**. In the top-right corner, click **+new source**.
3. On the Set up the source page, enter the name for the PostHog connector and select **PostHog** from the Source type dropdown.
4. Enter your `apikey`.
5. Enter your `start_date`.
5. Enter your `start_date`.
6. Change default `base_url` if self-hosted posthog instances is used
7. Click **Set up source**.

### For Airbyte OSS:

1. Navigate to the Airbyte Open Source dashboard.
2. Set the name for your source.
2. Set the name for your source.
3. Enter your `api_key`.
4. Enter your `start_date`.
4. Enter your `start_date`.
5. Change default `base_url` if self-hosted posthog instances is used
6. Click **Set up source**.

## Supported streams and sync modes

* [Projects](https://posthog.com/docs/api/projects)
* [Annotations](https://posthog.com/docs/api/annotations)
* [Cohorts](https://posthog.com/docs/api/cohorts)
* [Events](https://posthog.com/docs/api/events) \(Incremental\)
* [FeatureFlags](https://posthog.com/docs/api/feature-flags)
* [Insights](https://posthog.com/docs/api/insights)
* [Persons](https://posthog.com/docs/api/people)
- [Projects](https://posthog.com/docs/api/projects)
- [Annotations](https://posthog.com/docs/api/annotations)
- [Cohorts](https://posthog.com/docs/api/cohorts)
- [Events](https://posthog.com/docs/api/events) \(Incremental\)
- [FeatureFlags](https://posthog.com/docs/api/feature-flags)
- [Insights](https://posthog.com/docs/api/insights)
- [Persons](https://posthog.com/docs/api/people)

### Performance considerations

Expand All @@ -50,15 +51,16 @@ Please [create an issue](https://github.com/airbytehq/airbyte/issues) if you see

## Changelog

| Version | Date | Pull Request | Subject |
|:--------|:-----------| :--- |:-------------------------------------------------------------------------------------------------------------------|
| 0.1.9 | 2023-02-13 | [22906](https://github.com/airbytehq/airbyte/pull/22906) | Specified date formatting in specification |
| Version | Date | Pull Request | Subject |
| :------ | :--------- | :------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------- |
| 0.1.10 | 2023-04-15 | [24084](https://github.com/airbytehq/airbyte/pull/24084) | Increase `events` streams batch size |
| 0.1.9 | 2023-02-13 | [22906](https://github.com/airbytehq/airbyte/pull/22906) | Specified date formatting in specification |
| 0.1.8 | 2022-11-11 | [18993](https://github.com/airbytehq/airbyte/pull/18993) | connector migrated to low-code, added projects,insights streams, added project based slicing for all other streams |
| 0.1.7 | 2022-07-26 | [14585](https://github.com/airbytehq/airbyte/pull/14585) | Add missing 'properties' field to event attributes |
| 0.1.6 | 2022-01-20 | [8617](https://github.com/airbytehq/airbyte/pull/8617) | Update connector fields title/description |
| 0.1.5 | 2021-12-24 | [9082](https://github.com/airbytehq/airbyte/pull/9082) | Remove obsolete session_events and insights streams |
| 0.1.4 | 2021-09-14 | [6058](https://github.com/airbytehq/airbyte/pull/6058) | Support self-hosted posthog instances |
| 0.1.3 | 2021-07-20 | [4001](https://github.com/airbytehq/airbyte/pull/4001) | Incremental streams read only relevant pages |
| 0.1.2 | 2021-07-15 | [4692](https://github.com/airbytehq/airbyte/pull/4692) | Use account information for checking the connection |
| 0.1.1 | 2021-07-05 | [4539](https://github.com/airbytehq/airbyte/pull/4539) | Add `AIRBYTE_ENTRYPOINT` env variable for kubernetes support |
| 0.1.0 | 2021-06-08 | [3768](https://github.com/airbytehq/airbyte/pull/3768) | Initial Release |
| 0.1.6 | 2022-01-20 | [8617](https://github.com/airbytehq/airbyte/pull/8617) | Update connector fields title/description |
| 0.1.5 | 2021-12-24 | [9082](https://github.com/airbytehq/airbyte/pull/9082) | Remove obsolete session_events and insights streams |
| 0.1.4 | 2021-09-14 | [6058](https://github.com/airbytehq/airbyte/pull/6058) | Support self-hosted posthog instances |
| 0.1.3 | 2021-07-20 | [4001](https://github.com/airbytehq/airbyte/pull/4001) | Incremental streams read only relevant pages |
| 0.1.2 | 2021-07-15 | [4692](https://github.com/airbytehq/airbyte/pull/4692) | Use account information for checking the connection |
| 0.1.1 | 2021-07-05 | [4539](https://github.com/airbytehq/airbyte/pull/4539) | Add `AIRBYTE_ENTRYPOINT` env variable for kubernetes support |
| 0.1.0 | 2021-06-08 | [3768](https://github.com/airbytehq/airbyte/pull/3768) | Initial Release |