Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 New Source: Pardot #7091

Merged
merged 23 commits into from
Nov 19, 2021
Merged

🎉 New Source: Pardot #7091

merged 23 commits into from
Nov 19, 2021

Conversation

tuanchris
Copy link
Contributor

@tuanchris tuanchris commented Oct 16, 2021 •
edited
Loading

What

How

Added support for the following streams:

  • EmailClicks
  • Campaigns
  • ListMembership
  • Lists
  • Opportunities
  • ProspectAccounts
  • Prospects
  • Users
  • Visitors
  • VisitorActivities
  • Visits
  • EmailStats

All streams are running with basic normalization
image

Recommended reading order

  1. x.java
  2. y.python

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • [] Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • [] Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/SUMMARY.md
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions
  • Connector added to connector index like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • If adding a new generator, add it to the list of scaffold modules being tested
  • The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
  • Documentation which references the generator is updated as needed.

@github-actions github-actions bot added the area/connectors Connector related issues label Oct 16, 2021
@marcosmarxm
Copy link
Member

marcosmarxm commented Oct 16, 2021 •
edited
Loading

Amazing @tuanchris we're going to review this Monday! Do you mind applying ./gradlew format to correct some EOF and spaces?

@tuanchris
Copy link
Contributor Author

@marcosmarxm All done :D
image

@marcosmarxm marcosmarxm self-assigned this Oct 20, 2021
@marcosmarxm marcosmarxm mentioned this pull request Oct 22, 2021
Closed
@marcosmarxm
Copy link
Member

Please share the unit/integration test screen shoot showing the tests are passing.

@tuanchris
Copy link
Contributor Author

@marcosmarxm Here are the screenshots:
Integration test:
image
Unit test:
image

marcosmarxm
marcosmarxm reviewed Oct 25, 2021
View reviewed changes
airbyte-integrations/connectors/source-pardot/source_pardot/spec.json
"description": "UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.",
"type": "string",
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$",
"default": null,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this if doesn't exist a default value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some doc to make this clear. If this is null then the connector will not apply a time filter and integrate everything.

@tuanchris tuanchris requested a review from marcosmarxm October 26, 2021 00:06
@marcosmarxm
Copy link
Member

Hi Tuan so sorry for the delay in publishing this, we're waiting for the integration account to be able to publish this connector.

@marcosmarxm
Copy link
Member

Hello! Sorry to not finished your contribution before the date stipulated in the contest. All contributions made before 15-November are eligible to receive the award. We're trying to review your contribution as soon as possible.

@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Nov 16, 2021
@github-actions github-actions bot added area/api Related to the api area/platform issues related to the platform CDK Connector Development Kit normalization area/documentation Improvements or additions to documentation and removed area/documentation Improvements or additions to documentation area/api Related to the api CDK Connector Development Kit normalization area/platform issues related to the platform labels Nov 19, 2021
@github-actions github-actions bot removed the area/documentation Improvements or additions to documentation label Nov 19, 2021
marcosmarxm
marcosmarxm approved these changes Nov 19, 2021
View reviewed changes
Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @tuanchris thanks!

@marcosmarxm marcosmarxm temporarily deployed to more-secrets November 19, 2021 01:02 Inactive
@marcosmarxm marcosmarxm merged commit 7b2429d into airbytehq:master Nov 19, 2021
This was referenced Nov 19, 2021
Closed
Closed
Merged
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
@tuanchris @marcosmarxm @schlattk
🎉 New Source: Pardot (airbytehq#7091)
baa4b95 
* initial commit

* initial commit

* add more streams

* first streams with incremental

* working first two incremental models

* check point

* add visit child stream

* checkpoint

* add email_stats stream

* add email_id to schema

* clean up

* fix unit test & linting

* clean up

* lint

* add doc

* add documentation

* move docs and run format

* remove from summary

* update check_connection function

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
@marcosmarxm marcosmarxm mentioned this pull request May 4, 2023
Closed
@nataliekwong nataliekwong mentioned this pull request Nov 15, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marcosmarxm marcosmarxm marcosmarxm approved these changes

Assignees

@marcosmarxm marcosmarxm

Labels
area/connectors Connector related issues community connectors/source/mongodb-strict-encrypt connectors/source/pardot
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@tuanchris @marcosmarxm @octavia-squidington-iii @octavia-squidington-iv