Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source Klaviyo & Source Marketo - fix builds #18101

Merged
merged 1 commit into from
Oct 18, 2022
Merged

Source Klaviyo & Source Marketo - fix builds #18101

merged 1 commit into from
Oct 18, 2022

Conversation

davydov-d
Copy link
Collaborator

@github-actions github-actions bot added the area/connectors Connector related issues label Oct 18, 2022
@davydov-d
Copy link
Collaborator Author

davydov-d commented Oct 18, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-klaviyo

🕑 connectors/source-klaviyo https://github.com/airbytehq/airbyte/actions/runs/3272691377
✅ connectors/source-klaviyo https://github.com/airbytehq/airbyte/actions/runs/3272691377
Python tests coverage:

Name                         Stmts   Miss  Cover
------------------------------------------------
source_klaviyo/__init__.py       2      0   100%
source_klaviyo/streams.py      136     10    93%
source_klaviyo/source.py        16      3    81%
------------------------------------------------
TOTAL                          154     13    92%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     10    87%   15-16, 24-30, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1351    451    67%

Build Passed

Test summary info:

All Passed

@davydov-d
Copy link
Collaborator Author

davydov-d commented Oct 18, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-marketo

🕑 connectors/source-marketo https://github.com/airbytehq/airbyte/actions/runs/3272691901
✅ connectors/source-marketo https://github.com/airbytehq/airbyte/actions/runs/3272691901
Python tests coverage:

Name                         Stmts   Miss  Cover
------------------------------------------------
source_marketo/__init__.py       2      0   100%
source_marketo/utils.py         33      1    97%
source_marketo/source.py       271     29    89%
------------------------------------------------
TOTAL                          306     30    90%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     10    87%   15-16, 24-30, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1351    451    67%

Build Passed

Test summary info:

All Passed

@davydov-d davydov-d merged commit 9d23d48 into master Oct 18, 2022
@davydov-d davydov-d deleted the ddavydov/#17506-fix-klaviyo-and-marketo-builds branch October 18, 2022 12:04
YatsukBogdan1 pushed a commit that referenced this pull request Oct 18, 2022
@davydov-d @YatsukBogdan1
#17506 fix klaviyo & marketo expected_records (#18101)
a8137fa
YatsukBogdan1 added a commit that referenced this pull request Oct 19, 2022
@YatsukBogdan1 @edmundito
@terencecho @ryankfu @gosusnp @xpuska513 @evantahler @girarda @suhomud @octavia-squidington-iii @github-actions @sfc-gh-pkommini @benmoriceau @Amruta-Ranade @artem1205 @bazarnov @brianjlai @mfsiega-airbyte @alexander-marquardt @davydov-d
🪟 🧹 Implement ColumnSortButton component (#17798)
444f060 
* Implement ColumnSortButton component

* Updates component name; Moves component to ui/Table folder; Refactors formattedMessageId property into using render content as children directly; Removes minor SortIcon component

* Update airbyte-webapp/src/App.tsx

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

* Updates next properties: wasActive -> isActive, lowToLarge -> isAscending

* Skip psql stop in acceptance test for gke (#18023)

* Checks for iterator hasNext element (#18041)

* Checks for iterator hasNext element

* Fix linter with newline

* Add Message Migration to Destination Connection Checks (#17954)

* Add Message Migration to Destination Connection Checks

* Fix test setup

* Update helm release workflow (#18048)

* Update workflow

* Update trigger rules

* fix: Update release workflow with abillity to add tags

* Update workflow

* Remove unused `airbyte-cli` (#18009)

* 🐛  [low-code] $options shouldn't overwrite values that are already defined (#18060)

* fix

* Add missing test

* remove prints

* extract to method

* rename

* Add missing test

* rename

* bump

* Update helm chart comments (#18072)

* Update helm charts (#18073)

* add test

* fix chart.yaml

* 16250 Destination Redis: Add SSH support (#17951)

* 16250 Destination Redis: Add SSH support

* 16250 Resolve port issue

* 11679 Bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Bump helm chart version reference to 0.40.20 (#18074)

* Bump helm chart version reference to 0.40.20

* remove binary

Co-authored-by: xpuska513 <xpuska513@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* Helm Chart: Create service annotations for airbyte-server (#17932)

* Support annotations for airbyte-server as well, update version and update docs.

* Fix auto-indent.

Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* Bmoric/remove dep server worker (#17894)

* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* TMP

* Add missing dependencies

* PR comments

* Tmp

* [ci skip] Tmp

* Fix acceptance test and add the seed dependency

* Fix build

* For diff

* tmp

* Build pass

* make the worker to be  on the platform only

* fix setting.yaml

* Fix pmd

* Fix Cron

* Add chart

* Fix cron

* Fix server build.gradle

* Fix jar conflict

* PR comments

* Add cron micronaut environemnt

* Updated connector catalog page (#18076)

* Move the port forward outside of the main docker-compose (#17864)

* Bump Airbyte version from 0.40.14 to 0.40.15 (#17970)

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>

* 🎉 Source Shopify: Add metafield streams (#17962)

* 🎉 Source Shopify: Add metafield streams

* Source Shopify: fix unittest

* Source Shopify: docs update

* Source Shopify: fix backward compatibility test

* Source Shopify: fix schemas

* Source Shopify: fix state filter

* Source Shopify: refactor & optimize

* Source Shopify: fix test privileges

* Source Shopify: fix stream filter

* Source Shopify: fix streams

* Source Shopify: update abnormal state

* Source Shopify: fix abnormal state streams

* Source Shopify: fix streams

* updated methods, formated code

* Source Shopify: typo fix

* auto-bump connector version

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* fix check for streams that do not use a stream slicer (#18080)

* fix check for streams that do not use a stream slicer

* increment version and changelog before publish

* tolerate database nulls in webhook operation configs (#18084)

* Implement webhook operation in the sync workflow (#18022)

Implements the webhook operation as part of the sync workflow.

- Introduces the new activity implementation
- Updates the various interfaces that pass input to get the relevant configs to the sync workflow
- Hooks the new activity into the sync workflow
- Passes the webhook configs along into the sync workflow job

* Bump helm chart version reference to 0.40.22 (#18077)

* Added new "filters" python file, along with a "hash" filter. This can… (#18000)

* Added new "filters" python file, along with a "hash" filter. This can be extended to include other custom filters in the future.

* Added additional comments

* Moved usage of the hash_obj inside the conditional that confirms it exists

* Moved the hash function call inside a condition to ensure that it exists

* Fixed the application of the salt , so that it does not modify the hash unless it is actually passed in.

* Added unit tests to validate new jinja hash functionality

* Updated unit test to pass numeric value as a float instead of string

* Removed unreferenced import to pytest

* Updated version

* format

* format

* format

* format

* format

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>

* Bump helm chart version reference to 0.40.24 (#18081)

* Bump helm chart version reference to 0.40.24

* Update .gitignore

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* SATs: allow new records in a sequential read for full refresh test (#17660)

* SATs: allow new records in a sequential read for full refresh test

* SATs: upd changelog

* SATs: change the output when failing full refresh test

* SATs: upd according to code review

* Source facebook-marketing: remove `pixel` from custom conversions stream (#18045)

* #744 source facebook-marketing: rm pixel from custom conversions stream

* #744 source fb marketing: upd changelog

* #744 source facebook-marketing - add custom_conversions to the test catalog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* #17506 fix klaviyo & marketo expected_records (#18101)

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>
Co-authored-by: terencecho <3916587+terencecho@users.noreply.github.com>
Co-authored-by: Ryan Fu <ryan.fu@airbyte.io>
Co-authored-by: Jimmy Ma <gosusnp@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Yevhen Sukhomud <suhomud@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: xpuska513 <xpuska513@users.noreply.github.com>
Co-authored-by: Prasanth <72515998+sfc-gh-pkommini@users.noreply.github.com>
Co-authored-by: Benoit Moriceau <benoit@airbyte.io>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com>
Co-authored-by: Alexander Marquardt <alexander.marquardt@gmail.com>
Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Oct 20, 2022
Merged
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
@davydov-d @jhammarstedt
#17506 fix klaviyo & marketo expected_records (airbytehq#18101)
c0f1daf
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
@YatsukBogdan1 @edmundito
@terencecho @ryankfu @gosusnp @xpuska513 @evantahler @girarda @suhomud @octavia-squidington-iii @github-actions @sfc-gh-pkommini @benmoriceau @Amruta-Ranade @artem1205 @bazarnov @brianjlai @mfsiega-airbyte @alexander-marquardt @davydov-d @jhammarstedt
🪟 🧹 Implement ColumnSortButton component (airbytehq#17798)
e670c0f 
* Implement ColumnSortButton component

* Updates component name; Moves component to ui/Table folder; Refactors formattedMessageId property into using render content as children directly; Removes minor SortIcon component

* Update airbyte-webapp/src/App.tsx

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

* Updates next properties: wasActive -> isActive, lowToLarge -> isAscending

* Skip psql stop in acceptance test for gke (airbytehq#18023)

* Checks for iterator hasNext element (airbytehq#18041)

* Checks for iterator hasNext element

* Fix linter with newline

* Add Message Migration to Destination Connection Checks (airbytehq#17954)

* Add Message Migration to Destination Connection Checks

* Fix test setup

* Update helm release workflow (airbytehq#18048)

* Update workflow

* Update trigger rules

* fix: Update release workflow with abillity to add tags

* Update workflow

* Remove unused `airbyte-cli` (airbytehq#18009)

* 🐛  [low-code] $options shouldn't overwrite values that are already defined (airbytehq#18060)

* fix

* Add missing test

* remove prints

* extract to method

* rename

* Add missing test

* rename

* bump

* Update helm chart comments (airbytehq#18072)

* Update helm charts (airbytehq#18073)

* add test

* fix chart.yaml

* 16250 Destination Redis: Add SSH support (airbytehq#17951)

* 16250 Destination Redis: Add SSH support

* 16250 Resolve port issue

* 11679 Bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Bump helm chart version reference to 0.40.20 (airbytehq#18074)

* Bump helm chart version reference to 0.40.20

* remove binary

Co-authored-by: xpuska513 <xpuska513@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* Helm Chart: Create service annotations for airbyte-server (airbytehq#17932)

* Support annotations for airbyte-server as well, update version and update docs.

* Fix auto-indent.

Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* Bmoric/remove dep server worker (airbytehq#17894)

* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* TMP

* Add missing dependencies

* PR comments

* Tmp

* [ci skip] Tmp

* Fix acceptance test and add the seed dependency

* Fix build

* For diff

* tmp

* Build pass

* make the worker to be  on the platform only

* fix setting.yaml

* Fix pmd

* Fix Cron

* Add chart

* Fix cron

* Fix server build.gradle

* Fix jar conflict

* PR comments

* Add cron micronaut environemnt

* Updated connector catalog page (airbytehq#18076)

* Move the port forward outside of the main docker-compose (airbytehq#17864)

* Bump Airbyte version from 0.40.14 to 0.40.15 (airbytehq#17970)

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>

* 🎉 Source Shopify: Add metafield streams (airbytehq#17962)

* 🎉 Source Shopify: Add metafield streams

* Source Shopify: fix unittest

* Source Shopify: docs update

* Source Shopify: fix backward compatibility test

* Source Shopify: fix schemas

* Source Shopify: fix state filter

* Source Shopify: refactor & optimize

* Source Shopify: fix test privileges

* Source Shopify: fix stream filter

* Source Shopify: fix streams

* Source Shopify: update abnormal state

* Source Shopify: fix abnormal state streams

* Source Shopify: fix streams

* updated methods, formated code

* Source Shopify: typo fix

* auto-bump connector version

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* fix check for streams that do not use a stream slicer (airbytehq#18080)

* fix check for streams that do not use a stream slicer

* increment version and changelog before publish

* tolerate database nulls in webhook operation configs (airbytehq#18084)

* Implement webhook operation in the sync workflow (airbytehq#18022)

Implements the webhook operation as part of the sync workflow.

- Introduces the new activity implementation
- Updates the various interfaces that pass input to get the relevant configs to the sync workflow
- Hooks the new activity into the sync workflow
- Passes the webhook configs along into the sync workflow job

* Bump helm chart version reference to 0.40.22 (airbytehq#18077)

* Added new "filters" python file, along with a "hash" filter. This can… (airbytehq#18000)

* Added new "filters" python file, along with a "hash" filter. This can be extended to include other custom filters in the future.

* Added additional comments

* Moved usage of the hash_obj inside the conditional that confirms it exists

* Moved the hash function call inside a condition to ensure that it exists

* Fixed the application of the salt , so that it does not modify the hash unless it is actually passed in.

* Added unit tests to validate new jinja hash functionality

* Updated unit test to pass numeric value as a float instead of string

* Removed unreferenced import to pytest

* Updated version

* format

* format

* format

* format

* format

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>

* Bump helm chart version reference to 0.40.24 (airbytehq#18081)

* Bump helm chart version reference to 0.40.24

* Update .gitignore

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* SATs: allow new records in a sequential read for full refresh test (airbytehq#17660)

* SATs: allow new records in a sequential read for full refresh test

* SATs: upd changelog

* SATs: change the output when failing full refresh test

* SATs: upd according to code review

* Source facebook-marketing: remove `pixel` from custom conversions stream (airbytehq#18045)

* airbytehq#744 source facebook-marketing: rm pixel from custom conversions stream

* airbytehq#744 source fb marketing: upd changelog

* airbytehq#744 source facebook-marketing - add custom_conversions to the test catalog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* #17506 fix klaviyo & marketo expected_records (airbytehq#18101)

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>
Co-authored-by: terencecho <3916587+terencecho@users.noreply.github.com>
Co-authored-by: Ryan Fu <ryan.fu@airbyte.io>
Co-authored-by: Jimmy Ma <gosusnp@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Yevhen Sukhomud <suhomud@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: xpuska513 <xpuska513@users.noreply.github.com>
Co-authored-by: Prasanth <72515998+sfc-gh-pkommini@users.noreply.github.com>
Co-authored-by: Benoit Moriceau <benoit@airbyte.io>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com>
Co-authored-by: Alexander Marquardt <alexander.marquardt@gmail.com>
Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arsenlosenko arsenlosenko arsenlosenko approved these changes

@grubberr grubberr Awaiting requested review from grubberr

Assignees
No one assigned
Labels
area/connectors Connector related issues connectors/source/klaviyo connectors/source/marketo
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@davydov-d @arsenlosenko @octavia-squidington-iv