Source Greenhouse: support incremental syncs #16338

davydov-d · 2022-09-05T19:26:32Z

What

Support incremental syncs and fix SATs

How

by implementing custom stream slicers

…ncremental-syncs

davydov-d · 2022-09-05T19:28:50Z

/test connector=connectors/source-greenhouse

🕑 connectors/source-greenhouse https://github.com/airbytehq/airbyte/actions/runs/2995635061
✅ connectors/source-greenhouse https://github.com/airbytehq/airbyte/actions/runs/2995635061
Python tests coverage:

Name                              Stmts   Miss  Cover
-----------------------------------------------------
source_greenhouse/source.py           4      0   100%
source_greenhouse/__init__.py         2      0   100%
source_greenhouse/components.py      68     34    50%
-----------------------------------------------------
TOTAL                                74     34    54%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1322    463    65%

Build Passed

Test summary info:

All Passed

lazebnyi · 2022-09-05T20:38:36Z

/test connector=connectors/source-greenhouse

🕑 connectors/source-greenhouse https://github.com/airbytehq/airbyte/actions/runs/2995923549
✅ connectors/source-greenhouse https://github.com/airbytehq/airbyte/actions/runs/2995923549
Python tests coverage:

Name                              Stmts   Miss  Cover
-----------------------------------------------------
source_greenhouse/source.py           4      0   100%
source_greenhouse/__init__.py         2      0   100%
source_greenhouse/components.py      68      5    93%
-----------------------------------------------------
TOTAL                                74      5    93%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1322    463    65%

Build Passed

Test summary info:

All Passed

lazebnyi · 2022-09-05T20:55:51Z

/publish connector=connectors/source-greenhouse run-tests=false

🕑 Publishing the following connectors:
connectors/source-greenhouse
https://github.com/airbytehq/airbyte/actions/runs/2995988139

Connector	Did it publish?	Were definitions generated?
connectors/source-greenhouse	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

* airbytehq#1386 Source Greenhouse: support incremental syncs - first try * airbytehq#1386 Source Greenhouse: implement incremental syncs * airbytehq#1386 source greenhouse: upd changelog * Increased unittest to 90 * Updated link in spec * auto-bump connector version [ci skip] * Updated release stage Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

davydov-d added 3 commits September 2, 2022 14:26

#1386 Source Greenhouse: support incremental syncs - first try

9e95b6c

#1386 Source Greenhouse: implement incremental syncs

96cc4d7

Merge branch 'master' into ddavydov/#1386-source-greenhouse-support-i…

66565f1

…ncremental-syncs

octavia-squidington-iii added the connectors/source/greenhouse label Sep 5, 2022

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Sep 5, 2022

#1386 source greenhouse: upd changelog

9a13188

davydov-d linked an issue Sep 5, 2022 that may be closed by this pull request

Support incremental sync for greenhouse #1386

Closed

5 tasks

davydov-d self-assigned this Sep 5, 2022

davydov-d requested review from girarda, sherifnada, bazarnov, brianjlai, lazebnyi and roman-yermilov-gl September 5, 2022 19:29

lazebnyi removed request for girarda, sherifnada and brianjlai September 5, 2022 19:30

Increased unittest to 90

a2149dc

lazebnyi approved these changes Sep 5, 2022

View reviewed changes

lazebnyi removed request for bazarnov and roman-yermilov-gl September 5, 2022 20:39

Updated link in spec

7a6bcdb

lazebnyi mentioned this pull request Sep 5, 2022

Update fields in source-connectors specifications: greenhouse #9210

Closed

16 tasks

octavia-squidington-iii and others added 2 commits September 5, 2022 21:11

auto-bump connector version [ci skip]

3159676

Updated release stage

aad41e9

lazebnyi merged commit 8b5362e into master Sep 5, 2022

lazebnyi deleted the ddavydov/#1386-source-greenhouse-support-incremental-syncs branch September 5, 2022 21:13

lazebnyi temporarily deployed to more-secrets September 5, 2022 21:15 Inactive

octavia-squidington-iii mentioned this pull request Sep 12, 2022

Bump Airbyte version from 0.40.4 to 0.40.5 #16610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Greenhouse: support incremental syncs #16338

Source Greenhouse: support incremental syncs #16338

davydov-d commented Sep 5, 2022

davydov-d commented Sep 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Sep 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Sep 5, 2022 •

edited by github-actions bot

Loading

Source Greenhouse: support incremental syncs #16338

Source Greenhouse: support incremental syncs #16338

Conversation

davydov-d commented Sep 5, 2022

What

How

davydov-d commented Sep 5, 2022 • edited by github-actions bot Loading

Build Passed

lazebnyi commented Sep 5, 2022 • edited by github-actions bot Loading

Build Passed

lazebnyi commented Sep 5, 2022 • edited by github-actions bot Loading

davydov-d commented Sep 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Sep 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Sep 5, 2022 •

edited by github-actions bot

Loading