🐛 Prevent sources from hanging if they have orphaned threads #12544

edgao · 2022-05-03T18:23:32Z

What

Closes #5754, #4322, #8218; supersedes #11910

Automatically kill sources (after a delay) if they have no main thread. We've had this in destinations for a while; add it to sources as well.

How

Use the same watchForOrphanThreads wrapper that destinations use when consuming input.

edgao · 2022-05-03T18:24:37Z

@marcosmarxm fyi. lmk how you want to handle publishing this once it's approved, since I don't think we actually want to publish every single source (?)

tuliren

This is awesome!

Maybe run one source integration test for each source type before merging just to be safe.

marcosmarxm

can you run mysql and bump the connector to the user test this too?

marcosmarxm · 2022-05-03T19:17:47Z

I don't we need to bump every connector version though

edgao · 2022-05-03T19:36:55Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/2265956342
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/2265956342
No Python unittests run

edgao · 2022-05-03T19:37:02Z

/test connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2265956841
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/2265956841
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  80     17    79%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              285    106    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  896    259    71%
Name                                       Stmts   Miss  Cover
--------------------------------------------------------------
source_google_ads/__init__.py                  2      0   100%
source_google_ads/google_ads.py               68     10    85%
source_google_ads/streams.py                 136     25    82%
source_google_ads/source.py                   80     27    66%
source_google_ads/custom_query_stream.py      75     50    33%
--------------------------------------------------------------
TOTAL                                        361    112    69%
Name                                       Stmts   Miss  Cover
--------------------------------------------------------------
source_google_ads/__init__.py                  2      0   100%
source_google_ads/source.py                   80      4    95%
source_google_ads/streams.py                 136     10    93%
source_google_ads/custom_query_stream.py      75      6    92%
source_google_ads/google_ads.py               68      7    90%
--------------------------------------------------------------
TOTAL                                        361     27    93%

Python short test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:56: Skipping TestIncremental.test_two_sequential_reads because not found in the config
================== 21 passed, 1 skipped in 367.32s (0:06:07) ===================

edgao · 2022-05-03T19:37:26Z

/test connector=connectors/source-s3

🕑 connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/2265958466
✅ connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/2265958466
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  80     17    79%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              285    106    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  896    259    71%
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
source_s3/source_files_abstract/formats/parquet_spec.py               9      0   100%
source_s3/source_files_abstract/formats/csv_spec.py                  16      0   100%
source_s3/s3_utils.py                                                19      0   100%
source_s3/__init__.py                                                 2      0   100%
source_s3/source.py                                                  28      1    96%
source_s3/source_files_abstract/storagefile.py                       23      1    96%
source_s3/s3file.py                                                  37      2    95%
source_s3/source_files_abstract/formats/abstract_file_parser.py      35      2    94%
source_s3/source_files_abstract/stream.py                           184     11    94%
source_s3/stream.py                                                  43      3    93%
source_s3/source_files_abstract/formats/csv_parser.py                74     18    76%
source_s3/source_files_abstract/file_info.py                         26      8    69%
source_s3/utils.py                                                   29     10    66%
source_s3/source_files_abstract/source.py                            37     14    62%
source_s3/source_files_abstract/spec.py                              42     22    48%
source_s3/source_files_abstract/formats/parquet_parser.py            61     44    28%
-------------------------------------------------------------------------------------
TOTAL                                                               665    136    80%
Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
source_s3/source_files_abstract/formats/parquet_spec.py               9      0   100%
source_s3/source_files_abstract/formats/csv_spec.py                  16      0   100%
source_s3/source_files_abstract/formats/abstract_file_parser.py      35      0   100%
source_s3/source.py                                                  28      0   100%
source_s3/__init__.py                                                 2      0   100%
source_s3/source_files_abstract/formats/parquet_parser.py            61      3    95%
source_s3/source_files_abstract/storagefile.py                       23      5    78%
source_s3/source_files_abstract/formats/csv_parser.py                74     18    76%
source_s3/utils.py                                                   29      8    72%
source_s3/source_files_abstract/file_info.py                         26     10    62%
source_s3/source_files_abstract/source.py                            37     15    59%
source_s3/s3file.py                                                  37     18    51%
source_s3/source_files_abstract/stream.py                           184     91    51%
source_s3/source_files_abstract/spec.py                              42     22    48%
source_s3/s3_utils.py                                                19     13    32%
source_s3/stream.py                                                  43     30    30%
-------------------------------------------------------------------------------------
TOTAL                                                               665    233    65%

edgao · 2022-05-03T21:44:55Z

/publish connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/2266525402
❌ Failed to publish connectors/source-mysql
❌ Couldn't auto-bump version for connectors/source-mysql

edgao · 2022-05-04T00:04:33Z

/publish connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/2267033265
🚀 Successfully published connectors/source-mysql
🚀 Auto-bumped version for connectors/source-mysql
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/2267033265

wrap source in watchForOrphanThreads

7d32e66

edgao requested a review from tuliren May 3, 2022 18:23

edgao changed the title ~~wrap source in watchForOrphanThreads~~ Prevent sources from hanging if they have orphaned threads May 3, 2022

edgao changed the title ~~Prevent sources from hanging if they have orphaned threads~~ 🐛 Prevent sources from hanging if they have orphaned threads May 3, 2022

formatting

f882be5

tuliren approved these changes May 3, 2022

View reviewed changes

marcosmarxm approved these changes May 3, 2022

View reviewed changes

bump version

49ec2af

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels May 3, 2022

auto-bump connector version

1854e13

octavia-squidington-iii temporarily deployed to more-secrets May 4, 2022 00:42 Inactive

edgao merged commit 3d41612 into master May 4, 2022

edgao deleted the edgao/source_orphan_thread_watcher branch May 4, 2022 01:48

octavia-squidington-iii mentioned this pull request May 4, 2022

Bump Airbyte version from 0.36.7-alpha to 0.36.8-alpha #12553

Merged

marcosmarxm mentioned this pull request May 9, 2022

Explicitly exit to stop hung background threads in MySQL source connector #11910

Closed

38 tasks

suhomud pushed a commit that referenced this pull request May 23, 2022

🐛 Prevent sources from hanging if they have orphaned threads (#12544)

a1f6c54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Prevent sources from hanging if they have orphaned threads #12544

🐛 Prevent sources from hanging if they have orphaned threads #12544

edgao commented May 3, 2022

edgao commented May 3, 2022

tuliren left a comment

marcosmarxm left a comment

marcosmarxm commented May 3, 2022

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 4, 2022 •

edited by github-actions bot

Loading

🐛 Prevent sources from hanging if they have orphaned threads #12544

🐛 Prevent sources from hanging if they have orphaned threads #12544

Conversation

edgao commented May 3, 2022

What

How

edgao commented May 3, 2022

tuliren left a comment

Choose a reason for hiding this comment

marcosmarxm left a comment

Choose a reason for hiding this comment

marcosmarxm commented May 3, 2022

edgao commented May 3, 2022 • edited by github-actions bot Loading

edgao commented May 3, 2022 • edited by github-actions bot Loading

edgao commented May 3, 2022 • edited by github-actions bot Loading

edgao commented May 3, 2022 • edited by github-actions bot Loading

edgao commented May 4, 2022 • edited by github-actions bot Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 3, 2022 •

edited by github-actions bot

Loading

edgao commented May 4, 2022 •

edited by github-actions bot

Loading