Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source Files: decrease give up rate #23723

Merged
merged 11 commits into from
Mar 10, 2023
Merged

Source Files: decrease give up rate #23723

merged 11 commits into from
Mar 10, 2023

Conversation

darynaishchenko
Copy link
Collaborator

Updated description in spec, made more user-friendly error messages and docs.

@darynaishchenko darynaishchenko self-assigned this Mar 3, 2023
@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/file labels Mar 3, 2023
@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 3, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-file

🕑 connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4323229106
❌ connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4323229106
🐛 https://gradle.com/s/q3kznmumqi3jm

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_source.py::test_check_wrong_reader_options - assert Ai...
	 �[31m=================== �[31m�[1m1 failed�[0m, �[32m35 passed�[0m, �[33m1 warning�[0m�[31m in 2.82s�[0m�[31m ====================�[0m

@darynaishchenko
Copy link
Collaborator Author

/test connector=connectors/source-file

@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 3, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-file

🕑 connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4324618398
✅ connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4324618398
Python tests coverage:

Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/utils.py         13      1    92%
source_file/source.py        81      7    91%
source_file/client.py       283     37    87%
---------------------------------------------
TOTAL                       379     45    88%
Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/client.py       283     40    86%
source_file/utils.py         13      5    62%
source_file/source.py        81     46    43%
---------------------------------------------
TOTAL                       379     91    76%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: Incremental syncs are not supported on this connector.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
=================== 33 passed, 2 skipped in 67.43s (0:01:07) ===================

alafanechere
alafanechere requested changes Mar 3, 2023
View reviewed changes
Copy link
Contributor

@alafanechere alafanechere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really get the description of the url field 😄

airbyte-integrations/connectors/source-file/source_file/source.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-file/source_file/spec.json Outdated Show resolved Hide resolved
alafanechere
alafanechere approved these changes Mar 6, 2023
View reviewed changes
Copy link
Contributor

@alafanechere alafanechere left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider my latest suggestion. Approving to not block.

airbyte-integrations/connectors/source-file/source_file/spec.json Outdated Show resolved Hide resolved
@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 7, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-file

🕑 connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4352406765
✅ connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4352406765
Python tests coverage:

Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/utils.py         13      1    92%
source_file/source.py        81      7    91%
source_file/client.py       283     37    87%
---------------------------------------------
TOTAL                       379     45    88%
Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/client.py       283     40    86%
source_file/utils.py         13      5    62%
source_file/source.py        81     46    43%
---------------------------------------------
TOTAL                       379     91    76%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: Incremental syncs are not supported on this connector.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
=================== 33 passed, 2 skipped in 67.97s (0:01:07) ===================

@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 8, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-file


if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 9, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-file-secure

🕑 connectors/source-file-secure https://github.com/airbytehq/airbyte/actions/runs/4373428928
❌ connectors/source-file-secure https://github.com/airbytehq/airbyte/actions/runs/4373428928
🐛 https://gradle.com/s/digpmxukvwqrs

Build Failed

Test summary info:

Could not find result summary

@darynaishchenko darynaishchenko requested a review from lazebnyi March 9, 2023 10:43
@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 9, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-file

🕑 Publishing the following connectors:
connectors/source-file
https://github.com/airbytehq/airbyte/actions/runs/4376769767


Connector Did it publish? Were definitions generated?
connectors/source-file ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 9, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-file-secure

🕑 Publishing the following connectors:
connectors/source-file-secure
https://github.com/airbytehq/airbyte/actions/runs/4377114445


Connector Did it publish? Were definitions generated?
connectors/source-file-secure ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@darynaishchenko
Copy link
Collaborator Author

darynaishchenko commented Mar 9, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-file-secure

🕑 Publishing the following connectors:
connectors/source-file-secure
https://github.com/airbytehq/airbyte/actions/runs/4377453508


Connector Did it publish? Were definitions generated?
connectors/source-file-secure ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@darynaishchenko darynaishchenko merged commit 5f0d9f0 into master Mar 10, 2023
@darynaishchenko darynaishchenko deleted the daryna/source-file/decrease-give-up-rate branch March 10, 2023 09:13
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this pull request Mar 13, 2023
@darynaishchenko @octavia-squidington-iii @danielduckworth
Source Files: decrease give up rate (airbytehq#23723)
f141823 
* updated spec, docs, error messages

* added change log

* updated error message in unit tests

* added allowedHosts

* refactored error message and url descriprion

* updated example for url field in spec

* bump source-file verion for source-file-secure

* auto-bump connector version

* updated spec.json for source-file-secure

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
adriennevermorel pushed a commit to adriennevermorel/airbyte that referenced this pull request Mar 17, 2023
@darynaishchenko @octavia-squidington-iii @adriennevermorel
Source Files: decrease give up rate (airbytehq#23723)
3b59c10 
* updated spec, docs, error messages

* added change log

* updated error message in unit tests

* added allowedHosts

* refactored error message and url descriprion

* updated example for url field in spec

* bump source-file verion for source-file-secure

* auto-bump connector version

* updated spec.json for source-file-secure

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lazebnyi lazebnyi lazebnyi approved these changes

@alafanechere alafanechere alafanechere approved these changes

@davydov-d davydov-d Awaiting requested review from davydov-d

Assignees

@darynaishchenko darynaishchenko

Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/file connectors/source/file-secure
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@darynaishchenko @alafanechere @lazebnyi @octavia-squidington-iii