Faker emits AirbyteEstimateTraceMessage

[evantahler]

Extracted from #18630, part of https://github.com/airbytehq/airbyte-internal-issues/issues/1095.

[evantahler]

/test connector=connectors/source-faker

🕑 connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/3424661172
❌ connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/3424661172
🐛 https://gradle.com/s/yhjjw5yu4e5co

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestConnection::test_check[inputs0] - docker.errors.Cont...
FAILED test_core.py::TestConnection::test_check[inputs1] - Failed: DID NOT RA...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
============ 2 failed, 24 passed, 1 skipped, 27 warnings in 25.12s =============

[evantahler]

cc @alafanechere can you give me a pointer to how to pass this failing SAT test? Since I'm not really using the CDK tooling, I'm not sure how I'm supposed to exit the container in this case

[alafanechere]

Hey @evantahler, the problem was that in case of check failure the returned object was a generator (which is an iterable) because yield was used. And this generator does not have the status attribute accessed downstream.
I was able to make SAT run locally by:

• Return a status object instead of a generator
• Change the expected status to be failed instead of exception.
  I also remove the exit(1) instruction.

Are these changes impacting the logic you've been trying to implement in this PR?
Moreover, I don't see anything preventing this connector to use the CDK, which will be better for long term maintenance. (Just changing check to check_connection and discover to streams, with adapted return value, would do the job I think).

[alafanechere]

/test connector=connectors/source-faker

🕑 connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/3460147348
✅ connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/3460147348
Python tests coverage:

Name                       Stmts   Miss  Cover
----------------------------------------------
source_faker/__init__.py       2      0   100%
source_faker/source.py       150      7    95%
----------------------------------------------
TOTAL                        152      7    95%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       139      5    96%   87, 93, 235, 239-240
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1508    348    77%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 26 passed, 1 skipped, 27 warnings in 27.20s ==================

[evantahler]

Thank you @alafanechere!

[evantahler]

These changes are not logical changes, other than the emitting of stream size estimates at the beginning of the sync.

Yes, this connector should totally be moved to a more "normal" CDK connector. I made it a long time ago back when I didn't know what I was doing 😇... but that's another PR (#19389)!

[evantahler]

/publish connector=connectors/source-faker

🕑 Publishing the following connectors:
connectors/source-faker
https://github.com/airbytehq/airbyte/actions/runs/3474088026

Connector	Did it publish?	Were definitions generated?
connectors/source-faker	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️