Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Mailchimp: enable high test strictness level in SAT #19167

Conversation

alafanechere
Copy link
Contributor

What

A test_strictness_level field was introduced to Source Acceptance Tests (SAT).
Mailchimp is a generally_available connector, we want it to have a high test strictness level.

This will help:

  • maximize the SAT coverage on this connector.
  • document its potential weaknesses in term of test coverage.

How

  1. Migrate the existing acceptance-test-config.yml file to the latest configuration format. (See instructions here)
  2. Enable high test strictness level in acceptance-test-config.yml. (See instructions here)

⚠️ ⚠️ ⚠️
If tests are failing please fix the failing test by changing the acceptance-test-config.yml file or use bypass_reason fields to explain why a specific test can't be run.

Please open a new PR if the new enabled tests help discover a new bug.
Once this bug fix is merged please rebase this branch and run /test again.

You can find more details about the rules enforced by high test strictness level here.

Review process

Please ask the connector-operations teams for review.

@alafanechere
Copy link
Contributor Author

alafanechere commented Nov 10, 2022

/test connector=connectors/source-mailchimp

🕑 connectors/source-mailchimp https://github.com/airbytehq/airbyte/actions/runs/3435013867
❌ connectors/source-mailchimp https://github.com/airbytehq/airbyte/actions/runs/3435013867
🐛 https://gradle.com/s/6h6tmdez3zn26

Build Failed

Test summary info:

=========================== short test summary info ============================
ERROR test_incremental.py::TestIncremental - Failed: TestIncremental.test_two...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 1.03s ===============================

@davydov-d
Copy link
Collaborator

Implemented here

@davydov-d davydov-d closed this Jan 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants