enable parallelism on bigquery tests

[edgao]

Make destination-bigquery tests fast again by reenabling parallelism.

I'm still not sure why destination-snowflake tests are also slower. It already has a gradle.properties file with parallelism set to 4. Metabase says destination-snowflake tests got slower around the same time, so presumably https://github.com/airbytehq/airbyte/pull/32108/files#diff-a3dd6be9c3819c1fcfd593b0f11c29663912e686e0dd74cae14610ac9c282480 did something? But I'm not sure what exactly.

Would it break things to set testExecutionConcurrency=-1 by default? My understanding is that tests need to opt into concurrent execution (with @Execution(ExecutionMode.CONCURRENT)).

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)		Nov 15, 2023 11:57pm

[github-actions]

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

• PR name follows PR naming conventions
• Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
• Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
• You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
• Secrets in the connector's spec are annotated with airbyte_secret
• All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
• Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
• Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
• If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

1. Check for hidden checklists in your PR description

2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

[gisripa]

lgtm.
Its confusing so
systemProperty 'junit.jupiter.execution.parallel.enabled', 'true' is already set but this systemProperty 'junit.jupiter.execution.parallel.config.fixed.parallelism', concurrency can take precedence and limit it ?

[postamar]

I'm happy if the CI is happy! Presumably, this fix can be applied to other destinations whose tests are thread-safe.

@gisripa there are two layers of concurrency: gradle will happily spawn an arbitrary number of workers and partition the test set into each of their JVMs, however JUnit offers in-JVM concurrency. The former is safer because you don't have to worry about thread-safety, and it remains the default. The junit properties are a bit confusing but they enable developers to opt out of the gradle concurrency and into the junit concurrency by setting just one property.

[edgao]

confirmed that test runtime is back down to 8 minutes

CI failures:

• I didn't bump the connector version

• AvroSerializedBufferTest > testSnappyAvroWriter() FAILED
      org.opentest4j.AssertionFailedError: Expected size between 964 and 985, but actual size was 991 ==> expected: <true> but was: <false>
  

neither is relevant. approve-and-merging.

[edgao]

/approve-and-merge reason="irrelevant CI failures #32580 (comment)"

[octavia-approvington]

Send it!