feat: speed up router-tests

[alepane21]

Motivation and Context

It is important to make the tests as fast as possible. This PR addresses parallelization issues in the test suite, improves test reliability, and optimizes resource usage. Using insights from this library to identify tests that were not running in parallel, I was able to uncover additional issues such as unnecessary locks, data races, and flaky test behavior under load.

Checklist

• I have discussed my proposed changes in an issue and have received approval to proceed.
• I have followed the coding standards of the project.
• Tests or benchmarks have been added or updated.
• Documentation has been updated on https://github.com/wundergraph/cosmo-docs.
• I have read the Contributors Guide.

Key Changes

1. Test Parallelization Analysis

   • Leveraged the Go test parallelism library to identify and resolve tests not running in parallel effectively.

2. Shared Kafka and NATS Instances

   • All tests now share a single Kafka and NATS instance to reduce overhead.

3. Removed Unnecessary Locks

   • Eliminated the mutex in createTestEnv by leveraging the Freeport library.

4. Concurrent Safety Enhancements

   • Improved demo server handling of port allocation and ensured thread safety (previously affected by shared mutable state).

5. Integration Test Job

   • Created a dedicated GitHub job for integration tests.
   • Removed Redis from other jobs

6. Improved Parallelization

   • Reorganized tests across packages to facilitate parallel execution.

7. Retries for Flaky Tests

   • Added nick-fields/retry@v3 action to automatically retry flaky tests.

Future Improvements (Separate PR)

1. Use external Kafka and NATS services (similar to Redis).
2. Further reduce test flakiness, especially in telemetry, Kafka, and NATS-related tests.
3. Avoid duplicated work (the unit tests are run more than one time, as it is the image building process)

[github-actions]

Dependency Review

✅ No vulnerabilities or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details

Scanned Files