merge: #12430 #12445

12430: [Backport stable/8.2] ci(integration): split module and integration test jobs r=megglos a=backport-action # Description Backport of #12406 to `stable/8.2`. relates to #12028 12445: deps(maven): bump grpc-bom from 1.54.0 to 1.54.1 r=github-actions[bot] a=dependabot[bot] Bumps [grpc-bom](https://github.com/grpc/grpc-java) from 1.54.0 to 1.54.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/grpc/grpc-java/releases">grpc-bom's releases</a>.</em></p> <blockquote> <h2>v1.54.1</h2> <h2>Bug Fixes</h2> <ul> <li>core: Fix NPE race during hedging (<a href="https://redirect.github.com/grpc/grpc-java/pull/9853">grpc/grpc-java#9853</a>), fixing a Netty buffer memory leak for cancelled RPCs</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/grpc/grpc-java/commit/56d1c63802fd9706fece4b144969bfc9bdcfc99c"><code>56d1c63</code></a> Bump version to 1.54.1</li> <li><a href="https://github.com/grpc/grpc-java/commit/4a5605cea3d7cfc9196f35a36a7c60a5ce926eb2"><code>4a5605c</code></a> Update README etc to reference 1.54.1</li> <li><a href="https://github.com/grpc/grpc-java/commit/4b01c907bc72ca513b0f099fac58601addbe2dd3"><code>4b01c90</code></a> core: Fix NPE race during hedging</li> <li><a href="https://github.com/grpc/grpc-java/commit/92b4faed40344af6941e6ff506bd0670173e42e1"><code>92b4fae</code></a> Pass interop parameters to each langs run.sh as-is. run.sh should just pass t...</li> <li><a href="https://github.com/grpc/grpc-java/commit/1bf518af12f66322bd9bae63c667314a0ca41c94"><code>1bf518a</code></a> gcp-o11y: Remove monitored resource detection for logging (<a href="https://redirect.github.com/grpc/grpc-java/issues/10020">#10020</a>)</li> <li><a href="https://github.com/grpc/grpc-java/commit/5369df13a5e24c0e011e75e0f4ac59ab3835cec2"><code>5369df1</code></a> Removes the ExperimentalApi annotation from GcpObservability.</li> <li><a href="https://github.com/grpc/grpc-java/commit/6d21d71a257beaddf4805af27ed1d7f77ce86f2b"><code>6d21d71</code></a> use glob for example file names which is used in updating release versions (#...</li> <li><a href="https://github.com/grpc/grpc-java/commit/e955afe50a8ce720f89c0a58c85cc80038a1ce65"><code>e955afe</code></a> examples: Fix grpc version in gcp-observability</li> <li><a href="https://github.com/grpc/grpc-java/commit/5c09616aae5ff1620ed6bb5867717e0451c2d00f"><code>5c09616</code></a> xds: fix flaky wrr test (<a href="https://redirect.github.com/grpc/grpc-java/issues/10005">#10005</a>)</li> <li><a href="https://github.com/grpc/grpc-java/commit/7d5d25d34e20259962fb6bf0dbe501df761c6948"><code>7d5d25d</code></a> Bump version to 1.54.1-SNAPSHOT</li> <li>See full diff in <a href="https://github.com/grpc/grpc-java/compare/v1.54.0...v1.54.1">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=io.grpc:grpc-bom&package-manager=maven&previous-version=1.54.0&new-version=1.54.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com> Co-authored-by: Sebastian Bathke (Meggle) <sebastian.bathke@camunda.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
camunda · Apr 17, 2023 · a52c220 · a52c220
3 parents 1f0775c + e3576b6 + 17d28ac
commit a52c220
Show file tree

Hide file tree

Showing 8 changed files with 52 additions and 79 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -24,74 +24,38 @@ env:
 
 jobs:
   integration-tests:
-    name: Integration tests
-    runs-on: [ self-hosted, linux, amd64, "16" ]
-    timeout-minutes: 45
-    env:
-      TC_CLOUD_LOGS_VERBOSE: true
-      TC_CLOUD_TOKEN: ${{ secrets.TC_CLOUD_TOKEN }}
-      TC_CLOUD_CONCURRENCY: 2
-      ZEEBE_TEST_DOCKER_IMAGE: localhost:5000/camunda/zeebe:current-test
-    services:
-      registry:
-        image: registry:2
-        ports:
-          - 5000:5000
-    steps:
-      - uses: actions/checkout@v3
-      - uses: ./.github/actions/setup-zeebe
-        with:
-          maven-cache: 'true'
-          secret_vault_secretId: ${{ secrets.VAULT_SECRET_ID }}
-          secret_vault_address: ${{ secrets.VAULT_ADDR }}
-          secret_vault_roleId: ${{ secrets.VAULT_ROLE_ID }}
-      - uses: ./.github/actions/build-zeebe
-        id: build-zeebe
-        with:
-          maven-extra-args: -T1C
-      - uses: ./.github/actions/build-docker
-        with:
-          repository: localhost:5000/camunda/zeebe
-          version: current-test
-          push: true
-          distball: ${{ steps.build-zeebe.outputs.distball }}
-      - name: Prepare Testcontainers Cloud agent
-        if: env.TC_CLOUD_TOKEN != ''
-        run: |
-          curl -L -o agent https://app.testcontainers.cloud/download/testcontainers-cloud-agent_linux_x86-64
-          chmod +x agent
-          ./agent --private-registry-url=http://localhost:5000 '--private-registry-allowed-image-name-globs=*,*/*' > .testcontainers-agent.log 2>&1 &
-          ./agent wait
-      - name: Create build output log file
-        run: echo "BUILD_OUTPUT_FILE_PATH=$(mktemp)" >> $GITHUB_ENV
-      - name: Maven Test Build
-        run: >
-          ./mvnw -B -T2 --no-snapshot-updates
-          -D forkCount=5
-          -D maven.javadoc.skip=true
-          -D skipUTs -D skipChecks
-          -D failsafe.rerunFailingTestsCount=3 -D flaky.test.reportDir=failsafe-reports
-          -P parallel-tests,extract-flaky-tests
-          -pl '!qa/update-tests'
-          verify
-          | tee "${BUILD_OUTPUT_FILE_PATH}"
-      - name: Duplicate Test Check
-        uses: ./.github/actions/check-duplicate-tests
-        with:
-          buildOutputFilePath: ${{ env.BUILD_OUTPUT_FILE_PATH }}
-      - name: Upload test artifacts
-        uses: ./.github/actions/collect-test-artifacts
-        if: always()
-        with:
-          name: Integration Tests
-  qa-update-tests:
-    name: QA Update tests
+    name: "[IT] ${{ matrix.name }}"
+    timeout-minutes: 20
     runs-on: [ self-hosted, linux, amd64, "16" ]
-    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        group: [ modules, qa-integration, qa-update ]
+        include:
+          - group: modules
+            name: "Module Integration Tests"
+            maven-modules: "'!qa/integration-tests,!qa/update-tests'"
+            maven-build-threads: 2
+            maven-test-fork-count: 7
+            tcc-enabled: 'false'
+          - group: qa-integration
+            name: "QA Integration Tests"
+            maven-modules: "qa/integration-tests"
+            maven-build-threads: 1
+            maven-test-fork-count: 10
+            tcc-enabled: 'true'
+            tcc-concurrency: 2
+          - group: qa-update
+            name: "QA Update Tests"
+            maven-modules: "qa/update-tests"
+            maven-build-threads: 1
+            maven-test-fork-count: 10
+            tcc-enabled: 'true'
+            tcc-concurrency: 2
     env:
       TC_CLOUD_LOGS_VERBOSE: true
-      TC_CLOUD_TOKEN: ${{ secrets.TC_CLOUD_TOKEN }}
-      TC_CLOUD_CONCURRENCY: 2
+      TC_CLOUD_TOKEN: ${{ matrix.tcc-enabled == 'true' && secrets.TC_CLOUD_TOKEN || '' }}
+      TC_CLOUD_CONCURRENCY: ${{ matrix.tcc-concurrency }}
       ZEEBE_TEST_DOCKER_IMAGE: localhost:5000/camunda/zeebe:current-test
     services:
       registry:
@@ -127,12 +91,13 @@ jobs:
         run: echo "BUILD_OUTPUT_FILE_PATH=$(mktemp)" >> $GITHUB_ENV
       - name: Maven Test Build
         run: >
-          ./mvnw -B -T2 --no-snapshot-updates
+          ./mvnw -B -T ${{ matrix.maven-build-threads }} --no-snapshot-updates
+          -D forkCount=${{ matrix.maven-test-fork-count }}
           -D maven.javadoc.skip=true
           -D skipUTs -D skipChecks
           -D failsafe.rerunFailingTestsCount=3 -D flaky.test.reportDir=failsafe-reports
           -P parallel-tests,extract-flaky-tests
-          -pl qa/update-tests
+          -pl ${{ matrix.maven-modules }}
           verify
           | tee "${BUILD_OUTPUT_FILE_PATH}"
       - name: Duplicate Test Check
@@ -143,7 +108,7 @@ jobs:
         uses: ./.github/actions/collect-test-artifacts
         if: always()
         with:
-          name: QA Update Tests
+          name: "[IT] ${{ matrix.name }}"
   unit-tests:
     name: Unit tests
     runs-on: [ self-hosted, linux, amd64, "16" ]
@@ -185,7 +150,7 @@ jobs:
         with:
           name: "unit tests"
   smoke-tests:
-    name: Smoke tests on ${{ matrix.os }} with ${{ matrix.arch }}
+    name: "[Smoke] ${{ matrix.os }} with ${{ matrix.arch }}"
     timeout-minutes: 20
     runs-on: ${{ matrix.runner }}
     strategy:
@@ -243,7 +208,7 @@ jobs:
         uses: ./.github/actions/collect-test-artifacts
         if: always()
         with:
-          name: Smoke Tests on ${{ matrix.os }} with ${{ matrix.arch }}
+          name: "[Smoke] ${{ matrix.os }} with ${{ matrix.arch }}"
   property-tests:
     name: Property Tests
     runs-on: [ self-hosted, linux, amd64, "16" ]
@@ -489,7 +454,6 @@ jobs:
     runs-on: ubuntu-latest
     needs:
       - integration-tests
-      - qa-update-tests
       - unit-tests
       - smoke-tests
       - property-tests
@@ -509,7 +473,6 @@ jobs:
     runs-on: ubuntu-latest
     needs:
       - integration-tests
-      - qa-update-tests
       - unit-tests
       - smoke-tests
       - property-tests

diff --git a/...java/io/atomix/raft/RaftFailOverTest.java → ...t/java/io/atomix/raft/RaftFailOverIT.java b/...java/io/atomix/raft/RaftFailOverTest.java → ...t/java/io/atomix/raft/RaftFailOverIT.java
@@ -29,7 +29,7 @@
 import org.junit.runners.Parameterized.Parameters;
 
 @RunWith(Parameterized.class)
-public class RaftFailOverTest {
+public class RaftFailOverIT {
 
   @Rule @Parameter public RaftRule raftRule;
 

diff --git a/.../java/io/atomix/raft/zeebe/ZeebeTest.java → ...st/java/io/atomix/raft/zeebe/ZeebeIT.java b/.../java/io/atomix/raft/zeebe/ZeebeTest.java → ...st/java/io/atomix/raft/zeebe/ZeebeIT.java
@@ -52,7 +52,7 @@
 import org.junit.runners.Parameterized.Parameters;
 
 @RunWith(Parameterized.class)
-public class ZeebeTest {
+public class ZeebeIT {
 
   // rough estimate of how many entries we'd need to write to fill a segment
   // segments are configured for 1kb, and one entry takes ~30 bytes (plus some metadata I guess)

diff --git a/atomix/pom.xml b/atomix/pom.xml
@@ -73,6 +73,18 @@
           <suppressionsLocation>src/main/resources/suppression.xml</suppressionsLocation>
         </configuration>
       </plugin>
+
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-failsafe-plugin</artifactId>
+        <configuration>
+          <includes>
+            <include>**/IT*.java</include>
+            <include>**/*IT.java</include>
+            <include>**/*ITCase.java</include>
+          </includes>
+        </configuration>
+      </plugin>
     </plugins>
   </build>
 </project>
diff --git a/...unda/zeebe/backup/s3/CompressionTest.java → ...amunda/zeebe/backup/s3/CompressionIT.java b/...unda/zeebe/backup/s3/CompressionTest.java → ...amunda/zeebe/backup/s3/CompressionIT.java
@@ -40,7 +40,7 @@
 import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
 
 @Testcontainers
-final class CompressionTest {
+final class CompressionIT {
   private static final String ACCESS_KEY = "letmein";
   private static final String SECRET_KEY = "letmein1234";
   private static final int DEFAULT_PORT = 9000;

diff --git a/backup-stores/s3/src/test/java/io/camunda/zeebe/backup/s3/ConnectionErrorTest.java b/backup-stores/s3/src/test/java/io/camunda/zeebe/backup/s3/ConnectionErrorTest.java
@@ -18,10 +18,8 @@
 import org.apache.commons.lang3.RandomStringUtils;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
-import org.testcontainers.junit.jupiter.Testcontainers;
 import software.amazon.awssdk.regions.Region;
 
-@Testcontainers
 final class ConnectionErrorTest {
   private static final String ACCESS_KEY = "letmein";
   private static final String SECRET_KEY = "letmein1234";

diff --git a/...a/zeebe/backup/s3/CustomBasePathTest.java → ...nda/zeebe/backup/s3/CustomBasePathIT.java b/...a/zeebe/backup/s3/CustomBasePathTest.java → ...nda/zeebe/backup/s3/CustomBasePathIT.java
@@ -29,7 +29,7 @@
 import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
 
 @Testcontainers
-final class CustomBasePathTest {
+final class CustomBasePathIT {
   private static final String ACCESS_KEY = "letmein";
   private static final String SECRET_KEY = "letmein1234";
   private static final int DEFAULT_PORT = 9000;

diff --git a/parent/pom.xml b/parent/pom.xml
@@ -52,7 +52,7 @@
     <version.docker-java-api>3.2.14</version.docker-java-api>
     <version.elasticsearch>7.17.9</version.elasticsearch>
     <version.error-prone>2.18.0</version.error-prone>
-    <version.grpc>1.54.0</version.grpc>
+    <version.grpc>1.54.1</version.grpc>
     <version.gson>2.10.1</version.gson>
     <version.guava>31.1-jre</version.guava>
     <version.hamcrest>2.2</version.hamcrest>