Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not restart broker on snapshot test #9573

Merged
1 commit merged into from
Jun 22, 2022
Merged

Do not restart broker on snapshot test #9573

1 commit merged into from
Jun 22, 2022

Conversation

ChrisKujawa
Copy link
Member

@ChrisKujawa ChrisKujawa commented Jun 22, 2022

PR Description

Previously we took no snapshot if the exporter hasn't committed the positions, but this is no longer true (see #8176).
This means if we run the DebugExporter we still can take snapshots.

This allows to reduce the complexity in the SnapshotTest, as it is no longer necessary to restart the Broker
without the DebugExporter. This will of course also reduce the execution time of the test, and hopefully the flakiness.

TIL:

I thought I had to make the DebugExporter configurable so it will commit the exporting position if I want it to, which is why I implemented it here https://github.com/camunda/zeebe/tree/zell-debug-exporter

Later I realized (because I wrote tests 😁 ) that we take anyway snapshots, even if we have an exporter not committing the positions so I dropped the changes from the PR. Let me know whether we still want this, maybe worth for other things not sure 🤷‍♂️

Related issues

closes #9517

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

  • The changes are backwards compatibility with previous versions
  • If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/1.3) to the PR, in case that fails you need to create backports manually.

Testing:

  • There are unit/integration tests that verify all acceptance criterias of the issue
  • New tests are written to ensure backwards compatibility with further versions
  • The behavior is tested manually
  • The change has been verified by a QA run
  • The impact of the changes is verified by a benchmark

Documentation:

  • The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
  • New content is added to the release announcement
  • If the PR changes how BPMN processes are validated (e.g. support new BPMN element) then the Camunda modeling team should be informed to adjust the BPMN linting.

Please refer to our review guidelines.

Previously we took no snapshot if the exporter hasn't commited the positions, but this is no longer true.
This means if we run the DebugExporter we still can take snapshots.

This allows to reduce the complexity in the SnapshotTest, as it is no longer necessary to restart the Broker
without the DebugExporter. This will of course also reduce the execution time of the test, and hopefully the flakiness.
@ChrisKujawa ChrisKujawa requested a review from npepinpe June 22, 2022 10:26
@ChrisKujawa ChrisKujawa marked this pull request as ready for review June 22, 2022 10:28
@github-actions
Copy link
Contributor

Unit Test Results

   780 files  ±  0     780 suites  ±0   1h 28m 58s ⏱️ - 4m 34s
5 485 tests  - 57  5 478 ✔️  - 57  7 💤 ±0  0 ±0 
5 657 runs   - 57  5 650 ✔️  - 57  7 💤 ±0  0 ±0 

Results for commit 379e54f. ± Comparison against base commit 706e630.

@npepinpe
Copy link
Member

I feel dumb, I don't see how this was causing flakiness? 🙈

@ChrisKujawa
Copy link
Member Author

Based on the report:

Error:  Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 450.284 s <<< FAILURE! - in io.camunda.zeebe.test.SnapshotTest
Error:  io.camunda.zeebe.test.SnapshotTest.update(String, UpdateTestCase, ContainerState)[9]  Time elapsed: 26.233 s  <<< ERROR!
io.camunda.zeebe.client.api.command.ClientException: Timed out waiting on client response
	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:65)
	at io.camunda.zeebe.test.SnapshotTest.sendDummyMessageToEnforceSnapshot(SnapshotTest.java:77)
	at io.camunda.zeebe.test.SnapshotTest.update(SnapshotTest.java:54)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)

I haven't reproduced it locally, but I felt it is not necessary at all to restart the broker and send the additional message, which I hope reduces/removes the possibility to fail as well.

Copy link
Member

@npepinpe npepinpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Let's try this, it can only improve the situation.

@ChrisKujawa
Copy link
Member Author

BTW: Main Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 384.489 s - in io.camunda.zeebe.test.SnapshotTest
This branch: Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 231.777 s - in io.camunda.zeebe.test.SnapshotTest

@ChrisKujawa
Copy link
Member Author

bors r+

@ghost
Copy link

ghost commented Jun 22, 2022

Build succeeded:

@ghost ghost merged commit e82d2b8 into main Jun 22, 2022
@ghost ghost deleted the zell-snapshot-test branch June 22, 2022 11:33
@backport-action
Copy link
Collaborator

Backport failed for stable/1.3, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin stable/1.3
git worktree add -d .worktree/backport-9573-to-stable/1.3 origin/stable/1.3
cd .worktree/backport-9573-to-stable/1.3
git checkout -b backport-9573-to-stable/1.3
ancref=$(git merge-base 706e63079ce2a0d5ca8e27fe521aac8903690acc 379e54fe9f5e8673560caeb32caadb5019fd4d25)
git cherry-pick -x $ancref..379e54fe9f5e8673560caeb32caadb5019fd4d25

@backport-action
Copy link
Collaborator

Backport failed for stable/8.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin stable/8.0
git worktree add -d .worktree/backport-9573-to-stable/8.0 origin/stable/8.0
cd .worktree/backport-9573-to-stable/8.0
git checkout -b backport-9573-to-stable/8.0
ancref=$(git merge-base 706e63079ce2a0d5ca8e27fe521aac8903690acc 379e54fe9f5e8673560caeb32caadb5019fd4d25)
git cherry-pick -x $ancref..379e54fe9f5e8673560caeb32caadb5019fd4d25

ghost pushed a commit that referenced this pull request Jun 22, 2022
9580: [Backport 1.3]:  Do not restart broker on snapshot test r=npepinpe a=Zelldon

## Description
<!-- Please explain the changes you made here. -->
Backports #9573

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #9517



Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
ghost pushed a commit that referenced this pull request Jun 22, 2022
9581: [Backport 8.0]: Do not restart broker on snapshot test r=npepinpe a=Zelldon

## Description

Backports #9573 
<!-- Please explain the changes you made here. -->

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #9517 



Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
@lenaschoenburg lenaschoenburg added the version:1.3.11 Marks an issue as being completely or in parts released in 1.3.11 label Jun 27, 2022
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
version:1.3.11 Marks an issue as being completely or in parts released in 1.3.11
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SnapshotTest.update is flaky
4 participants