Introduce cache for process versions

[ChrisKujawa]

Description

When executing a process instance we often have to get the process model and related version to it.

When running a cluster for a while (creating a lot of state etc.) the process version will eventually be migrated to a low level of RocksDB (potentially L3), because most of the time process models are not deployed that often. In other words, if a key is not updated it will be moved to a lower level by RocksDB. Accessing lower levels of RocksDB is slower than accessing higher levels or mem tables.

You might ask why is it slow, even if we repeatedly access via RocksDB, why is it not in the cache? There are multiple reasons for it.

1. We only have caches for L0, and L1 configured (not for lower levels)
2. We have limited the cache sizes to a certain amount which might cause continuous eviction

In order to avoid running into issues with cold data, which is mostly static data, we can introduce our own caches, to work around this. This allows us to avoid unnecessary RocksDB access, unnecessary io, etc.

This PR does the following:

• Refactors the NextValueManager, including renaming as its only purpose is to be used for ProcessVersions
• Introduce a new cache for the version of each process, in order to avoid access to cold data.

Performance:

We run again a JMH benchmark with the changes and can see that the performance slightly increased (potentially 8%), not significant but it will likely come into play with other changes later.

See more details here

Related issues

closes #12034

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/1.3) to the PR, in case that fails you need to create backports manually.

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The change has been verified by a QA run
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• If the PR changes how BPMN processes are validated (e.g. support new BPMN element) then the Camunda modeling team should be informed to adjust the BPMN linting.

Other teams:
If the change impacts another team an issue has been created for this team, explaining what they need to do to support this change.

• Operate
• Tasklist
• Web Modeler
• Desktop Modeler
• Optimize

Please refer to our review guidelines.

[ChrisKujawa]

Just for posterity:

Today we had a sync PR review with @berkaycanbc and @koevskinikola where I explained all the details and reasoning behind it. We had a good discussion where I also was able to answer some open questions.

There was no blocker found during this call, so I expect we can go ahead to merge the PR after it is accepted by any of the reviewers.

[megglos]

LGTM!

[megglos]

bors r+

[ghost]

Build succeeded:

• Test summary

[backport-action]

Backport failed for stable/8.1, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin stable/8.1
git worktree add -d .worktree/backport-12475-to-stable/8.1 origin/stable/8.1
cd .worktree/backport-12475-to-stable/8.1
git checkout -b backport-12475-to-stable/8.1
ancref=$(git merge-base 9dbedea61c4e96297ee9076c159d0902195d9257 743bec2ca5c74372c3d0d359106d06396c41b008)
git cherry-pick -x $ancref..743bec2ca5c74372c3d0d359106d06396c41b008

[backport-action]

Successfully created backport PR for stable/8.2:

• [Backport stable/8.2] Introduce cache for process versions #12495