User can't deploy two different DMN models in one deployment

[ChrisKujawa]

Describe the bug

Hi,

I’m having an issue trying to write a process test using Zeebe-process-test-extension in Camunda Cloud 8. (v.8.0.2)
The following code runs fine if it’s just 1 bpmn and 1 dmn. But fails when i try adding second dmn:
return client.newDeployResourceCommand()
.addResourceFromClasspath(“test.bpmn”)
.addResourceFile("/dmn/test1.dmn")
.addResourceFile("/dmn/test2.dmn")
.send()
.join();

Here is the error: io.camunda.zeebe.db.ZeebeDbInconsistentException: Key DbLong{2251799813685250} in ColumnFamily DMN_DECISION_REQUIREMENTS already exists

I tried using addResourceFromClasspath and AddResourceFile - both methods result in the same error.

https://forum.camunda.io/t/fail-to-load-multiple-dmns-in-zeebe-process-test-extension-test-in-camunda-8/37483/4?u=zelldon

To Reproduce

return client.newDeployResourceCommand()
.addResourceFromClasspath(“dummy_bpmn.bpmn”)
.addResourceFromClasspath(“dummy1.dmn”)
.addResourceFromClasspath(“dummy2.dmn”)
.send()
.join();

dummy1.dmn.txt
dummy2.dmn.txt

dummy-bpmn.bpmn.txt

Expected behavior

No error on deploying two different dmn models.

Log/Stacktrace

Full Stacktrace

Error:
io.camunda.zeebe.client.api.command.ClientStatusException: Command 'CREATE' rejected with code 'INVALID_ARGUMENT': Expected to deploy new resources, but encountered the following errors:
'dummy2.dmn': Key DbLong{2251799813685250} in ColumnFamily DMN_DECISION_REQUIREMENTS already exists

	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.transformExecutionException(ZeebeClientFutureImpl.java:93)
	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:50)
	at io.camunda.c8.test.DeterminationProcessTest.initDeployment(DeterminationProcessTest.java:48)
	at io.camunda.c8.test.DeterminationProcessTest.testDeployment(DeterminationProcessTest.java:53)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Command 'CREATE' rejected with code 'INVALID_ARGUMENT': Expected to deploy new resources, but encountered the following errors:
'dummy2.dmn': Key DbLong{2251799813685250} in ColumnFamily DMN_DECISION_REQUIREMENTS already exists
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
	at io.camunda.zeebe.client.impl.ZeebeClientFutureImpl.join(ZeebeClientFutureImpl.java:48)
	... 71 more

Environment:

• zeebe-process-test:8.0.1

[saig0]

Let's analyze the issue first with the given resources. The goal is to fix the issue until the next minor version.

[OlikPolik]

Hi,

i think i now ran into probably the same bug now with ZeebeDeployment annotation:
@ZeebeDeployment(resources = {"classpath*:.bpmn","classpath:*.dmn"})

this fails with the same error as the above bud when there is more than 1 dmn found to be deployed.

Error:
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Command 'CREATE' rejected with code 'INVALID_ARGUMENT': Expected to deploy new resources, but encountered the following errors:
'tooth.dmn': Key DbLong{2251799813685252} in ColumnFamily DMN_DECISION_REQUIREMENTS already exists

[OlikPolik]

Ah oops, i see now someone already reported that as well

[remcowesterhoud]

In the DmnResourceTransformer we will transform each resource, so each dmn file. In here we have the method appendMetadataToDeploymentEvent . This method will add a DecisionRequirementsMetadataRecord to the DeploymentRecord. Once the metadata has been added we write the records:

private void writeRecords(final DeploymentRecord deployment, final DeploymentResource resource) {

    for (final DecisionRequirementsMetadataRecord drg : deployment.decisionRequirementsMetadata()) {
      if (!drg.isDuplicate()) {
        stateWriter.appendFollowUpEvent(
            drg.getDecisionRequirementsKey(),
            DecisionRequirementsIntent.CREATED,
            new DecisionRequirementsRecord()
                .setDecisionRequirementsKey(drg.getDecisionRequirementsKey())
                .setDecisionRequirementsId(drg.getDecisionRequirementsId())
                .setDecisionRequirementsName(drg.getDecisionRequirementsName())
                .setDecisionRequirementsVersion(drg.getDecisionRequirementsVersion())
                .setNamespace(drg.getNamespace())
                .setResourceName(drg.getResourceName())
                .setChecksum(drg.getChecksumBuffer())
                .setResource(resource.getResourceBuffer()));
      }
    }

As you can see this method takes the DeploymentRecord as a parameter and iterates over all the DecisionRequirementsMetadataRecords that have been added. So what happens when we deploy multiple dmns:

1. Send the deploy resource command. We create a new DeploymentRecord with an empty list of DecisionRequirementsMetadataRecords.
2. We start transforming dmn 1. A new DecisionRequirementsMetadataRecord gets appended to the DeploymentRecord
3. We write a record for all metadatas in the DeploymentRecord . At this point this is only the metadata of dmn 1
4. We start transforming dmn 2. A new DecisionRequirementsMetadataRecord gets appended to the DeploymentRecord
5. We write a record for all metadatas in the DeploymentRecord. At this point this record contains the metadata of both dmn 1 and dmn 2, resulting in a CREATED event for dmn 2 containing the metadata keys of dmn 1