feat: added ImmutableRecordCopier

- updated README and tests to use ImmutableRecordCopier

README.md

@@ -1,5 +1,3 @@
-[![Build Status](https://travis-ci.org/zeebe-io/zeebe-protocol-immutables.svg?branch=master)](https://travis-ci.org/zeebe-io/zeebe-protocol-immutables)
-
 # zeebe-protocol-immutables
 
 This library provides an implementation of the Zeebe protocol which can be serialized and
@@ -61,9 +59,8 @@ For example, an exporter could do the following:
 ```java
 private static final ObjectMapper MAPPER = new ObjectMapper();
 
-public void export(Record record) {
-
-  final Record clone = ImmutableRecord.builder().from(record);
+public void export(Record<?> record) {
+  final Record<?> clone = ImmutableRecordCopier.deepCopyOf(record);
 
   try(final OutputStream out = createOutputStream()) {
     MAPPER.writeValue(out, clone);  
@@ -73,17 +70,28 @@ public void export(Record record) {
 
 You could then have configured the `ObjectMapper` to write YAML, CBOR, etc., beforehand.
 
-### Limitations
+### Copying and comparing
+
+If you want to compare two `Record<T>` instances with potentially different implementations, the
+recommended way is to first convert them both to `ImmutableRecord<T>`.
+
+The easiest way is to use the `ImmutableRecordCopier` utility class. You can do a deep copy of any 
+`Record<>` as is, and it will return an equivalent `ImmutableRecord<>`.
+
+```java
+final Record<?> record = ...;
+final ImmutableRecord<?> copiedRecord = ImmutableRecordCopier.deepCopyOf(record);
+```
 
-There are some known limitations regarding serialization. Serializing as shown above will not
-serialize nested types, e.g. the value itself. If you want to do so recursively, you first have to
-clone the value itself and set it in the builder. I'd like to improve this, but haven't gotten to it
-yet, as I was mostly focused on deserialization.
+If you just want to copy the record value, or if you want to have a 
+`ImmutableRecord<ImmutableJobRecordValue>`, for example, then you can copy the value first and copy
+the record yourself as:
 
-Another known limitation is that an `ImmutableRecord` is not necessarily equal to the `Record`, even
-if they are logically equal. That is to say, serializing a record, then deserializing will not give
-two records which are `Object#equals()`. This is expected as `Record` is just an interface, but can
-be surprising nonetheless.
+```java
+final Record<DeploymentRecordValue> record = ...;
+final ImmutableDeploymentRecordValue copiedValue = ImmutableRecordCopier.deepCopyOf(record.getValueType(), record.getValue()); 
+final ImmutableRecord<ImmutableDeploymentRecordValue> copiedRecord = ImmutableRecord.builder().from(record).value(copiedValue).build();
+```
 
 ## Development
 
@@ -122,17 +130,45 @@ the generated `Immutable*` versions of these classes.
 
 ### Deserialization
 
-The `Record` class in the protocol is typed; the value's concrete class and the intent's concrete 
-enum are both derived from the `Record#getValueType`. As such, both fields are annotated using
-`@JsonTypeInfo` which points to that property, and are given a corresponding type resolver (see
-`ValueTypeIdResolver` and `IntentTypeIdResolver`), which allows Jackson to properly deserialize a 
-`Record<DeploymentRecordValue>` concretely as `ImmutableRecord<ImmutableDeploymentRecordValue>`.
+Since `Record<T>` is a typed interface, we need to resolve `T` during deserialization. The way to do
+so in Zeebe is by using the `Record#getValueType()`.
+
+> You can look at `ValueTypeIdResolver` to see how we resolve the value type to the right value
+> class.
+
+The `Intent` of the record is an interface, which also needs to be resolved to the correct type
+during deserialization.
+
+> You can look at `IntentTypeIdResolver` to see how raw intents are mapped to the right type.
+
+With these out of the way, you can then easily deserialize a raw JSON payload into an
+`ImmutableRecord<T>`, where all types are properly resolved. See [usage](#usage) for more.
 
 ## Testing
 
-Currently testing is sort of a playground - I decided to go for property based testing using jqwik, 
-and I'm learning as I go, so there are bound to be mistakes in how I'm doing this.
-Contributions are more than welcome :)
+We assume that the `immutables` library works properly, and as such focus primarily on the
+serialization capabilities.
+
+To test this, we use two built-in exporters: specifically the `DebugHttpExporter` and
+the `RecordingExporter`.
+
+We start a Zeebe broker, run a sample workload, then wait for all records to be exported.
+
+> NOTE: waiting for all records to be exported is difficult due to the black box nature of the test
+> infrastructure, so we simply wait up until some seconds have passed since the last record was
+> exported. This is relatively safe since the `ExporterIntegrationRule` already waits until the
+> workload is finished and exported to the `RecordingExporter`, so waiting just a few seconds more
+> is mostly just to be safe.
+
+The `DebugHttpExporter` provides an endpoint where we can get all exported records as JSON. This is
+our sample data set from which we can test the deserialization capabilities of the library.
+
+The `RecordingExporter` provides us with the raw exported records against which we can then compare.
+
+To simplify the comparison, the raw records are first converted to an equivalent
+`ImmutableRecord<?>` representation. This may seem tautological, but as mentioned, we assume the
+generated code is valid (which includes the builders), and just want to verify that serialization
+works as expected.
 
 ## Code of Conduct
 

pom.xml

@@ -138,6 +138,13 @@
       <scope>test</scope>
     </dependency>
 
+    <dependency>
+      <groupId>org.junit.jupiter</groupId>
+      <artifactId>junit-jupiter-params</artifactId>
+      <version>${version.junit}</version>
+      <scope>test</scope>
+    </dependency>
+
     <dependency>
       <groupId>org.assertj</groupId>
       <artifactId>assertj-core</artifactId>

src/main/java/io/zeebe/protocol/immutables/ImmutableRecordCopier.java

@@ -0,0 +1,198 @@
+/*
+ * Copyright © 2020 camunda services GmbH (info@camunda.com)
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.zeebe.protocol.immutables;
+
+import io.zeebe.protocol.immutables.record.ImmutableDeployedWorkflow;
+import io.zeebe.protocol.immutables.record.ImmutableDeploymentRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableDeploymentResource;
+import io.zeebe.protocol.immutables.record.ImmutableErrorRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableIncidentRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableJobBatchRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableJobRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableMessageRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableMessageStartEventSubscriptionRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableMessageSubscriptionRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableRecord;
+import io.zeebe.protocol.immutables.record.ImmutableRecord.Builder;
+import io.zeebe.protocol.immutables.record.ImmutableTimerRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableVariableDocumentRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableVariableRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableWorkflowInstanceCreationRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableWorkflowInstanceRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableWorkflowInstanceResultRecordValue;
+import io.zeebe.protocol.immutables.record.ImmutableWorkflowInstanceSubscriptionRecordValue;
+import io.zeebe.protocol.record.Record;
+import io.zeebe.protocol.record.RecordValue;
+import io.zeebe.protocol.record.ValueType;
+import io.zeebe.protocol.record.value.DeploymentRecordValue;
+import io.zeebe.protocol.record.value.ErrorRecordValue;
+import io.zeebe.protocol.record.value.IncidentRecordValue;
+import io.zeebe.protocol.record.value.JobBatchRecordValue;
+import io.zeebe.protocol.record.value.JobRecordValue;
+import io.zeebe.protocol.record.value.MessageRecordValue;
+import io.zeebe.protocol.record.value.MessageStartEventSubscriptionRecordValue;
+import io.zeebe.protocol.record.value.MessageSubscriptionRecordValue;
+import io.zeebe.protocol.record.value.TimerRecordValue;
+import io.zeebe.protocol.record.value.VariableDocumentRecordValue;
+import io.zeebe.protocol.record.value.VariableRecordValue;
+import io.zeebe.protocol.record.value.WorkflowInstanceCreationRecordValue;
+import io.zeebe.protocol.record.value.WorkflowInstanceRecordValue;
+import io.zeebe.protocol.record.value.WorkflowInstanceResultRecordValue;
+import io.zeebe.protocol.record.value.WorkflowInstanceSubscriptionRecordValue;
+import io.zeebe.protocol.record.value.deployment.DeployedWorkflow;
+import io.zeebe.protocol.record.value.deployment.DeploymentResource;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Utility class to perform deep copy of any {@link Record<>} implementation to an equivalent {@link
+ * ImmutableRecord<>} implementation.
+ *
+ * <p>This is necessary as by default the {@code copyOf()} methods generated by the library perform
+ * only shallow copies to some extent - that is, they will correctly detect when a member has an
+ * equivalent {@code Immutable*} type, but not when said member is a collection or a container.
+ *
+ * <p>If you want to perform deep copies, for example to compare two implementations, you can use
+ * the methods below.
+ */
+public final class ImmutableRecordCopier {
+
+  private ImmutableRecordCopier() {}
+
+  @SuppressWarnings("unchecked")
+  public static <T extends RecordValue, U extends T> ImmutableRecord<U> deepCopyOfRecord(
+      final Record<T> record) {
+    final U value = (U) deepCopyOfRecordValue(record.getValueType(), record.getValue());
+    final Builder<T> originalBuilder = ImmutableRecord.<T>builder().from(record);
+    final Builder<U> convertedBuilder = (Builder<U>) originalBuilder;
+
+    return convertedBuilder.value(value).build();
+  }
+
+  @SuppressWarnings("unchecked")
+  public static <T extends RecordValue, U extends T> T deepCopyOfRecordValue(
+      final ValueType type, final T value) {
+    switch (type) {
+      case JOB:
+        return (U) ImmutableJobRecordValue.builder().from((JobRecordValue) value).build();
+      case DEPLOYMENT:
+        return (U) deepCopyOfDeploymentRecordValue((DeploymentRecordValue) value);
+      case WORKFLOW_INSTANCE:
+        return (U)
+            ImmutableWorkflowInstanceRecordValue.builder()
+                .from((WorkflowInstanceRecordValue) value)
+                .build();
+      case INCIDENT:
+        return (U) ImmutableIncidentRecordValue.builder().from((IncidentRecordValue) value).build();
+      case MESSAGE:
+        return (U) ImmutableMessageRecordValue.builder().from((MessageRecordValue) value).build();
+      case MESSAGE_SUBSCRIPTION:
+        return (U)
+            ImmutableMessageSubscriptionRecordValue.builder()
+                .from((MessageSubscriptionRecordValue) value)
+                .build();
+      case WORKFLOW_INSTANCE_SUBSCRIPTION:
+        return (U)
+            ImmutableWorkflowInstanceSubscriptionRecordValue.builder()
+                .from((WorkflowInstanceSubscriptionRecordValue) value)
+                .build();
+      case JOB_BATCH:
+        return (U) deepCopyOfJobBatchRecordValue((JobBatchRecordValue) value);
+      case TIMER:
+        return (U) ImmutableTimerRecordValue.builder().from((TimerRecordValue) value).build();
+      case MESSAGE_START_EVENT_SUBSCRIPTION:
+        return (U)
+            ImmutableMessageStartEventSubscriptionRecordValue.builder()
+                .from((MessageStartEventSubscriptionRecordValue) value)
+                .build();
+      case VARIABLE:
+        return (U) ImmutableVariableRecordValue.builder().from((VariableRecordValue) value).build();
+      case VARIABLE_DOCUMENT:
+        return (U)
+            ImmutableVariableDocumentRecordValue.builder()
+                .from((VariableDocumentRecordValue) value)
+                .build();
+      case WORKFLOW_INSTANCE_CREATION:
+        return (U)
+            ImmutableWorkflowInstanceCreationRecordValue.builder()
+                .from((WorkflowInstanceCreationRecordValue) value)
+                .build();
+      case ERROR:
+        return (U) ImmutableErrorRecordValue.builder().from((ErrorRecordValue) value).build();
+      case WORKFLOW_INSTANCE_RESULT:
+        return (U)
+            ImmutableWorkflowInstanceResultRecordValue.builder()
+                .from((WorkflowInstanceResultRecordValue) value)
+                .build();
+      case SBE_UNKNOWN:
+      case NULL_VAL:
+      default:
+        throw new IllegalArgumentException("Unknown value type " + type);
+    }
+  }
+
+  private static ImmutableDeploymentRecordValue deepCopyOfDeploymentRecordValue(
+      final DeploymentRecordValue value) {
+    final List<DeployedWorkflow> workflows = new ArrayList<>();
+    final List<DeploymentResource> resources = new ArrayList<>();
+
+    for (final DeployedWorkflow workflow : value.getDeployedWorkflows()) {
+      final ImmutableDeployedWorkflow immutableWorkflow;
+      if (workflow instanceof ImmutableDeployedWorkflow) {
+        immutableWorkflow = (ImmutableDeployedWorkflow) workflow;
+      } else {
+        immutableWorkflow = ImmutableDeployedWorkflow.builder().from(workflow).build();
+      }
+
+      workflows.add(immutableWorkflow);
+    }
+
+    for (final DeploymentResource resource : value.getResources()) {
+      final ImmutableDeploymentResource immutableResource;
+      if (resource instanceof ImmutableDeploymentResource) {
+        immutableResource = (ImmutableDeploymentResource) resource;
+      } else {
+        immutableResource = ImmutableDeploymentResource.builder().from(resource).build();
+      }
+
+      resources.add(immutableResource);
+    }
+
+    return ImmutableDeploymentRecordValue.builder()
+        .from(value)
+        .resources(resources)
+        .deployedWorkflows(workflows)
+        .build();
+  }
+
+  private static ImmutableJobBatchRecordValue deepCopyOfJobBatchRecordValue(
+      final JobBatchRecordValue value) {
+    final List<JobRecordValue> jobs = new ArrayList<>();
+
+    for (final JobRecordValue job : value.getJobs()) {
+      final ImmutableJobRecordValue immutableJob;
+      if (job instanceof ImmutableJobRecordValue) {
+        immutableJob = (ImmutableJobRecordValue) job;
+      } else {
+        immutableJob = ImmutableJobRecordValue.builder().from(job).build();
+      }
+
+      jobs.add(immutableJob);
+    }
+
+    return ImmutableJobBatchRecordValue.builder().from(value).jobs(jobs).build();
+  }
+}

src/main/java/io/zeebe/protocol/immutables/record/ZeebeStyle.java

@@ -34,6 +34,7 @@
     validationMethod = ValidationMethod.NONE,
     defaultAsDefault = true,
     headerComments = true,
-    clearBuilder = true)
+    clearBuilder = true,
+    deepImmutablesDetection = true)
 @JsonSerialize
 public @interface ZeebeStyle {}

src/test/java/io/zeebe/protocol/immutables/ImmutableRecordCopierTest.java

@@ -0,0 +1,41 @@
+/*
+ * Copyright © 2020 camunda services GmbH (info@camunda.com)
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.zeebe.protocol.immutables;
+
+import static org.assertj.core.api.Assertions.assertThatCode;
+
+import io.zeebe.protocol.record.ValueType;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.EnumSource;
+import org.junit.jupiter.params.provider.EnumSource.Mode;
+
+final class ImmutableRecordCopierTest {
+
+  /**
+   * This test checks that every known record value type is handled by the {@link
+   * ImmutableRecordCopier}, and should fail if it isn't. This is a smoke test when updating Zeebe
+   * versions to detect new {@link io.zeebe.protocol.record.ValueType} instances.
+   */
+  @EnumSource(
+      value = ValueType.class,
+      names = {"NULL_VAL", "SBE_UNKNOWN"},
+      mode = Mode.EXCLUDE)
+  @ParameterizedTest
+  void shouldHandleEveryKnownValueType(final ValueType type) {
+    assertThatCode(() -> ImmutableRecordCopier.deepCopyOfRecordValue(type, null))
+        .isNotInstanceOf(IllegalArgumentException.class);
+  }
+}