Introduce Tensor API, Tensor Utilities and compatiblity with ONNX RT #369

mikepapadim · 2024-03-29T14:22:06Z

Description

This pull request introduces a new Tensor API v0.1, providing a comprehensive set of classes and utilities for working with tensors of various data types along with compatibility with ONNX RT Java bindings.

The primary changes include:

AbstractTensor Interface: An abstract base class that defines the common contract for all tensor implementations.
Tensor Classes: A set of concrete tensor classes for different data types, including TensorFloat32, TensorFloat16, TensorInt32, TensorInt16, TensorInt64, TensorFloat64, and TensorByte. These classes extend the AbstractTensor interface and provide type-specific functionality.
DType Class: A utility class that encapsulates data type information for tensors, facilitating type-safe operations and conversions.
Shape Class: A class representing the shape (dimensions) of a tensor.

Also, this PR adds test to show compatibility with ONNX RT Java bindings->

  public void testOnnxCompatibility() throws OrtException {
        Shape shape = new Shape(1, 3, 224, 224);
        TensorFloat32 tornadoTensor = new TensorFloat32(shape);

        tornadoTensor.init(2f);

        OnnxTensor outputTensor = null;

        try (OrtEnvironment env = OrtEnvironment.getEnvironment()) {
            // Load the MobileNet V2 ONNX model
            OrtSession session = env.createSession(MODEL_PATH, new OrtSession.SessionOptions());

            OnnxTensor inputTensor = OnnxTensor.createTensor(env, tornadoTensor.getFloatBuffer(), shape.dimensions());

            Map<String, OnnxTensor> inputMap = new HashMap<>();
            inputMap.put(INPUT_TENSOR_NAME, inputTensor);

            // Run the model inference
            try (OrtSession.Result outputMap = session.run(inputMap)) {
                Optional<OnnxValue> optionalOutputTensor = outputMap.get(OUTPUT_TENSOR_NAME);
                if (optionalOutputTensor.isEmpty()) {
                    throw new IllegalArgumentException("Output tensor not found in model output.");
                }
                outputTensor = (OnnxTensor) optionalOutputTensor.get();

            }
        } finally {
            Assert.assertNotNull(outputTensor);
        }
    }

Overview of the Tensor API Architecture:

Notes:
It adds a dependency to ONNX runtime in the tornado-unittests packages, as an alternative we could move the compatibility tests into a seperate repo or submodule.

Backend/s tested

Mark the backends affected by this PR.

OpenCL
PTX
SPIRV

OS tested

Mark the OS where this PR is tested.

Linux
OSx
Windows

How to test the new patch?

Provide instructions about how to test the new patch.

make jdk21 backends=opencl

# Test how to use Tornado Tensor Types with ONNX RT
tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorAPIWithOnnx

# Test Tensor Types for all the supported DTypes
tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes

Implemented Tensor, DType classes in `uk.ac.manchester.tornado.api.types.tensors` package. Additionally, a tensor testing class TestTensorTypes was added in `uk.ac.manchester.tornado.unittests.tensors`. The Tensor class contains methods relating to tensor data type and shape, while DType enum lists all supported data types.

This refactor turns the data types enum (DType) into several separate class files under a dtype package, aligning with good OOP practices by giving each datatype its own class. It also adds new Tensor classes with dtype and shape properties, and updates relevant tests and segments accordingly.

This commit removes the Tensor classes (`FloatTensor`, `HalfFloatTensor`, `Tensor`) and adds specific data type names to the respective DType subclasses (`Bool`, `Double`, `Float`, `HalfFloat`, `Int16`, `Int32`, `Int64`, `Int8`, `QInt32`, `QInt8`, `QUInt8`). This design change simplifies the overall architecture by removing unnecessary classes and enhances code readability and maintainability.

This commit renames the `TensorArray` class to `Tensor`. It also removes the necessity for an explicit cast method by taking `DType` as a parameter upon initialization. This refactor improves clarity of code, simplifies the class structure and makes it easier to work with tensors of different data types.

This commit changes the name of the `HalfFloat` class to `HF` for conciseness. All the references to `HalfFloat` have also been updated to reflect this change such as in the `DType` class. The renaming will make the code more succinct and manageable.

Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity.

This update introduces support for the HalfFloat data type in the Tensor class and includes modifications to multiple components for this purpose. The code has also been refactored for improved clarity, including the relocation of node replacement functions in the TornadoHalfFloatReplacement section.

This commit introduces streamlined support for HalfFloat data type within the Tensor class and includes necessary adjustments across various components. The reshuffling of certain functions, particularly within the TornadoHalfFloatReplacement section, has also been undertaken to improve code clarity

…Wrapper classes This commit includes several improvements in the handling of OpenCL devices and object wrapping. The unnecessary verbose logs have been removed from both classes. The code of OCLTornadoDevice has been simplified using variable capturing and switch expressions. The handling of task types and atomic variables has been streamlined for better readability and efficiency.

Implemented the instance of pattern in the TornadoDataflowAnalysis class to simplify repetitive if-else structures and enhance code readability. Also, removed redundant else statements to further improve the code maintainability and clarity.

Streamlined code in TornadoHalfFloatFixedGuardElimination by utilizing the instance of pattern for PiNode placeholderInput. This produces cleaner, easier to read code and eliminates unnecessary casting operations. We also removed unneeded "if" conditions for improved efficiency and clarity.

The array creation methods in DType (Tornado API) have been removed. These changes make the code cleaner and easier to read, as these removed methods seemed to be extras that are not needed, hence streamlining the code.

An additional unit test for tensor addition with floating point numbers is added. In the same file, some non-essential print statements and commented out verifying code have been removed to avoid redundancy and make the code cleaner.

The switch-case code block in OCLHotSpotBackendFactory.java is refactored to use the new Java Switch Expressions. This makes the code easier to read and understand while preserving the original functionality.

The function 'calculateSize' in the DType.java file has been removed. It seems like it was superfluous code as it was not being used anywhere, clearing it out for better code readability and maintainability.

Functionality was added to allow creation of a Tensor directly from a float array in Tensor.java. This is done through the static method 'fromArray' and the helper method 'createSegment'. This necessitated updates to existing unit tests with new tests written for the added functionality.

New tensor types (Int32, Int64, Float32, etc.) have been added to support various data types. The TornadoNativeArray class permissions were adjusted to accommodate these new tensor types. The existing Tensor class was also extended to implement the AbstractTensor interface.

…oVM into feat/tensors_api_v0.1

jjfumero · 2024-04-08T08:50:04Z

Is this PR ready for a second review?

mikepapadim · 2024-04-08T08:54:19Z

Is this PR ready for a second review?

Yes

stratika · 2024-04-08T09:06:47Z

I have a problem when running the following tests with OpenCL:

tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes
tornado --jvm "-Xmx6g -Dtornado.recover.bailout=False -Dtornado.unittests.verbose=True "  -m  tornado.unittests/uk.ac.manchester.tornado.unittests.tools.TornadoTestRunner  --params "uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes"
WARNING: Using incubator modules: jdk.incubator.vector
[ERROR] clEnqueueReadBuffer, code = -5 n[TornadoVM-OCL-JNI] ERROR : clEnqueueReadBuffer -> Returned: [JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> -5CL_OUT_OF_RESOURCES error executing CL_COMMAND_READ_BUFFER on NVIDIA RTX A2000 8GB Laptop GPU (Device 0).


[TornadoVM-OCL-JNI] ERROR : clFlush -> Returned: -9999
[JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> Unknown error executing clFlush on NVIDIA RTX A2000 8GB Laptop GPU (Device 0).

[TornadoVM-OCL-JNI] ERROR : clFinish -> Returned: -36
[TornadoVM-OCL-JNI] ERROR : clEnqueueWriteBuffer -> Returned: -5
[JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> CL_OUT_OF_RESOURCES error executing CL_COMMAND_WRITE_BUFFER on NVIDIA RTX A2000 8GB Laptop GPU (Device 0).

Do they work for you?

jjfumero · 2024-04-08T09:11:03Z

No. Same error in my configuration (OpenCL)

tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes 
tornado --jvm "-Xmx6g -Dtornado.recover.bailout=False -Dtornado.unittests.verbose=True "  -m  tornado.unittests/uk.ac.manchester.tornado.unittests.tools.TornadoTestRunner  --params "uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes"
WARNING: Using incubator modules: jdk.incubator.vector
[ERROR] clEnqueueReadBuffer, code = -5 n[TornadoVM-OCL-JNI] ERROR : clEnqueueReadBuffer -> Returned: [JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> -5CL_OUT_OF_RESOURCES error executing CL_COMMAND_READ_BUFFER on NVIDIA GeForce RTX 3070 (Device 0).


[TornadoVM-OCL-JNI] ERROR : clFlush -> Returned: -9999
[JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> Unknown error executing clFlush on NVIDIA GeForce RTX 3070 (Device 0).

[TornadoVM-OCL-JNI] ERROR : clFinish -> Returned: -36
[TornadoVM-OCL-JNI] ERROR : clEnqueueWriteBuffer -> Returned: [JNI] uk.ac.manchester.tornado.drivers.opencl> notify error:
[JNI] uk.ac.manchester.tornado.drivers.opencl> CL_OUT_OF_RESOURCES error executing CL_COMMAND_WRITE_BUFFER on NVIDIA GeForce RTX 3070 (Device 0).

jjfumero · 2024-04-08T09:11:41Z

This is related to offsets and sizes when moving data from the host to the device.

mikepapadim · 2024-04-08T09:33:26Z

if i run the tests individually, they are passing:

tornado-test --printKernel -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes#testTensorInt64Add

mikepapadim · 2024-04-08T10:05:30Z

Now, it is fixed

stratika

Added some suggested fixes for style issues in the current code base.

stratika · 2024-04-08T13:00:27Z

...s/opencl/src/main/java/uk/ac/manchester/tornado/drivers/opencl/runtime/OCLTornadoDevice.java

@@ -57,6 +57,7 @@
 import uk.ac.manchester.tornado.api.types.arrays.FloatArray;
 import uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray;
 import uk.ac.manchester.tornado.api.types.arrays.IntArray;
+import uk.ac.manchester.tornado.api.types.tensors.TensorFP16;


Remove this, seems to be unused.

stratika · 2024-04-08T13:08:01Z

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorTypes.java

+        tensorA.init(20l);
+        tensorB.init(3000l);
+        tensorA.init(0l);


By running,

make checkstyle: [INFO] There are 5 errors reported by Checkstyle 9.3 with tornado-assembly/src/etc/checkstyle.xml ruleset. [ERROR] src/main/java/uk/ac/manchester/tornado/unittests/api/TestConcat.java:[128,16] (misc) UpperEll: Should use uppercase 'L'. [ERROR] src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorAPIWithOnnx.java:[112,28] (misc) ArrayTypeStyle: Array brackets at illegal position. [ERROR] src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorTypes.java:[283,22] (misc) UpperEll: Should use uppercase 'L'. [ERROR] src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorTypes.java:[284,22] (misc) UpperEll: Should use uppercase 'L'. [ERROR] src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorTypes.java:[285,22] (misc) UpperEll: Should use uppercase 'L'.

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorTypes.java

...nittests/src/main/java/uk/ac/manchester/tornado/unittests/tensors/TestTensorAPIWithOnnx.java

…ests/tensors/TestTensorTypes.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…ests/tensors/TestTensorAPIWithOnnx.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…ests/tensors/TestTensorTypes.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

jjfumero · 2024-04-09T07:09:55Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/tensors/Tensor.java

+    }
+
+    @Override
+    public MemorySegment getSegment() {


I still don't get the purpose of the implementation of these methods, Can you set to abstract method and then all subtypes implement these methods using the right fields?

I ll change this

this is fixed now

jjfumero · 2024-04-09T07:10:55Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/tensors/TensorFP16.java

+     */
+
+    public TensorFP16(Shape shape) {
+        super(DType.HALF_FLOAT, shape);


FP16 is 2 bytes. so it shoult be SHORT, correct?

Indeed, but the DType class has the dtypes in format that are stored in model files

jjfumero · 2024-04-09T07:18:10Z

SPIR-V backend:

tornado-test -V --fast uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes
tornado --jvm "-Xmx6g -Dtornado.recover.bailout=False -Dtornado.unittests.verbose=True "  -m  tornado.unittests/uk.ac.manchester.tornado.unittests.tools.TornadoTestRunner  --params "uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes"
WARNING: Using incubator modules: jdk.incubator.vector
Test: class uk.ac.manchester.tornado.unittests.tensors.TestTensorTypes
	Running test: testHelloTensorAPI         ................  [PASS] 
	Running test: testTensorFloat16Add       ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<5.0>
	Running test: testTensorFloat32Add       ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<3000.0>
	Running test: testTensorFloat64Add       ................  [FP64 UNSUPPORTED FOR CURRENT DEVICE] 
	Running test: testTensorInt16Add         ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<300.0>
	Running test: testTensorInt32Add         ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<300.0>
	Running test: testTensorInt64Add         ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<3000.0>
	Running test: testTensorByte             ................  [FAILED] 
		\_[REASON] expected:<0.0> but was:<44.0>

I am using the Intel compute runtime: 24.09.28717.12

jjfumero · 2024-04-09T10:00:03Z

Pending:

Investigate the failures in the SPIR-V backend.

stratika

LGTM, let's proceed and extend it with fix for the pending issues.

mikepapadim · 2024-04-09T10:13:59Z

Pending: fix for FP16

mikepapadim · 2024-04-09T13:04:57Z

Pending: fix for FP16

this is fixed now

Improvements ~~~~~~~~~~~~~~~~~~ - [beehive-lab#369](beehive-lab#369): Introduction of Tensor types in TornadoVM API and interoperability with ONNX Runtime. - [beehive-lab#370](beehive-lab#370): Array concatenation operation for TornadoVM native arrays. - [beehive-lab#371](beehive-lab#371): TornadoVM installer script ported for Windows 10/11. - [beehive-lab#372](beehive-lab#372): Add support for ``HalfFloat`` (``Float16``) in vector types. - [beehive-lab#374](beehive-lab#374): Support for TornadoVM array concatenations from the constructor-level. - [beehive-lab#375](beehive-lab#375): Support for TornadoVM native arrays using slices from the Panama API. - [beehive-lab#376](beehive-lab#376): Support for lazy copy-outs in the batch processing mode. - [beehive-lab#377](beehive-lab#377): Expand the TornadoVM profiler with power metrics for NVIDIA GPUs (OpenCL and PTX backends). - [beehive-lab#384](beehive-lab#384): Auto-closable Execution Plans for automatic memory management. Compatibility ~~~~~~~~~~~~~~~~~~ - [beehive-lab#386](beehive-lab#386): OpenJDK 17 support removed. - [beehive-lab#390](beehive-lab#390): SapMachine OpenJDK 21 supported. - [beehive-lab#395](beehive-lab#395): OpenJDK 22 and GraalVM 22.0.1 supported. - TornadoVM tested with Apple M3 chips. Bug Fixes ~~~~~~~~~~~~~~~~~~ - [beehive-lab#367](beehive-lab#367): Fix for Graal/Truffle languages in which some Java modules were not visible. - [beehive-lab#373](beehive-lab#373): Fix for data copies of the ``HalfFloat`` types for all backends. - [beehive-lab#378](beehive-lab#378): Fix free memory markers when running multi-thread execution plans. - [beehive-lab#379](beehive-lab#379): Refactoring package of vector api unit-tests. - [beehive-lab#380](beehive-lab#380): Fix event list sizes to accommodate profiling of large applications. - [beehive-lab#385](beehive-lab#385): Fix code check style. - [beehive-lab#387](beehive-lab#387): Fix TornadoVM internal events in OpenCL, SPIR-V and PTX for running multi-threaded execution plans. - [beehive-lab#388](beehive-lab#388): Fix of expected and actual values of tests. - [beehive-lab#392](beehive-lab#392): Fix installer for using existing JDKs. - [beehive-lab#389](beehive-lab#389): Fix ``DataObjectState`` for multi-thread execution plans. - [beehive-lab#396](beehive-lab#396): Fix JNI code for the CUDA NVML library access with OpenCL.

mikepapadim added 30 commits February 27, 2024 18:06

wip

e704149

add tensor

4950843

Update Tensor to support HalfFloat type

367a6dd

Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity.

Update Tensor to support HalfFloat type

3f05f59

Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity.

Update Tensor to support HalfFloat type

96fc95c

Added support for the HalfFloat type in the Tensor class. This includes a new constructor to create Tensors with MemorySegment and HalfFloat type, and methods to set and get HalfFloat values in the Tensor. Also refactor a few methods for clarity.

WIP

e73a7cd

Remove array creation methods in DType

dc3dabd

The array creation methods in DType (Tornado API) have been removed. These changes make the code cleaner and easier to read, as these removed methods seemed to be extras that are not needed, hence streamlining the code.

Refactor code block in OCLHotSpotBackendFactory.java

91c1d87

The switch-case code block in OCLHotSpotBackendFactory.java is refactored to use the new Java Switch Expressions. This makes the code easier to read and understand while preserving the original functionality.

Remove calculateSize method from DType.java

5f18365

The function 'calculateSize' in the DType.java file has been removed. It seems like it was superfluous code as it was not being used anywhere, clearing it out for better code readability and maintainability.

Merge branch 'mikepapadim/segment_slice' into mikepapadim/tensors_v0.5

086de25

Add detailed comments and improve documentation in Tensor modules

5b6aabc

Refactor tensor unit tests and add random data population

6ba50f8

Refactor DType.java and enhance documentation

4520e84

Add tensor constructor and float buffer conversion.

9e2af7d

Fix merge conflicts

4655bd5

Update and extend tensor types for data handling.

f32c116

mairooni and others added 2 commits April 5, 2024 13:23

Merge branch 'feat/tensors_api_v0.1' of github.com:mikepapadim/Tornad…

efa781d

…oVM into feat/tensors_api_v0.1

add fix for tensorfp16

ff3189c

Remove TensorFP16 handling in OCLTornadoDevice

7033276

stratika reviewed Apr 8, 2024

View reviewed changes

mikepapadim and others added 6 commits April 8, 2024 14:40

Update tornado-unittests/src/main/java/uk/ac/manchester/tornado/unitt…

2cd3e6a

…ests/tensors/TestTensorTypes.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-unittests/src/main/java/uk/ac/manchester/tornado/unitt…

43ce7d1

…ests/tensors/TestTensorTypes.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-unittests/src/main/java/uk/ac/manchester/tornado/unitt…

f2e295a

…ests/tensors/TestTensorAPIWithOnnx.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-unittests/src/main/java/uk/ac/manchester/tornado/unitt…

cec52f9

…ests/tensors/TestTensorTypes.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Remove TensorFP16 handling in OCLTornadoDevice

42688dd

Update Tensor import

68c10c5

jjfumero requested changes Apr 9, 2024

View reviewed changes

mikepapadim added 2 commits April 9, 2024 10:08

Update Tensor import

ed4c6d9

Update Tensor class

86ecf73

jjfumero approved these changes Apr 9, 2024

View reviewed changes

stratika approved these changes Apr 9, 2024

View reviewed changes

jjfumero merged commit 6e22c3f into beehive-lab:develop Apr 9, 2024
2 checks passed

mairooni mentioned this pull request Apr 9, 2024

Handle cases where the class field is a half float array #373

Merged

6 tasks

jjfumero mentioned this pull request Apr 30, 2024

[release] TornadoVM v1.0.4 #398

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Tensor API, Tensor Utilities and compatiblity with ONNX RT #369

Introduce Tensor API, Tensor Utilities and compatiblity with ONNX RT #369

mikepapadim commented Mar 29, 2024 •

edited

Loading

jjfumero commented Apr 8, 2024

mikepapadim commented Apr 8, 2024

stratika commented Apr 8, 2024

jjfumero commented Apr 8, 2024

jjfumero commented Apr 8, 2024

mikepapadim commented Apr 8, 2024

mikepapadim commented Apr 8, 2024 •

edited

Loading

stratika left a comment

stratika Apr 8, 2024

stratika Apr 8, 2024

jjfumero Apr 9, 2024

mikepapadim Apr 9, 2024 •

edited

Loading

mikepapadim Apr 9, 2024

jjfumero Apr 9, 2024

mikepapadim Apr 9, 2024

jjfumero commented Apr 9, 2024

jjfumero commented Apr 9, 2024

stratika left a comment

mikepapadim commented Apr 9, 2024

mikepapadim commented Apr 9, 2024

Introduce Tensor API, Tensor Utilities and compatiblity with ONNX RT #369

Introduce Tensor API, Tensor Utilities and compatiblity with ONNX RT #369

Conversation

mikepapadim commented Mar 29, 2024 • edited Loading

Description

Backend/s tested

OS tested

How to test the new patch?

jjfumero commented Apr 8, 2024

mikepapadim commented Apr 8, 2024

stratika commented Apr 8, 2024

jjfumero commented Apr 8, 2024

jjfumero commented Apr 8, 2024

mikepapadim commented Apr 8, 2024

mikepapadim commented Apr 8, 2024 • edited Loading

stratika left a comment

Choose a reason for hiding this comment

stratika Apr 8, 2024

Choose a reason for hiding this comment

stratika Apr 8, 2024

Choose a reason for hiding this comment

jjfumero Apr 9, 2024

Choose a reason for hiding this comment

mikepapadim Apr 9, 2024 • edited Loading

Choose a reason for hiding this comment

mikepapadim Apr 9, 2024

Choose a reason for hiding this comment

jjfumero Apr 9, 2024

Choose a reason for hiding this comment

mikepapadim Apr 9, 2024

Choose a reason for hiding this comment

jjfumero commented Apr 9, 2024

jjfumero commented Apr 9, 2024

stratika left a comment

Choose a reason for hiding this comment

mikepapadim commented Apr 9, 2024

mikepapadim commented Apr 9, 2024

mikepapadim commented Mar 29, 2024 •

edited

Loading

mikepapadim commented Apr 8, 2024 •

edited

Loading

mikepapadim Apr 9, 2024 •

edited

Loading