Implement memory limit functionality per ExecutionPlan in the TornadoVM API #323

mikepapadim · 2024-02-01T18:58:39Z

Description

This PR introduces withMemoryLimit() option in TornadoVM's execution plan. With this feature, the API provides the means to execute a TaskGraph only if fullfil specific memory constraints.

The use of the withMemoryLimit() will cause TornadoVM to throw an OutOfMemoryException in order to resize its buffers or increase the device memory limit or use the batch processing utilities.

 executionPlan.withMemoryLimit("1GB").execute();

Backend/s tested

Mark the backends affected by this PR.

OpenCL
PTX
SPIRV

OS tested

Mark the OS where this PR is tested.

Linux
OSx
Windows

How to test the new patch?

tornado-test -V uk.ac.manchester.tornado.unittests.memoryplan.TestWithMemoryLimit

Implemented a new method to handle specification of memory limit for tasks in the Tornado execution pipeline. The method parses the value and unit from a string input, calculates the size in bytes, and throws an exception for unsupported units. Ensure units "MB" and "GB" are supported. Also added related changes to the `TornadoExecutionContext` and `TornadoExecutionPlan` classes.

Integrate memory limit specification into the Tornado task graph. The update includes a parser to extract value and units from a string input, and computes the memory limit size in bytes. The parser throws exceptions for unsupported units; currently, "MB" and "GB" are supported. The changes affect the `TornadoExecutionContext` and `TornadoExecutionPlan` classes as well.

A new class named TestWithMemoryLimit has been introduced in the tornado-unittests module. This class is an extension of the TornadoTestBase class and is a part of the uk.ac.manchester.tornado.unittests.memoryplan package. This addition is part of work to integrate tests related to memory limitation in Tornado.

This commit introduces the TestWithMemoryLimit class in the tornado-unittests module. The class extends TornadoTestBase and tests the memory limit functionality using two new methods, `testWithMemoryLimitOver` and `testWithMemoryLimitUnder`. These

This commit brings more readability to TornadoVM code by using string interpolation instead of string concatenation. Additionally, it updates the way devices are added to the list in `mapAllTasksToSingleDevice` method making the insertion order clearer.

This update adds a method in TornadoTaskGraph to check if memory limit is exceeded and alters the checkMaxHeapAllocation method in TestBatches to checkMaxHeapAllocationOnDevice for clarity. It also refactors exception handling and debug print statements for improved readability.

The update introduces a new method that checks if a memory limit has been exceeded in the TornadoTaskGraph. It also includes exception handling related to this new feature. Lastly, the test cases in the TestBatches class have been refactored, with the checkMaxHeapAllocation method being altered for clearer understanding and readability.

The code difference shows the reactivation of a test case in the TornadoTaskGraph. This test is focused on checking if the system correctly throws a TornadoMemoryException when the memory limit is exceeded. The previously commented test has been updated to recursively check if the memory limit is under the allotted limit.

mikepapadim · 2024-02-01T19:14:48Z

I need to pass the formatter properly.

The updated code reintroduces a check in the TornadoExecutionContext to ensure it doesn't exceed the execution plan memory limit. This additional check will only run if the memory limit is not -1, which signifies there's a defined limit, and only then it will verify if the total size surpasses the memory limit. This change improves the system's exception handling for memory limit related issues.

The NUM_ELEMENTS value in TestWithMemoryLimit class has been increased. Additionally, changes have been made to increase totalSize incrementally in TornadoExecutionContext to ensure that it does not exceed the execution plan memory limit.

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoExecutionPlan.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TaskGraph.java

tornado-runtime/src/main/java/uk/ac/manchester/tornado/runtime/common/BatchConfiguration.java

jjfumero · 2024-02-02T11:00:34Z

...ittests/src/main/java/uk/ac/manchester/tornado/unittests/memoryplan/TestWithMemoryLimit.java

+        a = new IntArray(NUM_ELEMENTS);
+        b = new IntArray(NUM_ELEMENTS);
+        c = new IntArray(NUM_ELEMENTS);
+


Apply the code formatter

Add the test in the tornado-test.py script.

…ory limits

jjfumero · 2024-02-05T08:30:26Z

Is this PR ready?

jjfumero · 2024-02-05T09:07:25Z

...ittests/src/main/java/uk/ac/manchester/tornado/unittests/memoryplan/TestWithMemoryLimit.java

+ * </code>
+ * </p>
+ */
+public class TestWithMemoryLimit extends TornadoTestBase {


Suggested change

public class TestWithMemoryLimit extends TornadoTestBase {

public class TestMemoryLimits extends TornadoTestBase {

jjfumero · 2024-02-05T09:10:29Z

...ittests/src/main/java/uk/ac/manchester/tornado/unittests/memoryplan/TestWithMemoryLimit.java

+        a = new IntArray(NUM_ELEMENTS);
+        b = new IntArray(NUM_ELEMENTS);
+        c = new IntArray(NUM_ELEMENTS);
+


Add the test in the tornado-test.py script.

mikepapadim · 2024-02-06T06:49:26Z

This is ready now

mairooni

LGTM

Improvements ~~~~~~~~~~~~~~~~~~ - `#323 <https://github.com/beehive-lab/TornadoVM/pull/323>`_: Set Accelerator Memory Limit per Execution Plan at the API level - `#328 <https://github.com/beehive-lab/TornadoVM/pull/328>`_: Javadoc API to run with concurrent devices and memory limits - `#340 <https://github.com/beehive-lab/TornadoVM/pull/340>`_: New API calls to enable ``threadInfo`` and ``printKernel`` from the Execution Plan API. - `#334 <https://github.com/beehive-lab/TornadoVM/pull/334>`_: Dynamically enable/disable profiler after first run Compatibility ~~~~~~~~~~~~~~~~~~ - `#337 <https://github.com/beehive-lab/TornadoVM/pull/337>`_ : Initial support for Graal and JDK 21.0.2 Bug Fixes ~~~~~~~~~~~~~~~~~~ - `#322 <https://github.com/beehive-lab/TornadoVM/pull/322>`_: Fix duplicate thread-info debug message when the debug option is also enabled. - `#325 <https://github.com/beehive-lab/TornadoVM/pull/325>`_: Set/Get accesses for the ``MatrixVectorFloat4`` type fixed - `#326 <https://github.com/beehive-lab/TornadoVM/pull/326>`_: Fix installation script for running with Python >= 3.12 - `#327 <https://github.com/beehive-lab/TornadoVM/pull/327>`_: Fix Memory Limits for all supported Panama off-heap types. - `#329 <https://github.com/beehive-lab/TornadoVM/pull/329>`_: Fix timers for the dynamic reconfiguration policies - `#330 <https://github.com/beehive-lab/TornadoVM/pull/330>`_: Fix the profiler logs when silent mode is enabled - `#332 <https://github.com/beehive-lab/TornadoVM/pull/332>`_: Fix Batch processing when having multiple task-graphs in a single execution plan.

mikepapadim added 8 commits February 1, 2024 10:12

mikepapadim added the API label Feb 1, 2024

mikepapadim requested review from jjfumero and mairooni February 1, 2024 18:58

mikepapadim self-assigned this Feb 1, 2024

mikepapadim added 2 commits February 1, 2024 19:45

jjfumero requested changes Feb 2, 2024

View reviewed changes

mikepapadim added 2 commits February 2, 2024 11:36

Update naming conventions for newly introduced methods related to mem…

57078cc

…ory limits

Refactor DataTypeEnum into a seperate class and modify usages

fd94dba

jjfumero reviewed Feb 5, 2024

View reviewed changes

jjfumero requested changes Feb 5, 2024

View reviewed changes

mikepapadim added 4 commits February 5, 2024 12:32

Add withMemoryLimit API tests to the tornado-test script

8d89f38

Apply code formatter

41174a4

Refactor common class DataTypeSize and its usages

b60f6b5

Apply code formatter

d97397c

jjfumero approved these changes Feb 6, 2024

View reviewed changes

mairooni approved these changes Feb 6, 2024

View reviewed changes

jjfumero merged commit 191c129 into beehive-lab:develop Feb 6, 2024
2 checks passed

jjfumero mentioned this pull request Feb 29, 2024

[release] TornadoVM 1.0.2 #341

Merged

mikepapadim mentioned this pull request Mar 13, 2024

Refactor memory limit checks to take into account primitve type wrappers when using the withMemoryLimit API #352

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement memory limit functionality per ExecutionPlan in the TornadoVM API #323

Implement memory limit functionality per ExecutionPlan in the TornadoVM API #323

mikepapadim commented Feb 1, 2024 •

edited

Loading

mikepapadim commented Feb 1, 2024

jjfumero Feb 2, 2024

jjfumero Feb 5, 2024

jjfumero commented Feb 5, 2024

jjfumero Feb 5, 2024

jjfumero Feb 5, 2024

mikepapadim commented Feb 6, 2024

mairooni left a comment

	public class TestWithMemoryLimit extends TornadoTestBase {
	public class TestMemoryLimits extends TornadoTestBase {

Implement memory limit functionality per ExecutionPlan in the TornadoVM API #323

Implement memory limit functionality per ExecutionPlan in the TornadoVM API #323

Conversation

mikepapadim commented Feb 1, 2024 • edited Loading

Description

Backend/s tested

OS tested

How to test the new patch?

mikepapadim commented Feb 1, 2024

jjfumero Feb 2, 2024

Choose a reason for hiding this comment

jjfumero Feb 5, 2024

Choose a reason for hiding this comment

jjfumero commented Feb 5, 2024

jjfumero Feb 5, 2024

Choose a reason for hiding this comment

jjfumero Feb 5, 2024

Choose a reason for hiding this comment

mikepapadim commented Feb 6, 2024

mairooni left a comment

Choose a reason for hiding this comment

mikepapadim commented Feb 1, 2024 •

edited

Loading