[Feature] Select graph from a multi-graph execution plan to launch. #601

jjfumero · 2024-12-10T14:18:22Z

Description

This PR extends the TornadoExecutionPlan API with a new method to select a graph to launch from a multi-graph execution plan configuration. This type of code is useful when we need to share context (e.g., command queues, cuda streams, code cache, etc) for the whole execution plan, but we only need to execute a task-graph at the time.

The two methods added are as follows:

try (TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(tg1.snapshot(), tg2.snapshot())) {

      // Select graph 1 (tg2) to execute
      // Once selected, every time we call the execute method,
      // TornadoVM will launch the passed task-graph.
      executionPlan.withGraph(1).execute();
  
      // Select the graph 0 (tg1) to execute
      executionPlan.withGraph(0).execute();

      // Select all graphs (tg1 and tg2) to execute.
      // Since we selected individual task-graphs, we should be
      // able to reverse this action and invoke all task-graph
      // again. This is achieved with the `withAllGraphs` from the
      // execution plan.
       executionPlan.withAllGraphs().execute();
 }

This allows the following pattern in TornadoVM:

for (numIterations):
   executionPlan.withGraph(1).execute(); 
}

executionPlan.withGraph(2).execute();
...

Problem description

This path expands the capabilities of TornadoVM towards sharing state across different task-graph within a single execution plan.

Backend/s tested

Mark the backends affected by this PR.

OpenCL
PTX
SPIRV

OS tested

Mark the OS where this PR is tested.

Linux
OSx
Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

Yes
No

How to test the new patch?

tornado-test --printKernel --threadInfo -V uk.ac.manchester.tornado.unittests.executor.TestExecutor#test06

mikepapadim · 2024-12-11T08:29:13Z

is the original .execute() still works or we need to replace all instnaces with executionPlan.withAllGraphs().execute();?

mikepapadim

LGTM, thanks @jjfumero

stratika · 2024-12-11T08:45:51Z

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java

+        a.init(1);
+        b.init(2);


can we please test the outcome when the elements of the arrays are initialised with different values?

I added a new test with random values.

stratika · 2024-12-11T08:48:56Z

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java

+            for (int i = 0; i < c.getSize(); i++) {
+                assertEquals(a.get(i) + b.get(i), c.get(i));
+            }
+
+            // Select the graph 0 (tg1) to execute
+            executionPlan.withGraph(0).execute();
+            for (int i = 0; i < c.getSize(); i++) {
+                assertEquals(a.get(i) + b.get(i), c.get(i));
+            }


in this case, all elements in c should be 3, right? I think we should include a test that has different graphs, and potentially the second graph to use the outcome of the first graph.

Does this make sense? In my view this would be a good test to assess the consistency of data used across graphs.

It depends on how accessor type associated with the data. If it is read only for both graphs, then yes. Otherwise,a dependency is carried for the next graph. I added a unit-test to check this.

stratika · 2024-12-11T08:48:58Z

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java

+        TaskGraph tg1 = new TaskGraph("s0") //
+                .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b) //
+                .task("t0", TestHello::add, a, b, c) //
+                .transferToHost(DataTransferMode.EVERY_EXECUTION, c);
+
+        TaskGraph tg2 = new TaskGraph("s1") //
+                .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b) //
+                .task("t1", TestHello::add, a, b, c) //
+                .transferToHost(DataTransferMode.EVERY_EXECUTION, c);


is it possible to add a test in which the graphs have different compute method and they do not share the same variables?

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoExecutionPlan.java

jjfumero · 2024-12-11T11:16:03Z

is the original .execute() still works or we need to replace all instnaces with executionPlan.withAllGraphs().execute();?

No need to use withAllGraphs to execute all graphs. This is the action by default. The method withAllGraphs is only used when a specific graph was selected a a run, and then the developer wants to select all of them in subsequent runs.

jjfumero · 2024-12-11T11:38:04Z

Comment addressed.

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java

…ests/executor/TestExecutor.java

stratika

thanks, LGTM

Improvements ============= - beehive-lab#573: Enhanced output of unit-tests with a summary of pass-rates and fail-rates. - beehive-lab#576: Extended support for 3D matrices. - beehive-lab#580: Extended debug information for execution plans. - beehive-lab#584: Added helper menu for the ``tornado`` launcher script when no arguments are passed. - beehive-lab#589: Enable partial loop unrolling for all backends. - beehive-lab#594: Added RISC-V 64 CPU port support to run OpenCL with vector instructions RVV 1.0 (using the Codeplay OCK Toolkit). - beehive-lab#598: OpenCL low-level buffers tagged as read, write and read/write based on the data dependency analysis. - beehive-lab#601: Feature to select an immutable task graph to execute from a multi-task graph execution plan. Compatibility ============= - beehive-lab#570: Extended timeout for all suite of unit-tests. - beehive-lab#579: Removed legacy JDK 8 and JDK11 build options from the TornadoVM installer. - beehive-lab#582: Restored tornado runner scripts for IntellIJ. - beehive-lab#583: Automatic generation of IDE IntelliJ configuration runner files from the TornadoVM command. - beehive-lab#597: Updated white-list of unit-test and checkstyle improved. Bug Fixes ============= - beehive-lab#571: Fix issues with bracket closing for if/loops conditions. - beehive-lab#572: Fix for printing default execution plans (execution plans with default parameters). - beehive-lab#575: Fix the Level Zero version used for building the SPIR-V backend. - beehive-lab#577: Fix checkstyle. - beehive-lab#587: Fix thread scheduler for new NVIDIA Drivers. - beehive-lab#592: Fix ``Float.POSITIVE_INFINITY`` and ``Float.NEGATIVE_INFINITIVE`` constants for the OpenCL, CUDA and SPIR-V backends. - beehive-lab#596: Fix extra closing bracket during the code-generation for the FPGAs. - Remove the intermediate CUDA pinned memory regions in the JNI code: [link](beehive-lab@9c3f8ce) - Fix bitwise negation operations for the PTX backend: [link](beehive-lab@0db1cd3) - `GetBackendImpl::getAllDevices` thread-safe: [link](beehive-lab@0d44252) - Check size elements for memory segments: [link](beehive-lab@4360385)

jjfumero added 2 commits December 10, 2024 15:06

[api] Graph Selection for the Execution Plan supported

3b246a9

[javadoc] withGraph selection

c6f7a27

jjfumero added the API label Dec 10, 2024

jjfumero requested review from mikepapadim, mairooni and stratika December 10, 2024 14:18

jjfumero self-assigned this Dec 10, 2024

Merge branch 'develop' into feat/api/multi-graph

4a45877

mikepapadim approved these changes Dec 11, 2024

View reviewed changes

stratika reviewed Dec 11, 2024

View reviewed changes

jjfumero added 2 commits December 11, 2024 12:34

[test] Multi-Graph copy dependency from one graph to another

21a984f

minor change

85ad9f6

stratika reviewed Dec 11, 2024

View reviewed changes

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/executor/TestExecutor.java Outdated Show resolved Hide resolved

Update tornado-unittests/src/main/java/uk/ac/manchester/tornado/unitt…

97234fd

…ests/executor/TestExecutor.java

stratika approved these changes Dec 11, 2024

View reviewed changes

jjfumero merged commit 2228f74 into beehive-lab:develop Dec 11, 2024
2 checks passed

jjfumero deleted the feat/api/multi-graph branch December 11, 2024 12:11

jjfumero mentioned this pull request Dec 20, 2024

[release] TornadoVM 1.0.9 #602

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Select graph from a multi-graph execution plan to launch. #601

[Feature] Select graph from a multi-graph execution plan to launch. #601

jjfumero commented Dec 10, 2024

mikepapadim commented Dec 11, 2024

mikepapadim left a comment

stratika Dec 11, 2024

jjfumero Dec 11, 2024

stratika Dec 11, 2024

jjfumero Dec 11, 2024

stratika Dec 11, 2024

jjfumero commented Dec 11, 2024

jjfumero commented Dec 11, 2024

stratika left a comment

[Feature] Select graph from a multi-graph execution plan to launch. #601

[Feature] Select graph from a multi-graph execution plan to launch. #601

Conversation

jjfumero commented Dec 10, 2024

Description

Problem description

Backend/s tested

OS tested

Did you check on FPGAs?

How to test the new patch?

mikepapadim commented Dec 11, 2024

mikepapadim left a comment

Choose a reason for hiding this comment

stratika Dec 11, 2024

Choose a reason for hiding this comment

jjfumero Dec 11, 2024

Choose a reason for hiding this comment

stratika Dec 11, 2024

Choose a reason for hiding this comment

jjfumero Dec 11, 2024

Choose a reason for hiding this comment

stratika Dec 11, 2024

Choose a reason for hiding this comment

jjfumero commented Dec 11, 2024

jjfumero commented Dec 11, 2024

stratika left a comment

Choose a reason for hiding this comment