Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYSTEMML-1325] Support for GPU in JMLC and Harmonize Script Execution Pipelines #830

Closed
wants to merge 1 commit into from

Conversation

thomas9t
Copy link
Contributor

This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript (command line) APIs. Specifically, the following changes were made:

  1. All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved.
  2. All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to execute the compiled program. Previously, JMLC called the Script.execute method directly.
  3. jmlc.Connection.prepareScript now supports compiling a script to use GPU. Compiling a GPU enabled program requires modifying static flags in DMLScript. Because of this, it is necessary to synchronize compilation when using GPU to ensure only a single program is being compiled at any one time. Otherwise, a script being compiled to use CPU only may incorrectly use GPU enabled opertors. To avoid unnecessary synchronization in the more common case that multiple JMLC scripts are being compiled to use CPU only, we perform a static check at startup to see if the user has included the JCuda dependency in the pipeline. If so, we assume they may want to use GPU and compilation is synchronized. Otherwise, compilation remains unsynchronized.
  4. A PreparedScript is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that a PreparedScript must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run.

@niketanpansare, @bertholdreinwald, @mboehm7 and @deroneriksson can you please take a look over these changes and let me know if you have comments/questions.

thomas9t referenced this pull request Aug 23, 2018
Although we allow that intermediate matrices and frames are passed from
one script invocation to another, every script execution initializes the
scratch-space and buffer pool which cleanups up all existing files. This
might delete files that still backup pending rdd operations or
matrix/frame objects. This patch fixes this issue by initializing the
scratch space only on the first execution. Furthermore, we now also
properly cleanup the scratch_space on mlcontext close.
@mboehm7
Copy link
Contributor

mboehm7 commented Aug 23, 2018

Overall, this is a great start. Let's also flag this with SYSTEMML-1325 during the merge, because we want to consolidate the replicated compilation chain for a while now.

There is just one scenario where the current approach wouldn't work. JMLC allows for dynamic recompilation. If there are multiple JMLC streams with different GPU configurations (which are also used for recompilation during runtime), the approach of static variables and synchronized compilation is not sufficient. Maybe we could promote these flags to thread-local compiler configurations similar to the recompilation configuration?

…n Pipelines

This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript (command line) APIs. Specifically, the following changes were made:

 - All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved.
- All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to execute the compiled program. Previously, JMLC called the Script.execute method directly.
- jmlc.Connection.prepareScript now supports compiling a script to use GPU. Compiling a GPU enabled program requires modifying static flags in DMLScript. Because of this, it is necessary to synchronize compilation when using GPU to ensure only a single program is being compiled at any one time. Otherwise, a script being compiled to use CPU only may incorrectly use GPU enabled opertors. To avoid unnecessary synchronization in the more common case that multiple JMLC scripts are being compiled to use CPU only, we perform a static check at startup to see if the user has included the JCuda dependency in the pipeline. If so, we assume they may want to use GPU and compilation is synchronized. Otherwise, compilation remains unsynchronized.
 - A PreparedScript is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that a PreparedScript must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run.

Closes apache#830.
@thomas9t thomas9t changed the title Support for GPU in JMLC and Harmonize Script Execution Pipelines [SYSTEMML-1325] Support for GPU in JMLC and Harmonize Script Execution Pipelines Aug 31, 2018
@thomas9t thomas9t closed this Sep 1, 2018
@thomas9t thomas9t deleted the jmlc_gpu branch September 1, 2018 00:06
niketanpansare pushed a commit to niketanpansare/systemml that referenced this pull request Oct 14, 2018
Support to JMLC

This PR adds support for compilation and execution of GPU enabled
scripts in JMLC and harmonizes the pipeline used to compile and execute
DML programs across the JMLC, MLContext and DMLScript. Specifically, the
following changes were made:

1. All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to
compile DML scripts. The original logic in MLContext and JMLC for
pinning inputs and persisting outputs has been preserved.
2. All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to
execute the compiled program. Previously, JMLC called the Script.execute
method directly.
3. jmlc.Connection.prepareScript now supports compiling a script to use
GPU. Note that following apache#832 the issue noted in apache#830 has been resolved.
4. A PreparedScript is now statically assigned a GPU context when it is
compiled and instatiated. This has potential performance implications
because it means that a PreparedScript must be executed on a specific
GPU. However, it reduces overhead from creating a GPU context each time
a script is executed and unsures that a user cannot compile a script to
use GPU and then forget to assign a GPU context when the script is run.
5. Per (3) I have added a unit test which compiles and executes a GPU
enabled script in JMLC both with and without pinned data and just
asserts that no errors occur.

Closes apache#836.
asfgit pushed a commit that referenced this pull request Oct 14, 2018
Support to JMLC

This PR adds support for compilation and execution of GPU enabled
scripts in JMLC and harmonizes the pipeline used to compile and execute
DML programs across the JMLC, MLContext and DMLScript. Specifically, the
following changes were made:

1. All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to
compile DML scripts. The original logic in MLContext and JMLC for
pinning inputs and persisting outputs has been preserved.
2. All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to
execute the compiled program. Previously, JMLC called the Script.execute
method directly.
3. jmlc.Connection.prepareScript now supports compiling a script to use
GPU. Note that following #832 the issue noted in #830 has been resolved.
4. A PreparedScript is now statically assigned a GPU context when it is
compiled and instatiated. This has potential performance implications
because it means that a PreparedScript must be executed on a specific
GPU. However, it reduces overhead from creating a GPU context each time
a script is executed and unsures that a user cannot compile a script to
use GPU and then forget to assign a GPU context when the script is run.
5. Per (3) I have added a unit test which compiles and executes a GPU
enabled script in JMLC both with and without pinned data and just
asserts that no errors occur.

Closes #836.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants