-
Notifications
You must be signed in to change notification settings - Fork 479
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SYSTEMML-1325] Support for GPU in JMLC and Harmonize Script Execution Pipelines #830
Conversation
Although we allow that intermediate matrices and frames are passed from one script invocation to another, every script execution initializes the scratch-space and buffer pool which cleanups up all existing files. This might delete files that still backup pending rdd operations or matrix/frame objects. This patch fixes this issue by initializing the scratch space only on the first execution. Furthermore, we now also properly cleanup the scratch_space on mlcontext close.
Overall, this is a great start. Let's also flag this with SYSTEMML-1325 during the merge, because we want to consolidate the replicated compilation chain for a while now. There is just one scenario where the current approach wouldn't work. JMLC allows for dynamic recompilation. If there are multiple JMLC streams with different GPU configurations (which are also used for recompilation during runtime), the approach of static variables and synchronized compilation is not sufficient. Maybe we could promote these flags to thread-local compiler configurations similar to the recompilation configuration? |
…n Pipelines This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript (command line) APIs. Specifically, the following changes were made: - All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved. - All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to execute the compiled program. Previously, JMLC called the Script.execute method directly. - jmlc.Connection.prepareScript now supports compiling a script to use GPU. Compiling a GPU enabled program requires modifying static flags in DMLScript. Because of this, it is necessary to synchronize compilation when using GPU to ensure only a single program is being compiled at any one time. Otherwise, a script being compiled to use CPU only may incorrectly use GPU enabled opertors. To avoid unnecessary synchronization in the more common case that multiple JMLC scripts are being compiled to use CPU only, we perform a static check at startup to see if the user has included the JCuda dependency in the pipeline. If so, we assume they may want to use GPU and compilation is synchronized. Otherwise, compilation remains unsynchronized. - A PreparedScript is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that a PreparedScript must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run. Closes apache#830.
Support to JMLC This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript. Specifically, the following changes were made: 1. All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved. 2. All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to execute the compiled program. Previously, JMLC called the Script.execute method directly. 3. jmlc.Connection.prepareScript now supports compiling a script to use GPU. Note that following apache#832 the issue noted in apache#830 has been resolved. 4. A PreparedScript is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that a PreparedScript must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run. 5. Per (3) I have added a unit test which compiles and executes a GPU enabled script in JMLC both with and without pinned data and just asserts that no errors occur. Closes apache#836.
Support to JMLC This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript. Specifically, the following changes were made: 1. All three APIs now call ScriptExecutorUtils.compileRuntimeProgram to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved. 2. All three APIs now use ScriptExecutorUtils.executeRuntimeProgram to execute the compiled program. Previously, JMLC called the Script.execute method directly. 3. jmlc.Connection.prepareScript now supports compiling a script to use GPU. Note that following #832 the issue noted in #830 has been resolved. 4. A PreparedScript is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that a PreparedScript must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run. 5. Per (3) I have added a unit test which compiles and executes a GPU enabled script in JMLC both with and without pinned data and just asserts that no errors occur. Closes #836.
This PR adds support for compilation and execution of GPU enabled scripts in JMLC and harmonizes the pipeline used to compile and execute DML programs across the JMLC, MLContext and DMLScript (command line) APIs. Specifically, the following changes were made:
ScriptExecutorUtils.compileRuntimeProgram
to compile DML scripts. The original logic in MLContext and JMLC for pinning inputs and persisting outputs has been preserved.ScriptExecutorUtils.executeRuntimeProgram
to execute the compiled program. Previously, JMLC called theScript.execute
method directly.jmlc.Connection.prepareScript
now supports compiling a script to use GPU. Compiling a GPU enabled program requires modifying static flags inDMLScript
. Because of this, it is necessary to synchronize compilation when using GPU to ensure only a single program is being compiled at any one time. Otherwise, a script being compiled to use CPU only may incorrectly use GPU enabled opertors. To avoid unnecessary synchronization in the more common case that multiple JMLC scripts are being compiled to use CPU only, we perform a static check at startup to see if the user has included the JCuda dependency in the pipeline. If so, we assume they may want to use GPU and compilation is synchronized. Otherwise, compilation remains unsynchronized.PreparedScript
is now statically assigned a GPU context when it is compiled and instatiated. This has potential performance implications because it means that aPreparedScript
must be executed on a specific GPU. However, it reduces overhead from creating a GPU context each time a script is executed and unsures that a user cannot compile a script to use GPU and then forget to assign a GPU context when the script is run.@niketanpansare, @bertholdreinwald, @mboehm7 and @deroneriksson can you please take a look over these changes and let me know if you have comments/questions.