Java JNI (JPype) Wrapper

.gitignore

@@ -187,7 +187,7 @@ testing/scripts/tensorflow
 testing/scripts/run.log
 testing/scripts/my-model/
 wrappers/s2i/python/_python/
-wrappers/s2i/python-conda/_python/
+incubating/wrappers/s2i/java-jni/_python/
 
 seldon-controller/go
 

.pre-commit-config.yaml

@@ -9,4 +9,4 @@ repos:
     rev: stable
     hooks:
       - id: black
-        args: ['python/', 'testing/', 'operator/helm', 'operator/seldon-operator/hack', '--exclude', '(testing/scripts/proto|seldon_core/proto/|.eggs)']
+        args: ['python/', 'testing/', '--exclude', '(testing/scripts/proto|seldon_core/proto/|.eggs)']

doc/source/index.rst

@@ -86,6 +86,7 @@ Documentation Index
    :caption: Incubating Projects
 
    Java Language Wrapper <java/README.md>
+   Java (JNI) Language Wrapper [ALPHA] <java-jni/README.md>
    R Language Wrapper [ALPHA] <R/README.md>
    NodeJS Language Wrapper [ALPHA] <nodejs/README.md>
    Go Language Wrapper [ALPHA] <go/go_wrapper_link.rst>

doc/source/java-jni/README.md

@@ -0,0 +1,237 @@
+# Packaging a Java model for Seldon Core using s2i
+
+In this guide, we illustrate the steps needed to wrap your own Java model in a
+docker image ready for deployment with Seldon Core using [source-to-image app
+s2i](https://github.com/openshift/source-to-image) and the JNI wrapper.
+
+The JNI wrapper leverages the [Python inference server](../python) to
+communicate through JNI with your Java model.
+The Python wrapper usually receives updates much faster than the [native Java
+one](../java/README.md).
+Therefore it allows you to have faster access to the latest Seldon Core
+features.
+
+If you are not familiar with s2i you can read [general instructions on using
+s2i](../wrappers/s2i.md) and then follow the steps below.
+
+## Step 1 - Install s2i
+
+[Download and install s2i](https://github.com/openshift/source-to-image#installation)
+
+ * Prerequisites for using s2i are:
+   * Docker
+   * Git (if building from a remote git repo)
+
+To check everything is working you can run
+
+```bash
+s2i usage seldonio/s2i-java-jni-build:0.3.0
+```
+
+## Step 2 - Create your source code
+
+To use our s2i builder image to package your Java model you will need:
+
+ * A Maven project that depends on `io.seldon.wrapper` library (>= `0.3.0`)
+ * A class that implements `io.seldon.wrapper.api.SeldonPredictionService` for the type of component you are creating
+ * .s2i/environment - model definitions used by the s2i builder to correctly wrap your model
+
+We will go into detail for each of these steps:
+
+### Maven Project
+Create a Spring Boot Maven project and include the dependency:
+
+```XML
+<dependency>
+	<groupId>io.seldon.wrapper</groupId>
+	<artifactId>seldon-core-wrapper</artifactId>
+	<version>0.3.0</version>
+</dependency>
+```
+
+A full example can be found at `incubating/wrappers/s2i/java-jni/test/model-template-app/pom.xml`.
+
+### Prediction Class
+To handle requests to your model or other component you need to implement one
+or more of the methods in `io.seldon.wrapper.api.SeldonPredictionService`, in
+particular:
+
+```java
+default public SeldonMessage predict(SeldonMessage request);
+default public SeldonMessage route(SeldonMessage request);
+default public SeldonMessage sendFeedback(Feedback request);
+default public SeldonMessage transformInput(SeldonMessage request);
+default public SeldonMessage transformOutput(SeldonMessage request);
+default public SeldonMessage aggregate(SeldonMessageList request);
+```
+
+There is a full H2O example in
+`examples/models/h2o_mojo/src/main/java/io/seldon/example/h2o/model`, whose
+implementation is shown below:
+
+```java
+public class H2OModelHandler implements SeldonPredictionService {
+	private static Logger logger = LoggerFactory.getLogger(H2OModelHandler.class.getName());
+	EasyPredictModelWrapper model;
+
+	public H2OModelHandler() throws IOException {
+		MojoReaderBackend reader =
+                MojoReaderBackendFactory.createReaderBackend(
+                  getClass().getClassLoader().getResourceAsStream(
+                     "model.zip"),
+                      MojoReaderBackendFactory.CachingStrategy.MEMORY);
+		MojoModel modelMojo = ModelMojoReader.readFrom(reader);
+		model = new EasyPredictModelWrapper(modelMojo);
+		logger.info("Loaded model");
+	}
+
+	@Override
+	public SeldonMessage predict(SeldonMessage payload) {
+		List<RowData> rows = H2OUtils.convertSeldonMessage(payload.getData());
+		List<AbstractPrediction> predictions = new ArrayList<>();
+		for(RowData row : rows)
+		{
+			try
+			{
+				BinomialModelPrediction p = model.predictBinomial(row);
+				predictions.add(p);
+			} catch (PredictException e) {
+				logger.info("Error in prediction ",e);
+			}
+		}
+        DefaultData res = H2OUtils.convertH2OPrediction(predictions, payload.getData());
+
+		return SeldonMessage.newBuilder().setData(res).build();
+	}
+
+}
+
+```
+
+The above code:
+
+  * Loads a model from the local resources folder on startup
+  * Converts the proto buffer message into H2O RowData using provided utility classes.
+  * Runs a BionomialModel prediction and converts the result back into a `SeldonMessage` for return
+
+#### H2O Helper Classes
+
+We provide H2O utility class `io.seldon.wrapper.utils.H2OUtils` in
+seldon-core-wrapper to convert to and from the seldon-core proto buffer message
+types.
+
+#### DL4J Helper Classes
+
+We provide a DL4J utility class `io.seldon.wrapper.utils.DL4JUtils` in
+seldon-core-wrapper to convert to and from the seldon-core proto buffer message
+types.
+
+### .s2i/environment
+
+Define the core parameters needed by our Java S2I images to wrap your model. 
+An example is:
+
+```bash
+API_TYPE=REST
+SERVICE_TYPE=MODEL
+JAVA_IMPORT_PATH=io.seldon.example.model.ExampleModelHandler
+```
+
+These values can also be provided or overridden on the command line when building the image.
+
+## Step 3 - Build your image
+
+Use `s2i build` to create your Docker image from source code.
+You will need Docker installed on the machine and optionally git if your source
+code is in a public git repo.
+
+Using s2i you can build directly from a git repo or from a local source folder.
+See the [s2i
+docs](https://github.com/openshift/source-to-image/blob/master/docs/cli.md#s2i-build)
+for further details.
+The general format is:
+
+```bash
+s2i build \
+  <git-repo | src-folder> \
+  seldonio/s2i-java-jni-build:0.3.0 \
+  --runtime-image seldonio/s2i-java-jni-runtime:0.3.0 \
+  <my-image-name> 
+```
+
+An example invocation using the test template model inside `seldon-core`:
+
+```bash
+s2i build \
+  https://github.com/seldonio/seldon-core.git \
+  --context-dir=incubating/wrappers/s2i/java-jni/test/model-template-app \
+  seldonio/s2i-java-jni-build:0.3.0 \
+  --runtime-image seldonio/s2i-java-jni-runtime:0.3.0 \
+  jni-model-template:0.1.0
+```
+
+The above s2i build invocation:
+
+ * uses the `seldon-core` [GitHub repo](https://github.com/seldonio/seldon-core)
+   and the directory `incubating/wrappers/s2i/java-jni/test/model-template-app`
+   inside that repo.
+ * uses the builder image `seldonio/s2i-java-jni-build`
+ * uses the runtime image `seldonio/s2i-java-jni-runtime`
+ * creates a docker image `jni-template-model`
+
+
+For building from a local source folder, an example where we clone the seldon-core repo:
+
+```bash
+git clone https://github.com/seldonio/seldon-core.git
+cd seldon-core
+s2i build \
+  incubating/wrappers/s2i/java-jni/test/model-template-app \
+  seldonio/s2i-java-jni-build:0.3.0 \
+  --runtime-image seldonio/s2i-java-jni-runtime:0.3.0 \
+  jni-model-template:0.1.0
+```
+
+For more help see:
+
+```bash
+s2i usage seldonio/s2i-java-jni-build:0.3.0
+s2i build --help
+```
+
+## Reference
+
+## Environment Variables
+The required environment variables understood by the builder image are
+explained below.
+You can provide them in the `.s2i/environment` file or on the `s2i build`
+command line.
+
+### JAVA_IMPORT_PATH
+
+Import path for your Java model implementation.
+For instance, in the example above, this would be
+`io.seldon.example.model.ExampleModelHandler`.
+
+### API_TYPE
+
+API type to create.
+Can be REST or GRPC.
+
+### SERVICE_TYPE
+
+The service type being created.
+Available options are:
+
+ * MODEL
+ * ROUTER
+ * TRANSFORMER
+ * COMBINER
+
+
+## Creating different service types
+
+### MODEL
+
+ * [A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/incubating/wrappers/s2i/java/test/model-template-app)
+ * [Example H2O MOJO](../examples/h2o_mojo.html)

doc/source/python/python_wrapping_s2i.md

@@ -35,7 +35,7 @@ To use our s2i builder image to package your python model you will need:
 We will go into detail for each of these steps:
 
 ### Python file
-Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at ```wrappers/s2i/python/test/model-template-app/MyModel.py```:
+Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at `wrappers/s2i/python/test/model-template-app/MyModel.py`:
 
 ```python
 class MyModel(object):
@@ -127,7 +127,7 @@ PERSISTENCE=0
 These values can also be provided or overridden on the command line when building the image.
 
 ## Step 3 - Build your image
-Use ```s2i build``` to create your Docker image from source code. You will need Docker installed on the machine and optionally git if your source code is in a public git repo. You can choose from three python builder images
+Use `s2i build` to create your Docker image from source code. You will need Docker installed on the machine and optionally git if your source code is in a public git repo. You can choose from three python builder images
 
  * Python 3.6 : seldonio/seldon-core-s2i-python36:1.2.3-dev seldonio/seldon-core-s2i-python3:1.2.3-dev
    * Note there are [issues running TensorFlow under Python 3.7](https://github.com/tensorflow/tensorflow/issues/20444) (Nov 2018) and Python 3.7 is not officially supported by TensorFlow (Dec 2018).
@@ -149,9 +149,9 @@ s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i
 
 The above s2i build invocation:
 
- * uses the GitHub repo: https://github.com/seldonio/seldon-core.git and the directory ```wrappers/s2i/python/test/model-template-app``` inside that repo.
- * uses the builder image ```seldonio/seldon-core-s2i-python3```
- * creates a docker image ```seldon-core-template-model```
+ * uses the GitHub repo: https://github.com/seldonio/seldon-core.git and the directory `wrappers/s2i/python/test/model-template-app` inside that repo.
+ * uses the builder image `seldonio/seldon-core-s2i-python3`
+ * creates a docker image `seldon-core-template-model`
 
 
 For building from a local source folder, an example where we clone the seldon-core repo:
@@ -174,10 +174,11 @@ s2i build --help
 To ensure Keras models with the Tensorflow backend work correctly you may need to call `_make_predict_function()` on your model after it is loaded. This is because Flask may call the prediction request in a separate thread from the one that initialised your model. See the [keras issue](https://github.com/keras-team/keras/issues/6462) for further discussion.
 
 ## Environment Variables
-The required environment variables understood by the builder image are explained below. You can provide them in the ```.s2i/environment``` file or on the ```s2i build``` command line.
+The required environment variables understood by the builder image are explained below. You can provide them in the `.s2i/environment` file or on the `s2i build` command line.
 
 
 ### MODEL_NAME
+
 The name of the class containing the model. Also the name of the python file which will be imported.
 
 ### API_TYPE
@@ -200,17 +201,36 @@ Set either to 0 or 1. Default is 0. If set to 1 then your model will be saved pe
 
 ### EXTRA_INDEX_URL
 
-For installing packages from private/self-hosted PyPi registry.
+.. Warning:: 
+   ``EXTRA_INDEX_URL`` is recommended to be passed as argument to ``s2i``
+   command rather than adding in ``.s2i/environment`` as a practice of avoiding
+   checking in credentials in the code.
 
-#### NOTE: EXTRA_INDEX_URL is recommended to be passed as argument to `s2i` command rather than adding in `.s2i/environment` as a practice of avoiding checking in credentials in the code.
+For installing packages from private/self-hosted PyPi registry.
 
 ### PIP_TRUSTED_HOST
 
 For adding private/self-hosted unsecured PyPi registry by adding it to pip trusted-host.
 
 ```bash
-s2i build -e EXTRA_INDEX_URL=https://<pypi-user>:<pypi-auth>@mypypi.example.com/simple -e PIP_TRUSTED_HOST=mypypi.example.com <src-folder> seldonio/seldon-core-s2i-python3:1.2.3-dev <my-image-name>
+s2i build \
+   -e EXTRA_INDEX_URL=https://<pypi-user>:<pypi-auth>@mypypi.example.com/simple \
+   -e PIP_TRUSTED_HOST=mypypi.example.com \
+   <src-folder> \
+   seldonio/seldon-core-s2i-python3:1.2.3-dev \
+   <my-image-name>
 ```
+
+### PAYLOAD_PASSTHROUGH
+
+If enabled, the Python server won't try to decode the request payload nor
+encode the response back.
+That means that the `predict()` method of your `SeldonComponent` model will
+receive the payload as-is and it will be responsible to decode it.
+Likewise, the return value of `predict()` must be a serialised response.
+
+By default, this option will be disabled.
+
 ## Creating different service types
 
 ### MODEL
@@ -231,7 +251,7 @@ s2i build -e EXTRA_INDEX_URL=https://<pypi-user>:<pypi-auth>@mypypi.example.com/
 ## Advanced Usage
 
 ### Model Class Arguments
-You can add arguments to your component which will be populated from the ```parameters``` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
+You can add arguments to your component which will be populated from the `parameters` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
 
 ```python
 class TfServingProxy(object):
@@ -278,24 +298,24 @@ These arguments can be set when deploying in a Seldon Deployment. An example can
 ```
 
 
-The allowable ```type``` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
+The allowable `type` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
 
 
 ### Local Python Dependencies
-```from version 0.5-SNAPSHOT```
+`from version 0.5-SNAPSHOT`
 
 To use a private repository for installing Python dependencies use the following build command:
 
 ```bash
 s2i build -i <python-wheel-folder>:/whl <src-folder> seldonio/seldon-core-s2i-python3:1.2.3-dev <my-image-name>
 ```
 
-This command will look for local Python wheels in the ```<python-wheel-folder>``` and use these before searching PyPI.
+This command will look for local Python wheels in the `<python-wheel-folder>` and use these before searching PyPI.
 
 ### Custom Metrics
-```from version 0.3```
+`from version 0.3`
 
-To add custom metrics to your response you can define an optional method ```metrics``` in your class that returns a list of metric dicts. An example is shown below:
+To add custom metrics to your response you can define an optional method `metrics` in your class that returns a list of metric dicts. An example is shown below:
 
 ```python
 class MyModel(object):
@@ -312,9 +332,9 @@ For more details on custom metrics and the format of the metric dict see [here](
 There is an [example notebook illustrating a model with custom metrics in python](../examples/custom_metrics.html).
 
 ### Custom Request Tags
-```from version 0.3```
+`from version 0.3`
 
-To add custom request tags data you can add an optional method ```tags``` which can return a dict of custom meta tags as shown in the example below:
+To add custom request tags data you can add an optional method `tags` which can return a dict of custom meta tags as shown in the example below:
 
 ```python
 class MyModel(object):