Skip to content

deroneriksson/stf4j

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

STF4J (Simplified TensorFlow for Java)

Table of Contents

Overview

STF4J is a lightweight Java API that simplifies the running of pretrained TensorFlow SavedModels from Java.

Design goals:

  1. REPL friendly (see useful output when objects are retrieved).
  2. Easy access to SavedModel signature information.
  3. Input type coercion from Java scalars, arrays, and multidimensional arrays to Tensors.
  4. Output type coercion from Tensors to Java scalars, arrays, and multidimensional arrays.
  5. Use friendly keys (e.g., probabilities, classes, image, input) rather than variable names (e.g., Placeholder:0, ArgMax:0, Softmax:0, input_tensor:0, softmax_tensor:0) for model inputs and outputs.
  6. Retrieve values by output key.
  7. Minimize external dependencies (only TensorFlow/Protocol Buffer libraries and log4j).

Hello World

The stf4j-test-models project contains several prebuilt SavedModels that are used for testing. Here, we use the add_string model to concatenate two input strings, hello and world.

package org.codait.example;

import org.codait.stf4j.TFModel;

public class Example {

	public static void main(String[] args) {
		TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_string").sig("serving_default");
		String result = model.in("input1", "hello").in("input2", "world").out("output").run().getString("output");
		System.out.println(result);
	}

}

Output:

helloworld

Build

STF4J is a standard maven Java project. The project can be cloned from GitHub and built using maven. Here, we skip testing. After building, the STF4J jar file is located in the target directory.

git clone https://github.com/CODAIT/stf4j.git
cd stf4j
mvn clean package -DskipTests

For convenience when trying out STF4J, the shade profile builds 1) an uber STF4J jar that contains the TensorFlow dependencies and log4j and 2) a tf STF4J jar that contains the TensorFlow dependencies with no log4j.

mvn clean package -DskipTests -Pshade

Currently STF4J only has the following library dependencies: tensorflow, libtensorflow, libtensorflow_jni, proto, protobuf-java, and log4j.

Testing

The stf4j-test-models project contains several prebuilt SavedModels for testing. The stf4j-test-models project contains information about how to rebuild the models if necessary.

For full testing, MNIST and CIFAR-10 data should be added to the stf4j-test-models project. Instructions to do this are located in the stf4j-test-models project. If the data is not present, the relevant tests will be skipped.

git clone https://github.com/CODAIT/stf4j-test-models.git
git clone https://github.com/CODAIT/stf4j.git
cd stf4j
mvn clean test

Examples

Java

Introduction

For our Java examples, we will start by installing the STF4J jar file into the local maven repository so that we can reference STF4J from Java projects using maven. If STF4J has been deployed to the maven central repository, this step is unnecessary.

git clone https://github.com/CODAIT/stf4j.git
cd stf4j
mvn clean install -DskipTests

We can now add the stf4j dependency to the pom.xml file of our example project.

<dependency>
	<groupId>org.codait.stf4j</groupId>
	<artifactId>stf4j</artifactId>
	<version>1.10.0-SNAPSHOT</version>
</dependency>

Our first Java example references the add_int32 model in the simple_saved_models project that adds two int Tensors. We create our TFModel object by referencing the path to the model directory. We specify that we will use the "serving_default" signature definition. The available signature definitions can conveniently be displayed by the toString method of the TFModel after it has been created. We print this information to the console after we create the model object.

We set input values to the model using the in() method. The first parameter of the in() method specifies the input key which we can see in the "serving_default" signature definition in the console. The second parameter to in() specifies the value. In this example, we input two scalar int values, "input1" and "input2", which will be added together. Outputs are specified by the out() method, where the output keys are specified.

The model is executed by the run() method, which returns the results in the form of a TFResults object. The toString() method of TFResults displays the output keys, the corresponding output variable names, and information about the Tensor data returned. This value is output to the console in the below example.

The result of the calculation is a scalar int with the key output. We can obtain this value by calling getInt("output") on the TFResults object. This sum is displayed to the console.

package org.codait.stf4j.example;

import org.codait.stf4j.TFModel;
import org.codait.stf4j.TFResults;

public class STF4JExample {

	public static void main(String[] args) {
		TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_int32").sig("serving_default");
		System.out.println("MODEL: " + model);
		TFResults results = model.in("input1", 1).in("input2", 2).out("output").run();
		System.out.println("RESULTS: " + results);
		int sum = results.getInt("output");
		System.out.println("SUM:" + sum);
	}

}

The above example generates the following output:

2018-08-20 13:35:36.171780: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: ../stf4j-test-models/simple_saved_models/add_int32
2018-08-20 13:35:36.176215: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2018-08-20 13:35:36.176377: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
2018-08-20 13:35:36.177207: I tensorflow/cc/saved_model/loader.cc:113] Restoring SavedModel bundle.
2018-08-20 13:35:36.177254: I tensorflow/cc/saved_model/loader.cc:123] The specified SavedModel has no variables; no checkpoints were restored.
2018-08-20 13:35:36.177262: I tensorflow/cc/saved_model/loader.cc:148] Running LegacyInitOp on SavedModel bundle.
2018-08-20 13:35:36.177275: I tensorflow/cc/saved_model/loader.cc:233] SavedModel load for tags { serve }; Status: success. Took 5513 microseconds.
MODEL: Model directory: ../stf4j-test-models/simple_saved_models/add_int32

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input1
    dtype: DT_INT32
    shape: ()
    name: input1:0
  input key: input2
    dtype: DT_INT32
    shape: ()
    name: input2:0
outputs:
  output key: output
    dtype: DT_INT32
    shape: ()
    name: output:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

RESULTS: SignatureDef Key: serving_default
Outputs:
  [1] output (output:0): INT32 tensor with shape []

SUM:3

Next, let's look at feeding in two 1-dimensional int arrays and obtaining the resulting 1-dimensional int array.

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_int32").sig("serving_default");
int[] result = model.in("input1", new int[] { 1, 2 }).in("input2", new int[] { 3, 4 }).out("output").run()
		.getIntArray("output");
System.out.println("RESULT: " + Arrays.toString(result));

Output:

RESULT: [4, 6]

STF4J also handles multidimensional arrays. In this example, we feed in 2 3-dimensional int arrays and obtain the resulting 3-dimensional int array.

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_int32").sig("serving_default");
int[][][] input1 = new int[][][] { { { 1, 2 }, { 3, 4 } }, { { 5, 6 }, { 7, 8 } } };
int[][][] input2 = new int[][][] { { { 1, 2 }, { 3, 4 } }, { { 5, 6 }, { 7, 8 } } };
int[][][] result = (int[][][]) model.in("input1", input1).in("input2", input2).out("output").run()
		.getIntArrayMultidimensional("output");
System.out.println("RESULT: " + Arrays.deepToString(result));

Output:

RESULT: [[[2, 4], [6, 8]], [[10, 12], [14, 16]]]

STF4J will implicitly perform type coercion where possible for inputs and outputs. In the previous example, we can obtain the result as a 3-dimensional float array by calling the getFloatArrayMultidimensional("output") method rather than the getIntArrayMultidimensional("output") method.

float[][][] result = (float[][][]) model.in("input1", input1).in("input2", input2).out("output").run()
		.getFloatArrayMultidimensional("output");
System.out.println("RESULT: " + Arrays.deepToString(result));

Output:

RESULT: [[[2.0, 4.0], [6.0, 8.0]], [[10.0, 12.0], [14.0, 16.0]]]

As another example of implicit type coercion using the previous example, although the model adds two int Tensors, we can pass in two 3-dimensional float arrays and STF4J will convert these to 3-dimensional int Tensors. Although the result is a 3-dimensional int Tensor, here we convert it to a 3-dimensional float array using the getFloatArrayMultidimensional("output") method.

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_int32").sig("serving_default");
float[][][] input1 = new float[][][] { { { 1.0f, 2.0f }, { 3.0f, 4.0f } }, { { 5.0f, 6.0f }, { 7.0f, 8.0f } } };
float[][][] input2 = new float[][][] { { { 1.0f, 2.0f }, { 3.0f, 4.0f } }, { { 5.0f, 6.0f }, { 7.0f, 8.0f } } };
float[][][] result = (float[][][]) model.in("input1", input1).in("input2", input2).out("output").run()
		.getFloatArrayMultidimensional("output");
System.out.println("RESULT: " + Arrays.deepToString(result));

Output:

RESULT: [[[2.0, 4.0], [6.0, 8.0]], [[10.0, 12.0], [14.0, 16.0]]]

Here, we see an example of inputting two 5-dimensional String arrays and outputting a 5-dimensional String array.

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_string").sig("serving_default");
String[][][][][] s1 = new String[][][][][] { { { { { "Lorem ", "dolor " }, { "amet, ", "adipiscing " },
		{ "sed ", "eiusmod " }, { "incididunt ", "labore " }, { "dolore ", "aliqua" } } } } };
String[][][][][] s2 = new String[][][][][] { { { { { "ipsum ", "sit " }, { "consectetur ", "elit, " },
		{ "do ", "tempor " }, { "ut ", "et " }, { "magna ", "." } } } } };
String[][][][][] result = (String[][][][][]) model.in("input1", s1).in("input2", s2).out("output").run()
		.getStringArrayMultidimensional("output");
System.out.println("RESULT: " + Arrays.deepToString(result));

Output:

RESULT: [[[[[Lorem ipsum , dolor sit ], [amet, consectetur , adipiscing elit, ], [sed do , eiusmod tempor ], [incididunt ut , labore et ], [dolore magna , aliqua.]]]]]

Next, we'll look at a basic model that returns multiple outputs. The boolean_logic model takes two boolean Tensor inputs and outputs five possible Tensor outputs. These outputs represent the following boolean operations on the two input Tensors: AND, OR, XOR, NOT AND, and NOT OR. We will retrieve three of these outputs using the following signature definition output keys: "and", "or", and "xor".

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/boolean_logic").sig("serving_default");
System.out.println("MODEL: " + model);
boolean[][] input1 = new boolean[][] { { true, true }, { false, false } };
boolean[][] input2 = new boolean[][] { { true, false }, { true, false } };
TFResults results = model.in("input1", input1).in("input2", input2).out("and", "or", "xor").run();
System.out.println("INPUT1: " + Arrays.deepToString(input1));
System.out.println("INPUT2: " + Arrays.deepToString(input2));
System.out.println("RESULTS: " + results);
System.out.println("AND: " + Arrays.deepToString((boolean[][]) results.getBooleanArrayMultidimensional("and")));
System.out.println("OR: " + Arrays.deepToString((boolean[][]) results.getBooleanArrayMultidimensional("or")));
System.out.println("XOR: " + Arrays.deepToString((boolean[][]) results.getBooleanArrayMultidimensional("xor")));

In the output, we see that the model toString() displays information about the "serving_default" signature definition, which lists the two inputs and the five outputs. Since we specify three outputs in the out("and", "or", "xor") method call, we see that boolean Tensors representing these outputs are available in the results. We obtain the results as two-dimensional boolean arrays using the getBooleanArrayMultidimensional() method and display their values to the console.

MODEL: Model directory: ../stf4j-test-models/simple_saved_models/boolean_logic

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input2
    dtype: DT_BOOL
    shape: ()
    name: input2:0
  input key: input1
    dtype: DT_BOOL
    shape: ()
    name: input1:0
outputs:
  output key: not_and
    dtype: DT_BOOL
    shape: ()
    name: output_not_and:0
  output key: and
    dtype: DT_BOOL
    shape: ()
    name: output_and:0
  output key: xor
    dtype: DT_BOOL
    shape: ()
    name: output_xor:0
  output key: not_or
    dtype: DT_BOOL
    shape: ()
    name: output_not_or:0
  output key: or
    dtype: DT_BOOL
    shape: ()
    name: output_or:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

INPUT1: [[true, true], [false, false]]
INPUT2: [[true, false], [true, false]]
RESULTS: SignatureDef Key: serving_default
Outputs:
  [1] and (output_and:0): BOOL tensor with shape [2, 2]
  [2] or (output_or:0): BOOL tensor with shape [2, 2]
  [3] xor (output_xor:0): BOOL tensor with shape [2, 2]

AND: [[true, false], [false, false]]
OR: [[true, true], [true, false]]
XOR: [[false, true], [true, false]]

MNIST

In this Java example, we'll use the MNISTUtil class to load the standard 10,000 MNIST test labels and images. Each image is treated as a 28x28 int value, and the entire 10,000 image dataset is loaded as a three-dimensional int array, where the first dimension is the image number, the second dimension is the row, and the third dimension is the column.

In the signature definition for "serving_default" (and "classify"), we see that the input image data should be float data (which is a little strange since it is really integer data). However, STF4J will automatically coerce int data to float data so we can feed in int data painlessly into a float Tensor.

Here, we'll perform a prediction on the first of the 10,000 images and display the label and the prediction to the console.

TFModel mnist = new TFModel("../stf4j-test-models/mnist_saved_model/").sig("serving_default");
System.out.println("MNIST MODEL: " + mnist);
int[] labels = MNISTUtil.getLabels("../stf4j-test-models/mnist_data/t10k-labels-idx1-ubyte");
int[][][] images = MNISTUtil.getImages("../stf4j-test-models/mnist_data/t10k-images-idx3-ubyte");
int label = labels[0];
int prediction = mnist.in("image", images[0]).out("classes").run().getInt("classes");
System.out.println("Label: " + label + ", Prediction: " + prediction);

Output:

MNIST MODEL: Model directory: ../stf4j-test-models/mnist_saved_model/

SignatureDef key: classify
method name: tensorflow/serving/predict
inputs:
  input key: image
    dtype: DT_FLOAT
    shape: (-1, 28, 28)
    name: Placeholder:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (-1, 10)
    name: Softmax:0
  output key: classes
    dtype: DT_INT64
    shape: (-1)
    name: ArgMax:0
SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: image
    dtype: DT_FLOAT
    shape: (-1, 28, 28)
    name: Placeholder:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (-1, 10)
    name: Softmax:0
  output key: classes
    dtype: DT_INT64
    shape: (-1)
    name: ArgMax:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

Label: 7, Prediction: 7

Since any number of images can be fed into the model, let's feed in all 10,000 test images and get back the 10,000 predictions as an int array.

int[] predictions = mnist.in("image", images).out("classes").run().getIntArray("classes");
System.out.println("PREDICTIONS: " + Arrays.toString(predictions));

Output:

PREDICTIONS: [7, 2, 1, 0, 4, 1, 4, 9, 5, 9, 0, 6, 9, 0, 1, 5, 9, 7, 3, 4, 9, 6, 6, 5, 4, 0, 7, 4, 0, 1, 3, 1, 3, 4, 7, 2, 7, ... ]

CIFAR-10

Next, let's have a look at the TensorFlow CIFAR-10 SavedModel using STF4J. A copy of this model is saved in the stf4j-test-models project in the cifar10_saved_model directory. This model performs image predictions on color images that are 32 rows by 32 columns by 3 channels. These images need to be preprocessed before feeding the images into the model. The labels and images can be obtained using the CIFAR10Util class, as we see below.

In this example, we obtain the 10,000 test labels as an int array and the 10,000 test images as a 4-dimensional float array, where the first dimension is the image number, the second dimension in the rows, the third dimension is the columns, and the fourth dimension is the channels, and we obtain these images after preprocessing using the CIFAR10Util.getPreprocessedImages() method.

We perform a prediction on the first CIFAR-10 image and display the label and prediction for the first image to the console.

TFModel cifar10 = new TFModel("../stf4j-test-models/cifar10_saved_model/").sig("serving_default");
System.out.println("CIFAR-10 MODEL: " + cifar10);
String testDataFile = "../stf4j-test-models/cifar10_data/cifar-10-batches-bin/test_batch.bin";
int[] labels = CIFAR10Util.getLabels(testDataFile);
float[][][][] images = CIFAR10Util.getPreprocessedImages(testDataFile, CIFAR10Util.DimOrder.ROWS_COLS_CHANNELS);
int label = labels[0];
int prediction = cifar10.in("input", images[0]).out("classes").run().getInt("classes");
System.out.println("Label: " + label + ", Prediction: " + prediction);

Output:

CIFAR-10 MODEL: Model directory: ../stf4j-test-models/cifar10_saved_model/

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input
    dtype: DT_FLOAT
    shape: (128, 32, 32, 3)
    name: input_tensor:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (128, 10)
    name: softmax_tensor:0
  output key: classes
    dtype: DT_INT64
    shape: (128)
    name: ArgMax:0
SignatureDef key: predict
method name: tensorflow/serving/predict
inputs:
  input key: input
    dtype: DT_FLOAT
    shape: (128, 32, 32, 3)
    name: input_tensor:0
outputs:
  output key: classes
    dtype: DT_INT64
    shape: (128)
    name: ArgMax:0
  output key: probabilities
    dtype: DT_FLOAT
    shape: (128, 10)
    name: softmax_tensor:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

Label: 3, Prediction: 3

Next, we'll feed in all 10,000 test images and obtain the predictions as an int array.

int[] predictions = cifar10.in("input", images).out("classes").run().getIntArray("classes");
System.out.println("PREDICTIONS: " + Arrays.toString(predictions));

The truncated console output is shown here:

PREDICTIONS: [3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 6, 5, 7, 8, 6, 7, 0, 4, 9, 5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, ...]

Graph Inputs and Outputs

Normal interactions with STF4J should occur through the TFModel and TFResults classes. If direct interactions with graph operations are required, this can be accomplished using the TFGraph and TFGraphResults classes.

In the following example, we first add two int arrays using the SavedModel referenced by TFModel, and we obtain our results through TFResults which gives us the resulting int array.

After this, we display the model's graph's operations to the console. We obtain a TFGraph object from TFModel. We specify two inputs, input1 and input2, and we specify one output, add. Note that the TFGraph input() and output() methods only take input and output data as Tensors.

We obtain our graph results through TFGraphResults. For convenience, we use the ArrayUtil.intTensorToIntArray() method to convert our Integer Tensor to an int array.

int[] i1 = new int[] { 1, 3, 5 };
int[] i2 = new int[] { 2, 4, 6 };

TFModel model = new TFModel("../stf4j-test-models/simple_saved_models/add_int32").sig("serving_default");
TFResults result = model.in("input1", i1).in("input2", i2).out("output").run();
int[] output = result.getIntArray("output");
System.out.println("Model Output: " + Arrays.toString(output));

Iterator<Operation> operations = model.graph().operations();
System.out.println("\nGraph Operations:");
while (operations.hasNext()) {
	Operation operation = operations.next();
	System.out.println("  Operation: " + operation);
}

Tensor<Integer> t1 = Tensor.create(i1, Integer.class);
Tensor<Integer> t2 = Tensor.create(i2, Integer.class);

TFGraph graph = model.tfGraph();
TFGraphResults res = graph.input("input1", t1).input("input2", t2).output("add").run();
Tensor<Integer> t = res.getTensor("add", Integer.class);
int[] intArray = ArrayUtil.intTensorToIntArray(t);
System.out.println("\nGraph Output: " + Arrays.toString(intArray));

The console output is shown here:

Model Output: [3, 7, 11]

Graph Operations:
  Operation: <Placeholder 'input1'>
  Operation: <Placeholder 'input2'>
  Operation: <Add 'add'>
  Operation: <Identity 'output'>

Graph Output: [3, 7, 11]

Note that a TFGraph object can be created based on an existing graph in the file system.

TFGraph graph = new TFGraph("./example/example_graph.pb");

Scala

Introduction

To use STF4J with the Scala REPL, we can build the STF4J uberjar and start the Scala REPL with the uberjar on the classpath.

mvn clean package -DskipTests -Pshade
scala -cp target/stf4j-uber-1.10.0-SNAPSHOT.jar

First, we'll import the org.codait.stf4j package, which contains the TFModel and TFResults classes. Next, we create a TFModel object based on the add_float32 model, which adds two Float Tensors. We set the signature definition to be "serving_default".

import org.codait.stf4j._
val model = new TFModel("../stf4j-test-models/simple_saved_models/add_float32")
model.sig("serving_default")
val result = model.in("input1", 1.0f).in("input2", 2.0f).out("output").run()
val sum = result.getFloat("output")

Below we see the REPL console output. Notice that when we create our model object, the contained signature definitions are displayed. We see a single signature definition which has the "serving_default" key. The input and output keys are displayed, along with their types, shapes, and underlying variable names.

Inputs are set by calls to the model's in method, and outputs are set by calls to the out method. The model is run by the run() method, which returns a TFResults object, which maps keys to output Tensors. Notice that the returned TFResults object's toString() displays information about the contained outputs. We see in the console output that the result contains an output key which maps to the output:0 variable and that the result is a FLOAT tensor containing a scalar float value. We obtain the float scalar result by calling the TFResults getFloat("output") method with the desired output key specified.

scala> import org.codait.stf4j._
import org.codait.stf4j._

scala> val model = new TFModel("../stf4j-test-models/simple_saved_models/add_float32")
2018-08-19 12:59:09.927801: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: ../stf4j-test-models/simple_saved_models/add_float32
2018-08-19 12:59:09.927952: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2018-08-19 12:59:09.928248: I tensorflow/cc/saved_model/loader.cc:113] Restoring SavedModel bundle.
2018-08-19 12:59:09.928271: I tensorflow/cc/saved_model/loader.cc:123] The specified SavedModel has no variables; no checkpoints were restored.
2018-08-19 12:59:09.928278: I tensorflow/cc/saved_model/loader.cc:148] Running LegacyInitOp on SavedModel bundle.
2018-08-19 12:59:09.928285: I tensorflow/cc/saved_model/loader.cc:233] SavedModel load for tags { serve }; Status: success. Took 487 microseconds.
model: org.codait.stf4j.TFModel =
Model directory: ../stf4j-test-models/simple_saved_models/add_float32

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input2
    dtype: DT_FLOAT
    shape: ()
    name: input2:0
  input key: input1
    dtype: DT_FLOAT
    shape: ()
    name: input1:0
outputs:
  output key: output
    dtype: DT_FLOAT
    shape: ()
    name: output:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

scala> model.sig("serving_default")
res50: org.codait.stf4j.TFModel =
Model directory: ../stf4j-test-models/simple_saved_models/add_float32

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input2
    dtype: DT_FLOAT
    shape: ()
    name: input2:0
  input key: input1
    dtype: DT_FLOAT
    shape: ()
    name: input1:0
outputs:
  output key: output
    dtype: DT_FLOAT
    shape: ()
    name: output:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInfo() method.

scala> val result = model.in("input1", 1.0f).in("input2", 2.0f).out("output").run()
result: org.codait.stf4j.TFResults =
SignatureDef Key: serving_default
Outputs:
  [1] output (output:0): FLOAT tensor with shape []

scala> val sum = result.getFloat("output")
sum: Float = 3.0

Using the existing model, let's input two Float arrays and obtain the resulting Float array.

val input1 = Array(1.0f, 2.0f)
val input2 = Array(0.1f, 0.2f)
val sum = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArray("output")

In the console output, we see the elements in the two Float arrays have been added, and the resulting Float array is returned by the call to the TFResults getFloatArray("output") method.

scala> val input1 = Array(1.0f, 2.0f)
input1: Array[Float] = Array(1.0, 2.0)

scala> val input2 = Array(0.1f, 0.2f)
input2: Array[Float] = Array(0.1, 0.2)

scala> val sum = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArray("output")
sum: Array[Float] = Array(1.1, 2.2)

Next, let's input two multidimensional Float arrays and obtain the resulting multidimensional Float array. Here, we input two 3d arrays and obtain the resulting 3d array. We print the values of the individual array elements.

val input1 = Array(Array(Array(1.0f, 2.0f), Array(3.0f, 4.0f)), Array(Array(5.0f, 6.0f), Array(7.0f, 8.0f)))
val input2 = Array(Array(Array(0.1f, 0.2f), Array(0.3f, 0.4f)), Array(Array(0.5f, 0.6f), Array(0.7f, 0.8f)))
val result = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArrayMultidimensional("output")
val sum = result.asInstanceOf[Array[Array[Array[Float]]]]
print(sum(0)(0)(0))
print(sum(0)(0)(1))
print(sum(0)(1)(0))
print(sum(0)(1)(1))
print(sum(1)(0)(0))
print(sum(1)(0)(1))
print(sum(1)(1)(0))
print(sum(1)(1)(1))

Output:

scala> val input1 = Array(Array(Array(1.0f, 2.0f), Array(3.0f, 4.0f)), Array(Array(5.0f, 6.0f), Array(7.0f, 8.0f)))
input1: Array[Array[Array[Float]]] = Array(Array(Array(1.0, 2.0), Array(3.0, 4.0)), Array(Array(5.0, 6.0), Array(7.0, 8.0)))

scala> val input2 = Array(Array(Array(0.1f, 0.2f), Array(0.3f, 0.4f)), Array(Array(0.5f, 0.6f), Array(0.7f, 0.8f)))
input2: Array[Array[Array[Float]]] = Array(Array(Array(0.1, 0.2), Array(0.3, 0.4)), Array(Array(0.5, 0.6), Array(0.7, 0.8)))

scala> val result = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArrayMultidimensional("output")
result: Object = Array(Array(Array(1.1, 2.2), Array(3.3, 4.4)), Array(Array(5.5, 6.6), Array(7.7, 8.8)))

scala> val sum = result.asInstanceOf[Array[Array[Array[Float]]]]
sum: Array[Array[Array[Float]]] = Array(Array(Array(1.1, 2.2), Array(3.3, 4.4)), Array(Array(5.5, 6.6), Array(7.7, 8.8)))

scala> print(sum(0)(0)(0))
1.1
scala> print(sum(0)(0)(1))
2.2
scala> print(sum(0)(1)(0))
3.3
scala> print(sum(0)(1)(1))
4.4
scala> print(sum(1)(0)(0))
5.5
scala> print(sum(1)(0)(1))
6.6
scala> print(sum(1)(1)(0))
7.7
scala> print(sum(1)(1)(1))
8.8

STF4J implicitly performs type coercion where possible. In the example below, two multidimensional Int arrays are input into the add_float32 model. Since the inputs need to be FLOAT Tensors, STF4J converts the multidimensional Int arrays to FLOAT Tensors.

val input1 = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))
val input2 = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))
val result = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArrayMultidimensional("output")
val sum = result.asInstanceOf[Array[Array[Array[Float]]]]
print(sum(0)(0)(0))
print(sum(0)(0)(1))
print(sum(0)(1)(0))
print(sum(0)(1)(1))
print(sum(1)(0)(0))
print(sum(1)(0)(1))
print(sum(1)(1)(0))
print(sum(1)(1)(1))

We see that the 3d Int arrays have been converted to FLOAT Tensors and added together by the model, and the resulting 3d Float array is obtained by the call to getFloatArrayMultidimensional("output").

scala> val input1 = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))
input1: Array[Array[Array[Int]]] = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))

scala> val input2 = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))
input2: Array[Array[Array[Int]]] = Array(Array(Array(1, 2), Array(3, 4)), Array(Array(5, 6), Array(7, 8)))

scala> val result = model.in("input1", input1).in("input2", input2).out("output").run().getFloatArrayMultidimensional("output")
result: Object = Array(Array(Array(2.0, 4.0), Array(6.0, 8.0)), Array(Array(10.0, 12.0), Array(14.0, 16.0)))

scala> val sum = result.asInstanceOf[Array[Array[Array[Float]]]]
sum: Array[Array[Array[Float]]] = Array(Array(Array(2.0, 4.0), Array(6.0, 8.0)), Array(Array(10.0, 12.0), Array(14.0, 16.0)))

scala> print(sum(0)(0)(0))
2.0
scala> print(sum(0)(0)(1))
4.0
scala> print(sum(0)(1)(0))
6.0
scala> print(sum(0)(1)(1))
8.0
scala> print(sum(1)(0)(0))
10.0
scala> print(sum(1)(0)(1))
12.0
scala> print(sum(1)(1)(0))
14.0
scala> print(sum(1)(1)(1))
16.0

In the previous example, the resulting 3d Float array could also be retrieved as a 3d Int array if desired, since STF4J can perform the type coercion automatically.

val result = model.in("input1", input1).in("input2", input2).out("output").run().getIntArrayMultidimensional("output")
val sum = result.asInstanceOf[Array[Array[Array[Int]]]]

Output:

scala> val result = model.in("input1", input1).in("input2", input2).out("output").run().getIntArrayMultidimensional("output")
result: Object = Array(Array(Array(2, 4), Array(6, 8)), Array(Array(10, 12), Array(14, 16)))

scala> val sum = result.asInstanceOf[Array[Array[Array[Int]]]]
sum: Array[Array[Array[Int]]] = Array(Array(Array(2, 4), Array(6, 8)), Array(Array(10, 12), Array(14, 16)))

Next, let's have a look at a model that returns multiple outputs. The boolean_logic model in the stf4j-test-models project has 2 boolean inputs and 5 boolean outputs. The model takes the 2 boolean inputs and performs the following boolean logic operations on the inputs and provides them as outputs: "and", "or", "xor", "not_and", "not_or".

Here, we input true for input1 and false for input2. We retrieve the values of the inputs ANDed and ORed together.

val model = new TFModel("../stf4j-test-models/simple_saved_models/boolean_logic").sig("serving_default")
val result = model.in("input1", true).in("input2", false).out("and", "or").run()
val true_and_false = result.getBoolean("and")
val true_or_false = result.getBoolean("or")

Output:

scala> val model = new TFModel("../stf4j-test-models/simple_saved_models/boolean_logic").sig("serving_default")
2018-08-19 14:25:23.109043: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: ../stf4j-test-models/simple_saved_models/boolean_logic
2018-08-19 14:25:23.109690: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2018-08-19 14:25:23.110120: I tensorflow/cc/saved_model/loader.cc:113] Restoring SavedModel bundle.
2018-08-19 14:25:23.110689: I tensorflow/cc/saved_model/loader.cc:123] The specified SavedModel has no variables; no checkpoints were restored.
2018-08-19 14:25:23.110704: I tensorflow/cc/saved_model/loader.cc:148] Running LegacyInitOp on SavedModel bundle.
2018-08-19 14:25:23.110723: I tensorflow/cc/saved_model/loader.cc:233] SavedModel load for tags { serve }; Status: success. Took 1683 microseconds.
model: org.codait.stf4j.TFModel =
Model directory: ../stf4j-test-models/simple_saved_models/boolean_logic

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input1
    dtype: DT_BOOL
    shape: ()
    name: input1:0
  input key: input2
    dtype: DT_BOOL
    shape: ()
    name: input2:0
outputs:
  output key: not_and
    dtype: DT_BOOL
    shape: ()
    name: output_not_and:0
  output key: and
    dtype: DT_BOOL
    shape: ()
    name: output_and:0
  output key: xor
    dtype: DT_BOOL
    shape: ()
    name: output_xor:0
  output key: not_or
    dtype: DT_BOOL
    shape: ()
    name: output_not_or:0
  output key: or
    dtype: DT_BOOL
    shape: ()
    name: output_or:0
Note: SignatureDef info can be obtained by calling TFModel's signatureDefInf...
scala> val result = model.in("input1", true).in("input2", false).out("and", "or").run()
result: org.codait.stf4j.TFResults =
SignatureDef Key: serving_default
Outputs:
  [1] and (output_and:0): BOOL tensor with shape []
  [2] or (output_or:0): BOOL tensor with shape []

scala> val true_and_false = result.getBoolean("and")
true_and_false: Boolean = false

scala> val true_or_false = result.getBoolean("or")
true_or_false: Boolean = true

MNIST

Next, we'll take a look at doing predictions using the TensorFlow MNIST model. We'll start the Scala REPL with more memory allocated since we load 10,000 MNIST test images into memory in this example.

scala -cp target/stf4j-uber-1.10.0-SNAPSHOT.jar -J-Xmx4g

In the Scala REPL, in addition to the primary STF4J package, we import the util package since it contains the MNISTUtil class that can be used to load the 10,000 MNIST test labels and images.

In this example, we load the MNIST model, load the test labels and test images, and then run the model on the first image. Even though the model requires Floats, STF4J allows the images to be fed in as Ints, since automatic type coercion is performed.

The prediction value, classes, is defined as an INT64 (Long value). STF4J allows the value to be retrieved and type coerced as an Int using the getInt() method. We display the label and the prediction to the console.

import org.codait.stf4j._
import org.codait.stf4j.util._
val mnist = new TFModel("../stf4j-test-models/mnist_saved_model/").sig("serving_default")
val labels = MNISTUtil.getLabels("../stf4j-test-models/mnist_data/t10k-labels-idx1-ubyte")
val images = MNISTUtil.getImages("../stf4j-test-models/mnist_data/t10k-images-idx3-ubyte")
val label = labels(0);
val prediction = mnist.in("image", images(0)).out("classes").run().getInt("classes")
print("Label: " + label + ", Prediction: " + prediction)

Output:

scala> import org.codait.stf4j._
import org.codait.stf4j._

scala> import org.codait.stf4j.util._
import org.codait.stf4j.util._

scala> val mnist = new TFModel("../stf4j-test-models/mnist_saved_model/").sig("serving_default")
2018-08-19 15:18:04.456606: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: ../stf4j-test-models/mnist_saved_model/
2018-08-19 15:18:04.457571: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2018-08-19 15:18:04.458685: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
2018-08-19 15:18:04.461273: I tensorflow/cc/saved_model/loader.cc:113] Restoring SavedModel bundle.
2018-08-19 15:18:04.493649: I tensorflow/cc/saved_model/loader.cc:148] Running LegacyInitOp on SavedModel bundle.
2018-08-19 15:18:04.497168: I tensorflow/cc/saved_model/loader.cc:233] SavedModel load for tags { serve }; Status: success. Took 40572 microseconds.
mnist: org.codait.stf4j.TFModel =
Model directory: ../stf4j-test-models/mnist_saved_model/

SignatureDef key: classify
method name: tensorflow/serving/predict
inputs:
  input key: image
    dtype: DT_FLOAT
    shape: (-1, 28, 28)
    name: Placeholder:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (-1, 10)
    name: Softmax:0
  output key: classes
    dtype: DT_INT64
    shape: (-1)
    name: ArgMax:0
SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: image
    dtype: DT_FLOAT
    shape: (-1, 28, 28)
    name: Placeholder:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (-1, 10)
    name: Softmax:0
  output key: classes
    dtype: DT_INT64
    shape: (-1)
    name: ArgMax:0
Note: SignatureDef info can b...
scala> val labels = MNISTUtil.getLabels("../stf4j-test-models/mnist_data/t10k-labels-idx1-ubyte")
labels: Array[Int] = Array(7, 2, 1, 0, 4, 1, 4, 9, 5, 9, 0, 6, 9, 0, 1, 5, 9, 7, 3, 4, 9, 6, 6, 5, 4, 0, 7, 4, 0, 1, 3, 1, 3, 4, 7, 2, 7, 1, 2, 1, 1, 7, 4, 2, 3, 5, 1, 2, 4, 4, 6, 3, 5, 5, 6, 0, 4, 1, 9, 5, 7, 8, 9, 3, 7, 4, 6, 4, 3, 0, 7, 0, 2, 9, 1, 7, 3, 2, 9, 7, 7, 6, 2, 7, 8, 4, 7, 3, 6, 1, 3, 6, 9, 3, 1, 4, 1, 7, 6, 9, 6, 0, 5, 4, 9, 9, 2, 1, 9, 4, 8, 7, 3, 9, 7, 4, 4, 4, 9, 2, 5, 4, 7, 6, 7, 9, 0, 5, 8, 5, 6, 6, 5, 7, 8, 1, 0, 1, 6, 4, 6, 7, 3, 1, 7, 1, 8, 2, 0, 2, 9, 9, 5, 5, 1, 5, 6, 0, 3, 4, 4, 6, 5, 4, 6, 5, 4, 5, 1, 4, 4, 7, 2, 3, 2, 7, 1, 8, 1, 8, 1, 8, 5, 0, 8, 9, 2, 5, 0, 1, 1, 1, 0, 9, 0, 3, 1, 6, 4, 2, 3, 6, 1, 1, 1, 3, 9, 5, 2, 9, 4, 5, 9, 3, 9, 0, 3, 6, 5, 5, 7, 2, 2, 7, 1, 2, 8, 4, 1, 7, 3, 3, 8, 8, 7, 9, 2, 2, 4, 1, 5, 9, 8, 7, 2, 3, 0, 4, 4, 2, 4, 1, 9, 5, 7, 7, 2,...
scala> val images = MNISTUtil.getImages("../stf4j-test-models/mnist_data/t10k-images-idx3-ubyte")
images: Array[Array[Array[Int]]] = Array(Array(Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0, 0, 0, 0, 84, 185, 159, 151, 60, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), Array(0, 0, 0...
scala> val label = labels(0);
label: Int = 7

scala> val prediction = mnist.in("image", images(0)).out("classes").run().getInt("classes")
prediction: Int = 7

scala> print("Label: " + label + ", Prediction: " + prediction)
Label: 7, Prediction: 7

Since the MNIST model takes any number of 28x28 images (indicated by the -1 first dimension value of the image input shape), we can pass the entire array of MNIST test images to the model and obtain the entire set of predictions.

val predictions = mnist.in("image", images).out("classes").run().getIntArray("classes")

Output:

scala> val predictions = mnist.in("image", images).out("classes").run().getIntArray("classes")
predictions: Array[Int] = Array(7, 2, 1, 0, 4, 1, 4, 9, 5, 9, 0, 6, 9, 0, 1, 5, 9, 7, 3, 4, 9, 6, 6, 5, 4, 0, 7, 4, 0, 1, 3, 1, 3, 4, 7, 2, 7, 1, 2, 1, 1, 7, 4, 2, 3, 5, 1, 2, 4, 4, 6, 3, 5, 5, 6, 0, 4, 1, 9, 5, 7, 8, 9, 3, 7, 4, 6, 4, 3, 0, 7, 0, 2, 9, 1, 7, 3, 2, 9, 7, 7, 6, 2, 7, 8, 4, 7, 3, 6, 1, 3, 6, 9, 3, 1, 4, 1, 7, 6, 9, 6, 0, 5, 4, 9, 9, 2, 1, 9, 4, 8, 7, 3, 9, 7, 9, 4, 4, 9, 2, 5, 4, 7, 6, 7, 9, 0, 5, 8, 5, 6, 6, 5, 7, 8, 1, 0, 1, 6, 4, 6, 7, 3, 1, 7, 1, 8, 2, 0, 2, 9, 9, 5, 5, 1, 5, 6, 0, 3, 4, 4, 6, 5, 4, 6, 5, 4, 5, 1, 4, 4, 7, 2, 3, 2, 7, 1, 8, 1, 8, 1, 8, 5, 0, 8, 9, 2, 5, 0, 1, 1, 1, 0, 9, 0, 3, 1, 6, 4, 2, 3, 6, 1, 1, 1, 3, 9, 5, 2, 9, 4, 5, 9, 3, 9, 0, 3, 6, 5, 5, 7, 2, 2, 7, 1, 2, 8, 4, 1, 7, 3, 3, 8, 8, 7, 9, 2, 2, 4, 1, 5, 9, 8, 7, 2, 3, 0, 4, 4, 2, 4, 1, 9, 5, 7, ...

CIFAR-10

Next, we'll perform predictions using the TensorFlow CIFAR-10 model. The CIFAR-10 test data consists of 10,000 32x32 3-channel images. We'll load the entire test dataset into memory, so let's start the Scala REPL with additional memory allocated.

scala -cp target/stf4j-uber-1.10.0-SNAPSHOT.jar -J-Xmx4g

We start by creating a TFModel object for the CIFAR-10 SavedModel. We specify "serving_default" as the signature definition to use. The possible signature definitions are displayed by the TFModel's toString() method, which is displayed to the REPL when the model object is created.

We'll utilize the CIFAR10Util class to obtain the 10,000 CIFAR-10 test labels and images. After that, we'll perform a prediction on the first image and output the label and prediction to the console.

import org.codait.stf4j._
import org.codait.stf4j.util._
val cifar10 = new TFModel("../stf4j-test-models/cifar10_saved_model/").sig("serving_default")
val testDataFile = "../stf4j-test-models/cifar10_data/cifar-10-batches-bin/test_batch.bin"
val labels = CIFAR10Util.getLabels(testDataFile);
val images = CIFAR10Util.getPreprocessedImages(testDataFile, CIFAR10Util.DimOrder.ROWS_COLS_CHANNELS);
val label = labels(0);
val prediction = cifar10.in("input", images(0)).out("classes").run().getInt("classes")
print("Label: " + label + ", Prediction: " + prediction)

In the console output, notice that when we create the model object, we see the possible SignatureDef keys, serving_default and predict. Also, notice that the input image data is expected to be Floats. We see that the images are expected to be 32 rows by 32 columns by 3 channels. Although the input shape specifies 128 input images, any number of images can be fed into the model. The classes output is specified to be an INT64 (Long) value. However, we implicitly convert it to an Int value by calling the TFResults getInt method.

scala> import org.codait.stf4j._
import org.codait.stf4j._

scala> import org.codait.stf4j.util._
import org.codait.stf4j.util._

scala> val cifar10 = new TFModel("../stf4j-test-models/cifar10_saved_model/").sig("serving_default")
2018-08-20 10:30:55.876988: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: ../stf4j-test-models/cifar10_saved_model/
2018-08-20 10:30:55.892737: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2018-08-20 10:30:55.901850: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
2018-08-20 10:30:55.919080: I tensorflow/cc/saved_model/loader.cc:113] Restoring SavedModel bundle.
2018-08-20 10:30:55.948538: I tensorflow/cc/saved_model/loader.cc:148] Running LegacyInitOp on SavedModel bundle.
2018-08-20 10:30:55.969714: I tensorflow/cc/saved_model/loader.cc:233] SavedModel load for tags { serve }; Status: success. Took 92737 microseconds.
cifar10: org.codait.stf4j.TFModel =
Model directory: ../stf4j-test-models/cifar10_saved_model/

SignatureDef key: serving_default
method name: tensorflow/serving/predict
inputs:
  input key: input
    dtype: DT_FLOAT
    shape: (128, 32, 32, 3)
    name: input_tensor:0
outputs:
  output key: probabilities
    dtype: DT_FLOAT
    shape: (128, 10)
    name: softmax_tensor:0
  output key: classes
    dtype: DT_INT64
    shape: (128)
    name: ArgMax:0
SignatureDef key: predict
method name: tensorflow/serving/predict
inputs:
  input key: input
    dtype: DT_FLOAT
    shape: (128, 32, 32, 3)
    name: input_tensor:0
outputs:
  output key: classes
    dtype: DT_INT64
    shape: (128)
    name: ArgMax:0
  output key: probabilities
    dtype: DT_FLOAT
    shape: (128, 10)
    name: softmax_tensor:...
scala> val testDataFile = "../stf4j-test-models/cifar10_data/cifar-10-batches-bin/test_batch.bin"
testDataFile: String = ../stf4j-test-models/cifar10_data/cifar-10-batches-bin/test_batch.bin

scala> val labels = CIFAR10Util.getLabels(testDataFile);
labels: Array[Int] = Array(3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 8, 5, 7, 8, 6, 7, 0, 4, 9, 5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, 1, 9, 5, 4, 6, 5, 6, 0, 9, 3, 9, 7, 6, 9, 8, 0, 3, 8, 8, 7, 7, 4, 6, 7, 3, 6, 3, 6, 2, 1, 2, 3, 7, 2, 6, 8, 8, 0, 2, 9, 3, 3, 8, 8, 1, 1, 7, 2, 5, 2, 7, 8, 9, 0, 3, 8, 6, 4, 6, 6, 0, 0, 7, 4, 5, 6, 3, 1, 1, 3, 6, 8, 7, 4, 0, 6, 2, 1, 3, 0, 4, 2, 7, 8, 3, 1, 2, 8, 0, 8, 3, 5, 2, 4, 1, 8, 9, 1, 2, 9, 7, 2, 9, 6, 5, 6, 3, 8, 7, 6, 2, 5, 2, 8, 9, 6, 0, 0, 5, 2, 9, 5, 4, 2, 1, 6, 6, 8, 4, 8, 4, 5, 0, 9, 9, 9, 8, 9, 9, 3, 7, 5, 0, 0, 5, 2, 2, 3, 8, 6, 3, 4, 0, 5, 8, 0, 1, 7, 2, 8, 8, 7, 8, 5, 1, 8, 7, 1, 3, 0, 5, 7, 9, 7, 4, 5, 9, 8, 0, 7, 9, 8, 2, 7, 6, 9, 4, 3, 9, 6, 4, 7, 6, 5, 1, 5, 8, 8, 0, 4, 0, 5, 5, 1, 1, 8, 9, 0, 3, 1, 9, 2, 2, 5, 3, 9, 9, 4, 0, 3,...
scala> val images = CIFAR10Util.getPreprocessedImages(testDataFile, CIFAR10Util.DimOrder.ROWS_COLS_CHANNELS);
images: Array[Array[Array[Array[Float]]]] = Array(Array(Array(Array(1.0636618, 0.077478275, -1.2731644), Array(1.0851005, 0.0560395, -1.316042), Array(1.2137332, 0.16323337, -1.2302868), Array(1.235172, 0.20611091, -1.1874093), Array(1.1065394, 0.077478275, -1.3374807), Array(1.0207843, 0.0131619545, -1.4446746), Array(1.1494169, 0.14179459, -1.316042), Array(1.0851005, 0.098917045, -1.3589195), Array(1.0636618, 0.0560395, -1.3803582), Array(1.0851005, 0.098917045, -1.4446746), Array(1.1279781, 0.16323337, -1.4446746), Array(1.1065394, 0.0560395, -1.2088481), Array(1.1279781, 0.0560395, -1.2731644), Array(1.235172, 0.18467213, -1.4446746), Array(1.2994883, 0.18467213, -1.3589195), Array(1.3209271, 0.22754969, -1.3803582), Array(1.2566108, 0.18467213, -1.4661133), Array(1.1494169, 0.0989...
scala> val label = labels(0);
label: Int = 3

scala> val prediction = cifar10.in("input", images(0)).out("classes").run().getInt("classes")
prediction: Int = 3

scala> print("Label: " + label + ", Prediction: " + prediction)
Label: 3, Prediction: 3

Here, we feed in all 10,000 images as a 4-dimensional Float array. In the background, this is converted to a 4-dimensional Tensor and this is fed into the model. The 10,000 predictions are returned as an Int array.

val predictions = cifar10.in("input", images).out("classes").run().getIntArray("classes")

Output:

scala> val predictions = cifar10.in("input", images).out("classes").run().getIntArray("classes")
predictions: Array[Int] = Array(3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 6, 5, 7, 8, 6, 7, 0, 4, 9, 5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, 1, 9, 5, 4, 6, 5, 6, 0, 9, 3, 9, 7, 6, 9, 8, 5, 3, 8, 8, 7, 7, 7, 3, 7, 3, 6, 3, 6, 2, 1, 2, 3, 7, 2, 6, 8, 8, 0, 2, 9, 3, 3, 8, 8, 1, 1, 7, 2, 5, 2, 7, 8, 9, 0, 3, 8, 6, 4, 3, 6, 0, 0, 7, 4, 5, 6, 3, 1, 1, 3, 6, 8, 7, 4, 0, 6, 2, 1, 3, 0, 4, 2, 7, 8, 3, 1, 2, 8, 1, 8, 3, 3, 2, 4, 1, 8, 9, 1, 2, 9, 7, 2, 9, 6, 5, 6, 3, 8, 2, 6, 6, 5, 2, 8, 9, 6, 0, 0, 5, 2, 9, 3, 4, 2, 1, 6, 6, 0, 4, 8, 4, 5, 8, 9, 0, 9, 8, 9, 9, 3, 7, 2, 0, 0, 5, 2, 2, 3, 8, 6, 3, 4, 0, 5, 8, 0, 1, 7, 2, 8, 8, 7, 8, 5, 1, 8, 7, 1, 3, 0, 5, 7, 9, 7, 4, 5, 9, 0, 0, 7, 9, 8, 2, 7, 6, 9, 4, 3, 9, 0, 4, 7, 6, 5, 1, 3, 8, 8, 0, 4, 7, 5, 5, 1, 1, 8, 9, 0, 3, 1, 9, 2, 2, 5, 3, 9, 9, 4, ...

About

Simplified TensorFlow for Java

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages