Neural Network Quickstart (Java)

Let's build a simple Java NN :

---

1. Configuration

Before working with tensors it might be useful to configure Neureka to fit your needs.
Accessing settings can be done as follows :

Settings settings = Neureka.get().settings();

For more information take a look at the settings documentation.

---

2. Devices

Before getting started with tensors we first have to consider the following question : Where do we want our tensors to be stored and executed?

The answer to this question is usually quite simple : By default, they are stored in primitive arrays on the JVM, so in your RAM! However! What if you want them to run on your GPU? In that case we have to use the Device interface to get a device on which our tensors ought to live :

Device device = Device.find("first gpu").orElse(CPU.get());

Devices are components of tensors and tensors are elements of a device. Therefore we can marry instances of these two types via the following two ways :

device.store( myTensor );

or

myTensor.to( device );

For simplicity reasons the next steps will not include the Device type when handling tensors.

---

3. Mock Data

Neural networks are known for their hunger for data. So let's prepare a snack :

Tsr X = Tsr.of(
            new double[][]{
                {0.6667, 1.0000},
                {0.3333, 0.5556},
                {1.0000, 0.6667}
            }
        );
Tsr y = Tsr.of(
            new double[][]{
                {0.9200},
                {1.0000},
                {0.8900}
            }
        );

If you want to know more about feeding data into your tensors consider looking at the FileDevice class.

---

4. Weights

For this network we are going to need 2 weight tensors :
First we need a 2 by 3 matrix for the weights between the input layer and the hidden layer. This weight matrix shall be called W1 Besides that we then create the second weight tensor which is a 3 by 1 matrix named W2.

Tsr W1 = Tsr.of(
            new double[][]{
                {-1.1843,  0.0146, -1.4647},
                {-1.4020, -1.0129,  0.6256}
            }
         ).setRqsGradient(true);
Tsr W2 = Tsr.of(
            new double[][]{
                {1.8095},
                {-0.4269},
                {-1.1110}
            }
         ).setRqsGradient(true);

---

5. Activation Function

The last setup step is the activation function. For this little quickstart we will use the sigmoid activation function 👍

Function sig = Function.of("sig(I[0])");

---

6. Finally! A neural network! 😃

private List<String> errors = new ArrayList<>();
private List<String> losses = new ArrayList<>();

public Tsr<?> forwardAndBackward( Tsr x )
{ 
    Tsr z1 = x.matMul(W1);
    Tsr hidden = sig.call(z1);
    Tsr z2 = hidden.matMul(W2);
    Tsr pred = sig.call(z2);
    Tsr error = y.minus(pred);
    errors.add(error.toString());
    Tsr loss = error.power(2).mean();
    losses.add(loss.toString());
    pred.backward(error); // This is where Neurekas autograd magic happend! 
    W1.applyGradient();
    W2.applyGradient();
    return loss;
}

---

7. Training

Let's train it over 100 epochs :

for ( int i = 0; i < 100; i++ ) {
    Tsr loss = forwardAndBackward(X);
    System.out.println(loss);
}

---