V1.2 (#8)

v1.2

README.md

@@ -2,7 +2,7 @@
 
 Made in Vancouver, Canada by [Picovoice](https://picovoice.ai)
 
-Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from speech commands within a given context of
+Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a given context of
 interest in real-time. For example, given a speech command "*Can I have a small double-shot espresso with a lot of sugar
  and some milk*" it infers that the user wants to *order a drink* with the following specific requirements.
 
@@ -20,16 +20,15 @@ Rhino is
 
 * intuitive. It allows users to utter their intention in a natural and conversational fashion.
 * using deep neural networks trained in **real-world situations**.
-* compact and computationally-efficient making it suitable for **IoT** applications. It can run with as low as 100 KB of RAM.
-* cross-platform. It is implemented in fixed-point ANSI C. Currently **ARM Cortex-M**, **ARM Cortex-A**,
-**Raspberry Pi**, **Android**, **iOS**, **watchOS**, **Linux**, **Mac**, **Windows**, and **WebAssembly** are supported.
+* compact and computationally-efficient making it suitable for **IoT** applications. It can run with as low as 90 KB of
+RAM on an MCU.
+* cross-platform. It is implemented in fixed-point ANSI C. Currently **Raspberry Pi**, **Beagle Bone** **Android**,
+**iOS**, **Linux**, **Mac**, **Windows**, and **web browsers** (**WebAssembly**) are supported. Additionally support for
+various **ARM Cortex-A**, **ARM Cortex-M** (M4/M7) and **DSP cores** is available for commercial customers.
 * customizable. It can be customized for any given domain.
 
 [![Rhino in Action](https://img.youtube.com/vi/WadKhfLyqTQ/0.jpg)](https://www.youtube.com/watch?v=WadKhfLyqTQ)
 
-NOTE: Currently Raspberry Pi, Android, and Linux builds are available to the open-source community. But we do have plans
-to make other platforms available as well in upcoming releases.
-
 ## Table of Contents
 * [Try It Out](#try-it-out)
 * [Motivation](#motivation)
@@ -43,9 +42,11 @@ to make other platforms available as well in upcoming releases.
 * [Running Demo Applications](#running-demo-applications)
  * [Running Python Demo Application](#running-python-demo-application)
  * [Running C Demo Application](#running-c-demo-application)
+ * [Runnning Android Demo Application](#running-android-demo-application)
 * [Integration](#integration)
  * [C](#c)
  * [Python](#python)
+ * [Android](#android)
 * [Releases](#releases)
 * [License](#license)
 
@@ -64,17 +65,17 @@ requires significant CPU and memory for an on-device implementation.
 
 Rhino solves this problem by providing a tightly-coupled speech recognition and NLU engine that are jointly optimized
 for a specific domain (use case). Rhino is quite lean and can even run on small embedded processors
-(think ARM Cortex-M or fixed-point DSPs) with very limited RAM (as low as 100 KB) making it ideal for
+(think ARM Cortex-M or fixed-point DSPs) with very limited RAM (as low as 90 KB) making it ideal for
 resource-constrained IoT applications.
 
 ## Metrics
 
-The table shows the average CPU usage on three different platforms (1) Raspberry Pi zero, (2) Raspberry Pi 3, and an
-Ubuntu box (i5-6500 CPU @ 3.20GHz). You can recreate this using the [C demo application](/demo/c).
+The table shows the average CPU usage on two different platforms (1) Raspberry Pi zero and (2) Raspberry Pi 3. You can
+recreate this using the [C demo application](/demo/c).
 
-Raspberry Pi zero | Raspberry Pi 3 | Ubuntu Desktop (i5-6500 CPU @ 3.20GHz)
-:---: | :---: | :---:
-48.7% | 8.9% | 1.2%
+Raspberry Pi zero | Raspberry Pi 3
+:---: | :---:
+46.4% | 7.2%
 
 ## Terminology
 
@@ -93,8 +94,8 @@ of spoken commands:
 
 ### Expression
 
-A context is made of a collection of spoken commands mapped to the user's intent. An expression is an entity that defines a mapping between
-a (or a set of) spoken commands and its (their) corresponding intent. For example
+A context is made of a collection of spoken commands mapped to the user's intent. An expression is an entity that defines
+a mapping between a (or a set of) spoken commands and its (their) corresponding intent. For example
 
 * {turnCommand} the lights. -> {turnIntent}
 * Make the {location} light {intensityChange}. -> {changeIntensityIntent}
@@ -143,7 +144,7 @@ python demo/python/rhino_demo.py \
 --rhino_context_file_path ./resources/contexts/linux/coffee_maker_linux.rhn \
 --porcupine_library_path ./resources/porcupine/lib/linux/x86_64/libpv_porcupine.so \
 --porcupine_model_file_path ./resources/porcupine/lib/common/porcupine_params.pv \
---porcupine_keyword_file_path ./resources/porcupine/resources/keyword_files/linux/hey_alfred_linux.ppn
+--porcupine_keyword_file_path ./resources/porcupine/resources/keyword_files/linux/hey\ pico_linux.ppn
 ```
 
 The following runs the engine on a *Raspberry Pi 3* to infer intent within the context of smart lighting system
@@ -155,14 +156,19 @@ python demo/python/rhino_demo.py \
 --rhino_context_file_path ./resources/contexts/raspberrypi/coffee_maker_raspberrypi.rhn \
 --porcupine_library_path ./resources/porcupine/lib/raspberry-pi/cortex-a53/libpv_porcupine.so \
 --porcupine_model_file_path ./resources/porcupine/lib/common/porcupine_params.pv \
---porcupine_keyword_file_path ./resources/porcupine/resources/keyword_files/raspberrypi/hey_alfred_raspberrypi.ppn
+--porcupine_keyword_file_path ./resources/porcupine/resources/keyword_files/raspberrypi/hey\ pico_raspberrypi.ppn
 ```
 
 ### Running C Demo Application
 
 This [demo application](demo/c) is mainly used to show how Rhino can be integrated into an efficient C/C++ application.
 Furthermore it can be used to measure runtime metrics of the engine on various supported platforms.
 
+### Running Android Demo Application
+
+Using Android Studio open [demo/android](/demo/android) as an Android project and then run the application. Note that
+you need an android phone with developer options enabled connected to your machine in order to run the application.
+
 ## Integration
 
 Below are code snippets showcasing how Rhino can be integrated into different applications.
@@ -282,8 +288,52 @@ collector.
 rhino.delete()
 ```
 
+### Android
+
+Rhino provides a binding for Android using JNI. It can be initialized using.
+
+```java
+ final String modelFilePath = ... // It is available at lib/common/rhino_params.pv
+ final String contextFilePath = ...
+
+ Rhino rhino = new Rhino(modelFilePath, contextFilePath);
+```
+
+once initialized `rhino` can be used for intent inference.
+
+
+```java
+ private short[] getNextAudioFrame();
+
+ while (rhino.process(getNextAudioFrame()));
+
+ if (rhino.isUnderstood()) {
+ RhinoIntent intent = rhino.getIntent();
+ // logic to perform an action given the intent object.
+ } else {
+ // logic for handling out of context or unrecognized command
+ }
+```
+
+when finalized the processing be sure to reset the object before processing a new stream of audio via
+
+```java
+ rhino.reset()
+```
+
+finally, prior to exiting the application be sure to release resources acquired via
+
+```java
+ rhino.delete()
+```
+
 ## Releases
 
+### v1.2.0 April 26, 2019
+
+* Accuracy improvements.
+* Runtime optimizations.
+
 ### v1.1.0 December 23rd, 2018
 
 * Accuracy improvements.

binding/android/rhino/src/main/java/ai/picovoice/rhino/Rhino.java

@@ -17,19 +17,18 @@
 
 package ai.picovoice.rhino;
 
-import java.util.HashMap;
+import java.util.LinkedHashMap;
 import java.util.Map;
 
 /**
- * Binding for Picovoice's speech-to-intent engine (aka Rhino).
- * The object directly infers intent from speech commands within a given context of interest in
- * real-time. It processes incoming audio in consecutive frames (chunks) and at the end of each
- * frame indicates if the intent extraction is finalized. When finalized, the intent can be
- * retrieved as structured data in form of an intent string and pairs of slots and values
- * representing arguments (details) of intent. The number of samples per frame can be attained by
- * calling {@link #frameLength()}. The incoming audio needs to have a sample rate equal to
- * {@link #sampleRate()} and be 16-bit linearly-encoded. Furthermore, Rhino operates on single
- * channel audio.
+ * Binding for Picovoice's speech-to-intent engine (aka Rhino). The object directly infers intent
+ * from speech commands within a given context of interest in real-time. It processes incoming audio
+ * in consecutive frames (chunks) and at the end of each frame indicates if the intent extraction is
+ * finalized. When finalized, the intent can be retrieved as structured data in form of an intent
+ * string and pairs of slots and values representing arguments (details) of intent. The number of
+ * samples per frame can be attained by calling {@link #frameLength()}. The incoming audio needs to
+ * have a sample rate equal to {@link #sampleRate()} and be 16-bit linearly-encoded. Furthermore,
+ * Rhino operates on single channel audio.
  */
 public class Rhino {
  static {
@@ -40,10 +39,11 @@ public class Rhino {
 
  /**
  * Constructor.
- * @param modelFilePath Absolute path to file containing model parameters.
- * @param contextFilePath Absolute path to file containing context parameters. A context
- * represents the set of expressions (commands), intents, and intent
- * arguments (slots) within a domain of interest.
+ *
+ * @param modelFilePath Absolute path to file containing model parameters.
+ * @param contextFilePath Absolute path to file containing context parameters. A context
+ * represents the set of expressions (commands), intents, and intent
+ * arguments (slots) within a domain of interest.
  * @throws RhinoException On failure.
  */
  public Rhino(String modelFilePath, String contextFilePath) throws RhinoException {
@@ -55,7 +55,8 @@ public Rhino(String modelFilePath, String contextFilePath) throws RhinoException
  }
 
  /**
- * Destructor. This is needs to be called explicitly as we do not rely on garbage collector.
+ * Destructor. This needs to be called explicitly as we do not rely on garbage collector.
+ *
  * @throws RhinoException On failure.
  */
  public void delete() throws RhinoException {
@@ -69,7 +70,8 @@ public void delete() throws RhinoException {
  /**
  * Processes a frame of audio and emits a flag indicating if the engine has finalized intent
  * extraction. When finalized, {@link #isUnderstood()} should be called to check if the command
- * was valid (is within context of interest).
+ * was valid (is within context of interest) and is understood.
+ *
  * @param pcm A frame of audio samples. The number of samples per frame can be attained by
  * calling {@link #frameLength()}. The incoming audio needs to have a sample rate
  * equal to {@link #sampleRate()} and be 16-bit linearly-encoded. Furthermore,
@@ -79,7 +81,7 @@ public void delete() throws RhinoException {
  */
  public boolean process(short[] pcm) throws RhinoException {
  try {
- return process(object, pcm) == 1;
+ return process(object, pcm);
  } catch (Exception e) {
  throw new RhinoException(e);
  }
@@ -88,13 +90,14 @@ public boolean process(short[] pcm) throws RhinoException {
  /**
  * Indicates if the spoken command is valid, is within the domain of interest (context), and the
  * engine understood it.
+ *
  * @return Flag indicating if the spoken command is valid, is within the domain of interest
  * (context), and the engine understood it.
  * @throws RhinoException On failure.
  */
  public boolean isUnderstood() throws RhinoException {
  try {
- return isUnderstood(object) == 1;
+ return isUnderstood(object);
  } catch (Exception e) {
  throw new RhinoException(e);
  }
@@ -105,21 +108,22 @@ public boolean isUnderstood() throws RhinoException {
  * string and pairs of slots and their values. It should be called only after intent extraction
  * is finalized and it is verified that the spoken command is valid and understood via calling
  * {@link #isUnderstood()}.
+ *
  * @return Inferred intent object.
  * @throws RhinoException On failure.
  */
  public RhinoIntent getIntent() throws RhinoException {
  final String intentPacked = getIntent(object);
  String[] parts = intentPacked.split(",");
  if (parts.length == 0) {
- throw new RhinoException(String.format("Failed to retrieve intent from %s", intentPacked));
+ throw new RhinoException(String.format("failed to retrieve intent from %s", intentPacked));
  }
 
- Map<String, String> slots = new HashMap<>();
+ Map<String, String> slots = new LinkedHashMap<>();
  for (int i = 1; i < parts.length; i++) {
  String[] slotAndValue = parts[i].split(":");
  if (slotAndValue.length != 2) {
- throw new RhinoException(String.format("Failed to retrieve intent from %s", intentPacked));
+ throw new RhinoException(String.format("failed to retrieve intent from %s", intentPacked));
  }
  slots.put(slotAndValue[0], slotAndValue[1]);
  }
@@ -130,6 +134,7 @@ public RhinoIntent getIntent() throws RhinoException {
  /**
  * Resets the internal state of the engine. It should be called before the engine can be used to
  * infer intent from a new stream of audio.
+ *
  * @throws RhinoException On failure.
  */
  public void reset() throws RhinoException {
@@ -143,6 +148,7 @@ public void reset() throws RhinoException {
  /**
  * Getter for expressions. Each expression maps a set of spoken phrases to an intent and
  * possibly a number of slots (intent arguments).
+ *
  * @return Expressions.
  * @throws RhinoException On failure.
  */
@@ -154,35 +160,38 @@ public String getContextExpressions() throws RhinoException {
  }
  }
 
- private native long init(String model_file_path, String context_file_path);
-
- private native long delete(long object);
-
- private native int process(long object, short[] pcm);
-
- private native int isUnderstood(long object);
-
- private native String getIntent(long object);
-
- private native boolean reset(long object);
-
- private native String contextExpressions(long object);
-
  /**
  * Getter for length (number of audio samples) per frame.
+ *
  * @return Frame length.
  */
  public native int frameLength();
 
  /**
  * Audio sample rate accepted by Picovoice.
+ *
  * @return Sample rate.
  */
  public native int sampleRate();
 
  /**
  * Getter for version string.
+ *
  * @return Version string.
  */
  public native String version();
+
+ private native long init(String model_file_path, String context_file_path);
+
+ private native void delete(long object);
+
+ private native boolean process(long object, short[] pcm);
+
+ private native boolean isUnderstood(long object);
+
+ private native String getIntent(long object);
+
+ private native boolean reset(long object);
+
+ private native String contextExpressions(long object);
 }

binding/python/rhino.py

@@ -20,7 +20,7 @@
 
 
 class Rhino(object):
- """Python binding for Picovoice's Speech to Intent (a.k.a Rhino) engine."""
+ """Python binding for Picovoice's Speech-to-Intent (a.k.a Rhino) engine."""
 
  class PicovoiceStatuses(Enum):
  """Status codes corresponding to 'pv_status_t' defined in 'include/picovoice.h'"""

binding/python/test_rhino.py

@@ -88,25 +88,34 @@ def _library_path(cls):
  system = platform.system()
  machine = platform.machine()
 
- if system == 'Linux':
+ if system == 'Darwin':
+ return cls._abs_path('lib/mac/x86_64/libpv_rhino.dylib')
+ elif system == 'Linux':
  if machine == 'x86_64':
  return cls._abs_path('lib/linux/x86_64/libpv_rhino.so')
  elif machine.startswith('arm'):
  return cls._abs_path('lib/raspberry-pi/arm11/libpv_rhino.so')
-
- raise NotImplementedError('Rhino is not supported on %s/%s yet!' % (system, machine))
+ elif system == 'Windows':
+ return cls._abs_path('lib/windows/amd64/libpv_rhino.dll')
+ else:
+ raise NotImplementedError('Rhino is not supported on %s/%s yet!' % (system, machine))
 
  @classmethod
  def _context_file_path(cls):
  system = platform.system()
  machine = platform.machine()
 
- if system == 'Linux' and machine == 'x86_64':
- return cls._abs_path('resources/contexts/linux/coffee_maker_linux.rhn')
- elif system == 'Linux' and machine.startswith('arm'):
- return cls._abs_path('resources/contexts/raspberrypi/coffee_maker_raspberrypi.rhn')
-
- raise NotImplementedError('Rhino is not supported on %s/%s yet!' % (system, machine))
+ if system == 'Darwin':
+ return cls._abs_path('resources/contexts/mac/coffee_maker_mac.rhn')
+ elif system == 'Linux':
+ if machine == 'x86_64':
+ return cls._abs_path('resources/contexts/linux/coffee_maker_linux.rhn')
+ elif machine.startswith('arm'):
+ return cls._abs_path('resources/contexts/raspberrypi/coffee_maker_raspberrypi.rhn')
+ elif system == 'Windows':
+ return cls._abs_path('resources/contexts/windows/coffee_maker_windows.rhn')
+ else:
+ raise NotImplementedError('Rhino is not supported on %s/%s yet!' % (system, machine))
 
 
 if __name__ == '__main__':