Dialog system

This article shows how to use the dialog system. With the dialog system you can ask amy something with speech or the web browser and the dialog system process the input. When some informations are missing, amy ask a question und you can answer. When all information is there, the method from the correct plugin is executed. The function is explained in the following article.

General things

XML files

Main XML file

this file must be in this folder: src\main\resources of the plugin and has the following name schema: NAMEOFTHESPEECHFILE.aim.xml. The name is arbitrary. A plugin can have more than one aim.xml file The main speech xml file consists of several section. A intent is ...
A intent consists of:

• a reference to the method that should be called at the end of the intent. (required)
• a grammar to match the possible commands. For grammar format see below. (required)
• zero or more entity templates. the content of a entity template can accessed in the referenced method from the intent. you can use a predefined entity template (see below) in your entity template. an entity template consists of:
  • an entity template id. this is the id to find the entity in the referenced method. Important: the id must be in lower case letters(required)
  • an attribute if entity template is required. if false the entity does not need to be filled and no prompt can't be called. If you try to get a not filled entity you get a null value. If true, the entity must filled do complete the intent. A prompt is necessary if the entity is in an optional-group. (required)
  • an entity grammar. this is the grammar from a entity Template. This is a separate grammar object and has no relation to intent grammar. This will be put into the given place. (required)
  • a prompt. the prompt is called if a required entity is not filled. It delivers a query and wait for a new input. The order of the questions depends on the order of the entity templates from the intent. A prompt consists of:
    • a grammar to match the input. This is a separate grammar object and has no relation to intent grammar. (required)
    • a query text to output in some way. (required)

A full example you can see here

natlangMeta XML

this file must be in this folder: src\main\resources\META-INF of the plugin and has the following name schema: io.github.amyassist.PLUGINNAME.natlangMeta This file must contain all speech aim.xml file names. This is necessary for the dialog system to find the dialog xmls. The file has following structer:

.aims:
SPEECHFILE1.aim.xml
SPEECHFILE2.aim.xml

@Intent annotation

This annotation is used to declare methods as speech methods. All methods referenced in the xml must have this annotation. Otherwise the method can't call from the manager.

@EntityProvider annotation

With this annotation in a speech class you can define grammars during runtime. the annotation needs a name from an entity template that is defined in the xml and overwrite the grammar from this entity template. You can only use this entity template in prompt grammar and not in the intent grammar. Example:

@EntityProvider("tags")
public List<String> provideTags(){
    List<String> tags = getSomeTags();
    return tags;
}

Grammar Format

The syntax is derived from jsgf with special characters for pre-defined grammar rules and some limitations like no Kleene star.

Basics

Sentences are Strings containing any number of OR Groups, Optional Groups, Words and Rules in any order.
OR groups and optional groups may contain one or more sentences them selves. This means recursion is possible.
For example: @Grammar("test (test2 | test3 [test4|test5])") is a valid grammar in AGF.
Where test (test2 | test3 [test4|test5]) , test2, test3 [test4|test5], test4 and test5 are sentences.

Longer AGF grammars will most likely be harder for the speech recognition to spot. Try to keep them light.

Internally AGF grammars are represented by tree structures.
More details can be found in the natlang package in core. (These structures can be parsed in valid JSGF Grammars, which are used by the Speech Recognizer.)

The validity of AGF strings will be checked after loading the plugin.

OR Groups

These groups may contain one or more sentences separated by |.

Example:

• (word1|word2) - word1 or word2 has to be used
• word6 (word7|word8) - possible sentences: "word6 word7", "word6 word8"

Optional Groups

Optional groups work just like OR groups, but they are not necessary in the input.
They may contain one or more sentences separated by |

Example:

• [word3] - word3 is optional
• [word4|word5] - word4 or word5 can be used

Wildcard

at the end of a grammar or alone in a prompt grammar can placed a wildcard with *

Example:

• play *

Special wildcard with a maximum length of 5

this wildcard may not be used at the end of a grammar. It's necessary that is a word after this wildcard. The matching text can be 0-5 words long. The symbol for this is +

Example:

• from + to + at

Usage hints

• try to avoid too general keywords like ("turn on" or "activate")

More Examples

Predefined Entities

The dialog system provides a set of predefined entities that can used for the grammar xml. Do not use the predefined entities in an intent or prompt grammar directly. The best practice is to create a own entity and put the predefined entity in the grammar from the entity.
Example:

<EntityTemplate id="number" required="true">
	<gram>{amyinteger}</gram>
</EntityTemplate>

Integer

with {amyinteger} you can get a integer with entityData.getNumber().
Example input: 100 you get a integer value of 100

Time

with {amytime} you can get a LocalTime object with entityData.getTime().
The gramar for the time is: (HOUR (x|oh|:) MINUTE| ((HOUR|quarter|half)(past|to) MINUTE )|HOUR [o clock]|now)[am|pm|p.m.|a.m.]
Example Inputs:

• 10:00 a.m.
• 10 oh 10 p.m.
• 11 to 5
• quarter past 10
• 12 o clock
• now

Date

with {amydate} you can get a LocalDate object with entityData.getDate().
The grammar for the time is : ([DAYOFWEEK [the]] DAYOFMONTH [of|.] (MONTHASNUMBER|MONTHASWORD) [[.]YEAR]|today|tomorrow)
Example Inputs for the 3.9.2018:

• 3.9.2018
• 3.9
• 3th of september 2018
• 3th of september
• monday the 3th of september 2018
• monday the 3.9.2018
• tomorrow

Date and Time

with {amydatetime} you can get a LocalDateTime obejct with entityData.getDateTime.
The grammar fo the date and time is: DATE at TIME
DATE and TIME see above. Example inputs for the 3.9.2018 at 10:00 am:

• 3th of september 2018 at 10:00 a.m.
• 3.9 at 10 o clock