Skip to content
This repository has been archived by the owner on Nov 8, 2019. It is now read-only.

Commit

Permalink
* implemented pre-compilation for templates (eHTML)
Browse files Browse the repository at this point in the history
   * completed implementation for storing & loading compiled template
files
   * added pre-compilation code for template-parsing task (in ANT
parse.xml)
   * added mechanism for loading pre-compiled templates
   * configuration for enabling/disabling usage of pre-compiled files
   * included up-to-date check (using MD5-checksums) to verify that
compiled templates are up-to-date
 * grammars:
   * added English example grammar (from/used in smart-case example)
   * example Japanese dictionary and (minimal) grammar
 * test page for semantic-interpreter: FIX in language-selection menu
(dropdown box): select currently set language on page-load (before:
first entry in the list was selected)
 * CommonUtils:
   * BUGFIX create instance with new (avoid var-leaking into global
namespace)
   * updated network-checking code (to reflect/use current
Cordova-implementation)
 * ControllerManager: refactored info-object creation (for preparing
controller-/view-loading)
 * BUGFIX load input-manager asynchronously
 * BUGFIX prevent default-click behavior in DEFAULT-BUTTON-HANDLER (see
vclick handling in executeAfterEachPageIsLoaded())
 * cordovaAudioOutput
   * removed some commments
   * BUGFIX for implementation of isPaused() function
   * slightly modified stop() implementation
   * removed unnecessary/obsolete callback-id-list mechanism
   * FIX: remove onCanPlay listener after first invocation
   * FIXED/IMPROVED status-dependent functions (stop() etc);
   * NOTE intialization only works error-free in combinaiton with
modification in Cordova resources so that on-init event for media/audio
is correctly fired in case of async-preparation
   * FIX included private field for audio status in order to avoid
Exceptions in stop() function
 * Cordova (MODIFIACATION for 2.8.1 JAR)
   * added listeners in Audio impl. in order to avoid ERROR output (for
normal/regular behavior) to LogCat
   * BUGFIX in onPreparation() in case of async preparation: only signal
MEDIA_STARTING if the current state is lower (e.g. do nothing if already
playing)
   * FIX for media/audio initialization: in case of async-preparation
now INIT-callback is triggered when prepared-listener is invoked (not
directly when async-preparation is started as before)
 * html5AudioOutput
   * remove console debug output
   * FIX / WORKAROUND for stop/replay: in case audio does not support
timeranges (dependent on server), audio is reloaded, since we cannot
reset currentTime to 0
   * BUGFIX need this-reference for removing listener in audio-object
   * BUGFIX for initialization listener (should only be called once)
 * maryTextToSpeech
   * BUGFIX trigger onEnd-callback in tts-function if cancel-function is
called (i.e. notify ending of TTS)
   * FIX set onEnd-callback to null, after calling (avoiding possible,
multiple calls to same callback)
   * FIX apply encodeURIComponents() to text/sentence before sending to
MARY (instead of just encoding space chars)
 * webkitAudioInput
   * BUGFIX for recognize() implementation
   * BUGFIX need to reset variable aborted, otherwise recognition
assumes abortion on every listener-callback, if abort/cancel has been
invoked once before
   * BUGFIX only restart asr if no speech is detected.
   * BUGFIX for BUG in case of more than one instance running, ASR would
get stuck in an infinite loop
 * Notification
   * BUGFIX for callback triggering (avoid null-pointer exceptions)
   * DISABLED releasing resources on pause (in order to be able to play
notification sounds as long as app is only paused and not exited yet)
     * added & implemented optional parameter isKeepOnPause for creating
notifications sounds (only relevant for ANDROID): if set, sounds are not
released when APP pauses   
   * added initSound() function (for avoiding double-triggering sound
upon first playing, e.g. BEEP feedback after starting/resuming APP)
   * REFACTORED private method for getting audio-object from internal
cache
 * removed language-specific functionality from ConfigurationManager
(set to @deprecated: instead the LanguageManager should be used
directly)
 * modified ANT tasks to use integrated SCXML-JS for compiling dialog-
and input-engine
 * buil-properties/-settings:
   * removed properties in build-setting files and ANT script that were
required for previous solution (i.e. usage of external SCXML-JS)
   * added/extended comments and added example for referencing nodejs in
standard MacOS-environment
  • Loading branch information
russaa committed Jul 11, 2014
1 parent fe51d80 commit ebd73c9
Show file tree
Hide file tree
Showing 237 changed files with 74,113 additions and 2,204 deletions.
468 changes: 468 additions & 0 deletions .README.md.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .project
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>StarterKit-public</name>
<name>MMIG-StarterKit</name>
<comment></comment>
<projects>
</projects>
Expand Down
2 changes: 1 addition & 1 deletion AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
android:name="de.dfki.iui.mmig.MMIGAppActivity"
android:configChanges="orientation|keyboardHidden"
android:label="@string/app_name"
android:launchMode="singleInstance" >
android:launchMode="singleTask" >

<intent-filter>
<action android:name="android.intent.action.MAIN" />
Expand Down
15 changes: 11 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MMIR - Multimodal Mobile Interaction and Rendering framework

----

This StarterKit is a small example application for the MMIR framework that can run locally in an (HTML5 capable) internet browser, and using the Cordova framework (http://cordova.apache.org/), it can also be run as an Android app.
This StarterKit is a small example application for the MMIR framework that can run locally in an (HTML5 capable) internet browser, and using the [Cordova framework][0], it can also be run as an Android app.


This is an ALPHA version for the MMIR framework (CAUTION: significant parts of the framework -- such as the API, used technology etc. -- may still be subject to change)
Expand All @@ -14,10 +14,17 @@ As a goal, this frameworks aims to provide a lightweight multi-modal dialog mana

A base principle is the MVC (Model View Controller) pattern. The framework provides a template mechanism similar to JSP (Java Server Pages), ASP (Active Server Pages), and alike.

The state of the dialog system can be tracked and manipulated based on a finite-state machine (FSM). This is realized using SCXML (State Chart XML) with SCION (https://github.com/jbeard4/SCION), a JavaScript based interpreter for SCXML files.
The state of the dialog system can be tracked and manipulated based on a finite-state machine (FSM). This is realized using SCXML (State Chart XML) with [SCION][1], a JavaScript based interpreter for SCXML files.

The StarterKit provides basic capabilities for recognizing speech input (ASR, using the Google Speech Recognition service), and speech output (TTS, using MARY http://mary.dfki.de/). Note, that for running the StarterKit in a browser environment currently only Google Chrome (version 25.x.x) allows access to the microphone resource for speech input.
The StarterKit provides basic capabilities for recognizing speech input (ASR, using the Google Speech Recognition service), and speech output (TTS, using [MARY][2]).
Note however, that there are some restrictions for running the StarterKit in a browser environment:
* microphone access: currently only Google Chrome (version 21.x.x), and Firefox (version 20.x) allow access to the microphone resource for speech input
* silence detection: currently only Google Chrome (version 25.x.x) and Firefox (version 26.x) provide an implementation for the AudioContext object, which is needed by our implementation for the end-of-speech detection (aka silence detection)

----

If not stated otherwise, the code, resource files etc. is provided under the MIT license (see the license file).
If not stated otherwise, the code, resource files etc. is provided under the MIT license (see license file).

[0]: http://cordova.apache.org/
[1]: https://github.com/jbeard4/SCION
[2]: http://mary.dfki.de/
3 changes: 2 additions & 1 deletion assets/www/config/configuration.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
{
"language": "en",
"usePrecompiledViews": "true",
"silenceDetectorNoiseTreshold": "0.1",
"silenceDetectorPauseCount": "3",
"silenceDetectorMinimalSpeachCount": "3",
"silenceDetectorResetCount": "15",
"HTML5InputSilenceBuffer": "2000000",
"HTML5InputWebSocketAddress": "ws://127.0.0.1:9999",
"HTML5InputWebSocketAddress": "ws://localhost:8347",
"HTML5InputSoundPackageSize": "4096",
"HTML5InputSilenceDetectorPath": "mmirf/env/media/silenceDetection.js",
"HTML5OutputServerBasePath": "http://mary.dfki.de:59125/",
Expand Down
2 changes: 1 addition & 1 deletion assets/www/config/directories.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"/controllers":["application.js","calendar.js"],"/views":["application","calendar","layouts"],"/views/application":["login.ehtml","registration.ehtml","welcome.ehtml","~languageMenu.ehtml"],"/views/calendar":["create_appointment.ehtml"],"/views/layouts":["application.ehtml"],"/models":["calendarModel.js","user.js"],"/config":["languages","statedef","configuration.json","directories.json"],"/config/languages":["de","en"],"/config/languages/de":["dictionary.dic","grammar.json","grammar.json.MD5","speaker.json"],"/config/languages/en":["dictionary.dic","speaker.json"],"/config/statedef":["dialogDescriptionSCXML.xml","dialogDescriptionSCXML.xml.MD5","example-view_transitions-dialogDescriptionSCXML.xml","inputDescriptionSCXML.xml","inputDescriptionSCXML.xml.MD5","readme.txt","scxml2js.bat","scxml2js.sh"],"/mmirf/plugins":["beepPlugin.js","cookiePlugin.js","directoryListing.js","multitouchPlugin.js","nativeASR.js","nativeTTS.js","nuance.js","queuePlugin.js","softkeyboard.js","video.js"],"/helpers":["applicationHelper.js"],"/gen/grammar":["de_grammar.js"]}
{"/controllers":["application.js","calendar.js"],"/views":["application","calendar","layouts"],"/views/application":["login.ehtml","registration.ehtml","welcome.ehtml","~languageMenu.ehtml"],"/views/calendar":["create_appointment.ehtml"],"/views/layouts":["application.ehtml"],"/models":["calendarModel.js","user.js"],"/config":["languages","statedef","configuration.json","directories.json"],"/config/languages":["de","en","jp"],"/config/languages/de":["dictionary.dic","grammar.json","grammar.json.MD5","grammar.json_minimal-example","speaker.json"],"/config/languages/en":["dictionary.dic","grammar.json","grammar.json.MD5","grammar.json_disabled","speaker.json"],"/config/languages/jp":["dictionary.dic","grammar.json","grammar.json.MD5","speaker.json"],"/config/statedef":["dialogDescriptionSCXML.xml","dialogDescriptionSCXML.xml.MD5","example-view_transitions-dialogDescriptionSCXML.xml","inputDescriptionSCXML.xml","inputDescriptionSCXML.xml.MD5","readme.txt","scxml2js.bat","scxml2js.sh"],"/mmirf/plugins":["VolumeControl.js","beepPlugin.js","cookiePlugin.js","directoryListing.js","multitouchPlugin.js","nativeTTS.js","nuance.js","queuePlugin.js","softkeyboard.js","video.js"],"/helpers":["applicationHelper.js"],"/gen/grammar":["de_grammar.checksum.txt","de_grammar.js","en_grammar.checksum.txt","en_grammar.js","jp_grammar.checksum.txt","jp_grammar.js"],"/gen/views":[]}
Loading

0 comments on commit ebd73c9

Please sign in to comment.