...
Our technology emphasizes natural language interaction, nonverbal behavior, and visual recognition and is broken up into the following main modules:
- Non-Player Character Editor (NPCEditor): A suite of tools which work together to create appropriate dialogue responses to users’ inputs. A text classifier selects responses based on cross-language relevance models; the authoring interface relates questions and answers; and a simple dialogue manager controls aspects of output behavior.
- Nonverbal Behavior Generator (NVBG): A rule-based behavior planner that infers communicative functions from the surface text and selects appropriate behaviors that augment and complement the characters’ dialogue.
- SmartBody (SB): A modular, controller-based character animation system that uses the Behavior Markup Language.
- Watson: A real-time visual feedback recognition library for interactive interfaces which uses the images from either a monocular or stereo camera to recognize eye and head gazes and gestures.
- Speech Client (AcquireSpeech): A tool to send audio, or text, to speech recognizers and to relay the information to the entire system. The toolkit uses PocketSphinx as a 3rd party speech recognition solution.
...