Page History
...
Our technology emphasizes natural language interaction, nonverbal behavior, and visual recognition and is broken up into the following main modules:
- Non-Player Character Editor (NPCEditor): A suite of tools which work together to create appropriate dialogue responses to users’ inputs. A text classifier selects responses based on cross-language relevance models; the authoring interface relates questions and answers; and a simple dialogue manager controls aspects of output behavior.
- Nonverbal Behavior Generator (NVBG): A rule-based behavior planner that infers communicative functions from the surface text and selects appropriate behaviors that augment and complement the characters’ dialogue.
- SmartBody (SB): A modular, controller-based character animation system that uses the Behavior Markup Language.
- Watson: A real-time visual feedback recognition library for interactive interfaces which uses the images from either a monocular or stereo camera to recognize eye and head gazes and gestures.
- Speech Client (AcquireSpeech): A tool to send audio, or text, to speech recognizers and to relay the information to the entire system. The toolkit uses PocketSphinx as a 3rd party speech recognition solution.
- MultiSense: A perception framework that enables multiple sensing and understanding modules to inter-operate simultaneously, broadcasting data through Perception Markup Language. It currently contains GAVAM, CLM FaceTracker, and FAAST which work with a webcam or Kinect.
- Rapport: An agent that provides nonverbal feedback based on human nonverbal and verbal input. Rapport has been used in a variety of international studies related to establishing rapport between real and virtual humans.
...
Please be aware that the toolkit consists of research software for which documentation and support is limited. However, both the software as well as the accompanying documentation are actively being developed and updated.
...
Toolkit
...
There are many [[Projects|ICT projects]] that use a subset of the technology provided with the toolkit. Below is a list of some examples:* [[Projects#SASO|
- SASO
...
...
...
...
...
...
...
...
...
...
- (with
...
...
- Tactical Questioning
In addition, many groups outside of ICT use some of the toolkit components, most notably [[SmartBody]] and [[Watson]]:
* Watson.
- University of Reykjavik
...
- German Research Center for Artificial Intelligence
...
- ArticuLab at Northwestern University
...
- Telecom Paris Tech
...
- Affective Computing Research group at MIT Media Lab
...
- ICSI/UCB Vision Group at UC Berkeley
...
- Human-Centered, Intelligent, Human-Computer Interaction group at Imperial College
...
- Worcester Polytechnic Institute
...
- Microsoft Research
...
- Relational Agents group at Northeastern University
...
- Component Analysis Lab at Carnegie Mellon University
| Recently Updated |
|---|
Overview
Content Tools
Activity