Page History

Overview

The Non-VerbalNonverbal-Behavior-Generator (NVBG) is the module that generates behavior other than speech, such as gestures, facial expressions and gazes.  The NVBG generates the aforementioned behavior based on the speech input and other messages that it receives. These   These behaviors serve to augment and emphasize spoken communication.  Based on who speaks and who listens, the NVBG can characterize the NPC as a listener, speaker or a bystander and generate appropriate behavior.  These behaviors are configurable using xml files provided as input to NVBG.  Using these xml files, we can specify which words or parts-of-speech trigger which animations.  We can specify idle animations and idle gazes that get triggered when the character is idle for a specified amount of time.

The NVBG is loosely coupled with a "parser" which gets the input sentence from NVBG and returns the parts of speech in the sentence. NVBG   NVBG can extract information from the lexical, syntactic, and semantic structure of the surface text that can support the generation of believable nonverbal behaviors.

The NVBG also has a saliency map which it uses to keep track of it's environment, important events and conversational topics.  This helps the NVBG generate behaviors that are appropriate to the current scene and helps it prioritize.  This saliency map is configurable through xml files too.

...

The NVBG now supports multiple characters within one process. You can toggle between these characters as desired.The character itself needs to be configured using the various confuguration configuration options that NVBG provides. You can do this using a config file or through VHMessages.

...

As we can see, the saliency map can contain multiple story-points each specifying which objects in the scene are of importance, the relation between spoken words and object in the scene and the emotion of the charctercharacter.

The above example contains only one story-point but in general it can contain many. NVBG can be notified of which storypoint story-point it should load so that the appropriate priorities are assigned to the objects in the scene.

...

The Nonverbal Behavior Generator (NVBG) is a tool a that automates the selection and timing of nonverbal behavior for Embodied Conversational Agents (aka Virtual Humans).  It uses a rule-based approach that generates behaviors given the information about the agent's cognitive processes but also by inferring communicative functions from a surface text analysis

...