Page History

Overview

VHBuilder is a tool that allows you to quickly author and review verbal and nonverbal behaviors for one or more characters. It 's purpose is to be a complete UI interface for authoring characters, editing behaviors and adding/recording dialog easily. It is aimed at quickly getting basic characters up and running; power users can then move on to using the NCPEditor NPCEditor or NVBG directly for more fine-grained control. 

It abstracts the end user from individual components of the Toolkit pipeline and offers a go-to place for easy authoring of characters. 

...

• Location: /tools/VHBuilder (latter for public Toolkit site)
• Language: C#
• Distribution: source
• Platform(s): Windows

Users

Creating a new scenario

The VHBuilder application requires the user to either create or load a scenario before authoring the virtual human characters. A scenario is basically a sandbox which includes all the files and data associated with it.

Launch the VHBuilder application from the corresponding tab in the launcher. It will load up with a message ' Please create or load a scenario'.

 Click on the File menu option on the top left and select the 'Create Scenario' option as shown below

On selecting the 'Create Scenario' option, you will be presented with a dialog as shown below. Please type in the name of the new Scenario you want to create, select where you want to create it and select 'Save' as shown. On hitting 'Save' above, it will create a scenario and it will also launch all the components required to create the new scenario.

...

You cannot begin editing the scenario unless you add new characters to the scene. In order to do this, you can use either the 'Query' or 'Add' button as shown below.

The 'Query' button works only if you have an instance of Unity running in the background with a scene consisting of Toolkit characters. The 'Query' button basically queries Unity for the characters that are currently loaded and it adds these characters to the VHBuilder's list of characters as shown below.

If you don't have a Unity instance running, another way to add the characters is hitting the 'Add' button next to the 'Query' button. This will pop up a window as shown below

...

In case you don't have FaceFx installed on your machine, the window you see will look different as VHBuilder will default to using the VisemeSchedulerSOX tool instead which is our in-house tool for generating basic lipsync.

Selecting

...

nonverbal behavior files

A 'behavior file', contains the mapping from the spoken words to animations. For each character, you can select the 'behavior file' that you want to use for that character from the list of available files in the drop-down list. e.g. in the example below, we have selected 'rule_input_Rachel_template.xml' for the selected character 'Rachel'. Once you select the behavior file for a character, the tool will remember it. 

Modifying nonverbal behavior rules

In order to modify a behavior rule, you select the 'Rule' that you want to modify and the 'Posture' that you want the rule to apply to, using the drop-down lists as shown below. The drop-down list will contain the available rules and postures for the character.

...

Now you can add words that will trigger this rule under the 'Words' section by typing the word in the text-box and hitting the enter key. You are also able to select the animations that will be triggered when the character speaks that particular word, by clicking on the animation name and selecting it. The list of selected animations for a particular rule is displayed in the bottom right as shown.

Notes:

a) The character only performs the animation when the words in the 'Words' list are spoken and he is in the specified posture.

...

d) You can select and hit enter to add animations and hit delete in the 'Selected Animations' list to remove animations.

Adding a new nonverbal behavior rule

You can add a new rule by hitting the '+' button next to the Rule drop-down list. This will pop up a dialog box as shown below.

...

Enter the name of the new rule. Hitting OK , will create a new rule and you can go ahead and add words to this rule and select animations for it. Rule names are mostly for organizational purposes allowing you to have different rule names for different set of words.

...

1) In order to save the scenario and all the changes made to it, select the File menu option and hit the 'Save Scenario' as shown below

...

Pinging background components

...

If for any reason, one of the components fails to function or exits you can relaunch the components using the 'Relaunch' menu option as shown below. This can be used to restart the system with all components functioning in the background.

Message API

Query Unity for characters : 

vht_get_characters

VHBuilder character_names Rachel Brad

Link question and answer : 

NPCEditor <script target="user">document.getModel().setLinkValue(document.getModel().getQuestions().findIndexOfWithID("Anybody-9"),document.getModel().getAnswers().findIndexOfWithID("utterance_20130206114053_AMYNAMEISITHAVEA"),6);</script>

Flow of messages sent in order to record speech and generate audio file and bml:

vhtspeechrecorder start
120528396 acquireSpeech startSession
120528396 acquireSpeech set mic_utteranceFile.enabled true
120528396 nvbg_set_option disable_nvbg true
120528396 render_text_overlay disable
120528494 vhtspeechrecorder wizard_text I am having a great time
120528495 acquireSpeech startUtterance mic
120528503 vrSpeech start user0001 user
120528503 acquireSpeech startedListening mic 20130906115616,033 user0001 1378494328500
120528522 vrSpeech partial user0001 1 1.0 normal
120528665 vrSpeech partial user0001 2 1.0 normal WHAT
120528799 TO_ASR <start id="temp">
120528799 TO_ASR <stop id="temp">
120528800 TO_ASR <start id="temp">
120528800 TO_ASR <stop id="temp">
120528808 TO_ASR <start id="temp">
120528859 vrSpeech partial user0001 3 1.0 normal WHAT
120529103 vrSpeech partial user0001 4 1.0 normal DON'T
120529301 vrSpeech partial user0001 5 1.0 normal DON'T WHAT
120529496 vrSpeech partial user0001 6 1.0 normal DON'T WHAT IS
120529691 vrSpeech partial user0001 7 1.0 normal DON'T WHAT IS THE
120529706 acquireSpeech stopUtterance mic
120529707 acquireSpeech stopSession
120529707 acquireSpeech set mic_utteranceFile.enabled false
120529710 vrSpeech finished-speaking user0001
120529710 acquireSpeech stoppedListening mic 20130906115616,033 user0001 1378494329709
120529711 acquireSpeech stoppedSession null 20130906115616,033 1378494329711
120529885 vrSpeech partial user0001 8 1.0 normal DON'T WHAT IS THE
120529983 vrSpeech partial user0001 9 1.0 normal DON'T WHAT IS THE
120529995 vrSpeech interp user0001 1 1.0 normal DON'T WHAT IS THE
120529995 vrSpeech asr-complete user0001
120530048 TO_ASR <stop id="temp">
120535424 VHBuilder audiofile_created utterance_20130906120529_DON'TWHATISTHE
120535427 vhtspeechrecorder stop
120535427 render_text_overlay enable
120535427 NPCEditor <script target="user">document.getModel().getAnswers().getUtterances().get(document.getModel().getAnswers().findIndexOfWithID("utterance_20130206114108_IAMHAVINGAGREATTO")).setID("utterance_20130906120529_DON'TWHATISTHE");</script>
120535428 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/ExampleScenario_dialog.plist").toURI().toURL();document.saveToURLOfTypeForSaveOperation(url, document.getApplication().fileTypeForURL(url), com.leuski.af.Document.SaveOperation.kSaveAs);</script>
120535428 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/QNA.xml").toURI().toURL();document.writeToURL(url, new edu.usc.ict.npc.editor.io.ClassifierJAXBFileType());</script>
120535575 nvbg_set_option disable_nvbg false
120535575 acquireSpeech startSession
120535585 acquireSpeech startedSession null 20130906120535,575 1378494335585
120535675 acquireSpeech startUtterance mic
120535679 vrSpeech start user0002 user
120535679 acquireSpeech startedListening mic 20130906120535,575 user0002 1378494335678
120535717 vrSpeech partial user0002 1 1.0 normal

Messages sent out on clicking Apply button

121025076 NPCEditor <script target="user">document.startTrainingAll();</script>
121025076 nvbg_create_character Rachel
121025077 nvbg_set_option Rachel rule_input_file rule_input_ChrRachel_template.xml
121025077 nvbg_set_option Rachel nvbg_POS_rules false
121025077 nvbg_set_option Rachel posture ChrBrad@Idle01
121025077 nvbg_set_option refresh_transform
121025077 sbm bml char Rachel posture ChrBrad@Idle01
121025077 nvbg_create_character Brad
121025078 nvbg_set_option Brad rule_input_file rule_input_ChrBrad_template.xml
121025078 nvbg_set_option Brad nvbg_POS_rules false
121025078 nvbg_set_option Brad posture ChrBrad@Idle01
121025078 nvbg_set_option refresh_transform
121025078 sbm bml char Brad posture ChrBrad@Idle01
121025078 acquireSpeech stopSession
121025078 vrKillComponent asr-server
121025080 vrProcEnd asr-server
121025081 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/ExampleScenario_dialog.plist").toURI().toURL();document.saveToURLOfTypeForSaveOperation(url, document.getApplication().fileTypeForURL(url), com.leuski.af.Document.SaveOperation.kSaveAs);</script>
121025081 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/QNA.xml").toURI().toURL();document.writeToURL(url, new edu.usc.ict.npc.editor.io.ClassifierJAXBFileType());</script>

Message sent out on posture change

sbm bml char Rachel posture ChrHarmony@IdleCrossedArms01

Message sent out on animation double click

sbm bml char Rachel anim ChrBrad@Idle01_BeatHighBt01

Message sent out on adding question

NPCEditor <script target="user">edu.usc.ict.npc.editor.model.Person domain = document.getModel().getDefaultSpeaker(); edu.usc.ict.npc.editor.model.EditorUtterance eu = new edu.usc.ict.npc.editor.model.EditorUtterance("oooga booga", "1987503103635144181104057941", domain, new Date()); document.getManagedObjectContext().insertObject(eu); document.getModel().getQuestions().getUtterances().add(eu);</script>
135510407 NPCEditor <script target="user">URL url = new

Message sent out on adding answer

NPCEditor <script target="user">edu.usc.ict.npc.editor.model.Person domain1 = document.getModel().getSpeakers().get(0); edu.usc.ict.npc.editor.model.EditorUtterance eu = new edu.usc.ict.npc.editor.model.EditorUtterance("ooga booga", "48309906635144181677535283", domain1, new Date());eu.addAnnotation(document.getModel().getCategoryWithID("speaker").tokenWithID("Rachel")); document.getManagedObjectContext().insertObject(eu); document.getModel().getAnswers().getUtterances().add(eu);</script>

Known Issues

• All answers within a given scenario need to be unique, regardless of whether different characters speak them.

FAQ

See the Main FAQ. Please use the Google Groups emailing list for unlisted questions.