Overview
The purpose of the VHBuilder tool is to be a complete UI interface for authoring characters easily, editing behaviors and adding/recording dialog.
Creating a new scenario
The VHBuilder application requires the user to either create or load a scenario before authoring the virtual human characters. A scenario is basically a sandbox which includes all the files and data associated with it..
- Launch the VHBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select the 'Create Scenario' option as shown below
2) On selecting the 'Create Scenario' option, you will be presented with a dialog as shown below. Please type in the name of the new Scenario you want to create, select where you want to create it and select 'Save' as shown. On hitting 'Save' above, it will create a scenario and it will also launch all the components required to create the new scenario.
On creating the scenario, the VHBuilder tool will launch all required components and while it is doing so, it will indicate that background processes are running, using the indicator on the top right (as shown below)
Once it is done loading the scenario, it will indicate that it is ready, again using the indicator on the top right as shown below.
3) You cannot begin editing the scenario unless you add new characters to the scene. In order to do this, you have to type in the name of the characters you want to add to the scene using the text-box on the top left.
Add as many characters as you want. These characters will get added to the character combo box as shown below.
Another way to add the characters is by querying the Unity editor. Open up a new toolkit scene (e.g. CampusEmpty), drag in the characters you want in the scenario and hit play. Note that this is a standard toolkit scene, which should have it's default objects like the SmartbodyManager, VHMsgManager etc. These are needed for a smartbody character to function. Once this is done, you can hit 'Refresh' button on the top left as shown below. This will query the characters currently loaded up in Unity and add them to the list.
5) As indicated by the 'red' button on the top, you now need to hit 'Apply' in order to have changes applied. The tool will create a language model, classify the NPCEditor and initialize NVBG with the current scenario. At this point you can communicate with the characters, but of course they still don't have any answers or questions to respond to. Editing the scenario is covered in the next section.
Editing a scenario
Once you have loaded a scenario or created a new one, you can modify the character dialogues and their behavior using the tool directly. This can be done as shown below
Adding a question and answer and linking them
1) In order to add a question/answer, first select the character that you want to add questions/answers for, from the character drop-down menu on the top left. Then type the question/answer in the respective text-box and hit enter. This will add the question/answer.
NOTE: The newly added answer is spoken by the character that is currently selected in the CharacterCombo box on the top left. e.g. in the image below, 'Rachel' is the currently selected character, so any answer that is entered will be spoken by her.
2) Once you have added the question and answer, you can link them together by selecting them and clicking the 'Link' button as shown below. You can select multiple questions and answers at the same time and link them as shown.
3) Once you are done adding and linking questions/answers, hit the apply button on the top as indicated by it's red color. This will apply your changes.
Recording the audio for an answer
1) In order to record audio for a newly added answer, first select the answer by clicking on it.This will highlight the answer. Now hold down the 'Record' button and speak into the microphone and release the button when you are done speaking.
2) On releasing the 'Record' button, you will see a window popup. This generates the bml with the lipsync for the audio you just recorded. Please wait until the tool indicates otherwise. Once the tool indicates it is done, the audio has been recorded and the lipsync generated. You should now be able to ask the corresponding question and hear the character respond in the recorded voice.
Modifying the character behavior
1) In order to modify the character behavior, please select the 'Non-Verbal' tab as shown below.
2) For each character, you can select the 'behavior file' that you want to use for that character. e.g. in the example below, we have selected 'rule_input_ChrRachel_template.xml' for the selected character 'Rachel'. Once you select the behavior file for a character, the tool will remember it.
3) In order to modify a rule, you select the 'Rule' and the 'Posture' that you want the rule to apply to. Now you can add words that will trigger this rule under the 'Words' section by typing the word in the text-box and hitting the enter key. You are also able to select the animations that will be triggered when the character speaks that particular word, by clicking on the animation name and hitting enter. The list of selected animations for a particular rule is displayed in the bottom right as shown.
Notes:
a) The character only performs the animation when the words are spoken and he is in the specified posture.
b) You can select multiple animations by holding down the 'ctrl' key and clicking on them.
c) If you have Unity running with the characters loaded, you can double click on the animations (or select a posture) and see how they play on the character.
d) You can select and hit delete in the 'Selected Animations' list to remove animations.
3) Once you are done adding the words and selecting the animations,
a) Hit the 'Save Changes To Behavior' button as indicated below. This will save the changes you made to the behavior file.
b) Then hit 'Apply'. This will configure NVBG.
On doing this, the character behavior has been applied. Now when the character speaks that particular word, he will perform one of the selected animations.
4) You can also add a rule by entering the rule name in the text-box as shown below. This will create a new rule and you can go ahead and add words to this rule and select animations for it. Rule names are mostly for organizational purposes allowing you to have different rule names for different set of words.
Loading an existing scenario
- Launch the VHBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select 'Load Scenario' as shown
3) A dialog box will pop up as shown below prompting you to select an existing scenario. Select a previously saved scenario folder to load up an existing scenario. e.g. In the toolkit trunk, under data/VHBuilder, there is an existing scenario called ExampleScenario. Please select this as shown
4) On selecting the example scenario and clicking 'OK', you should see a few windows show up and minimize. This should include the NVBG, NPCEditor, VHTSpeechRecorder, AcquireSpeech, ElvinParser, PocketSphinx-sonic-server and TTSRelay.
During this interval, the tool should indicate that it is busy loading. It does this using a 'wait' image on the top right as shown below
5) Once the scene is loaded and all the components are initialized, the tool will indicate that it is ready as shown below.
6) At this point, the scene has been loaded and you can begin interaction with the characters. Note that currently the tool does not load up the characters in Unity automatically (yet to be implemented). So you will have to open up an empty scene in the Unity editor (e.g. CampusEmpty) and drag the listed characters in the scene and hit play in Unity. The characters that are in the scene can be seen in the combo-box as shown below
7) Once Unity loads up with the characters, then you can begin interacting with the characters just as you always do. The characters will respond to the questions with the answers as they are linked.
Saving the scenario
1) In order to save the scenario and all the changes made to it, select the File menu option and hit the 'Save Scenario' as shown below
