Overview
The purpose of the virtual human authoring tool is to be a complete UI interface for authoring characters easily, editing behaviors and adding/recording dialog.
Loading an existing scenario
- Launch the vhBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select 'Load Scenario' as shown
3) A dialog box will pop up as shown below prompting you to select an existing scenario. Select a previously saved scenario folder to load up an existing scenario. e.g. In the toolkit trunk, under tools/vhBuilder, there is an existing scenario called ExampleScenario. Please select this as shown
4) On selecting the example sceanario and clicking 'OK', you should see a few windows show up and minimize. This should include the NVBG, NPCEditor, VHTSpeechRecorder, AcquireSpeech, ElvinParser, PocketSphinx-sonic-server and TTSRelay.
During this interval, the tool should indicate that it is busy loading. It does this using a 'wait' image on the top right as shown below
5) Once the scene is loaded and all the components are initialized, the tool will indicate that it is ready as shown below.
6) At this point, the scene has been loaded and you can begin interaction with the characters. Note that currently the tool does not load up the characters in Unity automatically (yet to be implemented). So you will have to open up an empty scene in the Unity editor (e.g. CampusEmpty) and drag the listed characters in the scene and hit play in Unity. The characters that are in the scene can be seen in the combo-box as shown below
7) Once Unity loads up with the characters, then you can begin interacting with the characters just as you always do. The characters will respond to the questions with the answers as they are linked.
Creating a new scenario
- Launch the vhBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select the 'Create Scenario' option as shown below
2) On selecting the 'Create Scenario' option, you will be presented with a dialog as shown below. Please type in the name of the new Scenario you want to create, select where you want to create it and select 'Save' as shown. On hitting 'Save' above, it will create a scenario, which is essentially a folder containing all relevant files, and it will also launch all the components required to create a new scenario.
3) You cannot begin editing the scenario unless you add new characters to the scene. In order to do this, you have to open up the Unity editor, open up and empty scene, drag in the characters you want in the scenario and hit play. Once this is done, you have to let the vhBuilder tool know which characters are loaded in Unity. This can be done using the 'Refresh' button on the top left as shown below.
4) On hitting the 'Refresh' button, you will see the character names in the drop-down box as shown below. Also the question and answer sections will have just a single question and answer linked to each other as an example.
5) As indicated by the 'red' button on the bottom left, you now need to hit 'Apply' in order to have the scenario setup. On doing this, the tool will create a language model, classify the NPCEditor and initialize NVBG with the current scenario. On doing this, the new scenario is set up.
Editing a scenario
Once you have loaded a scenario or created a new one, you can modify the character dialogues and their behavior using the tool directly. This can be done as shown below
Adding a question and answer and linking them
1) In order to add a question/answer, first select the character that you want to add questions/answers for, from the character drop-down menu on the top left. Then type the question/answer in the respective text-box and hit enter or the 'Add' button as shown below. This will add the question/answer.
NOTE: The answer is spoken by the character that is currently selected in the CharacterCombo box on the top left. e.g. in the image below, 'Rachel' is the currently selected character, so any answer that is entered will be spoken by her.
2) Once you have added the question and answer, you can link them together by selecting them and clicking the 'Link' button as shown below. You can select multiple questions and answers at the same time and link them as shown.
3) Once you are done adding and linking questions/answers, hit the apply button on the bottom left as indicated by it's red color. This will apply your changes in NPCEditor.
Recording the audio for an answer
1) In order to record audio for a newly added answer, first select the answer by clicking on it. This will highlight the answer. Now hold down the 'Record' button and speak into the microphone and release the button when you are done speaking.
2) On releasing the 'Record' button, you will see a window popup as shown below. This generates the bml with the lipsynch for the audio you just recorded. Please wait until the tool indicates otherwise. Once the tool indicates it is done, the audio has been recorded and the lipsynch generated. You should now be able to ask the corresponding question and hear the character respond in the recorded voice.
Modifying the character behavior
1) In order to modify the character behavior, please select the 'Non-Verbal' tab as shown below.
2) For each character, you can select the 'behavior file' that you want to use for that character. e.g. in the example below, we have selected 'rule_input_ChrRachel.xml' for the selected character 'Rachel'. As you can see, on selecting a rule file, the Animations section gets filled with the animations available for that character. Once you select the behavior file for a character, the tool will remember it.
3) In order to modify a rule, you select the 'Rule Type' and the posture that you want the rule to apply to. Now you can add words that will trigger this rule under the 'Words' section by typing the word in the text-box and hitting the enter key or the 'Add' button. You are also able to select the animations that will be triggered when the character speaks that particular word, by clicking on the animation name.
Notes:
a) The character only performs the animation when the word is spoken and he is in the specified posture.
b) You can select multiple animations by holding down the 'ctrl' key and clicking on them.
c) If you have Unity running with the characters loaded, you can double click on the animations and see how they play on the character.
d) At any time, you can hit 'R' on the 'Animations' box and it will reset to the original selected animations.
3) Once you are done adding the words and selecting the animations,
a) Hit the 'Save Changes To Behavior' button as indicated below. This will save the changes you made to the behavior file.
b) Then hit 'Apply'. This will configure NVBG.
On doing this, the character behavior has been applied. Now when the character speaks that particular word, he will perform one of the selected animations.
Saving the scenario
1) In order to save the scenario and all the changes made to it, select the File menu option and hit the 'Save Scenario' as shown below
