Page History
Overview
The purpose of the virtual human authoring VHBuilder tool is to be a complete UI interface for authoring characters easily, editing behaviors and adding/recording dialog.
...
Creating a new scenario
- Launch the vhBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select 'Load Scenario' as shown
3) A dialog box will pop up as shown below prompting you to select an existing scenario. Select a previously saved scenario folder to load up an existing scenario. e.g. In the toolkit trunk, under tools/vhBuilder, there is an existing scenario called ExampleScenario. Please select this as shown
4) On selecting the example sceanario and clicking 'OK', you should see a few windows show up and minimize. This should include the NVBG, NPCEditor, VHTSpeechRecorder, AcquireSpeech, ElvinParser, PocketSphinx-sonic-server and TTSRelay.
During this interval, the tool should indicate that it is busy loading. It does this using a 'wait' image on the top right as shown below
5) Once the scene is loaded and all the components are initialized, the tool will indicate that it is ready as shown below.
6) At this point, the scene has been loaded and you can begin interaction with the characters. Note that currently the tool does not load up the characters in Unity automatically (yet to be implemented). So you will have to open up an empty scene in the Unity editor (e.g. CampusEmpty) and drag the listed characters in the scene and hit play in Unity. The characters that are in the scene can be seen in the combo-box as shown below
7) Once Unity loads up with the characters, then you can begin interacting with the characters just as you always do. The characters will respond to the questions with the answers as they are linked.
Creating a new scenario
The VHBuilder application requires the user to either create or load a scenario before authoring the virtual human characters. A scenario is basically a sandbox which includes all the files and data associated with it..
- Launch the VHBuilder Launch the vhBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select the 'Create Scenario' option as shown below
2) On selecting the 'Create Scenario' option, you will be presented with a dialog as shown below. Please type in the name of the new Scenario you want to create, select where you want to create it and select 'Save' as shown. On hitting 'Save' above, it will create a scenario , which is essentially a folder containing all relevant files, and it will also launch all the components required to create a new scenario.the new scenario.
On creating the scenario, the VHBuilder tool will launch all required components and while it is doing so, it will indicate that background processes are running, using the indicator on the top right (as shown below)
Once it is done loading the scenario, it will indicate that it is ready, again using the indicator on the top right as shown below.
3) You cannot begin editing the scenario unless you add new characters to the scene. In order to do this, you have to open up the Unity editor, open to type in the name of the characters you want to add to the scene using the text-box on the top left.
Add as many characters as you want. These characters will get added to the character combo box as shown below.
Another way to add the characters is by querying the Unity editor. Open up and empty scene, drag in the characters you want in the scenario and hit play. Once this is done, you have to let the vhBuilder tool know which characters are loaded in Unity. This can be done using the can hit 'Refresh' button on the top left as shown below.
4) On hitting the 'Refresh' button, you will see the character names in the drop-down box as shown below. Also the question and answer sections will have just a single question and answer linked to each other as an example.
This will query the characters currently loaded up in Unity and add them to the list.
5) As indicated by the 'red' button on the bottom lefttop, you now need to hit 'Apply' in order to have the scenario setup. On doing this, the changes applied. The tool will create a language model, classify the NPCEditor and initialize NVBG with the current scenario. On doing this, the new scenario is set up. At this point you can communicate with the characters, but of course they still don't have any answers or questions to respond to. Editing the scenario is covered in the next section.
Editing a scenario
...
1) In order to add a question/answer, first select the character that you want to add questions/answers for, from the character drop-down menu on the top left. Then type the question/answer in the respective text-box and hit enter or the 'Add' button as shown below. This will add the question/answer.
NOTE: The newly added answer is spoken by the character that is currently selected in the CharacterCombo box on the top left. e.g. in the image below, 'Rachel' is the currently selected character, so any answer that is entered will be spoken by her.
2) Once you have added the question and answer, you can link them together by selecting them and clicking the 'Link' button as shown below. You can select multiple questions and answers at the same time and link them as shown.
3) Once you are done adding and linking questions/answers, hit the apply button on the bottom left top as indicated by it's red color. This will apply your changes in NPCEditor.
Recording the audio for an answer
1) In order to record audio for a newly added answer, first select the answer by clicking on it.This will highlight the answer. Now hold down the 'Record' button and speak into the microphone and release the button when you are done speaking.
2) On releasing the 'Record' button, you will see a window popup as shown below. This generates the bml with the lipsynch lipsync for the audio you just recorded. Please wait until the tool indicates otherwise. Once the tool indicates it is done, the audio has been recorded and the lipsynch lipsync generated. You should now be able to ask the corresponding question and hear the character respond in the recorded voice.
Modifying the character behavior
...
3) In order to modify a rule, you select the 'Rule Type' and the posture that you want the rule to apply to. Now you can add words that will trigger this rule under the 'Words' section by typing the word in the text-box and hitting the enter key or the 'Add' button. You are also able to select the animations that will be triggered when the character speaks that particular word, by clicking on the animation name.
Notes:
a) The character only performs the animation when the word is spoken and he is in the specified posture.
b) You can select multiple animations by holding down the 'ctrl' key and clicking on them.
c) If you have Unity running with the characters loaded, you can double click on the animations and see how they play on the character.
d) At any time, you can hit 'R' on the 'Animations' box and it will reset to the original selected animations.
3) Once you are done adding the words and selecting the animations,
...
On doing this, the character behavior has been applied. Now when the character speaks that particular word, he will perform one of the selected animations.
Loading an existing scenario
- Launch the vhBuilder application. It will load up with a message ' Please Create/Load a scenario' as shown below
2) Click on the File menu option on the top left and select 'Load Scenario' as shown
3) A dialog box will pop up as shown below prompting you to select an existing scenario. Select a previously saved scenario folder to load up an existing scenario. e.g. In the toolkit trunk, under tools/vhBuilder, there is an existing scenario called ExampleScenario. Please select this as shown
4) On selecting the example sceanario and clicking 'OK', you should see a few windows show up and minimize. This should include the NVBG, NPCEditor, VHTSpeechRecorder, AcquireSpeech, ElvinParser, PocketSphinx-sonic-server and TTSRelay.
During this interval, the tool should indicate that it is busy loading. It does this using a 'wait' image on the top right as shown below
5) Once the scene is loaded and all the components are initialized, the tool will indicate that it is ready as shown below.
6) At this point, the scene has been loaded and you can begin interaction with the characters. Note that currently the tool does not load up the characters in Unity automatically (yet to be implemented). So you will have to open up an empty scene in the Unity editor (e.g. CampusEmpty) and drag the listed characters in the scene and hit play in Unity. The characters that are in the scene can be seen in the combo-box as shown below
7) Once Unity loads up with the characters, then you can begin interacting with the characters just as you always do. The characters will respond to the questions with the answers as they are linked.
Saving the scenario
1) In order to save the scenario and all the changes made to it, select the File menu option and hit the 'Save Scenario' as shown below
...