VHToolkit

Home

Page tree

Versions Compared

Old Version 16

changes.mady.by.user Unknown User (ravikumar)

Saved on

compared with

New Version Current

changes.mady.by.user Arno Hartholt

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

You can add new lines by typing them into the text box on the utterance list and hitting enter.

For each line you can there are three ways of getting the character to speak the line:

  • Use text-to-speech. This is the default and you don't need to do anything for this. You can however change the voice from the Voice pull down menu in the Settings tab, and clicking the Set button

  • Record your own voice. Select the appropriate line. Hold down the Record button while speaking the line. Make sure toUse a prerecorded audio wave files. The spoken text in this file needs to match the text depicted in the line exactly. to:

    • Start speaking immediately after pushing the button
    • Hold the button continuously while speaking
    • Release immediately after done speaking
    • Read the line accurately, since both the text and audio is used for creating a lip synching schedulea lip syncing schedule
    • Note that a window pops up indicating automatic analyzes of the text and audio to generate a lip syncing schedule; you can change the method from Lipsync under the Options menu.
  • Use an existing audio file:
    • Select the appropriate line; ensure that the text in this line exactly matches the text in the audio
    • Click the Add button in the Audio section
    • Select the file and click OK
    • Note that a window pops up indicating automatic analyzes of the text and audio to generate a lip syncing schedule; you can change the method from Lipsync under the Options menu.

Play a line

Either double click the line or click the Play button in the Line panel.

...

Change Background Color

renderer color 0 255 255

 

Add audio file

vhtspeechrecorder wizard_text Hello, I am a virtual human.
vhtspeechrecorder processaudiofile C:\Build4272-08-06-2013-ci\vhtoolkit\core\vhtoolkitUnity\Assets\StreamingAssets\Sounds\brad_askmeabout.wav Hello, I am a virtual human.
VHBuilder audiofile_created utterance_20130906121840_Iamavirtualhuman.
vhtspeechrecorder stop
render_text_overlay enable
nvbg_set_option disable_nvbg false

 

 

Flow of messages sent in order to record speech and generate audio file and bml:

vhtspeechrecorder start
120528396 acquireSpeech startSession
120528396 acquireSpeech set mic_utteranceFile.enabled true
120528396 nvbg_set_option disable_nvbg true
120528396 render_text_overlay disable
120528494 vhtspeechrecorder wizard_text I am having a great time
120528495 acquireSpeech startUtterance mic
120528503 vrSpeech start user0001 user
120528503 acquireSpeech startedListening mic 20130906115616,033 user0001 1378494328500
120528522 vrSpeech partial user0001 1 1.0 normal
120528665 vrSpeech partial user0001 2 1.0 normal WHAT
120528799 TO_ASR <start id="temp">
120528799 TO_ASR <stop id="temp">
120528800 TO_ASR <start id="temp">
120528800 TO_ASR <stop id="temp">
120528808 TO_ASR <start id="temp">
120528859 vrSpeech partial user0001 3 1.0 normal WHAT
120529103 vrSpeech partial user0001 4 1.0 normal DON'T
120529301 vrSpeech partial user0001 5 1.0 normal DON'T WHAT
120529496 vrSpeech partial user0001 6 1.0 normal DON'T WHAT IS
120529691 vrSpeech partial user0001 7 1.0 normal DON'T WHAT IS THE
120529706 acquireSpeech stopUtterance mic
120529707 acquireSpeech stopSession
120529707 acquireSpeech set mic_utteranceFile.enabled false
120529710 vrSpeech finished-speaking user0001
120529710 acquireSpeech stoppedListening mic 20130906115616,033 user0001 1378494329709
120529711 acquireSpeech stoppedSession null 20130906115616,033 1378494329711
120529885 vrSpeech partial user0001 8 1.0 normal DON'T WHAT IS THE
120529983 vrSpeech partial user0001 9 1.0 normal DON'T WHAT IS THE
120529995 vrSpeech interp user0001 1 1.0 normal DON'T WHAT IS THE
120529995 vrSpeech asr-complete user0001
120530048 TO_ASR <stop id="temp">
120535424 VHBuilder audiofile_created utterance_20130906120529_DON'TWHATISTHE
120535427 vhtspeechrecorder stop
120535427 render_text_overlay enable
120535427 NPCEditor <script target="user">document.getModel().getAnswers().getUtterances().get(document.getModel().getAnswers().findIndexOfWithID("utterance_20130206114108_IAMHAVINGAGREATTO")).setID("utterance_20130906120529_DON'TWHATISTHE");</script>
120535428 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/ExampleScenario_dialog.plist").toURI().toURL();document.saveToURLOfTypeForSaveOperation(url, document.getApplication().fileTypeForURL(url), com.leuski.af.Document.SaveOperation.kSaveAs);</script>
120535428 NPCEditor <script target="user">URL url = new File("C:/VHToolkit/data/VHBuilder/Scenarios/ExampleScenario/QNA.xml").toURI().toURL();document.writeToURL(url, new edu.usc.ict.npc.editor.io.ClassifierJAXBFileType());</script>
120535575 nvbg_set_option disable_nvbg false
120535575 acquireSpeech startSession
120535585 acquireSpeech startedSession null 20130906120535,575 1378494335585
120535675 acquireSpeech startUtterance mic
120535679 vrSpeech start user0002 user
120535679 acquireSpeech startedListening mic 20130906120535,575 user0002 1378494335678
120535717 vrSpeech partial user0002 1 1.0 normal

 

 

Play Utterance

vrExpress Brad all 1623190679 <?xml version="1.0" encoding="UTF-8" standalone="no" ?><act><participant id="Brad" role="actor" /><fml><turn start="take" end="give" /><affect type="neutral" target="addressee"></affect><culture type="neutral"></culture><personality type="neutral"></personality></fml><bml><speech id="sp1" ref="utterance_20130906121840_Iamavirtualhuman." type="application/ssml+xml">Hello, I am a virtual human.</speech></bml></act>

 

Record start/stop

renderer_record start
renderer_record stop

 

Set Camera

renderer setcamera set Camera01_mediumCt

Gaze at camera

sbm bml char Brad <gaze target="Camera01_mediumCt" sbm:joint-range="HEAD EYES NECK" sbm:joint-speed="500"/>')

Set Voice

sbm python scene.getCharacter("Brad").setVoiceBackupCode("Festival_voice_cmu_us_jmk_arctic_clunits")

 

Set Facial Expression

sbm char * viseme au_1_left 0 0
sbm char * viseme au_1_right 0 0
sbm char * viseme au_2_left 0 0
sbm char * viseme au_2_right 0 0
sbm char * viseme au_4_left 0 0
sbm char * viseme au_4_right 0 0
sbm char * viseme au_5 0 0
sbm char * viseme au_6 0 0
sbm char * viseme au_7 0 0
sbm char * viseme au_10 0 0
sbm char * viseme au_12_left 0 0
sbm char * viseme au_12_right 0 0
sbm char * viseme au_26 0 0
sbm char * viseme au_112 0 0
sbm char * viseme au_130 0 0
sbm char * viseme au_124 0 0
sbm char * viseme au_129 0 0
sbm char * viseme au_136 0 0
sbm char * viseme au_103 0 0
sbm char * viseme au_102 0 0
sbm char * viseme au_101 0 0
sbm char * viseme au_100 0 0
sbm char * viseme au_126 0 0
sbm char * viseme au_127 0 0
sbm char * viseme au_131 0 0
sbm char * viseme au_132 0 0
sbm char * viseme au_133 0 0
sbm char * viseme au_134 0 0
sbm char * viseme au_112 1 0.2


Generate Gestures checkbox

nvbg_set_option Brad speaker_gesture false

nvbg_set_option Brad speaker_gesture true

 

Generate Facial expressions/head movement checkbox

nvbg_set_option Brad nvbg_POS_rules false

nvbg_set_option Brad nvbg_POS_rules true

 

Toggle subtitles

render_text_overlay disable

render_text_overlay enable


Toggle Saccades

sbm bml char Brad <saccade finish="true"/>

sbm bml char Brad <saccade mode="listen"/>

sbm bml char Brad <saccade mode="talk"/>

sbm bml char Brad <saccade mode="think"/>



FAQ

See Main FAQ for frequently asked questions regarding the installer.  Please use the Google Groups emailing list for unlisted questions.