VHToolkit

Home

Page tree

Versions Compared

Old Version 4

changes.mady.by.user Unknown User (ravikumar)

Saved on

compared with

New Version 5

changes.mady.by.user Unknown User (ravikumar)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This message is sent to NVBG by the NLU, NPCEditor or similar module. This message can be used to convey information about speech data, posture, status change, emotion change, gaze data etc. as shown below.


Speech

The speech messages are characterized by the speech tag within them. They are interpreted and the corresponding output bml is generated with the speech time marks, animations, head-nods, facial-movements etc. These animations are generated based on the content of the speech tag and the fml tag in the input message.


 vrExpress "harmony" "ranger" "harmony221" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="harmony" role="actor"/>
<fml>
<intention>
<object name="A316">
<attribute name="addressee">ranger</attribute>
<attribute name="speech-act">
<object name="A317">
<attribute name="content">
<object name="V28">
<attribute name="modality">
<object name="V29">
<attribute name="conditional">should</attribute>
</object>
</attribute>
<attribute name="polarity">negative</attribute>
<attribute name="attribute">jobAttribute</attribute>
<attribute name="value">bartender-job</attribute>
<attribute name="object-id">utah</attribute>
<attribute name="type">state</attribute>
<attribute name="time">present</attribute>
</object>
</attribute>
<attribute name="motivation">
<object name="V27">
<attribute name="reason">become-sheriff-harmony</attribute>
<attribute name="goal">address-problem</attribute>
</object>
</attribute>
<attribute name="addressee">ranger</attribute>
<attribute name="action">assert</attribute>
<attribute name="actor">harmony</attribute>
</object>
</attribute>
</object>
</intention>
</fml>
<bml>
<speech id="sp1" type="application/ssml+xml">ranger utah cant be bartender if he becomes sheriff</speech>
</bml>
</act>"



vrExpress "utah" "ranger" "utah200" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="utah" role="actor"/>
<fml>
<intention>
<object name="A131">
<attribute name="addressee">ranger</attribute>
<attribute name="speech-act">
<object name="A133">
<attribute name="reason">become-sheriff</attribute>
<attribute name="content">
<object name="V6">
<attribute name="content">
<object name="V7">
<attribute name="q-slot">polarity</attribute>
<attribute name="type">question</attribute>
<attribute name="prop">
<object name="P24">
<attribute name="type">event</attribute>
<attribute name="location">town</attribute>
<attribute name="theme">sheriff-job</attribute>
<attribute name="event">providePublicServices</attribute>
<attribute name="agent">utah</attribute>
<attribute name="time">future</attribute>
</object>
</attribute>
</object>
</attribute>
<attribute name="type">csa</attribute>
<attribute name="action">info-req</attribute>
<attribute name="actor">ranger</attribute>
<attribute name="addressee">utah</attribute>
</object>
</attribute>
<attribute name="motivation">
<object name="V5">
<attribute name="reason">become-sheriff</attribute>
<attribute name="goal">avoid</attribute>
</object>
</attribute>
<attribute name="action">avoid</attribute>
<attribute name="type">backward</attribute>
<attribute name="addressee">ranger</attribute>
<attribute name="actor">utah</attribute>
</object>
</attribute>
</object>
</intention>
</fml>
<bml>
<speech id="sp1" type="application/ssml+xml">im not sure about that</speech>
</bml>
</act>"



Posture change

These messages are characterized by the <body posture=""> tag which allows NVBG to know that there has been a change in posture.

vrExpress "harmony" "None" "??" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="harmony" role="actor" />
<bml>
<body posture="HandsAtSide" />
</bml>
</act>"


Status / request

The idle_behavior and all_behavior attributes within the request tag allows NVBG to keep track of whether or not to generate the corresponding behavior.

vrExpress "harmony" "None" "??" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="harmony" role="actor" />
<fml>
<status type="present" />
<request type="idle_behavior" value="off" />
</fml>
</act>"


Gaze


These gaze tags, if present within the input message are transferred unaltered to the output message.

vrExpress "harmony" "ranger" "constant103" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="harmony" role="actor" />
<fml>
<gaze type="weak-focus" target="ranger" track="1" speed="normal" > "listen_to_speaker" </gaze>
</fml>
</act>"


vrExpress "utah" "ranger" "gaze199" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="utah" role="actor" />
<fml>
<gaze type="avert" target="ranger" track="eyes-offset" speed="slower" > "planning_speech_hold_turn" </gaze>
</fml>
</act>"



Emotion

The affect tag contains data about the emotional state the character is currently in. This can be used to affect output behavior.

vrExpress "harmony" "None" "schererharmony17" "<?xml version="1.0" encoding="UTF-8" standalone="no" ?><act>
<participant id="harmony" role="actor" />
<fml>
<affect type="Fear" STANCE="LEAKED" intensity="110.475"></affect>
</fml>
<bml> </bml>
</act>"




 

 

nvbg_set_option

These are control messages to set options for NVBG. They are as shown below:

nvbg_set_option [char-name] all_behavior true/false - sets/unsets flag that allows all behavior generated by NVBG.

nvbg_set_option [char-name] saliency_glance true/false - sets/unsets flag that allows saliency map generated gazes. These gazes are based on content in the speech tag and the information in the saliency map.

nvbg_set_option [char-name] saliency_idle_gaze true/false - sets/unsets flag that allows idle gazes generated by the saliency map. These idle gazes are based on the priority of pawns in the saliency map and are generated when the character is idle.

nvbg_set_option [char-name] speaker_gaze true/false - sets/unsets flag that allows for the character to look at the person he's speaking to.

nvbg_set_option [char-name] speaker_gesture true/false - sets/unsets flag that allows gestures to be generated when speaking.

nvbg_set_option [char-name] listener_gaze true/false - sets/unsets flag that allows the listener to gaze at the speaker when he speaks.

nvbg_set_option [char-name] nvbg_POS_rules true/false - sets/unsets flag that allows behavior to be generated based on parts of speech returned by the parser.


Output

vrSpeak

Below is an example of the vrSpeak message that is output from NVBG. It contains animation/head-nods/facial behavior based on the text that is to be spoken/heard.

...