VHToolkit

Home

Page tree

Versions Compared

Old Version 24

changes.mady.by.user Edward Fast

Saved on

compared with

New Version Current

changes.mady.by.user Arno Hartholt

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Overview

vhToolkitUnity is the Unity3D Unity rendered version of the Virtual Human Toolkit.  This project integrates several research systems together in order for the user to interact with a virtual human character through usage of text to speech and language understanding. Some of the systems that are integrated into his project are the Smartbody animation system, text to speech engines, and AcquireSpeech.  Unity is a 3rd party proprietary rendering engine and is provided as binary-only with the Virtual Human Toolkit. You can download the free version of Unity here. For changing art assets you need to purchase Unity Pro. vhtoolkitUnity was created using the vhAssets package as a starting point.

...

  • W,A,S,D - camera movement. Q & E - camera up/down
  • J - mouse visibility toggle - mouse look mode
  • L - toggles the fake recognizer text box
  • O - toggles the user's recognized text
  • M - toggles speech recognition mode. When on, click and hold and talk in the mic. Release to quit talking.
  • X - reset camera
  • Z - show debug statistics
  • I - Toggles sbm character subtitles
  • P - Toggle entire GUI
  • Alt-enter - toggle windowed / fullscreen
  • Escape - quit

Accessing the Command Console

Use the ~ key to bring up the console.

See Using the Console

Developers

Loading VHToolkit Scene

  1. Open Unity
  2. File->Open Project (this may already be done for you if this is the first time you've launched Unity)
  3. On the Open Project tab, click the Open Other button.
  4. Navigate to the VHToolkit Unity Project here: <VHToolkit>\core\vhtoolkitUnity
  5. Click Select Folder
  6. In the Project column, double click on vhtoolkitScene.
  7. Click Play

Creating your own Project using the VHToolkit Unity Package

  1. Open Unity
  2. File->New Project
  3. You need 2 Unity Packages to start with.  You'll find all packages here: <VHToolkit>\bin\vhtoolkitUnity.  Copy them to <your project>\Assets. 
  4. First, is the vhAssets package.  This contains common scripts and libraries used in various Virtual Humans applications.  There is a different .unitypackage file for each platform.  
    1. vhAssetsPackage.unityPackage    (Windows)
    2. vhAssetsPackageOSX.unityPackage
    3. vhAssetsPackageIOS.unityPackage
    4. vhAssetsPackageAndroid.unityPackage
  5. Second, is the vhAssets Test Scene.  This contains assets and scripts that will initialize a very basic test scene environment.  This file is:  vhAssetsTestScenePackage.unityPackage
  6. Copy the 2 Unity Packages to this new project.
  7. In Unity, in the 'Project' column, double click on the 2 Unity Packages.
  8. Open Scenes->vhAssetsTestScene
  9. Click Play.  The virtual human should be animating in an idle pose.  Hit 'C' to show a debug menu.  You should be able to control him via the buttons on the debug menu.

 

These instructions are for both Unity Pro and Unity Free. When there are specific instructions for a particular version, they have been marked as such.

There are differences because Unity Pro uses an .dll version of smartbody, and runs within the process. For Unity Free, Smartbody is launched as a separate process and communicates to Unity via network traffic.

Steps:

...

...

 

 

 

  1. Now we are ready to test
    1. In Unity, hit Play button
    2. (Unity Free) Launch Smartbody as a separate process. Run <VHToolkit>\tools\launch-scripts\run-toolkit-sbm.bat
    3. Brad should appear and animating with his idle animation.
    4. Try and play an animation
    5. Open the Unity debug console by hitting tilde (~)
    6. Type the following command: vhmsg sbm bml char brad anim HandsAtSide_Arms_Chop

Creating your own Virtual Human

 

Using the Console

Hit the ~ key to bring up the console. Here you can see some debug information. Type 'help' for all available commands.

...

  • q - quit
  • play_intro - Play intro sequence
  • vhmsg ... - Send vhmsg out to system. 'vhmsg sbm ...' for sending a smartbody command
  • setresolution x y - Set resolution to 'x' x 'y'. Example 'setresolution 1024 768'
  • toggle_fullscreen - toggle windowed / fullscreen

Message API

Receives:

Sends:

Known Issues

  • ...

FAQ

See Main FAQ for frequently asked questions regarding the installer. Please use the Google Groups emailing list for unlisted questions.