...

Where can I get the Toolkit?

Please see the Download section on how to obtain the toolkit.

How do I

...

Please see the Installing the Toolkit tutorial.

How do I run the provided example scenario?

...

Some components, like SmartBody, Launcher, Logger and VHMsg, are open source, while others are binary-only. We might release the source code of these in the future, though, including NPCEditor, NVBG and MultiSense. SmartBody has its own SourceForge page which you can access here. See the License Agreement page for licensing information. 

...

The complete License Agreement and supporting documentation is found in the License section.

Is there a commercial version of the Toolkit?

...

The Toolkit as a whole is currently targeted for the Microsoft Windows platform only, in particular Windows 710, both 32 and 64 bit. Note that MultiSense works best on Windows 7 and is not supported on Windows 10. Some components are multi-platform, most notably SmartBody and the NPCEditor, as well as the Launcher and Logger. C# components should work on Mac OS when using Mono.  We do no currently support non-Windows platforms. Build instructions for other platforms can be found here. 

I know of this great I know of this great character at ICT that uses Toolkit technology. Can I get access to the data driving that character?

With the Toolkit we are aiming to provide a technical platform allowing researchers to more quickly create their own virtual humans. Whether project-specific agents will be made available to the public will be determined on a case by case basis, but will usually fall outside the scope of the Toolkit.

Installation Process

Why do some of the 3rd party installers not start?

Not all 3rd party software come with their own installers. Some might just copy over files to the correct location.

The installer tells me I should do something with the Registry; how do I do that?

If you installed the MS SAPI voices and you are running Windows XP 64 bit, some changes in the registry are required to be able to use the installed voices. You can do this by double clicking "Microsoft-Voices.reg" in "C:\vhtoolkit\scripts". You can check to see if the changes worked by going to the Speech settings in Windows (Start -> Control Panel -> Speech). Under "Voice Selection" you should have several options, including "Microsoft Mike".

Running the Example Scenario

Why can't I run the Launcher? It came up fine after the installation

It might be that the executable used to start the Launcher can not be found. Please make sure that javaw.exe is part of your path. The location is usually "C:\Program Files (x86)\Java\jre1.6.0_26\bin". The path is defined in your Environment Variables; right-click on My Computer, go to the Advanced tab and click Environment Variables. The Toolkit is currently only compatible with the 32 bit version of Java.

Why do I get many errors and are all of the Launcher rows orange?

Orange means that the Launcher has tried to launch a component, but hasn't received a confirmation message back indicating it is up and running. This is likely due to not running the ActiveMQ service. Please make sure it is running (Right click My Computer, select Manage, Services and Applications, Services). If it is not running, turn it on (the Play button at the top). If it is not present, please install using the Toolkit installer with 3rd party software.

Why won't the renderer load?

1. Be sure to install all 3rd party software that comes with the installer. 
2. If this doesn't work, make sure that you have the latest video drivers installed.
3. Make sure that your video card supports shader model 3 or higher. This is usually the case with newer dedicated video cards. Integrated video cards might not work.
4. See if the problem happens with both Unity and Ogre. If Ogre works in OpenGL mode, it might be a DirectX problem.
5. In rare cases, a virus scanner may block the Unity executable. Refer to the documentation of your virus scanner to see if the executable has been quarantined or deleted. Update your virus definitions.

Why don't I see Brad and Rachel, even though a renderer (Unity, Ogre) is running?

Is SmartBody running? You can see this by looking at the SmartBody row in the Launcher. If it's green, it is running. If it is orange, its status is unknown (usually, the Launcher tried running the component, but did not get confirmation it is actually up and running), or if the color is neutral it means it is not running. For Unity, SmartBody is a linked in library, so it should load automatically. For Ogre, make sure the renderer is up and running before you start SmartBody. Please also make sure the renderer is completely up and running before you start SmartBody. If you use the Run All button, you might have to kill SmartBody and launch it manually.

Why don't I hear Brad and Rachel?

See if the characters are moving their lips. If so, make sure your volume is not all the way down, that it is not muted, and that you know where sound should come from (i.e., desktop speakers, or headphones that have just been plugged in). If Brad is not moving his lips, make sure the Text To Speech module is running. If it is, look in its console window for any error messages that might give you an idea of what is going wrong. 

Why did Brad and Rachel stop reacting to me?

They might not have the correct answer for you, especially if you have tried asking the same question a couple of times. There might also be a delay in some of the modules. Try waiting a 30 seconds before trying again. If there is no response at all, make sure all modules are still green in the Launcher. If a module that is checked is neutral, try launching it. If one of the checked rows is orange, the module might have crashed. Find the associated window or process and quit it manually, then restart that module. If all checked rows are green, try to restart SmartBody, the NVBG and the NPCEditor. If this doesn't help, quit all modules and restart them. As a last resort, try rebooting your computer and start all modules. If this still doesn't resolve the issue, ask in the Google Group.

ActiveMQ

How can I check whether ActiveMQ is running?

Right-click on Computer in the start menu, select Manage, expand Services and Applications, click on Services and see if the ActiveMQ service is running.  If it is listed and not running, select it and click the Play icon above or right click and select Start. If it is not listed, use the Toolkit installer with 3rd party software and make sure the ActiveMQ box is selected. 

SmartBody

For all SmartBody related questions, see the dedicated SmartBody website. 

...

Running the Example Scenario

Why can't I run the Launcher? It came up fine after the installation

It might be that the executable used to start the Launcher can not be found. Please make sure that javaw.exe is part of your path. The location is usually "C:\Program Files (x86)\Java\jre1.6.0_26\bin". The path is defined in your Environment Variables; right-click on My Computer, go to the Advanced tab and click Environment Variables. The Toolkit is currently only compatible with the 32 bit version of Java.

Why do I get many errors and are all of the Launcher rows orange?

Orange means that the Launcher has tried to launch a component, but hasn't received a confirmation message back indicating it is up and running. This is likely due to not running the ActiveMQ service. Please make sure it is running (Right click My Computer, select Manage, Services and Applications, Services). If it is not running, turn it on (the Play button at the top). If it is not present, please install using the Toolkit installer with 3rd party software.

Why won't the renderer load?

1. Be sure to install all 3rd party software that comes with the installer. 
2. If this doesn't work, make sure that you have the latest video drivers installed.
3. Make sure that your video card supports shader model 3 or higher. This is usually the case with newer dedicated video cards. Integrated video cards might not work.
4. See if the problem happens with both Unity and Ogre. If Ogre works in OpenGL mode, it might be a DirectX problem.
5. In rare cases, a virus scanner may block the Unity executable. Refer to the documentation of your virus scanner to see if the executable has been quarantined or deleted. Update your virus definitions. The Unity executable is \vhtoolkit\bin\vhtoolkitUnity\vhtoolkitUnity.exe.

Why don't I hear Brad and Rachel?

See if the characters are moving their lips. If so, make sure your volume is not all the way down, that it is not muted, and that you know where sound should come from (i.e., desktop speakers, or headphones that have just been plugged in). If they are not moving their lips, make sure the NVGB, NPCEditor and speech recognition software are all running (check that their row is green in the Launcher). Try to identify where in the chain things break; when you talk, white text indicates the speech recognition result; yellow text indicates the characters are responding. You can look in the Unity console window (~ key) for any error messages that might give you an idea of what is going wrong. 

Why did Brad and Rachel stop reacting to me?

They might not have the correct answer for you, especially if you have tried asking the same question a couple of times. There might also be a delay in some of the modules. Try waiting a 30 seconds before trying again. If there is no response at all, make sure all modules are still green in the Launcher. If a module that is checked is neutral, try launching it. If one of the checked rows is orange, the module might have crashed. Restart them. If this doesn't help, quit all modules and restart them. As a last resort, try rebooting your computer and start all modules. 

ActiveMQ

How can I check whether ActiveMQ is running?

Right-click on Computer in the start menu, select Manage, expand Services and Applications, click on Services and see if the ActiveMQ service is running.  If it is listed and not running, select it and click the Play icon above or right click and select Start. If it is not listed, use the Toolkit installer with 3rd party software and make sure the ActiveMQ box is selected. 

SmartBody

For all SmartBody related questions, see the dedicated SmartBody website. 

NVBG

How do I add a new Character?

Edit the run-toolkit-NVBG-C#-all.bat by adding "-create_character <CharacterName> <CharacterFile>.ini" (without quotes).  The .ini files are found in data\nvbg-toolkit.  It's recommended to use Brad.ini or Rachel.ini to get started.

NPCEditor

How do I add a new Character?

1. Click on the Settings Tab
2. Click on Speaker in the Categories column on the left side of the application
3. Click the Add button under Tokens

I am asking my character a predefined question, but he doesn't respond. How can I solve that?

...

If you have installed previous versions of the toolkit and used it before on your system, then the NPCEditor would most likely have created user setting files on your machine. Once in a while, a change from our side to NPCEditor's settings will need to be propogated to the users, which will require them to delete these user files on their machines so that they don't interfere/override the new settings.

The NPCEditor creates .plist files which it stores in the user directory for Windows. These files contain NPCEditor specific information.
So if you need to delete these user settings from your machine, follow the below steps.

Depending on which OS you are using, these files will be present in the mentioned folders

Windows 7:

new settings.

The NPCEditor creates .plist files which it stores in the user directory for Windows. These files contain NPCEditor specific information.
So if you need to delete these user settings from your machine, follow the below steps.

Depending on which OS you are using, these files will be present in the mentioned folders

Windows 7:

If you are using Windows 7, then you should look under

Your Primary Drive:\Users_Your user name_\AppData\Roaming\NPCEditor\people

You should see files with the extension ".plist". These are the local files created by the NPCEditor on your machine to save user settings. Delete these files. If these files don't exist, then it shouldn't matter.

Windows XP:

Under Windows XP, These files should be present If you are using Windows 7, then you should look under

Your Primary Drive:\UsersDocuments and Settings_Your user name_\AppDataApplication Data\Roaming\NPCEditor\people

You should see files with the extension ".plist". These are the local files created by the NPCEditor on your machine to save user settings. Delete these files. If these files don't exist, then it shouldn't matter.

Windows XP:

Under Windows XP, These files should be present under

Your Primary Drive:\Documents and Settings_Your user name_\Application Data\NPCEditor\people

These are the local files created by the NPCEditor on your machine to save user settings. Delete these files. If these files don't exist, then it shouldn't matter.

Once you delete these files, then the NPCEditor will, by default, use the .plist files that the installer has copied within the toolkit folder, with the new settings.

Note: The AppData/ApplicationData folder under both OS's are hidden folders. So you will not be able to see them unless you make sure that hidden folders are visible in Windows Explorer. In order to do this, open any Explorer window and in the dropdown menu on top, go to 'Tools->Folder Options'. Go to the 'View' tab. Under 'Advanced settings', check the button that says "Show hidden files, folders, and drives".
This will display hidden folders in Windows Explorer.

Unity

What keyboard commands can I use in Unity?

• W,A,S,D - camera movement. Q & E - camera up/down
• J - mouse visibility toggle - mouse look mode
• L - toggles the fake recognizer text box
• O - toggles the user's recognized text
• M - toggles speech recognition mode. When on, click and hold and talk in the mic. Release to quit talking.
• X - reset camera
• Z - show debug statistics
• I - Toggles character subtitles
• P - Toggle entire GUI
• Alt-enter - toggle windowed / fullscreen
• Escape - quit

What console commands can I use in Unity?

Use the ~ key to bring up the console.

Commands:

...

t exist, then it shouldn't matter.

Once you delete these files, then the NPCEditor will, by default, use the .plist files that the installer has copied within the toolkit folder, with the new settings.

Note: The AppData/ApplicationData folder under both OS's are hidden folders. So you will not be able to see them unless you make sure that hidden folders are visible in Windows Explorer. In order to do this, open any Explorer window and in the dropdown menu on top, go to 'Tools->Folder Options'. Go to the 'View' tab. Under 'Advanced settings', check the button that says "Show hidden files, folders, and drives".
This will display hidden folders in Windows Explorer.

Unity

What keyboard commands can I use in Unity?

• W,A,S,D - camera movement. Q & E - camera up/down
• J - mouse visibility toggle - mouse look mode
• L - toggles the fake recognizer text box
• O - toggles the user's recognized text
• M - toggles speech recognition mode. When on, click and hold and talk in the mic. Release to quit talking.
• X - reset camera
• Z - show debug statistics
• I - Toggles character subtitles
• P - Toggle entire GUI
• Alt-enter - toggle windowed / fullscreen
• Escape - quit

What console commands can I use in Unity?

Use the ~ key to bring up the console.

Commands:

• q - quit
• play_intro - Play intro sequence
• vhmsg ... - Send vhmsg out to system. 'vhmsg sbm ...' for sending a smartbody command
• set_resolution x y - Set_resolution to 'x' x 'y'. Example 'set_resolution 1024 768'
• toggle_fullscreen - toggle windowed / fullscreen
• set_loco_char_name - ...

How do Unity and SmartBody interact?

We have a c++ dll, called the vhWrapper, that wraps API calls into SmartBody. This dll is loaded into Unity and its functions are exposed in the SmartbodyExternals.cs file in Unity.  These function are then called from the SmartbodyManager.cs class (also inside of unity).  This dll is used for 2-way interaction between Unity and SmartBody.  For example, you can send python commands from Unity to Smartbody using the function SmartbodyManager.PythonCommand(..) or play an animation using SmartbodyManager.SBPlayAnim(..);

Each frame, unity passes it’s time step to SmartBody and SmartBody returns to Unity all the positions/rotations of the joints of each character.  You can see the structures that are marshaled between the c# and c++ worlds in SmartbodyCharacterWrapper and UnityCharacterData structs.  These structs contain all the data that is passed from Unity to Smartbody and back each frame. 

Look at SmartbodyManager.LateUpdate and GetUnityCharacterData and UnitySmartbodyCharacter.OnBoneTransformations() functions to see how the data is used

...

Ogre

To what extent Is Ogre supported?

...

If all else fails

Look at the Support page for an overview of available support.

...

A run-time Toolkit component that is part of the toolkit architecture. Modules are essential for running a virtual character, as opposed to tools or libraries which just play a support role.

NVBG

Generates nonverbal behavior, based on textual input. One of the Toolkit modules. Often referred to with acronym NVBG. Developed at the Institute for Creative Technologies.

...