Table of Contents | ||
---|---|---|
|
The vhAssetsPackage vhAssets is a redistributable Unity3D Engine package file that contains functionality for allowing Smartbody and VHMsg to work from within the Unity3D engine.collection of redistributable Unity package files:
The Toolkit uses Unity as it's primary rendering engine. It can run in Unity Free or Unity Pro. For Unity Free, we interface with Smartbody SmartBody using 'bonebus mode'. For Unity Pro, we access Smartbody SmartBody directly using a .dll.
Some additional functionality Functionality that the vhAssetsPackage contains isprovides:
Any of these optional components are available when installing the vhAssetsPackage
Quick facts:
Unity is available within ICT. The latest version is available through SVN here:
https://svn.ict.usc.edu/svn_vh/trunk/core/Unity
This location has the latest up-to-date version of Unity, with the documentation pruned for space.
The entire Unity distribution is available here.
https://svn.ict.usc.edu/svn_vh/vendor/unity/current
Also, all previous versions of Unity are tagged in this vendor branch as well.
Please note that you need a Serial # for Unity Pro. Please contact Helpdesk for a Serial #.
For projects, it is recommended to use the version of Unity in SVN. It is virtually the same as installing Unity from the web site*. But you get the bonus of:
...
* The only difference in installing Unity vs using SVN is that certain file associations are not registered (in Windows). So, double-clicking on a .unitypackage will not open it. But right-click->Import New Package works fine.
More internal Unity documentation can be found here: Unity
vhAssets is distributed via the vhAssetsPackage.unitypackage. It is built regularly by our build server, so it will always contain the latest code. It can be found, along with all of our other Unity packages here:
\\vhbuild2\VHToolkit-Builds (sort by Date)
The source for this package is stored here at https://svn.ict.usc.edu/svn_vh/trunk/lib/vhunity
...
Through the Unity Editor, many variables of vhAssets script components can be customized
Captures all Smartbody SmartBody and Unity log ouput and display it to user. Also can be used to send commands. Type '?' for a list of available commands.
Variable Name | Purpose | Value Range |
---|---|---|
Percentage Of Screen | The amount of vertical space that the debug console GUI will span | float 0 - 1 |
Variable Name | Purpose | Value Range |
---|---|---|
Axes | Specifies which axes the camera can rotate around | MouseX, MouseY, MouseXAndY |
Sensitivity X/Y | Angular velocity for camera rotation | float |
Movement Speed | Linear velocity for camera movement | float |
Secondary Movement | Linear velocity for camera movement when holding the left shift key | float |
Maximum X//Y | Max angle the camera can rotate on an axis before being clamped | -360 to +360 |
Minimum X/Y | Min angle the camera can rotate on an axis before being clamped | -360 to +360 |
Camera Rotation On | Enables/Disables camera rotation from mouse movement: True or False | True/False |
Move Keys | Allows specification for which keys are used to make the camera move up, down, left, right, forward, backward, and toggle rotation on/off | KeyCode |
Provides functionality for full screen image
Allows for control of the call ordering of all MonoBehaviour messages sent from Unity.
Variable Name | Purpose | Value Range |
---|---|---|
Initial Priority | The ordering in which this script will be called compared to other VHBehaviours. A lower number entails a higher priority. i.e. priority 0 behaviours get processed before priority 10 behaviours | int |
The control system that allows for correct processing order of VHBehaviours. This object MUST be present in order for VHBehaviours to work
Variable Name | Purpose | Value Range |
---|---|---|
Speed | Linear movement rate at which you move between waypoints | float |
Turn Towards Target | Turn to face your next waypoint target. | True/False |
Ignore Height | Only move to next waypoint along the x/z axes | True/False |
Immediately Start Pathing | Start Pathing: As soon as the scene starts, start moving | True/False |
Loop Type | Action to take upon reaching the last way point of the path | Loop, Ping Pong, Stop |
Pather | The gameobject that will do the moving | GameObject |
WayPoints | The list of transforms that will be used as waypoints for the pather to move along | GameObject |
Works with VHWayPointNavigator and FpsCounter to track performance along a specified path in the scene and then uploads fps/performance data to a database
Variable Name | Purpose | Value Range |
---|---|---|
Time Demo Name | Identifying name of this time demo. Used for starting a specific time demo through the console or command line | string |
Performance Log Name | The filename for the output log from unity's performance tracking | string |
Project Name | Specifies the project this demo is used for | string |
Time Demo Length | The amount of time it will take for the time demo to complete once started | positive float |
Fps Sampling Rate | How often the fps will be sampled during the time demo | positive float |
...
Interfaces with smartbodySmartBody.dll to provide smartbody SmartBody character animation to unity
Variable Name | Purpose | Value Range |
---|---|---|
Path To SBM Files | The directory in which smartbody SmartBody should look for initialization files. Starting directory is the current unity project folder | string |
Character Load Path | Project subdirectory in which character prefabs should be instantiated from | string |
Position Scale | Units of measurement between smartbody SmartBody and unity can sometimes be different based on how the art was exported. If there is a difference, it can be mitigated with this variable | positive float |
Display Log Messages | Toggle for allowing smartbody SmartBody LOGs to be output to unity. | True/False |
All Facebone Characters | Toggle for making all unity smartbody SmartBody driven characters facebone drive or not | True/False |
Cam Settings | Used for the SBMonitor to duplicate the renderer viewport and camera | positive float |
Initial VHMsgs to Send | VHMsgs that will get sent as soon as smartbody SmartBody is initialized | string |
...
Works with SmartbodyManager SmartBodyManager to have a smartbody SmartBody driven character inside of unity
Variable Name | Purpose | Value Range |
---|---|---|
Bone Parent Name | Child gameobject name path that will lead to the skeleton root | string |
Is Face Bone Driven: | Toggle that informs smartbody SmartBody whether or not this character is face bone driven | True/False |
Interfaces with VHMsg that allows communication via strings between processes
Variable Name | Purpose | Value Range |
---|---|---|
Use Specified Host and Port | If checked, the Host and Port specified will be used on connection, otherwise localhost and 61611 will be used | True/False |
Host | Host to connect to | string |
Port | Port to connect to | positive int |
All vhAssets code can be modified directly in the repository https://svn.ict.usc.edu/svn_vh/trunk/lib/vhunity/vhAssets
Creating your own Virtual Human has different meanings to different groups. Some people want to use the existing characters that we supply, but only change certain features like giving them a different voice, or a different colored shirt. Others groups want to use a different character, but the character was created using a standard character package that we already support (Mixamo, etc). Others want to use a completely different character with a unique skeleton, etc. These instructions will attempt to explain the different features and how to customize based on your needs.
The easiest way to use your own Virtual Human is to create a Unity Project using the included .unitypackages as a starting point. From here, you can add/change features on the default character, or bring in your own and customize using the existing character for reference.
Every character should have a script associated with him which specifies the attributes of the character. This class is derived from the SmartbodyCharacterInit class.It specifies the following parameters
The parameters allow us to configure the character correctly with respect to smartbody. Please take a look at the 'InitBrad.cs' file for an example of how to configure the character
The character can use audio files on a per utterance basis, or it can use a TTS generated voice. If using audio files, the voiceType parameter in the above mentioned configuration will be set to 'audiofile' and the voiceCode parameter will point to the folder containing the sound files for the character.
e.g. in the case of the character Brad, His voice files are under the folder "Sounds". This folder contains the audio files and the corresponding .bml and .xml files which are the lipsynch schedule and the non-verbal behavior respectively.
If you want the character to use the TTS generated voice, you will set the voiceTypeBackup parameter to "remote" and set the voiceCodeBackup parameter to the name of the voice you want to use. The name of this voice can be obtained by looking at the TTSRelay application which prints out the available voices on launching.
When Smartbody cannot find audio files, it defaults to the TTS voice and uses the voice you specified as the characters voice.
Smartbody has the capability of online retargeting motions built for one character to another. This can be done in two steps.
After doing these two steps, you should be able to use one character's motions on another. In the above example, you should be able to play a motion like ChrBrad@Idle01_ArmStretch01.skm on Rachel, eg:
SmartbodyManager.SBPlayAnim("Rachel", "ChrBrad@Idle01_ArmStretch01.skm");
A new scene entitled "mecanim" has been added to the vhAssets package which displays 2 of the Brad character, one driven by the Smartbody animation system and the other by Unity's Mecanim animation system. This scene is currently in development but can be used as a guide for how to setup a mecanim character. This scene needs to be loaded through the MainMenu scene to work properly. Alternatively, you can manually activate the SmartbodyManager, VHMsgManager, and DebugConsole objects.
Select a gameobject and add the MecanimCharacter component to it. By adding this one component, other components required to make the character nod, blink, gaze, etc will automatically be added. Make sure there is a gameobject with the MecanimManager component in the scene as well.
For an example of how to setup a mecanim driven character, click on the gameobject ChrBradPrefab_Mecanim in the mecanim scene. Also add a child game object called "SoundNode" and attach an audio source to it. Again, see ChrBradPrefab_Mecanim as a reference.
Use the UI menu on the left side of the screen to test animations, nods, gazes, saccades, and lip sync.
For information on Mecanim, please visit Unity's documentation http://docs.unity3d.com/Manual/AnimationOverview.htmlHere are some examples for common functionality that is modified on a per project basis
Add a new command callback for an arbitrary command
Code Block | |
---|---|
| |
| |
m_Console.AddCommandCallback("some_command", new DebugConsole.ConsoleCallback(HandleConsoleMessage)); void HandleConsoleMessage(string commandEntered, DebugConsole console){ if { if (commandEntered.IndexOf("some_command") != -1) // Do Something }} |
To use the console, at run-time hit the ~ key. Type '?' and hit enter to see a list of commands. If you used the code above in your script, you should see "some_command" in the list. Note, you can pass additional arguments with these commands.
Send arguments with a command while typing in the console
Infocode |
---|
vhmsg someMessage |
Subscribe to a certain type of message
Code Block | |
---|---|
| |
| |
vhmsg.SubscribeMessage("someMessage"); |
Subscribe to all vhmsgs
Infocode |
---|
vhmsg.SubscribeMessage("*"); |
Send a vhmsg
Infocode |
---|
vhmsg.SendVHMsg("someMessage someParam1"); |
Handle a message
Code Blockinfo | ||
---|---|---|
| ||
vhmsg.AddMessageEventHandler(new VHMsgBase.MessageEventHandler(VHMsg_MessageEvent)); void VHMsg_MessageEvent(object sender, VHMsgBase.Message message){ string { string [] splitargs = message.s.Split( " ".ToCharArray() );if if (splitargs[0] == "someMessage") // Do Something } |
You can customize the build process by creating and modifying a BuildSettings.xml file.
In the same folder as the Assets folder within your Unity project create a .bat file with the following lines
Infocode |
---|
@setlocal set PROJECTPATH=%CD% pushd ..\Unity call runEditorBatch.bat -projectPath %PROJECTPATH% -batchmode -nographics -quit -executeMethod UnityStartup.PerformWindowsBuild popd @endlocal |
Use the following template for custom modification
Code Block | ||
---|---|---|
| ||
<?xml version="1.0" encoding="utf-8"?> <BuildSettings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.cpandl.com"> <ExternalAssetsPaths> <ExternalAssetsPaths> <string>Assets/SomeDirectory</string> <string>Assets <string>Assets/SomeOtherDirectory</string> </ExternalAssetsPaths> <BuildOutputPath>Builds/YourProject/YourProject.exe</BuildOutputPath> <PostBuildScript>somePostBuildScript.bat</PostBuildScript> <ConfigFiles> <ConfigFiles> <string>Assets/config.ini</string> </ConfigFiles> </BuildSettings> |
Field | Desrcription |
---|---|
ExternalAssetsPaths | Array of folder names that will be copied to the BuildOutputPath. (OPTIONAL) |
BuildOutputPath | Path specifying the name and location of the .exe for your build (REQUIRED) |
PostBuildScript | A .bat or .exe that will be run after the build process is completed. (OPTIONAL) |
ConfigFiles | Array of file names taht will be copied to the BuildOutputPath. (OPTIONAL) |
You can create a config .ini file in your project and read it using the IniParser class
Code Blockinfo | ||
---|---|---|
| ||
m_ConfigFile = new IniParser(configFileName); m_ConfigFile.GetSetting("SomeSetting") // returns a string with the value of that setting, if it exists |