Table of Contents | ||
---|---|---|
|
This page explains how to use the MultiSense framework (also known as multimodal framework, developed by Multicomp Lab) focusing on CLM (developed by Jason Saragih et .al) / Gavam (developed by Louis-Philippe Morency's Multicomp Lab) face-tracker, Kinect, FAAST (developed by Mixed Reality Group) which is based on the ssi lib (developed by Johannes Wagner, University of Augsburg). Each of these modules is implemented within the MultiSense framework and runs separate threads in a synchronized manner. The output from the MultiSense framework is PML (the pml xml schema can be found at PML.xsd) which is currently passed on using vrPerception message (based on VHMsg API).
...
MultiSense currently supports a lot of modules like vision (for tracking face features, smiles, gaze, attention, activity, gestures, among others), speech. To run the MultiSense, the user needs to run the MultiSense application which basically sends out the vrPerception messages (based on pml) which can be subscribed by any module for specific data.
There are two main steps involved in running MultiSense -
To run the MultiSense module, either
...
By default, only Gavam and CLM facetracker are turned on (for the toolkit; otherwise only the Gavam is turned on). If you wish to change the configuration for running other modules, you can navigate to \bin\multisense\ssi_vhmsger\bin\config.ini and change the useXXXX to true to make a module work. Note that if the dependencies for the module are not setup correctly, the ssi_vhmsger will crash. Another thing to note is that If you want to run FAAST, make sure the useKinect as well as useFAAST are both true in the config.ini. Make sure to have a web cam connected to the computer. If the application fails to start, one of dependencies might be missing. Currently the ssi_vhmsger will not get killed when you click the "Kill" button from the launcher. Also, if you try to kill it from the launcher, the ssi tool will turn yellow.
To run the PerceptionTest application, either
...
Please contact either Unknown User (suri) or Unknown User (stratou) if you have any issues in running the application
To setup the MultiSense, the user will need to checkout the repository from -
...
If there is a problem with installing MultiSense, please contact Unknown User (suri) or Unknown User (stratou).
vrPerception
vrPerceptionProtocol
See Main FAQ for frequently asked questions regarding the installer. Please use the Google Groups emailing list for unlisted questions.