The Toolkit provides an example of how to use the MultiSense framework (also known as multimodal framework, developed by Multicomp Lab).
Multisense uses these technologies:
The Toolkit includes two main components that use MultiSense:
MultiSense currently supports multiple modules, including a vision module (for tracking face features, smiles, gaze, attention, activity, gestures, among others), and a speech recognition module. To use MultiSense, simply run the MultiSense application, and it will broadcast the vrPerception messages (based on pml) generated by each module, which can be subscribed to by any module or external component.
Two components are required to make full use of MultiSense, the MultiSense Application, and the Perception Appliclation.
To run the MultiSense Application, either
By default, only Gavam and CLM facetracker are turned on (for the toolkit; otherwise only the Gavam is turned on). If you wish to change the configuration for running other modules, you can edit \bin\multisense\ssi_vhmsger\bin\config.ini and change the "useXXXX" parameter to "true" to make a module work. Note, that if the dependencies for the module are not setup correctly, the ssi_vhmsger will crash. Another thing to note is that If you want to run FAAST, make sure the "useKinect" as well as "useFAAST" parameters are both "true" in the config.ini. Make sure to have a web cam connected to the computer.
If the application fails to start, one of dependencies might be missing.
Currently the ssi_vhmsger will not get killed when you click the "Kill" button from the launcher. Also, if you try to kill it from the launcher, the ssi tool will turn yellow.
To run the PerceptionTest application, either
Please contact either Unknown User (suri) or Unknown User (stratou) if you have any issues in running the application
To setup the MultiSense, the user will need to checkout the repository from -
https://svn.ict.usc.edu/svn_vision/Projects/trunk/ssi-ict
After the user downloads the framework, they will need to build module which allows to send and receive VHMsgs (located in \tools\ssi_vhmsger; The solution file is located at \tools\ssi_vhmsger\build\ssi_vhmsger.sln). Make "ssi_vhmsger_test as the "Startup project" . The user will also need to set the working directory as "..\..\..\..\bin". Once the project builds successfully, the user will need to have a web cam connected to the computer to send vrPerception messages using activeMQ (note that this module has a dependency on activeMQ and it will need to be installed. The latest installer for apache-activemq can be found at \\vhbuild\SASO-Installs).
You can check out the detailed description of how to setup the project at the following location -
VrPerception messages with SSI-ICT framework
As an illustration of integration, the please check out the following link (integrates the toolkit with MultiSense)
If there is a problem with installing MultiSense, please contact Unknown User (suri) or Unknown User (stratou).
Sends:
vrComponent
Receives:
vrPerception
vrPerceptionProtocol
vrAllCall
vrKillComponent (does not react to it yet)
See Main FAQ for frequently asked questions regarding the installer. Please use the Google Groups emailing list for unlisted questions.