NPCEditor controls the spoken behavior of the characters in the Toolkit, as well as the structure and logic of the interaction through its dialog manager. It contains a list of user "Questions" and character "Answers", and the links between them. NPCEditor uses a statistical text classifier to determine the best character response to novel user input, allowing users to converse with the characters with reduced authoring effort.
In the launcher, click 'Launch' in the NPCEditor row under the 'Agents' group. It will also run by default when you launch 'Run Checked'
The quickest way to make a character speak a line is to double click an entry in the right side of the Utterances Tab, under 'Answers'. The character should speak the line you've selected, with lip sync and gestures provided by NonVerbal Behavior Generator.
All of Brad's, or any character's, lines of dialog are stored in a plist generated by NPCEditor. NPCEditor uses a tabbed layout, with the two main tabs for users being the Utterances Tab and the Chat Tab. Dialog is found in the Utterances Tab.
The left hand side of the Utterances Tab lists the Questions, while the right is dedicated to the Answers. Questions are examples of user utterances that the system expects to receive. This can include questions, "What is your name?", or statements, "Tell me about yourself." When NPCEditor receives user input, whether typed or spoken through AcquireSpeech, it uses a statistical classifier to compare the input to known Questions. If there is an exact match, NPCEditor automatically responds with the highest scoring Answer. If there is not an exact match, the classifier determines the closest line in the Questions tab and responds with the highest scored linked Answer. These scores are determined by the links between Questions and Answers.
Character dialog is listed as Answers on the right of the tab. Each of a character's lines of dialog has its own entry.
An Answer consists of a number of fields.:
Each of these fields and the values within can be edited in the Settings Tab.
The Chat Tab allows you to see a log of the latest discussion with an NPCEditor controlled character, and interact with a character independent of other tools. You do not need a renderer or any other components. The body of the tab is by default a maximized chat interface. Users may enter what they wish to say to a character at the bottom, and the classifier will select the best response and print it to the Conversation Log.
For example, as you speak to Brad with your microphone, you will see your speech and Brad's responses populate the log. If you wish to type an answer, you may do so here by selecting the user to speak as, typing your utterance, then clicking "Enter Question" in the bottom right. If Unity or another engine is running, Brad should speak and respond there, as well as in his window. Even without any other components running, you can test text interaction with a character through this interface.
Along the left hand side are other tools for seeing how the classifier within NPCEditor parses user input, analyzes it, and selects the best response.
There is no set way in how to author a character, but the general approach is:
The less specific you make your answers, the more appropriate they are in different situations. For example, if you ask Brad "Did you go to USC?", he can answers "Yes, I went to USC.". This is a perfectly fine and appropriate response for a yes-no question, but not for a related question, like "Where did you go to school?" As an author, you can either create an additional response, like "For college, I went to USC", or you can link both questions to a more generic answer like "I went to USC."
In general there is a balance you need to keep between making answers interesting and having them work in many different contexts. It can be very compelling when a virtual character responds exactly to what you say, especially by repeating part of the question (i.e. "Yes, I did go to USC! You too?"), but having an inappropriate response completely breaks the illusion. The best strategy seems to stick to generic and broad answers most of the time, while anticipating some very specific answers only once in a while. The generic answers should be stand alone statements that sound good in relative isolation and relate broadly to the topic inquired about.
The NPCEditor usually tries to not say the exact same line twice in a row, but it can only do that when an appropriate alternative is available. You should therefore make sure that you explicitly provide several paraphrases for a statement. Otherwise, the character can either repeat the exact same line (which is unrealistic), or it can select a different response which is statistically close to the answer your provided, but semantically very different.
The NPCEditor version provided with the toolkit treats any input (usually a question) without any context. That means a character is unaware of what has been said in the conversation thus far, nor does it know what it has just said (apart from a simple 'do not repeat yourself is possible' rule). As a result, a character cannot ask a yes / no question, because it has no way to relate the answer ("yes", "no" or "maybe") to the question just asked. To the character, the answer is just isolated input; it cannot discern to which of potentially many yes/no questions it is an answer to. Of course, if you have only one yes/no question in your domain, you are safe to assume any yes/no related input is an answer to that particular question. In addition, if you know you will get a very specific answer back, you can respond appropriately to that specific answer. For instance, apart from a couple of rhetorical questions, Brad only asks the user how he or she is doing. This works in most cases, because it can appropriately respond to answers like "I am fine" and "Not so well". However, when a user just says "OK", we need to make a choice to either treat that as a response to how that person is doing, or as an unrelated acknowledgement to any of Brad's statements.
There are ways around this particular problem that we are experimenting with within ICT. If you are interested, do not hesitate to contact us.
You can do this by double clicking on the answer in the right hand panel of the NPCEditor, or by selecting the row and clicking Send, at bottom right. You might notice that a particular TTS engine does not pronounce a certain word you expected it. If that's the case, you can either use a different word, or be more phonetic in your spelling. For instance, the Brad character has "ICT" spelled as "I C T" in order to force the TTS engine to pronounce each individual letter, rather than trying to create a sound as if it was a word.
Initially you can just have a couple of questions for each set of answers, but user testing will result in many different paraphrases for these questions. Collecting all these paraphrases and linking them to the right set of answers ensures that your character will get more and more robust.
NPCEditor provides a way to modify your dialogue manager using a groovy script. If you look under the Dialog Manager tab in NPCEditor, you can point to a script of your own and use it.
Groovy is a scripting language which uses java like syntax. You can find more about it at http://groovy.codehaus.org/
See these slides to get started: 2013-03.NPCEditor-secrets.key.pdf
The NPCEditor uses a .plist file format for loading.
The character names and ids can be found in this section
Lists all the user questions
Lists all the system answers that the virtual humans respond with. Speaker id is the index of categories → speaker → tokens array
Contains an array of dicts that maps a question from the questions section of the plist to an answer from the answer section of the plist using array indices.
See the NPCEditor section in the Main FAQ.