Basic Goals:
- Try and create animations for speech.
- Mimic realisitc mouth movements.
- Mini-Editing studio for the animation/keyframes.
- Look good!
Let's start with a demo!
- I call this my "androgynous hipster".
- Mostly mouth movements.
- Starting add few other tweaks to give it more life.
Now wasn't that interesting?
- Shorter words have pretty accurate syncing.
- Longer sentences tend to get a bit off.
- The face needs a lot more variation between the frames.
- How did I do it?
Key frame editor
- Custom UI Toolkit (built from the ground up)
| GUIButton | Provides an interface for an extensible, clickable button. Support images for normal, mouse-over and mouse-click. Each button can be assigned an action name which then gets handled upon clicking. Additionally, the cursor is modified when ever the mouse is over a button. Button text gets centered onto each button behind the scenes, so no need to worry about spacing. Width, height and position are fully adjustable. |
|---|---|
| GUIMenu | Allows multiple buttons to be added together into a single menu component. Menus can be either vertical or horizontal --supporting an optional header item as well. Contains flags for visible and hidden so you can toggle the visibility upon a right click action. Width, height and position are fully adjustable. |
| GUITextInput | Allows the input of text into a well-recognized text-box format. Simply supports the handling textual input and backspace for deleting characters. Width, height and position are fully adjustable. |
| GUITextPrompt | Gives the ability to prompt the user for input. While this prompt is active, the rest of the screen is grayed out to help draw focus to the prompt. Uses a GUITextBox for input and 2 GUIButtons (OK and Cancel) for handling actions. Also supports the ENTER key press in lieu of the OK button. Width, height and position are fully adjustable. |
| GUIManager | Encases all GUI components under 1 manager component. Use this to create and intialize all GUI Elements. Also here is were you would handling Processing's built in actions and pass that event down to each of the GUI elements. |
| GUIEditor | The container to hold the 2d objects and all associated points/lines. Has the ability to have points added, removed and edited. Allows for straight line mode, or curved shapes (currently only using 1 subdivision scheme, but others are trivial to implement). Each key frame will contain a list of this which make up the overall scene. |
Key frame creation
- Ending up using about 10 key frames to mimic all the different sounds.
- Suprisingly little number need to mimic accurate talking.
| Keyframe Name: | Represents: | Keyframe Name: | Represents: |
| AandI | a i | E | e |
|---|---|---|---|
| U | u oo ou | O | o aw |
| Consonants | c d g k n r s th y z | FandV | f ph gh v |
| L | l ll h | MBP | m b p |
| WandQ | w q | T | t |
Interpreter
- Take in text; output keyframes.
- Have exclusion rules (ex: don't mouth vowel if it is last letter).
- Flags for elongating or shortening a movement
(ex: oo make a tad bit longer).
Final Demo & Questions
- Lets play with a different model this time.
Finished!