With the weight lift animation finally completed, it was now time to move on to the second part of this assignment, the lip-sync, which we were to create using the new, updated, and recently released version (V3) of the ‘Rain’ rig for Blender 4.2.
Given creative freedom to choose our own 8-10 second audio clip, with so many options available from such a wide range of media, what should I choose? My first step was to therefore discover what makes a good audio clip for animation, and more importantly, what audio not to choose, with my research uncovering the following useful tips in the Sir Wade Neistadt post below:
Source: YouTube, 2021
The first was to steer clear from audio from a well-known movie, TV show or video game clip, as it’s likely many people have already seen it, and so the moment they hear the audio, they’re distracted from your work thinking, ‘I know that voice,’ instead of actually watching the animation. Also, compromising your work, in all probability they already know the plot and what the character is thinking, and will therefore compare your work with the original, especially if the audio originates from a well-known animated movie. Finally, to prevent distractions and allow the audience to focus on the animation, the clip should be of good quality, the dialogue audible, and where possible contain elements where you can easily relay emotion, and most of all, have fun.
With this in mind, I therefore headed over to the 11secondclub website (https://www.11secondclub.com/), where I was aware that in a monthly competition, short audio clips are posted, and animators upload their interpretations of this audio to be voted on by the website community.
Here, after extensively browsing the audio clip archives, I decided on a trimmed down version of the ‘muffins’ audio from their November 2023 competition, that would meet the 8-10 second assessment criteria.
This specific audio clip, where a girl is clearly struggling to relay her despair and frustration that while baking, she burned her tray of muffins, was chosen not only because it presented a fabulous opportunity for me to visually relay these emotions, but because of the contrast between her whispered confession, and her subsequent, emphatic shout of disbelief.
Also, as successful facial animation consists of many elements which must all work together to convey and elicit emotion, while my own animation would be unique, by choosing a voice clip from the 11secondclub website, if in doubt about any aspect of the lip-sync process, I had access to a multitude of different 2D and 3D animated examples which could be used to point me in the right direction.
The next stage was to obtain a live action reference that would help me capture and relay the storytelling, performance and emotion required. I therefore again enrolled my favourite amateur actor, and before recording, to ensure the body language matched the dialogue and what the scene was trying to convey, decided how best to interpret the voice clip and where contrasting poses could be used. While in the end not an exact interpretation of the audio, the reference proved to be of great help in understanding the action, and enabled me to enhance the believability of both my character and my overall animation.
Lip Sync Live Reference
Live action captures of how the face and mouth moves when repeating this dialogue were then also recorded from a front, and perspective view (this time with me in the starring role), and to help me achieve realistic expressions, throughout the animation process I always had a mirror to hand that could be used to obtain my own live references.
Lip-Sync Face and Mouth Reference (Front View)
Lip-Sync Face and Mouth Reference (Perspective View)
To assist in the blocking out stage, I first wrote down the dialogue and stage directions, before then breaking these down into the following phases; the body shaping for which then became the key frames in my animation:
- (Trembling sigh)
- ‘I was so distracted, I…’
- (Deep intake of breath, followed by a brief breath out)… ‘I’
- (Deep sigh and trembling intake of breath)
- ‘I burned my muffins’ (whispered)
- Voice off camera, ‘No,’
- ‘Yes’ (shouted).
Once the key frames were in place and timed to ensure the body movement matched the key beats of the performance, my character was then posed by blocking in the core neutral, anticipation, extreme, and settle poses, which as the software fills in the in-betweens for us, would allow me to sell the idea without, as in the past, manually adding umpteen additional in-between poses to create more smooth and natural transitions. Finally in this section, the timing and pacing was adjusted to set the tempo of the actions, and transitions.
Our ‘Is there alcohol in this?’ workshop exercise provided an excellent opportunity to practice the steps involved in this process, and to consider key elements such as silhouette, line of action, staging, and where appropriate, exaggeration. Part one of this exercise, including up to the point where the body movements have been completed, is shown below:
Part One: ‘Is There Alcohol in This?’ Class Exercise
While not frequently used in animation, and it often gets a bad press, intrigued by the information on symmetry or ‘twinning’ in the acting section of ‘The Animator’s Survival Kit’ below, and as we, as humans, naturally mirror our actions in real-life, I decided to use this technique, but to also create a little asymmetry by tweaking the timing of the left hand ever so slightly to soften the effect.
Source: Williams, 2001
Also, when animating the head turns, as we’d been taught in our 2D ‘head turn’ class exercise below, I ensured these moved in a smooth arc.
2D Head Turn Class Exercise
Adapted from: Newtown, 2016
https://design.tutsplus.com/tutorials/animation-for-beginners-how-to-animate-a-head-turn–cms-26487
Also, as ‘the movement of the head in relation to the body helps to create expression,’ and ‘the simple tilting of the head into a posture related to the emotion being expressed is capable of conveying a much broader emotional range’ (Blair 1994), posturing too was an important consideration.
The next stage was to add emotive facial expressions to support the foundational body movements, and of course, lip-sync detail in which ‘enabling them to be realistic, expressive, and communicate with the audience, the character’s mouth moves in time with the words they are saying’ (Brush Ninja, 2024).
Concentrating on first blocking out the facial expressions, so as to relay a full picture, after reading how ‘we can easily communicate with our eyes alone’ (Williams, 2001), I was particularly careful to show emotion and intent in the eyes, and reinforce the intensity of this with appropriate eyebrow movements. Also, to enhance body movement, the eyes were also animated to blink a few frames before each head turn.
Up and down jaw movements reflecting how the jaw moves according to the syllable and the emotion of the character were then added, and in-betweens mirroring the mouth shapes in the audio. During this process I found this resource, showing the different female mouth shapes helpful, and some useful tips in the online tutorial below.
Source: Zych, 2015
Source: YouTube, 2020
Of particular note was that in reality, when speaking, we don’t ‘ARR TICK YOO LATEH EVERY LITTLE SIL AH BULL AND LETARR’ (Williams, 2001), and as can be seen in the clip below (YouTube, 2020), as such exaggerated movement can be very distracting, unless creating a ‘cartoony’ animation, less is often more. Therefore, to ensure my final animation appeared realistic and believable, it was more important to focus on the words and phrasing in my audio clip, and to replicate the mouth shapes of the sounds these words made (phonemes) using correct timing to ensure that the animation synchronised with the dialogue (Wahl, 2024).
Source: YouTube, 2020
Again, our ‘Is there alcohol in this’ class tutorial and workshop exercise helped me to better understand the lip-sync process, with part two, the completed animation, shown below.
Part 2/Final Outcome: ‘Is There Alcohol in This?’ class exercise
Accents (frames of punctuation where preceding actions are used to emphasise the dialogue) were added next where the character whispers ‘I burned my muffins’ and shouts ‘Yes,’ and with polishing, including the addition of secondary animations using the principles of slow in and slow out, overlapping action, and follow-through, my initial animation pass was ready to upload to SyncSketch for feedback.
Initial Amination Pass
References:
Blair, P. (1994). Cartoon Animation. Tustin, California: Walter Foster Publishing Inc.
Brush Ninja (2024). Lip syncing. [online]. [Accessed 29 November 2024].
Available at: https://brush.ninja/glossary/animation/lip-syncing/
Newtown, P. (2016). Animation for Beginners: How to Animate a Head Turn. evanto tuts + [online]. [Accessed: 21 November 2024].
Available at: https://design.tutsplus.com/tutorials/animation-for-beginners-how-to-animate-a-head-turn–cms-26487
Wahl, S. (2024). Free Workshop: Mastering Lip Sync in Animation – Animation Mentor. [online]. [Accessed: 3 December 2024].
Available at: https://www.animationmentor.com/blog/free-workshop-mastering-lip-sync-in-animation/
Williams, R. E. (2001). The Animator’s Survival Kit. London: Faber and Faber. [online]. [Accessed 5 October 2024].
Available at: https://archive.org/details/TheAnimatorsSurvivalKitRichardWilliams
YouTube (2020). IMPROVE Your Lip Sync Animation in 3 Steps – Sir Wade Neistadt. [online]. [Accessed: 29 November 2024].
Available at: https://www.youtube.com/watch?v=fySj1JDAFZQ
YouTube (2021). The Secret Workflow for Animating Dialogue – Sir Wade Neistadt. [online]. [Accessed: 23 November 2024].
Available at: https://www.youtube.com/watch?v=5cIxEZwZmS4
Zych, Z. (2015). Mouth Reference Images Speaking – Pinterest. [online]. [Accessed: 24 November 2024].
Available at: https://uk.pinterest.com/pin/365917538450852995/