Starting The Project
Initial Ideas
I had 3 initial ideas that I had planned in my mind or on paper, which excited me for the project! My first one was using audio from It’s Always Sunny in Philadelphia (1.08-1.17), the second was from The Rocky Horror Picture Show (1.11-1.20) and the third was from The Adventure Zone (12.02-12.12).
Although I really liked this idea and thought using a podcast would be the best option, as I would have nothing but audio to work from, I was quite overwhelmed with the multiple characters when I tried to plan this one out. There would also be and issue with getting all 3 rigs, all different, into 1 scene (I could eliminate the 4th) and having to animate all of their parts. Unfortunately, I couldn’t think of any other monologue parts from The Adventure Zone, or easier memorable parts.
Chosen Audio
A friend suggested I take a look at What We Do in the Shadows, the TV show version. Its a mockumentary style, which would mean monologues, and I haven’t watched the show or seen clips from it so I thought this would be a good piece to sample from. The biggest issue with this audio though is the accent (when Nadja says ‘reignited’ ‘flames’, ‘brutally’), and the visuals/context: I felt as though there would be little room to reinterpret the clip, but It would also be much simpler, only having 1 acting character (I planned on having Link in the scene, but lying on the ground).
From 0.13 to 0.20 “-And every time he comes back our love is reignited like a big ball of flames… And then, he is brutally decapitated.”
I felt that watching my clip over and over again, and focussing on Nadja’s mouth movements, helped a lot when trying to understand the pacing and motions- I tried to note down every time her mouth closes (I wrote down the audio too, and marked every sound that has a special movement), pinches or widens, and particularly how sounds blend and how the mouth moves but never closes. If this wasn’t an animation course, this would sound very strange!
Although I’ve said the accent could hinder the animation, there are parts that may actually help me exaggerate the mouth shapes and add more interest to the lip sync, particularly on ‘flames’ and ‘decapitated’- I could draw out the motions and key most of the sounds for flames individually, for example.
Chosen Rigs
Breath of The Wild Link and Zelda by Christoph Schoch
Process – Personal Development Blog
Inspirations and References
Before starting this project I had thought that when doing lip syncs, 3D animators exaggerated the mouth shapes to make it clear what was being said. In reality, after watching some recent stylised animated films I could see that this wasn’t the case.
One thing I noticed with Turning Red and Luca was that the mouth shapes were very variated and fun, in Luca their mouths move around their faces a lot to sell the style (also, their teeth are always showing, I suppose I’m used to 2D where stuff like that disappears!) There’s a lot of movement, but it doesn’t seem too fast, but while when we speak there’s a lot of complex face movements, I noticed that Luca and Turning Red prioritised large, expressive poses for their lip sync.
In terms of acting, I liked Luca’s facial movements a lot! Their eyes and eyebrows are very fast I noted, and when looking at 1 spot their eyes are still moving- which is more realistic. Because of the stylised animation their body animation is also very fast but not jarring, I really liked this style of animation. There’s also some secondary animation, Luca’s hair being the most obvious, so I wanted to make sure I moved all I could (that made sense) with Zelda.
I also watched Disney’s Encanto, which has a much more subdued style of lip sync which I liked too;
https://www.youtube.com/watch?v=TDiBixlQHGs
Much more realistic animation compared to Luca, the motions are slower and it relies more on the lip and jaw movements. I also liked how Encanto animated the face to react as well to the lip sync, here Mirabel’s nose and cheeks move with the mouth and her eyebrows also react (they don’t only move when she makes an expression!). Mirabel’s mouth doesn’t open and close all that much, even in places where I imagined they would – like the Vs, instead they use the lip corner in/out for places like this! The acting in Encanto is also quite subdued and smooth in non-music parts, but then there’s the cloth, hair and jewellery that adds more motion.
Sir Wade’s tutorial on lip sync really helped me, I watched some of his other workflow videos but this one helped me understand better how lip sync should work with his components of language section:
- Corners in/out
- Lip shape
- Jaw open/close
- Tongue
Later on, when making my own animation I kept a note beside me to try use all of these components!
I looked at some other kinds of 3D animation, like realism in games and even more heavily stylised but I liked the look of Luca the best, but I wanted to keep it a bit more subtle like Encanto. With this style in my showreel, I would want to be able to apply for character animation roles for adverts/tv/film.
Keyframing
I had an idea in my mind of how I wanted this to look, so I skipped a storyboard phase and just wrote down my audio and annotated it (including drawings of certain movements, eyes etc.) and started blocking.
Although Alec has told us before that moving teeth shouldn’t be necessary when lip syncing, for a lot of the mouth poses I wanted the teeth to be visible without having her mouth move around her face too much, so in certain parts I have moved her teeth for a frame or 2; like the Fs and Vs, and at ES (flames).
“And every time he comes back”
After sending the block in to my university discord, it was suggested that I change her pose and bring the camera back- the rig comes with a little torch that I could use to tie into the audio, and I added a tree I made for my other project for Zelda to lean against.
I decided that I needed to show more of her teeth on the V in every, and possibly linger there for a frame extra so its clear what she’s saying. Tongues are also a very fast moving organ, but again to be able to register the movements I wanted the tongue to be more visible in L (and TH) sounds. I made sure the tongue was visible by having it curl over her teeth, I know when we speak, the tongue is usually behind the teeth but it was not as clear as I wanted it to be.
Using the animation layers feature, I started blocking body movements for the first half of the animation.
“And every time he comes back our love is reignited like a big ball of flames”
Polishing
After watching some more video tutorials on lip syncing, and noting down ideas from actual films and professional’s advice I tried to add some more details to my animation:
Lip Sync:
- Nose stretching or flaring on wide mouth positions (i.e., “flames”)
- Lip pursing on ‘O’ sounds
- Mouth moving up and down on face slightly, or around face for more 2D look.
Acting/extras:
- Faster fall on the blinks, with a slower open. Eyes also follow the lids down. Eyelids also rotated so the inner corner looks like its leading. When open wide at “flames” her eyes shake slightly. When eyes open, pupils dilate as reaction to light.
- Cheeks move up/squish slightly when blinking, ears also twitch slightly on the blink compress frame.
- Ears twitch when eyebrows are raised high or very low, they also bounce and follow the head a frame after when moved. Hair also follows head.
- Slight breathing loop
- Overlap on certain parts, i.e. her eyebrows for “and then-” have a frame difference.
I was very happy with this progress in the lip sync! and then Maya crashed and I lost a good chunk of it.
After redoing all the missing parts, and repositioning her with a torch and finishing the last quarter of audio, this is what I had! I added some more ear movements, hair motions and even a subtle infinity breathing cycle, thought it helped the look! I also had to set up cameras.
Fixing the Lighting
Using my own background.
Issues Faced
This was my first time using the animation layers feature, so I often lost my keys thinking they were elsewhere, having layers override others, having to delete all FV and Curl poses as I lost them, etc.
If I was to revisit this, I would maybe spend more time on the first quarter of lip sync, I feel as though there is too much fast movement and I should have held it a little longer. I would also liked to have had more expression, in her mouth or using her eyes/eyebrows/shoulders or ears maybe. I also would have loved to have a fire on the torch, but it proved to be too much for my laptop and having to constrain a 2D fire to the torch was much more finnicky than I thought, plus with the 2D fire, the flame didn’t wave with the motions. Even when trying a 3D fire, I was able to make it using this video below, but I didn’t know how to attach it to the torch. The rig came with a fire plane, but I didn’t like it and couldn’t figure out how it could even work as an animated fire.
I also maybe should have moved her eye shines about, they have a very subtle up/down movement but I maybe should have constraint them to the eye movements, rather than keep them in 1 point. I kept them still because shines don’t move with the iris but looking back on it, I could have made it more interesting having the shines move as at times it looks awkward.
Finally, I feel as though her arm moves too much, especially at around the second half and I maybe could have made that smoother and quicker, at points.
Without Editing
Final Render
I edited the final video a bit, so that there’s a longer beginning, end, and bigger pause at ‘flames’, I thought this would be nicer.
Be First to Comment