Lip Sync
I think I underestimated how difficult this would be!
For my audio, I picked a clip of Karlach from Baldur’s Gate 3, as at the time the game was pretty much constantly on my mind. The clip had a lot of emotion, I thought it could be good to try and portray that in my scene. Unfortunately, I am absolutely not an actor, so when it came to recording my reference video, I struggled to get across the movements I had in mind for my animation.
After getting a somewhat passable video- which I am far too self conscious about to post here!- I watched some lip-sync examples online. I found a few videos going into detail about the different ways people go about their dialogue animations, and tried to incorporate some of the things they mentioned into my blocking. I also watched clips from some of my favourite animated shows like arcane, to see how much detail they put into their characters’ faces and mouth movements. It’s very interesting seeing everyone’s different styles and techniques!
I’ll include some of the example short animations at the bottom!
Animating
We had already done a lip sync exercise in class, which gave us a good head start on our assignment.
I knew I was going to be framing my animation from the hips up at the very least, so I didn’t pay much attention to the feet beyond putting them in a suitable standing position to save time. I blocked out the main poses in line with the audio, the hands first as I have her doing these big sweeping gestures which were very easy to plot. In one of the videos I watched, it was suggested to finish all of the body poses before starting on the actual lip sync. I worked on the body, then the facial expressions, and the mouth posing and smaller details like the hair and tie very last. After this, I added in some more frames to refine movement and started testing my timing.
I had to go back and forth between constant and Bezier interpolation to get the movements to look right, I was so used to the direct snapping to a pose when I needed it, and I had so many keyframes in that sorting through them to make the movement flow as I intended for it to took a decent amount of time.
Actually animating the lip sync was a learning experience for me. I had seen in a few videos that to get the jaw movements correct, and to avoid over annunciation, some people held a hand to their chin when they spoke their dialogue out loud and paid attention to when their jaw actually opened. I used this to help key out where the jaw would be open and where I’d just be focusing on lip movement to portray the sound. I recorded a second reference video; this time just close up to my mouth to reference the lip movement. I made sure to not only use the main mouth rig but the secondary as well, so that I had better control of the corners of the mouth specifically, which I had been made aware are very important in getting across a sound. I had also seen someone demonstrate slowly scrubbing through their audio frame by frame, marking where each sound was being heard, so that they knew what mouth movements were actually needed- a lot of noises in a word are mumbled or slurred together when people talk, so rather than plotting out every noise that I know is in a word, this helped keep the focus on what was actually needing portrayed.
After I had all of this done, I noticed an issue with her hands. I think somewhere along the way while blocking, I had accidentally rotated the hand too much and hadn’t realised, so when I changed to Bezier interpolation the hands would spin weirdly between frames. I fixed most of them by copying and pasting the rotation values of the previous frames and then readjusting, or just spinning them all the way back manually, but there were a few instances where this just didn’t seem to want to work. In these cases, I just added keyframes all the way between the two I already had, and just manually adjusted, so it looked normal right up to the frame where the twist happened, so any weird rotations didn’t have any visible time to actually move. The below video isn’t the offending section, but of course I forgot to actually take any videos of it before fixing it!
As for her facial expressions, I wanted to portray anger, sadness, frustration, and the disappointment and blame she was putting on herself. I tried to make her eyebrows expressive, especially when her emotions ramp up in the latter end of the audio. I tried to portray her emotions with hew eyebrows, mouth, eyes, head and hand movements, and where she was looking, which is where I think I could have pushed things more. I think I’m often too reserved in my animating, trying too hard to make things look more believable than compelling, but maybe I’m reading into it too much!
And so here is the final!
Reflection
When you work on something for too long uninterrupted, I feel like it’s easy to lose perspective and stop noticing when things don’t look right. Because of my visits to the hospital this term, I fell behind in my work, particularly with my Character Creation assignment, which I had to take an extension for. I spent a lot of time trying to finish that one off, and it ate into my time for animating for the lip-sync and weight lift. I didn’t want to have to take another extension, as I had already taken them for the other assignments this year, so I spent the days between getting my Character Creation assignment done and the deadline for these animations just sitting working on them both as much as I could. But, because of this animating without time to let my eyes take breaks, I fear I haven’t had the chance to look at them freshly and so won’t have noticed mistakes that I had simply grown used to seeing.
Regardless of this, I am happy that I have managed to get everything done before the deadline, despite my push for time. I really hope I am of better health for this new semester, so that I can be as present in class and as caught up with my work as I can be. Nothing feels worse than feeling behind and overwhelmed, especially with important deadlines. I’m looking forward to doing more work with animating dialogue, and having more time to be able to feel fully happy with what I have made when my skills have developed.
A few reference animations!