In support of my portfolio / showreel, I animated a 3D scene to continue my personal practice & development. I animated a 3D shot in Maya focusing on dialogue and acting, for my last project I used a Miles Morales rig from the Agora Community, so I went there first to look for this project. I wasn’t sure what audio I wanted to animate, but I thought choosing a rig first would help narrow it down.
I found a Thor rig which seemed pretty cool, not too simple so I’d get experience with little bit more of a complex rig. I set up MGpicker, a very intuitive and animator friendly Maya picker tool, with the a powerful feature that enables you to create your own customised picker without any prior knowledge in coding and hooked this up to the Thor rig with the file included by Agora. I tested some facial expressions and liked the range I could achieve with the rig.
I also looked at an Aang rig, which was a little more simple and was advertised by Agora as being for facial animation. I attempted to recreate the same expression as before to see how I liked it. I didn’t like the blue eyes in the Thor rig, these eyes were much more expressive.
Aang had a lot more automated features in the rig, with sliders allowing you to pucker the lips or puff the cheeks out. It also had a studio library file included, with a bunch of pre-set facial expressions already set up for you. I decided to go for the Thor rig since I thought it would be more of a challenge, since it didn’t have any of this automation. Also because I haven’t seen it used much online in other showreels, so it’s a little bit more original.
I also noticed the Thor rig was game-ready so I tested it out in Unreal, I thought it might’ve been cool to render this shot in Unreal for my showreel, since it wouldn’t take long and you can make a pretty enough scene easily. I built a small scene using assets from the Vertical Slice project and brought Thor into Unreal, but I noticed the UV seams were pretty visible in Unreal and thought. if this was going in my portfolio, having scenes from my Vertical Slice project and my Lip-sync animation in the same environment might be a bit repetitive, so I decided against rendering in Unreal.
Sir Wade had a lot of useful videos online, going through how to find an audio and what to avoid/look for. His ‘secret’ workflow when it came to dialogue shots and lip-sync tips, like acting towards the camera, having the mouth shape before the audio happens (light is faster than sound) and having asymmetry/imperfections.
I found an interview Sir Wade did with the head character animator on How to Train Your Dragon, it gave an insight into acting through animation, his workflow and general tips on animating. This video from James Baxter (worked on a lot of Disney titles) was more useful and much shorter, mostly going over the mouth opening and closing as well as the timing for this, when to hold and when to close suddenly etc. in order to avoid a boring linear up and down. He also simplified the mouth shapes and combined shapes into one, so I guess you can get away with combining sounds/mouth shapes especially when the character is talking fast or slurring words, limiting these mouth shapes migth be easier in 2D animation though which is what he was covering. I also looked at a live action analysis of the over the shoulder shot, which I though would be useful to me since I had two characters talking but not enough time to animate both of them fully.
I also collected some material to look at in terms of the actual mouth shapes for sounds, finding a YouTube video breaking it down a bit and some pictures I could look at while working.
I watched a lot of interviews and clips from Chris Hemsworth since I thought it would be cool to have Thor’s actual voice in the dialogue, but not just taking from a scene in the MCU, since it would be recognizable and the watcher might compare my shot to that scene. Sir Wade mentioned have some sort of change in emotion/tempo or interesting sounds in the audio and I thought this clip was good work with and had an interesting popping sound in it.
I found an interview with a Dreamwork’s animator that was more focused on the lip-sync and acting than the previous interview. He went into this idea of subtext, going beyond just animating what is said and going into the thought process of the character, a useful exercise he mentioned was animating the same character and audio in 6 different ways (different emotions, gestures, emphasis etc.) I obviously don’t have time for this but it was useful just to think about. I took some notes similar to how he did in his example to help inform my animation, typing out a simple script and what I thought the character of Thor would be thinking. I also checked out twitter to see how other animators approach this, alot using reference.
I did some further research into the two rigs for examples of dialogue shots and manged to find some on YouTube and Art station. I really liked the first one, showing the capabilities of the Thor rig and how much personality can be added. I wanted to do something similar with the style of my animation, the cartoony/stylised 3D style in these clips were very engaging. From comparing the first and second rig, I thought the eyes made a big difference on the animations, with the first one (edited from original rig) being more appealing.
I recorded some reference footage (bunch of clips on timeline, maybe 12 takes) and reviewed what I liked and didn’t like about them. I had the idea to have the pop sound be from him removing his finger from his mouth after eating, seeing something similar in Spider-Verse. I used this as reference for the style, as well as when building my little scene too. I also changed the eyes, just applying different lambert materials to parts of the eye to humanise the character a bit.
Some embarrassing reference footage, while I didn’t like the acting much in this one (kind of boring) the timing and actions felt nice and I had a bunch of other clips I could reference that had more personality in them.
I started blocking out the scene and testing camera angles, if I established the framing it would help with the rest of the animation, since I could kind of cheat, animating towards the camera. I wanted to focus on the pop first since it was the main reason for choosing this audio.
I went straight in on the pop, blocking out the main poses and timing for it. I was happy with how it was looking already, it felt very satisfying and had a bunch of personality with the cocky/playful eyeroll.
Then I blocked out the poses leading up to the pop, where he would bite the food, I liked pushing the anticipation for the bite, it was pretty exaggerated but still felt believable in the animation. I’d push this a bit further later on too when adding more squash and stretch to the head. Animating with the animpicker was very useful since I could animate with no controls on screen.
Next I went in and added all the key poses I wanted to hit throughout the animation, I was worried about the flexing pose, and the pose after it, it felt like a lot of movement for this scene, but I didn’t want to stay with the same silhouette throughout, since he was already resting on a table so had a restricted enough range.
This is the rough blockout at the end, once I was happy with the pop. I think it is a little awkward but it felt nice enough to move on and the individual poses were quite strong and fit the audio.
Taking from my reference footage, I added the head shakes/nods throughout and some follow through/anticipation for the poses, just cleaning up parts and adding keyframes throughout, trying to be mindful of secondary actions and offsetting a lot of movement, since Sir Wade mentioned asymmetry and imperfection in his video.
Looking at the 11secondclub, some online livestreams focused on lip-syncing and lip-sync tutorials in Maya helped refresh me on animating the lip movements in my animation.
This was the first pass(?) of lip-sync, I tried to simplify and combine shapes as much as possible to see what I could get away with, so the mouth wouldn’t just be going up and down and he wouldn’t be overenunciating. I needed to fix the timing towards the end and add more mouth shapes. The ending definitely had some questionable timing and needed more keyframes.
I noticed my graph was looking ugly so I used auto tangents, and spline tangents on certain peaks, adjusting where I though necessary. I also played around with the timing using the dope sheet but I’m still not 100% from when he says arrows onwards so I had a bunch of iterations of the file, letting me experiment without losing previous progress.
I then went into the details, adding finger animation and making adjustments on these grey controls to add ‘contact’ between the table and the fingers and elbows. I tried to clean up the graphs but it was getting confusing with the amount of keyframes, since I had a lot of offset happening and probably didn’t have the cleanest workflow over these few days. I decided to bring the Aang character in and animate his “lip-sync” which was just vague mouth movements since it was an over the shoulder shot. I thought he was too bright and took away from Thor so I adjusted his materials exposure levels.
This is showing the collision I faked with the table, as well as the little ‘set’ I built. I had some difficulties with the Aang rig but managed to get it in by copy and pasting it from another maya file.
I added some camera animation, with subtle rotations and very slight zoom in to give it a more natural feel, as well as adding depth of field. I thought the depth of field would help take away focus from Aang since I wanted the main focus to be on Thor, especially because I spent only a few minutes on Aang’s ‘animation’.
I’m quite happy with how it turned out, there’s some things that could definitely be improved but I rushed through this and have spent too long looking at this animation now to progress much more. It’s similar in style to the stuff I was looking at, and definitely can see the spiderverse inspiration just in the framing and sucking ketchup from his finger, it’s not too derivative, I tried to keep it dissimilar in the rest of the scene, adding much more energy and movement. I’m not 100% on Aang being added, it feels a little weird and think it might be better without but I wanted the over the shoulder shot, hopefully people don’t actually look at Aang’s animation though. There’s some moments in Thor’s animation that are a little awkward but I’m happy enough, I might’ve pushed the fingers a little too much though, and had moments where they seemed quite stiff and lifeliss. I forgot to document but I added a sphere to his finger with a transparent blinn material to make ketchup, which had a null group constrained to the finger control, then I just keyframed it to shrink and hide once it went in his mouth. It definitely would’ve been useful to send it to one of the members of staff for some feedback but it was so rushed in the last few days I didn’t really have time to.