Final Year Project – Lil
Ideation
Goals
When thinking of what to create for my short, I first wanted to consider what I was hoping to achieve from it. I knew from the get go I wanted a heavily animation based film, with lots of movement and human characters. So the idea of an action based one shot was hard to shake. But also a scary prospect given the fact I had no idea what I was doing.
I also wanted to experiment and gain experience animating with cloth sim and hair sim. In order to have a better understanding of the difficulties within these workflows and just out of general interest to see how they might work.
My last goal was to create stylised characters, successfully. Modelling and sculpting is not my strong suit so I wanted to challenge myself in these areas. And force myself to learn more.
Getting Started
With these goals in mind I started looking for references and inspiration for the stylisation of the characters. Eventually finding these:
Detective Diaries Chapter 1: Fish Out of Water | Hololive
Watch Dogs Legion Cinematic Short | Tipping Point
Both of these styles stood out to me as something I would love to attempt recreating. They had a decent amount of room for error in regards to texturing and modelling, with a looser style for both. However both could still hold enough emotion to carry a serious scene. Once I figured out the style I was going for, I started drawing up some concepts in photoshop.
I wanted to create characters that didn’t stand out as too abnormal in their designs, given the realistic setting. It would be odd to push the boundaries of clothing styles and such, as I felt it would distract from the scene. So I kept their designs slightly more low key. I also tried to design each of them with the capabilities of Marvellous designer in mind, having done some research into what looks good and not in terms of simulation. During this time I was also looking into various methods of rendering, cloth and hair simulation. Also into methods of importing characters and character creation. I decided early on I was going to use Unreal to render my scene, not only for the lighting optimisation. But also the Hair simulation and environmental capabilities. Here are a few of the videos I watched at that time:
Based off this research i was confident that Maya, to Marvellous to Unreal Engine 5 could potentially work for my project. So I started watching videos and tutorials based on this. So by the time I was presenting, I had fully conceptualised my main character creation workflow.
Workflow Plan:
Model Layout – Maya
Sculpt Detail – Blender
Model Retopo and UV creation (UDIMS) – Maya
Texture creation – Substance Painter
Rigging – Advanced Skeleton
Animation – Maya
Hair creation – Xgen Maya
Cloth sim – Marvelous Designer
Import Alembics, Hair Cache and Skeletal Animation – Unreal
Tweak Sim settings/Render – Unreal
When I presented these ideas In presentation one. The feedback on the character designs, style and workflow was mostly positive. So I got started with modelling the first of my three characters.
Production
Character Creation – Lilly
Creating Lilly was a huge learning curve for me, as not only was she the first partially realistic character I had ever attempted to Model and Sculpt. She was also my first attempt at Rigging, Character Texturing and Cloth creation. So it was safe to assume she was going to go through many reiterations.
Sculpt
I began by laying out basic shapes to map out the proportions of the character. I then Moved this into Blender, and using its proportions I began detailing my sculpt. I watched many videos on sculpting during this time, here are a few listed below:
Retopo & UVS
Once Lils sculpt was finished I moved it back into Maya, and began Retopologising, and creating UVs. Retopology took a long time, I wanted to make sure I was doing it right, since I knew if I wanted to create one long shot, I would need to save every bit of space available. So I found a great course with lots of resources by Danny Mac 3D. I then UV’d this mesh for export to substance.
https://youtu.be/l1v0Vwpnxlw?si=_OvkktrPkynglTM3
Marvelous Creation/Experimentation & Substance Painter & Advanced Skeleton
I then began experimenting with Marvellous Designer, watching tutorials on how to create the desired assets and how the UVs work. At this point I was pressed for time with the upcoming presentation so I only got her shirt done to experiment with the cloth sim. Looking back the simulation settings were off and the cloth sim looks flat. However for the sake of this experiment the simulation settings didn’t overly matter. I then moved this into Substance where I textured using reference from similar styles I wanted to replicate. (Was very proud of how her eyes turned out) For all of my character textures I baked the Sculpt detail onto the Low Poly mesh. Which I feel gave me higher amounts of detail and better end results when working within Substance. I then attempted to create a first rough pass of her rig in Advanced Skeleton. Getting familiar with its workflow and extra features.
Hair creation Xgen/ Import tests
I then started to create the Xgen I would use for my character, this would end up being 1 of 3 I created, because the first 2 ended up being quite low quality for the type of simulation I was going to be using. This was due to mainly lack of overall knowledge using Xgen and Unreal Simulation. However I was again really happy with how this turned out. I learned how to create this Xgen from watching tutorials by JHill. One is linked below:
Exporting the hair from Maya and importing it into Unreal was a challenge for me, it took a lot of fixing but eventually I was able to get my head around it. Here are a few of the many videos I watched on this topic, I combined multiple methods until I found a workflow that worked consistently for my use.
Animation and Unreal Importing
I exported the Skeleton, then baked out the animation into a Skeleton Animation. This took a lot of trouble shooting to complete successfully. But eventually I got it working and was able to render.
I was super proud of this end result, because it was proof to me that my research and insight into my workflow had payed off. It also gave me plenty of time to fix visible problems with Hair sim, Rigs, body proportions and cloth physics in later versions of Lil. These changes could also be applied to the other characters before beginning them.
Scene Creation – Block Out
Now that I was confident my method for creating my scene was going to work, I began blocking out the scene. Figuring out where the camera was going to go and what the characters were going to roughly do. While doing this I started to layout the environment. I created the general layout, with windows, doors walls and roof etc. I also downloaded Shelves, ice cream drawer and a petrol pump from Sketchfab to add to the overall clarity in conveying what I was going for.
Once this was finished I started Sculpting the other two characters using the same extensive workflow I covered with Lilly already.
Hunter_01 – Character Creation
Sculpt
For Hunter_01s character, I wanted to maintain a stylistic approach, but wanted to also make him formidable and grounded. I extenuated the sharp areas of his face, and tried to add stylistic elements such as an over exaggerated brow and Scar.
Retopo & UVS
When Retopologising Hunter_01 I attempted to even further optimise his topology, creating more density in areas where needed and less in areas that aren’t seen. I also UV’d him, this time creating a separate mouth texture to help with fixing the abnormal AO maps around the lips which seemed to bake out wrong with Lil.
Bake
Photoshop fix
I then downloaded a scan of some shoes off, Sketchfab and retopologised them to be better optimised. Then UV’d them to be textured within substance.
Marvellous Creation
Created the clothing for him in Marvellous. This is the more recent clothing, after having gone through multiple feedback based changes to his collar and coat. I unfortunately deleted the old file. Since this was my first time using multiple assets to be textured in marvellous. I did some research into how to go about organising the materials/ textures since Alembic caches from marvellous don’t have per poly materials. I decided to go for UDIMS, using each square for a different material in Marvellous, I would then import this material as a virtual texture once its exported into Unreal Engine 5.
Texturing
Here you can see the original Jacket. When texturing Hunter_01, it was important for me that he looked sickly and drab. So I tried my best to replicate this look onto him. At the time I didn’t notice how glass eyed he was, however this was later fixed.
Rigging – Advanced Skelton
When rigging Hunter 01, I put more effort into creating effective skin weights to better sell his emotion. I watched various videos on this topic, and overall I am really happy with the result.
Animation / Marvelous Sim / Import Test
In order to stress test the Marvellous sim and challenge my import workflow again, I decided to create a quick running animation using Hunter 01, then ran it through a marvellous sim editing some of the simulation values. While in Marvellous I also edited the Garment, giving the sleeves and ridges of the clothing a double layer, giving the illusion of cloth depth. I then exported it all and imported it into unreal. Learned a lot about Marvellous during this test, specifically the 0.25 speed and thickness export option. Which instantly levelled up the finished look in my opinion.
Hunter_02 Character Creation
Sculpt
Sculpted Hunter_02, for each of the character sculpts I tried my best to capture their main emotion within their default expression. Since this means the rig doesn’t have to work as hard to achieve it. For Hunter_02 he is scared and anxious, so I tried my best to get this across in the sculpt as best I could.
Retopology/UVs
Again, I tried to level up my retopology from last, counted the amount of squares and tried to keep them as clean as I could. I was super proud of how his hand came out in particular. I feel retopology is something I have learned a lot about this year and that I have somewhat of a grasp on now, which is more than I could say when I was starting out this project.
Texturing/Marvellous
I wanted to try a trickier Marvellous clothing creation, so I attempted a Puffer Jacket following this tutorial:
And a beanie following this one:
Making the puffer jacket and Beanie was a serious learning experience for me. I learned about new tools and new ways of simulating clothing that I hadn’t heard of before. Showed me the capabilities of the program and opened my eyes that i have a long way to go when it comes to using Marvellous effectively for Garment creation.
In regards to texturing, regrettably I feel I rushed Hunter 02. I was anxious to get started animating and neglected the quality of his textures. I am hoping that by the end of year show I will have time to revisit his textures and bring a little more life to his face and eyes.
Lil Rig 2, 3, 4 and 5
Lilly was a tricky character to get right, her proportions went through various changes, her Marvellous kept evolving both in textures and fitting and her rigs were continuously changed to better suit her range of movement. I ended up doing a multitude of tests using the various different rigs, for instance here’s a test using a Rig and a groom that was scrapped completely.
Since I was learning as I went. A lot of time was spent on Lilly trying to get her right. While continuously, unknowingly pushing her in the wrong direction. For instance this was a change I made to the proportions of her legs.
The change is from left to right.
These halts were mostly down to my inexperience, pushing too far without proper planning, then being forced to take ten steps back. For instance I made a version of her that I was happy with, however when trying to match her scale wise with the hunters and the gun. The rig and its proportions, just weren’t right. And when working with Multiple programs, and Xgen grooms, Marvellous etc, these multiple set backs end up causing quite a bit of trouble. I have learned since to take my time when preplanning scales and proportions, before jumping into a fully fledged rig and xgen groom. 4 times…
I eventually was able to make a rig for her that I edited to include Double Knee joints, which helped a lot with her crawling and stumbling. Again, I spent a long time on the Skin weights for her, ensuring it looked decent while also trying to limit geometry going through itself in areas that were clothed to help with the simulation.
However because of this I was able to create a new Xgen hair groom which was made cleaner and in my opinion looks better than the first two.
I also overhauled her textures on the Marvellous garment. Due to it having to be remade and Re UVed, however the end result is a definite improvement in my opinion. I also experimented with opacity masks to feather the edges of the garment. But wasn’t able to bring them across successfully into Unreal.
Environment Tweaks
After the rigs were done it was time to fix the environment previs to make it more accurate with what the environment would actually look like. I modelled the store itself, the floor, and laid out the landscape loosely based on the block out. I modelled various debris on the floor and some planks for the window, as well as some cans that were strewn about the scene. I also downloaded various models off Sketchfab which I mostly re-UVd and would end up retexturing completely. However I stopped myself before I got too carried away modelling the scene. I was trying to manage my time, and I didn’t want to go too detailed yet as I wanted to leave room for the animation to change the proportions of the setting. Meaning the animation timings come first, setting moulds around them. This environment proportion change ended up happening a lot, so I was happy I stopped myself before making this mistake.
Unreal Tests, finalizing workflow
I wanted to be absolutely certain that I could invest as much time into the animation as I was going to, and in order to be certain of that, I had to work out a few more things within Unreal Engine 5.
Hair Sim Stress test
One of those, was hair simulation, colliding with my characters geometry. This proved harder than I initially thought it would, a lot of tutorials I followed, wouldn’t work. Other than this one which I found after a good few days of searching over the course of the project:
Because of this tutorial, I was able to set up my character to have fully working hair simulation. As shown in this render.
I also wanted to make a decision on whether or not to use a Painterly post process filter. So I tested it inside of unreal for a long time, playing with settings trying to make it work. I even rendered out a quick earlier animation with the post process effect applied. However I eventually decided against it after seeing how the filter interacts with camera focus pulls.
I also did a test playing about with camera settings and stuff a little bit more, this helped to inform me as to what film making style I was going to try replicating.
Prop Creation
To save time, and to get started animating as soon as possible. I downloaded the Crossbow, Gun and Walkie Talkie off Sketchfab. The gun model in particular had broken UVs so I had to Re UV the textures then use a UV set list to reset the squares to what they used to be on the geometry. Each of these props were edited within Substance painter to give them a new look that fit the style I was going for.
Animation
Without Reference
Starting my animation, I was convinced that I would be better off laying out my scene without reference first. The reason for this was so that I could get a better grasp of the timings of each scene, however looking back this was not the right approach. I first followed the “Blockout” with my character, changing the environment where necessary. Eventually I recognised that the actual animation was going to be a lot longer than I had initially anticipated. I also was attempting to tackle it using no reference which I would later come to regret. Here is an incredibly early playlist of the sequence
At this stage I hadn’t yet cut the beginning from my animation, so it would’ve started with her at night, watching Vinny die, then a match cut transition to the scene it starts in now. I was happy with this rough block out, so I began work on my second scene.
2 weeks of work.
The animation was moving, however I knew it was lacking rhythm and naturalness in its posing and timings. Around this time I went away on holiday for 2 weeks, the holiday made me re-evaluate my approach, I was going to record reference. It also gave me an excuse to work on some sound design stuff. I created two different sounds both of which I had hoped to use in the final audio edit. I only ended up using one however.
Riser End Used:
Train crescendo Not Used:
With Reference
When I returned home I immediately got stuck in recording references, recording took a full day since I made sure to properly pre plan not only the movements but also the amount of steps and general timings between each take. I really didn’t want to create more work for myself in the long run by having to refilm, re-edit and reimport. I then edited each of these for each character with all of them synced up so that when I eventually would bring it into Maya, I could use each reference simultaneously. This ended up being extremely useful since it meant when I was bored animating one person, instead of having to switch the image file, I could just hide or show which ever image plane I was using in the viewport. Making the animation workflow way easier.
However, since I added reference over the top of existing animation. Nothing was matching up. All the key poses were different and the timings were less interesting. I wrestled with the keyframes a bit but eventually ended up restarting the animation. Basically scrapping two weeks of work in order to rework it. However I take that as a valuable lesson to always use reference and as a reminder to myself to not get too carried away when you don’t have a solid plan.
1 week of Reference animation progress
4 weeks of Reference animation progress
After 4 weeks I had successfully retimed certain moments to make them faster or to have more ease in and ease out. This was as far as I could go without Voice actors.
Voice Actors
I began looking for voice actors on Fiverr the night I finished animating. I found the two I was going to use that night. Shomii and Matt. I rendered out the body mechanics in the unreal scene to give the voice actors a better idea of what the scene would look like. I also provided timed subtitles which they could use to base their acting off.
Once that was all sent off, I had to wait for them to get back with their lines so I moved on to setting up and texturing my environment.
Texturing Environment
Store
I began to texture the various areas within the store, creating materials for tiles on the ground and falling paint on the walls. I then set up substance to export straight into unreal to allow for an easier and much quicker feedback loop between the two software’s. This way I could better account for the lighting and texture map changes. I think overall the Store looks ok, however if I have time I would love to change a few of the textures in time for the end of year show. To add a little more colour and to reflect more natural light.
Ground Close Ups
For the close up ground meshes, I downloaded mega scans off of Sketchfab, I then retopologized them and edited their textures in Substance Painter as to better blend in with the environment and also to blend in with each other. I am super proud of how they turned out.
Voice Actors Finish
VA Audio – Hunter_01
Due to the file size restrictions. I am afraid I cant upload all of them but all in all I received over 25 minutes of audio from the Matt and Shomii.
After having received back the audio from both of the voice actors. I got started, gathering multiple takes and cutting them together to create the lines, at points using one word form a specific take and another word from a different one combining them to get the best inflections. Even pitching the vocal down slightly to get a deeper tone. I then started mixing the audio, creating a 5.1 sequence and keying spatial audio to each of their voices to give them more life and presence within the scene.
I rendered each of those out as Stereo Tracks to be used for facial animation in maya
4.5 weeks of reference animation progress (Final Submission Animation with Facial Animation)
Marvellous Designer Simulation
With the animation finished, it was now time to start creating some Marvellous clothing simulation. I started by exporting what I needed for each export for each character. For instance Hunter 01 needs his clothes to make contact with the arrows, and the crossbow. So I exported his Geo as an alembic, along with the specific frames that the alembics of the arrows and the crossbow would make contact. As seen here:
However for Lilly, she needs contact with the gun, the ground, the roots the ridge and at one point even Hunter 01. So optimizing the amount of frames suddenly becomes far more important.
The Simulation of the clothing took longer than I ever could have anticipated. For instance Hunter 01s first simulation took around 9 hours to finish, and that was only the first half of his animation. This meant time management, prioritizing getting marvellous done quickly was important to keep track of. Especially in the last few days leading up to submission. I also ran into errors importing directly from Marvellous to Unreal, however when I imported the Alembic from Marvellous to Maya then exported from Maya, and then imported into Unreal. It fixed this issue. It was also very hard to keep on top of the simulation in terms of fixing errors as they go. Especially with fast movements like when Lilly takes off running as she moves quite fast and the cloth sim very often couldn’t keep up. The alembic caches took a long time to export and the application often crashed or without warning would reset certain settings. Ruining the simulation. However like with any program, the more i practiced with Marvellous the more i learned how to fix these issues. It takes time until you recognise how to fix some of these issues, and I like to think I was able to get a good grasp on the basics of Marvellous Designer during my time using it. I also think the end result was for the most part worth the hassle, however even still in the final submission there are a few small errors with it, that I am hoping to fully polish using the combine amin layers option, before the end of year show.
Geometry Cache Export
When exporting the Crossbolt and Arrows. Due to the rig set up I had used I couldn’t just click on the geometry and hit export. Since the keyframes weren’t transforming on the mesh, when I would attempt this the cache would mess up. So I found a way of parenting a locator to the geometry, baking the movement transforms of that locator, then unparenting the crossbolt from his hand, and reparenting it onto the locator that I just added. Then simply baking the transforms from the locator onto the mesh. Which I could then export as a geometry cache successfully.
Unreal Engine
Imports
Due to the amount of preparation I put into my pre-production in terms of importing animations, skeletons and Hair Caches to Unreal Engine 5. I didn’t run into any issues at this stage when importing the animations or the environment. However the caches took a long time to import into my scene, at one point the import took about an hour. Which again meant time management was key if I wanted to get my project done on time. However these caches would also slow the system down tremendously. I eventually learned to import the caches once I was pretty much done creating the environment, since this gave me better run time when fixing the other Elements such as lighting, camera work and texture refinement.
Trees/Foliage
I combined various environmental packs when creating my environment for this short. Adjusting textures, saturation, base colour, roughness and vibrance to better match each of the packs together. My inspiration for the look of the forest was heavily inspired by the gameplay reveal for The Last of Us Part 2. But I also loved the over exposure effect present in the watch dogs legion short. I tried my best to have a healthy mix of them both in my scene. So I began placing foliage based on this reference, keeping in mind natural paths, leaves and where the trees would naturally grow more dense and less dense.
1st Pass Render Issues
After viewing my first render of the film, I noticed that I had forgot to un loop the walkie talkie cache, as a result the walkie talkie could be seen floating by the window for a few frames, ruining the immersion and distracting the viewers attention, as seen here:
So to fix this I: Checked which frames it was visible 1315 – 1385. Deleted them, then re-rendered the frames.
Sound
I immediately imported my render into adobe Premiere pro, along with the timed stereo directional exports of each of the characters voices. I then went on freesound.org to start looking for more specific samples that weren’t on the sound library.
I knew I wanted to have grounded and immersive audio, and so convincing Foley sounds were important to get right, both in mixing and in timing. I downloaded a range of cloth foley sounds, such as fabric crunching, towels flicking and even hands rubbing together, to create the sound of her clothes rustling as she moves. I focused on mixing that audio in an attempt to create convincing sounds that would hopefully ground Lilly in the scene. I did the same for the gun with small metallic scrapes of a belt buckle, and with slapping noises for her feet hitting the floor. Little details like this took ages, and with something this quiet and subtle it was important to put a good bit effort into making it tense and present, but also not distracting. Same could be said for the Hunters, who’s boots needed to scrape off the floor in a way that sounded convincing while also both having heavy coats and weighty presence.
I think given the time I had I tried my best with the audio, I do slightly feel that the mix is drowned out by the low rumbles too often, and that more detail to the individual sound elements would help to sell the scene more. However that being said, I feel that the audio correctly conveys the feeling of the short. Which is a huge success in my book. And I am really proud of some of the sound in the mix as it is.
Link to Final Film: https://drive.google.com/file/d/1J9ACVCWYAa7BQL8WgeCwNoXfhIzN8X_8/view?usp=sharing
What didn’t get finished?
Due to a time crunch towards the end of the deadline, I ended up having to rush the render and sound pass out as soon as possible. Meaning both the quality of the render, and the quality of the sound took a massive hit. Not only that but if I had better managed my time I was going to draw over in 2D to add some bullets and muzzle flashes to the guns. I was also hoping to add an extra alembic dirt explosion over the top of the fracture explosion I had used on the bullet impact with the ground along with some 2D lens dirt effects. However that didn’t end up happening for this submission. Looking forward to the end of year show I would love to be able to finish off these Elements as well as better refine the cloth sim and some of the environmental lighting and textures.
Reflection
Reflecting on this year, I feel I have worked hard, and had a clear goal from the start. I also feel that out of my three starting goals I mostly achieved all of them. However I feel that writing this has made me realise how often I jump head first into starting something without proper planning or consideration into other options. Whether it be the fully skinned rigs that were scrapped, due to overlooking flaws, or if its the 2 weeks worth of scrapped animation since I didn’t use reference when animating. If there’s a takeaway for me from this year, it would be to slow down and properly plan. Before investing time into something. That being said I am still incredibly proud of the end result, I wasn’t expecting to actually be able to pull off the one shot take format at all, so its safe to say I’m very happy with how that turned out. I also feel great about how the characters look in my short, I feel the inspirations shone through well and that the overall visual style in my film is consistent with itself somewhat. I look forward to further refining it as we head towards the final year show.
Quick thank you to all the lecturers who work hard to ensure we bring out the best in what we do. Very thankful to have you all as our teachers.
End