At the start of the year, I had an idea of what I wanted to achieve with my final film. Coming straight from placement year, at GRAHAM construction, I had mainly spent my time in unreal engine over the year that I was there. I have always been intrigued the most by real time rendering technology, and the possible current and future implications on the industry that it offers. With unreal engine fresh in my mind, I knew that I wanted to achieve a film that was visually impressive, while also improving my knowledge within the software, with advancing skill in environment setup and and things like material blueprints, and although I didn’t know it at the time, how to create and use Meta Humans in Unreal Engine, for character animation using facial scanning data from my iPhone. This Blog will capture the research and development leading to my major project’s completion, and delve deeper into the more technical aspects of my production process for this film.
Film Pitch
I started my project with research on my pitch and pitch presentation, I knew that I had a story roughly thought out, and I knew what art style I was aiming for, which was realism. My desire to achieve a realistic look was in response to how much I enjoyed the film, Top Gun: Maverick. Although I had already came up with the idea of some sort of military or air force conflict in my film before The films release, It played a massive role in the inspiration and references for my film. mainly when it came to the design of my environment, some equipment modelled for the pilot character and the cinematography. I was taken by how the cinematography could be executed to such a high standard and be so breath-taking while still conveying the sense of risk, speed and excitement of being in a fighter jet.
For my pitch, I would also have to pick references for my models, clothes and character. For my character, I didn’t have to complete any concept work, due to the fact that I was aiming for a realistic look and therefore used references of air force uniforms to get the general idea across for my aim for the uniform. The look of the character was yet to be decided.
Some research was done into what model of jet I would want to use as reference for both jets in my film. I decided on a Jet known as the F-35. I chose this particular jet because of it’s smooth surfaces and sharp angles. making it look aggressive, while also easier to model and texture later on during my production phase.
I gathered references from online images and also films such as Top Gun: Maverick and also films like, Pixar’s Planes. specifically an at sea aircraft carrier I noticed in one of the trailers for the film. Other references from top gun included an environment which seemed much more appealing to me, a desert mountain range. I chose this option because it would allow for better cinematography and create the sense of speed during low to the ground shots.
a reference I used for the recreation of a top gun style of cinematography, in unreal engine 5, is linked below. This was a great inspiration at the start of my project to see what was really possible.
Top Gun Maverick in Unreal Engine 5 (youtube.com)
My reference to Pixar’s Planes trailer is below
Disney’s Planes – Teaser Trailer (youtube.com)
Pre Production
Pre-vis:
The first thing I wanted to complete in the pre-production phase of my film, was a 3D Pre-vis. Originally, I was going to complete this pre vis in Maya, as I would always associate pre-vis animations with Maya as the industry standard software. After some thought and advice from my tutor, I completed the pre-vis in Unreal Engine. This helped me greatly in getting a feel for how the environment setup and animation would go if I were so solely use Unreal Engine to animate and render my Film. The goal for my pre-vis was to gain a rough idea of how I would want the setup of the scene and cinematography to look, and to rule out any style of shots that my lecturers and I didn’t think fit the dynamic of the scene. This became extremely helpful as a resource down the line, especially when it came to dogfighting sequences through the mountains.
A link to my pre-vis animation is below.
previs James Greer 1 (youtube.com)
Niagara Particle and texture tests:
I was very interested in using effects which utilised plugins in Unreal Engine, like Niagara. Niagara is Unreal Engine 5’s VFX system. Allowing the user to add visual particle effects such as water, explosions, smoke and more. The idea behind this section of my research was to be able to edit Niagara templates and blueprints to create a smoke trail effect for my missile shots in my film. This involved editing a template such as the fire thruster. I found that with enough adjustments, you could make the large thruster effect seem thin and wispy. The results from my research are shown below in a demo I took of this work in progress.
When it came to my other technical research for my preproduction progress, I needed to find a way to achieve the look of a fighter jet’s afterburner thruster. After some digging, I came across moving textures. Utilising a blueprint I put together through online research, I was able to combine two texture files, one representing the blue tinted flame of the after burner and the other the pattern in which it would animate to. In this case, a molten lava texture. the test of this animated texture is seen below, along with the first result of it being applied to a cone shaped object in Unreal Engine.
The outcome of this technical research was a positive one and confirmed the method I would use in order to create an animated afterburner out of the rear of the fighter jet.
Marvelous designer
As part of my pre-production, I needed to think about how I was going to create the pilots uniform. Originally, my thoughts looked to sculpting software, such as Blender or z-brush. Throughout my pre-production phase and in preparation for my pre-production presentation. I realised I needed to have a uniform with material that moved fluidly with the characters body movements when in the cockpit of the fighter jet. The solution to this problem, was marvelous designer.
I had never used marvelous designer before, making it more intriguing to me, pushing myself to gain experience in a new software would be beneficially not only for the project but for my career. I found a tutorial online on how to create a flight suit in marvelous designer, this was exactly the style of uniform I was looking for and had completed another step of my planning for the style of the flight suit.
The tutorial I found and used for the flight suit is linked below.
Basic flight suit. Marvelous Designer. Clo3d. (youtube.com)
The next stage was to decide on the software I would use to 3D model and texture my models. These two software’s were Maya and Substance Painter.
Production
3D Modelling
Before and during the Christmas break, I began production for my 3D models, the first thing was the main component of my film, the fighter jet. With my references of f-35 jets, I started out by compiling images of actual blue prints for the body shape of the f-35 lighting II. I completed a rough 3D model of the jet, to be edited later on in the production stage. The reference I used to complete the first rendition of the jet model is seen below, as well as the first version of the 3D modelled jet.
The next 3D model that needed to be completed was the interior of my jet model. This model was the most complex of the hard surface models due to the fact it had to include so many different components like buttons. switches, screens, plates and joy sticks. For this model, I used reference images online for the interiors of many different jet cockpits, but mainly used a 3D model displayed on SketchFab. This allowed me to look at the closer layout and details of the cockpit by navigating the model, like a viewport in a software. This modelling process was tricky due to some mesh issues I was having when exporting the model into substance, noticing the textures were not applying to the model as well as they should be. Upon further investigation, I found that there were N-Gon’s in the mesh. When these were removed, the textures all applied properly and baked correctly. The final model of the cockpit for my jet is seen below.
The link to the SketchFab model I used for reference here is below
https://skfb.ly/DZz7
I also had to model an oxygen mask for my character, I had already used a 3rd party asset from SketchFab for my pilot’s helmet, so I decided to model my oxygen mask off of a similar shape of the mask on the 3rd party asset helmet. Having the shapes for both the mask in the jet, and the oxygen mask in the hospital scene at the end of my film, be quite similar when sitting on the pilots face would help with the symmetry that is planned for the match cut at the end of the film.
The link to the pilots helmet on SketchFab is below, the original files for this downloaded asset and my edited version are included in my submission link on Blackboard in the ‘maya files’ folder.
https://skfb.ly/6Us6z
The model for the oxygen mask is seen below.
More models were made for my project, more simple then the previous ones seen in this blog, such as the jet missile and a simple 3D object which I have modelled to resemble the shape in which I would like my jet flame to shoot out as. Then I made two objects for the bed in the hospital scene, a simple pillow and a duvet cover. These are all seen below.
Sculpting
Earlier on in my project, I was pursuing the idea of sculpting my character myself, in blender. I had multiple versions of this character sculpted. This was because it was my first time sculpting a character and I found to be a hard challenge under the time limits I was setting for myself under my schedule. The main issues I had with the character, was symmetry. Even with the symmetry option on I always ended up making errors which would come back to bite me later on in the process. Causing me to attempt to fix, but start again, multiple times when sculpting. I also struggled to achieve the correct proportions for my character when sculpting him from the start. This was because I am very much used to a 3D modelling background and found the cross over of using brushes, and not vertices, edges and faces, to be slightly challenging at first.
Despite the challenges I faced during my first sculpting experience, I enjoyed the learning outcomes of the character sculpting process and advanced my knowledge in character modelling and sculpting by a large margin and will use these newly acquired skills in any aspect of character creation in the future. Images of my first character sculpt are below.
Retopologising
The next step in my production phase was to retopologise my character, before UV mapping and texturing. Retopologising is something I have experience with a lot with hard surface modelling, but not so much when it comes to character development. I used multiple methods to while trying to successfully retopologise the character model, first using two plug-in’s withing blender, called “B-surface” and “loop tools”. This was seemingly successful at the time of exporting the model from blender into maya, so that I could UV map the character, however when imported into maya, the software wasn’t recognising some of the retopologised quad drawn surface. After some trouble shooting, I decided to move the original sculpted mesh into maya and retopologise in that software instead. This was a lot more successful but proved difficult as I had never retopologised a sculpted mesh before and found issues with the symmetry while editing the retopology that I did not detect in blender.
Once the retopologising was complete, it was time to think about texturing my character. The retopologisation of my character’s head mesh is seen below.
The other models in my project were cases where retopology was not needed, as they were hard surface models, to which their components were made of simple enough geometry.
UV mapping
The UV mapping of my character was straight forward and didn’t take too much time. The o9nly process I had to make sure I was doing correctly was the splitting of the body parts. This needed to be as accurate as possible so that the textures would apply evenly. The results of the UV maps are seen below.
UV maps for my Jet are seen below. These UV maps look more messy due to there being a piece of geometry stretching across the centre of my UV tile. This appeared when I re-modelled the front portions of the jet, where there would be air intakes. these were changed as I wanted them to seem more sharp rather than soft edged.
Substance Painter
I learnt a lot about substance painter throughout this project. The main thing I took from the learning process was how to create human skin. I never knew previously, just how much detail in the layers of substance painter goes into the texturing of skin. I learned that you have to think a lot more about the real anatomy of a human being, starting with things like undertones of the skin such as the blood under the skin of the face, or the areas in which you would find the bones closer to the skin. Even thinking about what parts of the human face would have more fat, therefore increasing the undertones of yellow to the skin’s surface.
I learnt and practiced this process with my character mesh from watching this tutorial on YouTube, linked below.
Skin Texturing Tutorial in Substance Painter | Files available – YouTube
The results from my experience are also seen below.
For the eyeballs for the character, a simple 3D mesh was made and using the premade eyeball smart texture in substance painter, I created the eyes for my character. these are seen below.
In my other models, texturing in substance painter was very enjoyable as I was able to freely explore the different generators, smart materials and thing even things like decals, or stickers, which were applied to the surface of some of my models. The jet interior cockpit textures model is seen below.
The finished textures for my jet are seen below.
These jet textures were also edited to create the textures for the enemy jet, by editing the layers and removing or lowering attributes such as the metalness. Then, also, removing some decals from the wings or rear flaps and roughening up the edges of some of the metal edges around the model. Finally, setting the enemy apart from the protagonist, was the kaki green colouring. These textures are seen completed below.
Flight Suit
Texturing the flight suit was something that was done slightly later in my production phase, as I had to complete the flight suit in Marvelous Designer first which took me longer than expected. I attempted first to bring in the textures directly from marvelous to substance painter, which proved more difficult to setup properly than to start from scratch. This lead to me exporting the marvelous designer flight suit mesh as an OBJ, and bringing it into maya, where I setup the UV maps for texturing in substance painter.
UDIMS:
I wanted to setup the UV’s for the flight suit across multiple tiles. This would allow me to import the flight suits textures as UDIMS, so they can be setup as virtual textures in Unreal engine. This was an easier outcome for me when in unreal engine, as the flight suit contained many different materials as different zips, buttons and pockets throughout the suit itself. Importing them all as one set of textures was extremely helpful and sped up the process of importing and setting up the flight suit in Unreal engine. It also helped when setting up the flight suit with the alembic files. I made sure to add my own separate layers of actual smart materials with all attributes turned off apart from the height. This gave the flight suit a texture and slight shallow pattern of an actual clothing material, avoiding my unwanted look of a completely smooth material over the whole suit. However, the pockets around the zips still remained smooth, as I feel it provided contrast to the materials on the main limbs of the garments, giving it a more interesting and realistic look.
the completed flight suit materials in substance painter are seen below.
Marvelous Designer
Marvelous designer was a new experience for me, but was a necessary one due to the requirements for movement of the flight suits material with the pilots body movements. It posed few challenged for me and was mostly a problem of time spent in the software. Everything about the software was new for me and therefore took a lot longer to get used to. The main hindrance was the fact that I had planned for more interior shots of the jet and therefore wanted to put a lot of detail into the pockets, separate materials, zips and buttons of the flight suit. Even down to details like the stitching on the seams of the suit itself. This was done by following the tutorial I had linked above in this blog. It went into a lot of detail and, even though it took longer than I had originally anticipated, ended up saving me a lot of time in the long run.
The stitching of the seams of the flight suit was quite complicated as there were a lot of overlapping seams that lead to different materials, this was for the likes of combining layers of pockets or split zips and even the waste band, which lead from around the back to the front of the suit. Adding buttons and zips was a worthwhile thing for some shots that would end up in my final film, but if I were to complete a project like this again, I would spend less time on the details of the lower half of the flight suit. I think I got too caught up in what details the tutorial was including in the flight suit, and although I am happy with the outcome of the work, I would definitely re-evaluate the amount of detail needed for the shots that would be included in my film. For example, the zips on the flight suit were good for looks, but when rendering out an alembic of the body movements in marvelous, they would rarely ever be visible and the same is true for the final outcome in unreal engine.
The final flight suit design is below, as seen when completed in marvelous designer.
Animating the baked animation sequence in marvelous designer was an okay task. The only problem I had was when I imported the baked animation sequence first, I realised I had to bake a blank 50 frames or so as warm up time for the ‘a’ posed mesh to rise up into the actual body animations for rendering. This made sure the pose of the marvelous garment matched the starting pose of the baked animation sequence, letting it remain attached to the animated mesh from the start.
Unreal Engine 5
Metahuman Animator
So far, in this blog, you have seen me mention that I was creating my character myself from scratch. This is not how I continued working on my project. I decided after the Christmas break was over, I was not happy with my character at all. I already knew that I have a love for using unreal engine 5 and learning about new technical skills which would not only interest me, but greatly benefit my knowledge and portfolio for any future endeavours. This resulted in me choosing to use a Metahuman as my main character. I had been excited to learn how to use metahuman in unreal engine 5 in the past, so this was the perfect time to do so. This came with a challenge at first of figuring out how I would animate the character, as I knew that key frame animating, the facial animation especially, with the metahuman control rig in unreal would prove to be slightly taxing. I found a solution to this in the ‘Live Link Face’ app, for unreal engine.
Live Link
The ‘Live Link Face’ app, is an app that records and streams very high quality animations of your own facial movements into Unreal Engine 5. There is an option to export these facial scans into unreal engine 5, and then use the Metahuman animator plug-in to have the metahuman you designed perform these processed facial movements. The process is quite difficult to setup at first, but once I got used to the workflow, I thought it was a great solution to use to bring my character to life. Once I had made my metahuman, I imported him into unreal engine and began to setup the Live link performance with the metahuman animator.
First off, you have to record calibration takes. This is so unreal engine can use these takes to plot calibration points on your face in the different calibration poses. These takes would be you looking straight, then looking from side to side and smiling to shoe your teeth. You can then ‘promote’ these poses in your ‘Metahuman identity’, in unreal engine. after this you can select ‘prepare to solve’ within the metahuman identity. This then calibrates your facial positions you just selected, to the metahuman identity, which may look slightly like you. This will then be used to perform any of your facial animations.
The next step is creating a ‘metahuman performance’. This is where you will process any take you record on the live link face app for any other metahuman you wish to use. You still select your metahuman identity you have just created, but use the desired take from the live link app, instead of the calibration takes. Once you process this, it will be ready to create into a new level sequence, giving you the metahuman performance in that sequence. By then selecting the metahuman performance head in the scene and replacing the skeletal mesh with your own designed meta human, you can then see how the animation will look on the new metahuman facial rig.
From here, it’s pretty simple. You need to bake the animation from the metahuman performance and apply this to your new metahuman actor, once it is in the sequencer. Now you have a fully functioning facial animation, but it automatically detaches the head from the body when you add the facial animation and again when you add a baked body animation that you have animated through key frames in unreal engine. After many hours of trouble shooting, myself and many online sources, I figured out that you have to bake the animation sequences back onto the metahuman control rig, and replace the head and neck key frames on the metahuman body rig, with the ones from the facial rig, then delete them completely from the facial rig. This successfully combines the body and facial animations, while not losing any of the neck movements included in the recorded takes from your own face in the live link face app.
This experience with metahuman animator has shown me a way to create extremely realistic characters for my projects, while also learning new skills in the technical aspects of unreal engine 5. The application of the metahuman animator with the live link face app feels limitless, and has helped me achieve a better look and emotion for my final film than I could have ever hoped for. I am very happy with the result.
Below are some recordings and screenshots of my metahuman animator process.
A video that helped me a lot with the entire process is linked below.
MetaHuman Animator Tutorial | Unreal Engine 5 (youtube.com)
USD Assets from substance to Unreal Engine
There were a couple of models that I had issues with importing from substance into unreal engine, due to the fact that materials were too high in numbers to accurately see which ones belonged to what part of the model. For example, I made the mistake of not naming the materials on my jet interior cockpit model and therefore decided to import it as a USD asset. This meant that I would export not only as textures but also the mesh from substance, also giving me a single, combined, material to apply to the mesh itself. this was extremely helpful in the long run when preparing to animate the jet in my scene, as it saved a lot of time not having to go back and organise the materials to let me export and import them efficiently.
Unreal Engine environment setup
I made my environment landscape itself using Quixel Bridge assets. The models I used were all massive terrain and cliff based assets, utilising there materials with each other. For example, using the base colour texture from the sandstone desert mountains for the more red coloured desert cliff assets, so that they could match and still hold up quality of detail wise for certain distances.
as seen above in one of the action shots from my final film render, the environment matches to the same desert like colour. Whereas these rocks and cliffs that make up the archway you see here, used to be very orange and red at their base colours. Below are images of the environment models I used and edited for my final film, from Quixel Bridge.
I had an issue when rendering out some background assets at the end of production, these were that some of the environment assets were failing to show up in the renders when they were definitely key framed in the sequencer. I solved this by converting the static mesh actors themselves to ‘spawnable’ in the sequencer. This meant they would be visible in every shot as long as I went through and key framed the environment in the sequencer and made them spawnable. The same issue arose with the jets, they were also not showing in my renders. The solution for this was nearly the same, however I had to change the static mesh actors into blueprint class actors. Then, had to go through each shot and replace the animated jet mesh in the sequencer with the new, spawnable, blueprint class actor and paste the key frames into the new jets in the sequencer. Once I did this, I had no more trouble with objects not being visible in the sequencer.
Post production
When it came to the post production of my film, I edited in Premiere pro, This is because I have experience In premiere pro throughout university and previously in school with things like cutting together footage and mixing and editing or placing audio. A large part of my post production process however, was about compiling the right sounds hat would suit my film. All together I compiled nearly forty different audio files, all copyright and royalty free from websites like free sounds. Their website is linked below.
By the end of my audio editing process, these near forty sounds turning into nearly 230 separate edited sound audio tracks in the timeline. To get the right sound for the jets and all the effects of turning, shaking, wind and so much more, I compiled sounds into my film that I wouldn’t have thought of using before. Earthquake sounds, for example, were used to replicate the sound of the shakiness and dep rumble from both inside and outside the cockpit. Blow torch noises were used for the enhancement of the afterburner flame, firing back from the jet. Even simple whoosh and impact noises were used and scaled or pitched or reversed for the enhancement of some slight turns or banking movements of the jet.
I enjoyed the post production process greatly and think that It is what made my final film truly come to life. Making any sort of emotional reaction to what’s happening possible. Whether that’s through the sound of the roaring jet engine or the music towards the final scene, it transforms the experience as a whole, and I am very glad to have spent more time on it than I thought I needed, or wanted to, before beginning the process.
The full timeline of edited sounds and cut together clips for my final film edit is seen blow.
Conclusion
By the end of this Final year project, I have learnt more about the process of animation than ever. Whether it’s been learning about the pipeline or workflow with new software’s I’ve used, like marvelous designer. Or learning new technical skills that have helped change the way I think about creating animation or films, forever, with metahuman animator. This final year has been the best experience at ulster for me yet. Allowing myself to tackle a creative project on my own has been both, a reality check, and a great enjoyment for me. It has, with certainty, better prepared me more, for what is on the other side of university life. Thank you to all of the lecturers and tutors who have been there, for me and the rest of the class, throughout this final year project and also since the beginning. Your work and attention does not go unnoticed and I appreciate every bit of knowledge and help from all of you along the way. I am very happy with the outcome of my final film and think it is some the best work I have produced so far, and am satisfied with the research, planning and execution of this project overall.