Technical Wizardry – creating photo realistic foliage from photos [ue5]

Developing Skills – Practice 1

Continuing from my last blog on 3D modelling and photogrammetry practice, this week’s blog will use similar techniques to create photorealistic fallen leaves. In this blog I will go through the steps to create 3D models of leaves from single photos and create a photo realistic environment in Unreal Engine 5. This method can be used to create a wide range of meshes and showcases a simple way to import and create texture for 3D assets. The brief of the weekly project was about creating an article for 80.lv, a popular website for the creative industry which, amongst other resources, provides useful tutorials.

Image examples from Unreal Engine 5 of the leaves created using photos.

For this project I decided to use my phone’s camera, Blender, and Unreal Engine to create an autumn forest scene. I was inspired by my local parks in north London and my passion for photogrammetry. I thought that creating 3D models from a simple photograph shot with a phone is fantastic. The physical layout tool will then help to place the actors in the level procedurally, according to the collision data of the mesh. 

Initially, I went to the park and picked up a few autumn leaves to take photos. After, I imported them to Blender to cut out the silhouette. I used the image to plane add-on, created another plane mesh and merged it to center to start extruding the vertices along the edge of the leaf. 

Image 1 – Cutting the leaf’s silhouette in Blender

Once I had cut out the silhouette, I created a few faces using the knife tool and applied the normal weights and solidify modifier to create a very thin static mesh. Then, I used the smart UV unwrap feature and textured the mesh in the shader editor in Blender. This process could be repeated for as many leaves or objects as required by the project. In this instance, I only used one mesh. 

Image 2 – Blender Shader Editor

Once I finished working in the shader editor, I imported the image texture into a website called Photopea.com. Here I created a normal map for the leaf static mesh. 

Image 3 – Creating normal map from image texture in Photopea.com

Now that I have the base colour texture, normal map, and static mesh of the leaf, it is finally time to import it to Unreal Engine. Here, I created a material for the leaf with a world aligned blend node to give colour variation to the leaves based on their location and a couple of vectors 3 parameters to be able to change the colour and provide extra variation when using the material instance of the parent leaf material. I will be duplicating the static mesh and assign different instances of the material with colour variations before spawning the leaves with the Physical Layout Tool. 

Image 4 – Unreal Engine material editor for the leaf static mesh

Image 5 – The leaf static mesh in Unreal Engine’s editor

In the meantime, I created a landscape and started modelling it according to some autumn forest pictures I used as a reference. Afterward, I used the modelling tools in UE5 to bend the leaf and give it a more natural look. For this process I used the lattice tool available from the deform tab of the modelling mode. 

Image 6 – Bending the leaf using the modelling tools in UE

Before proceeding, if you do not have the Physical Layout Tool installed, you would need to browse to the Unreal Engine’s Marketplace and download it: Physical Layout Tool. This is a free tool that gives you the ability to spawn and move assets according to their collision data, allowing for physically accurate collisions.  

Once enabled, the plug in adds a new mode on the drop-down mode selection menu named Physical Layout Mode. In this mode, you will be able to add static meshes similarly to how the built in foliage mode works. Note that this mode could be intensive for the computer, so I do recommend saving the project before using the new tool as a precaution. Also, it is important to select the spawned meshes and click on the bake selected actor into instance mesh to increase performance. 

Before starting to spawn the meshes, remember to change the material of the static meshes with the material instances with colour variations and tick the place with gravity box to avoid the leaves from flying away. You should also add a minimum and maximum range for the rotation, position, and scale to create a more realistic look. 

Example of leaves placement in UE with the physical layout tool

Note: the physical layout tool can spawn multiple meshes simultaneously, example available here: Saeid Gholizade showcasing the Physical Layout Tool Video

References:

80 Level (2024) 80.lv Available at: https://80.lv/ (Accessed 13 Jan 2024)

Battiston, L. A. (2023) Autumn Fallen Leaves. Unpublished

Blender (2024) Blender. Available at: https://www.blender.org/ (Accessed 12 Jan 2024)

Epic Games (2024) Unreal Engine 5. Available at: https://www.unrealengine.com/en-US (Accessed 13 Jan 2024)

Ivan Kutskir (2012) Photopea.com Available at: https://www.photopea.com/ (Accessed 13 Jan 2024)

Saeid Gholizade (2021) Physical Layout Tool Available at: https://www.unrealengine.com/marketplace/en-US/product/349bf2d4b1a945fa8f31db80fabf8058? (Accessed 13 Jan 2024)