Kai Mallari

Given my extensive background using Photoshop, I quickly picked up the fundamentals of channel packing. Channels within an image are just opportunities to store greyscale texture data such as Specular, Roughness, Bump or AO.

By combining multiple textures into a single file, you decrease the overall number of texture files your application needs to handle leading to lower memory consumption.

Additionally, I discovered that you can extrapolate the Blue channel of a Normal map from the Red and Green channels using a bit of math.

Unfortunately, because we're not using the typical Normal map compression method, this will introduce compression artifacts -- that means being selective about where this use used; potential areas are terrain, foliage, rocks, and meshes which have a noisy texture which will also be receiving a detail Normal map.

Because we're using the Blue channel of the Normal map for a different texture, we'll need to reconstruct the lost data somehow.

In Unreal Engine 5, we do this by appending the Red and Green channels using the `AppendVector` node, then we normalize the values to -1 to 1 using the following:

• Multiply the result by 2
• Subtract the result by 1

Next, run the result through the `DeriveNormalZ` node and you've got yourself a Normal map!

In a 4k workflow using the full range of Specular, Roughness, Bump, Displacement, Emissive, Opacity and AO maps, texture packing is a great first step in reducing the memory required for a material.

By combining multiple textures into a single file, you decrease the overall number of texture files your application needs to handle; however, this may have a negative impact on your textures.

Bit depth is the number of bits used to define the colour of pixels in a digital display system. It determines the number of colours that can be displayed.

When talking about bit depth in images, we usually refer to the number of bits used for each colour channel. For example, in a standard RGB image, there are 8 bits used for each of the three colour channels (red, green, and blue). This setup allows each channel to express 256 different shades (2^8 = 256), resulting in a total of about 16.7 million possible colours (256^3).

Limitations of Channel Packing

Limited Colour Depth:
Each channel in an 8-bit image only holds 256 levels of grey, which might not provide sufficient precision for certain texture types. For textures requiring subtle gradations or high dynamic range, such as displacement maps or detailed normal maps, this limitation can lead to quantization errors or visible banding.

Compromise in Texture Quality:
When packing high-frequency detail textures, such as roughness maps, into an 8-bit channel, the limited bit depth can result in loss of fine details, affecting the material's visual fidelity in rendered scenes.

Potential for Data Crossover:
When packing different grayscale images into the colour channels of a single texture, there’s a risk that high-frequency data in one channel could bleed over or affect another, depending on the filtering method (like bilinear or trilinear filtering). This issue is typically minimal but can be noticeable in certain conditions.

Mipmap Generation Issues:
When textures are compressed or mipmaps are generated, colour channel bleeding can occur. This is particularly problematic in texture packing because different data types are stored in each channel. For instance, compression artifacts in one channel (like roughness) could inadvertently affect another channel (like metallic), leading to unexpected visual results.

Considerations

Lower Bit Depths:
Efficient for real-time applications because they require less memory and bandwidth. This makes them suitable for mobile or VR applications where performance is critical.

Higher Bit Depths:
Used for high-dynamic range imaging (HDR), depth maps, and detailed normal maps. Higher bit depths help prevent banding and allow for smoother gradients; providing greater precision, which is necessary for realistic lighting, fine surface details without artifacts. High-precision textures might require their own dedicated channels or even separate texture files.

One of the biggest problems I faced was trying to maintain fine details with the lower resolution Normal maps.

The solution I found was to break up my Normal maps into Macro and Micro detail. The Macro detail represents the broader large scale deformations while the Micro detail represents the smaller fine details.

To do this in Unreal Engine 5:
Multiply the `TexCoord` by the XY values of the `ObjectScale` node. You can do this with a `ComponentMask(RG)`

This allows the textures to tile the UV coordinates as the mech scales; allowing artists to scale a mesh while maintaining the same texture density. Using this technique, you're able to represent higher fidelity details using textures as small as 512px!

Unreal has many options for compression methods, but these are the three that came up most often when I was researching.

DXT1 Compression

Description:
Primarily used for compressing full-colour texture maps.

How it Works:
It compresses 16 pixels in a 4x4 block into 64 bits, effectively reducing the texture size by a factor of 8 from the original, without an alpha channel or with 1-bit alpha (two levels of transparency).

Advantages:
Extremely efficient in terms of storage, making it suitable for diffuse textures without transparency.

Limitations:
Does not support full alpha transparency; best suited for opaque textures or those with masked transparency.

Use Cases:
Ideal for diffuse textures without alpha channels. Since BC1 compresses colour data into smaller blocks, it’s a good choice for large surfaces that do not require high fidelity or detailed alpha transparency, like basic walls, floors, and simple objects in a scene.

DXT5 Compression

Description:
Designed to handle textures with complex alpha gradients.

How it Works:
Similar to DXT1 in using the 4x4 block compression method but allocates additional bits specifically for alpha channel data, providing smooth transitions in transparency.

Advantages:
Ideal for textures that require detailed alpha channels, such as foliage, cloth, or any material with semi-transparent properties.

Limitations:
Larger file size compared to DXT1 due to additional alpha information and can sometimes yield artifacts in complex alpha transitions.

Use Cases:
Frequently used for colour textures that include an alpha channel. BC3's ability to handle complex alpha transitions makes it suitable for detailed textures on objects with transparency, such as foliage, fences, fabric, or any asset where transparency gradients are crucial.

BC7 Compression

Description:
Part of the newer Block Compression (BC) series, BC7 offers higher quality texture compression with full support for high dynamic range (HDR) colours.

How it Works:
BC7 compresses data into a 4x4 block using a more complex algorithm that allows for up to 8 different types of encoding modes, significantly improving quality and flexibility.

Advantages:
Provides excellent quality for a wide range of textures, including colour maps, normal maps, and specular maps, with better handling of detailed colour and transparency.

Limitations:
More computationally intensive to encode and decode than DXT formats, making it less ideal for platforms with limited processing power.

Use Cases:
This format is used when high-quality colour and texture detail are necessary. BC7 offers fine-grained control over compression, making it suitable for high-resolution textures or UI elements where clarity is paramount. It's often used for materials that are seen up close or for textures where colour fidelity is crucial.

File formats play a critical role in image compression, as they determine how data is stored, read, and processed within game engines like Unreal Engine 5.

JPEG (JPG)

Bit Depth:
Supports 8 bits per channel, resulting in a 24-bit image for RGB.

Compression:
Uses lossy compression, which can introduce artifacts and reduce quality, particularly in areas with high colour contrast. This compression helps reduce file sizes significantly, making JPEG suitable when bandwidth and loading times are considerations.

Effect on DXT/BC Compression:
Not typically used directly because JPEG artifacts can interfere with the block compression algorithms, leading to poorer quality or inefficiency.

Preprocessing Requirement:
Often converted to a lossless format like PNG or TGA before applying DXT/BC to avoid amplifying compression artifacts.

Use Case:
Because of the lossy nature, JPEG is less ideal for textures in high-quality renders but may be used for background images or textures where detail is less critical.

PNG

Bit Depth:
Supports higher bit depths than JPEG, up to 16 bits per channel in PNG-48 (48-bit RGB) and even includes an alpha channel for transparency (PNG-64 with 16 bits per RGBA channel).

Compression:
Uses lossless compression, which preserves image quality perfectly but generally results in larger file sizes compared to JPEG.

Effect on DXT/BC Compression:
Suitable for use where lossless transparency or color detail is important. PNG's lossless nature means there are no artifacts to exacerbate during DXT/BC compression.

Preprocessing Requirement:
Direct usage in DXT/BC compression is common, especially for assets requiring transparency.

Use Case:
PNG is suitable for detailed textures where transparency is needed or where image quality must not be compromised, such as UI elements, decals, and alpha textures.

TIFF

Bit Depth:
Can handle up to 16 bits per channel and supports various colour format. TIFF files can also include multiple images and layers, similar to PSD files.

Compression:
Offers options for both lossless (LZW, ZIP) and lossy (JPEG) compression. This flexibility makes it a favorite for archiving and transferring images that require high quality.

Effect on DXT/BC Compression:
Not commonly used directly in game engines due to its complexity and the overhead of handling its versatile but bulky options.

Preprocessing Requirement:
TIFF files are often converted to a more straightforward format like DDS for direct use with DXT/BC compression in real-time rendering to streamline processing and reduce memory usage.

Use Case:
From what I've understood, TIFF is less relevant in real-time environments, instead focusing on TGA as a high-fidelity image format.

TGA (Targa)

Bit Depth:
TGA files support a range of bit depths, typically from 8-bit per channel up to 32-bit for true colour images with an alpha channel, providing flexibility in balancing image quality and file size.

Compression:
Offers RLE (Run-Length Encoding) compression, a form of lossless compression that reduces file size without affecting image quality. This is particularly effective for images with large areas of uniform colour.

Effect on DXT/BC Compression:
Often used in scenarios demanding high-quality texture assets due to its support for deep color and alpha.

Preprocessing Requirement:
Typically used directly due to its flexibility and compatibility with DXT/BC without introducing unwanted artifacts.

Use Case:
TGA is popular in digital artwork and texture creation for games because it supports high quality and alpha channels, making it ideal for detailed textures that require transparency, such as HUD elements, sprites, and complex overlays.

DDS (DirectDraw Surface)

Bit Depth:
DDS format supports a variety of bit depths and is unique in its ability to include compressed and uncompressed pixel formats. It is commonly used with DXT compression formats which reduce the texture size significantly.

Compression:
Can utilize a variety of block compression (BC) formats, such as BC1 (DXT1), BC3 (DXT5), and BC7, tailored for different types of textures and quality requirements. These compression types are directly supported by many graphics hardware, facilitating efficient GPU processing.

Effect on DXT/BC Compression:
Ideal for real-time rendering as it allows textures to be compressed offline and stored in GPU-friendly formats.

Preprocessing Requirement:
None, since DDS is intended to be used directly with DXT/BC compression.

Use Case:
DDS is extensively used for storing compressed textures because it allows direct loading of textures into video memory without conversion, speeding up the rendering process. Ideal for environment textures, character skins, and any application where performance is a priority.