Anatomy of a Texture

By Agentlien — Graphics Programmer

Introduction

"Last year, I was tasked with writing code to convert texture data across various gaming platforms. Going into the project, I seriously underestimated just how complex texture memory layouts actually are."

Currently, I am part of a team at 505 Games, where we are porting a proprietary PC game engine over to consoles. A significant portion of our technical hurdles has centered around texture conversion. While I was aware that the process involved various subtleties, the true scale of the complexity only became apparent when I had to implement working code that managed all these variables simultaneously.

This post aims to demystify the intricacies of texture memory layout and explain why such complexity is a necessity. I will approach this from the perspective of a developer debugging the memory address calculations for specific components of a console texture.

Prerequisites

To follow along, I assume you have a grasp of:

• General programming fundamentals.
• How memory layouts function.
• Basic digital imagery and video game technology.

---

The Case Study: A $1024 \times 1024$ Texture

Throughout this discussion, we will use a specific example to ground the theory: A $1024 \times 1024$ RGBA texture utilizing BC7 compression.

When moving this texture between platforms, we must track a variety of specific details. For this article, I will focus on:

• Block Compression
• Texel Ordering
• Mipmaps (Mips)
• Texture Tiles

I will also cover pitch, depth, and texture array indices. Actually, I'll ignore those; they mostly involve simple offsets that, while tedious, don't offer much theoretical interest.

Warning: If you were to ignore these complexities and interpret the resulting memory as a simple stream of color values, you wouldn't see an image—you would see chaotic rainbow noise.

---

Platform Specifics & Memory Logic

You might wonder: If this is about texture formats, why are we discussing memory locations?

The reason is that while different platforms use the same basic hierarchy of building blocks to decompose textures, the order of those elements in memory varies. Therefore, converting a texture is essentially a mathematical problem: calculating the exact memory address where each element should reside on the target platform.

The Texture Hierarchy

To visualize how a texture is broken down, consider this flow:

Note: Where platform-specific distinctions are required, I will refer to the DirectX 12 implementation.

---

The "Idealized" vs. Actual View

If you have a basic understanding of digital images, you might assume a 2D texture is stored linearly.

The Naive Assumption: An RGBA image with 32-bit depth is just a continuous stream of texels, sweeping from left to right, row by row, with each texel consisting of four consecutive bytes (R, G, B, A).

In modern gaming, this is rarely the case. Instead, several orthogonal techniques are used to optimize performance:

Cache Locality is the practice of storing data that will be accessed together physically close to one another in memory. Since memory transfer is one of the most expensive operations in hardware, maximizing locality is critical.

---

Deep Dive: The Single Texel & Block Compression

Let's examine how a texel is actually represented. In an uncompressed RGBA image with 8 bits per channel, a single texel occupies 32 bits (4 bytes).

To optimize this, we use Block Compression (BC). Specifically, BC7 is a lossy format that makes clever assumptions about the similarity of neighboring texels.

How BC7 Works:

1. Endpoints: Each block defines pairs of reference colors called end points.
2. Interpolation: Each individual texel within that block is assigned an index.
3. Calculation: The final color is an interpolated value between the end points based on that index.

The Math of BC7

A single BC7 block is 16 bytes in size and represents a $4 \times 4$ grid of texels (16 texels total).

$\text{Compression Ratio} = \frac{\text{Uncompressed Size}}{\text{Compressed Size}} = \frac{16 \text{ texels} \times 4 \text{ bytes}}{16 \text{ bytes}} = 4\times$

This results in an average of 1 byte per texel. Despite this 4x reduction, the visual difference is nearly imperceptible to the human eye, making it ideal for streaming massive textures.

Memory Layout Example

If we treat the texture as a series of 16-byte chunks, the memory looks like this conceptually:

struct BC7Block {
    uint8_t data[16]; // Represents a 4x4 texel area
};

// The texture is then an array of these blocks, 
// but ordered according to the platform's tiling scheme.
BC7Block textureMemory[TOTAL_BLOCKS];

By grouping texels into blocks, we further improve cache locality, as effects that sample adjacent texels will find the required data already loaded in the cache.