Understanding JPEG: A Deep Dive into Lossy Compression

By Jennings | Originally published September 9, 2017

In the fast-paced realm of technology, the concept of "forever" usually lasts about five years. It is quite remarkable, then, that the JPEG image standard, established in 1992, remains a dominant force today—surviving more than five "forevers."

This guide breaks down the core mechanics of JPEG using accessible language. To help you visualize these concepts, imagine an interactive compressor where you can manipulate settings and observe the results in real-time.

---

The Fundamentals of Compression

At its heart, compression is the art of identifying redundant patterns within data and substituting them with more compact representations. Think of it as using an acronym to replace a lengthy phrase.

However, audio and visual data are typically too "noisy" for simple substitutions to work effectively. This is why JPEG utilizes lossy compression.

Lossy vs. Lossless Compression

The Librarian Analogy: Imagine a librarian who has run out of shelf space. If she converts a bulky book into a microfiche slide, that is lossless compression. If she simply burns the books that nobody ever reads to make room, that is lossy compression.

The secret to high-quality lossy compression is knowing exactly which information the human brain ignores.

The Psychovisual Basis of JPEG

JPEG relies on two primary principles of human perception:

1. Luminance over Chrominance: We are far more sensitive to brightness than to color.
   • The Biology: The human retina possesses $\approx 120$ million rod cells (brightness) but only $\approx 6$ million cone cells (color).
2. Low-Frequency over High-Frequency: We perceive broad shapes and edges (low frequency) more accurately than intricate, busy textures (high frequency).
   • Example: Camouflage works by using high-frequency patterns to mask the low-frequency outlines of an animal.

---

The JPEG Pipeline

To achieve compression, JPEG transforms image data to isolate brightness and frequency, then applies a final layer of lossless compression.

Selecting Your Sample

When experimenting with these steps, you might choose from various images:

• Tower (Recommended: balanced frequencies)
• Flowers
• Hippos
• Custom Upload

---

Step 1: Isolating Color Information

Digital images are composed of pixels. In a standard format, each pixel is a combination of three primary light channels: Red, Green, and Blue (RGB).

The issue is that brightness information is distributed across all three RGB channels. To optimize this, JPEG converts the color space from RGB to YCbCr:

$\text{RGB} \rightarrow \text{YCbCr}$

• Y: The Luminance (brightness) channel.
• Cb: The blue-difference chroma channel.
• Cr: The red-difference chroma channel.

By moving all the definition to the Y channel, the Cb and Cr channels become "muddy," as they now only contain color data.

---

Step 2: Discarding Redundant Color

Because our eyes are less sensitive to color, JPEG can afford to throw some of it away. This is done by scaling down the Cb and Cr channels while keeping the Y channel at full resolution.

Common Scaling Options:

• Full resolution (No loss)
• 1/2 resolution
• 1/4 resolution (Used in this demonstration)

The Math of the Reduction: If we start with 3 full channels and reduce the two color channels to 1/4 size: 1 (Y) + 0.25 (Cb) + 0.25 (Cr) = 1.5 total channels

We have effectively halved the amount of data before even reaching the complex math!

---

Step 3: Shifting to the Frequency Domain

To leverage our lack of sensitivity to high-frequency detail, JPEG divides the Y, Cb, and Cr channels into small blocks.

# Conceptual representation of block division
image_channel = divide_into_blocks(size=(8, 8))
for block in image_channel:
    transform_to_frequency_domain(block)

Each block consists of an $8 \times 8$ grid of pixels. The goal is to move these blocks from the spatial domain (where we see pixel values) to the frequency domain.

• Spatial Domain: The actual value of each pixel in the grid.
• Frequency Domain: A representation of how rapidly the values change across the block.