Your Brain's Rendering Pipeline,
in 886 Lines of GLSL⚙

The Pipeline: Biology → GLSL

The visual system is vastly more complex than any current simulation captures — dozens of cortical areas, parallel processing streams, recurrent feedback loops. The shader models four stages chosen for their outsized impact on peripheral appearance: retinal spatial filtering, LGN attentional gating, V1 feature integration, and V4 color processing. Eccentricity from the gaze point drives every parameter.

Biological Primer

Much of what we know about these structures has been understood for decades — Kuffler described center-surround receptive fields in 1953, Hubel and Wiesel mapped orientation selectivity in 1962. The sections below summarize the minimum context for following the simulation. For deeper treatment, see Wandell's Foundations of Vision (1995), or this project's annotated bibliography, which maps each cited paper to its role in the shader implementation.

The Retina: Where Vision Begins

The retina is not a passive sensor — it's a layered neural circuit. Light hits photoreceptors (rods and cones), which convert photons to electrical signals. These signals pass through bipolar cells to retinal ganglion cells, whose axons form the optic nerve. In the fovea — a 1.5mm pit at the retina's center, densely packed with cones — each cone connects to its own ganglion cell (1:1 wiring). In the periphery, 100+ rods converge onto a single ganglion cell, trading spatial resolution for sensitivity. This convergence ratio is the fundamental reason peripheral vision is blurry.

Receptive Fields and Center-Surround🔬

A neuron's receptive field is the region of visual space that drives its response. Retinal ganglion cells have center-surround receptive fields (Kuffler, 1953): an excitatory center ringed by an inhibitory surround, or vice versa. This makes them bandpass filters — they respond to spatial frequencies matching their field size, not to uniform illumination. The Difference-of-Gaussians (DoG) is a mathematical model of this response profile. Critically, receptive field size scales with eccentricity: larger fields in the periphery means coarser frequency tuning further from fixation.

The Visual Pathway

Signals leave the retina via the optic nerve and reach the Lateral Geniculate Nucleus (LGN), a six-layered structure in the thalamus. The LGN is organized into magnocellular layers (fast, achromatic, motion-sensitive — fed by parasol ganglion cells) and parvocellular layers (slower, color-sensitive, high spatial frequency — fed by midget ganglion cells). From the LGN, signals project to V1 (primary visual cortex) in the occipital lobe, where orientation-selective neurons (Hubel & Wiesel, 1962) decompose the image into local features. Higher areas — V2, V4 — process increasingly complex properties: contours, color constancy, object recognition.

Eccentricity and the Three Zones

Eccentricity is the angular distance from fixation, measured in degrees of visual angle. The visual field is conventionally divided into functional zones: the fovea (0–2°, highest acuity, cone-dominated), the parafovea (2–5°, where crowding begins to impair recognition), and the periphery (>5°, rod-dominated, coarse spatial pooling). The cortex devotes disproportionate surface area to the fovea — a property called cortical magnification (M-scaling). This is why a letter at 10° eccentricity must be ~5x larger than a foveal letter to be equally legible.

Stage 1: The LGN Gate

"The LGN is not a relay station. It's a gate."
— Sherman & Guillery, The role of the thalamus in the flow of information to the cortex (2002)

The Lateral Geniculate Nucleus sits between retina and cortex. For decades, neuroscientists treated it as a simple relay — signals pass through unchanged. Sherman and Guillery showed it's actually an attentional gatekeeper: top-down signals from cortex modulate what gets through. If you've built information retrieval systems, this is a familiar pattern — a filter stage that decides what enters the processing pipeline at all, before any expensive downstream computation. In the shader, this gating is instantiated as two binary switches and a multiplication. The signal either passes or it doesn't.

Structure Masking⚙

The LGN gate needs to know what kind of content occupies each region of the page — and how that knowledge is represented constrains what operations are natural downstream. A content analysis pass (content-analysis.js) scans the DOM and encodes three signals into a structure map:

• Rhythm — text cadence and repetition (are there regular line breaks, list items, tabular patterns?)
• Density — visual weight per region (dense paragraph vs. whitespace vs. sparse navigation)
• Type — element classification (text, image, interactive control, empty)

When the LGN stage encounters low density — empty whitespace between content — it suppresses the downstream simulation entirely. The biological LGN doesn't gate blank space to cortex; the simulation mirrors this by zeroing the suppression factor for regions with no visual content to process.

// LGN Structure Masking — lines 319–323 of peripheral2.frag
if (config.lgn_use_structure_mask) {
    if (u_has_structure > 0.5 && signal.density < 0.1) {
        signal.suppressionFactor = 0.0;  // Skip peripheral simulation for empty space
    }
}

Saliency Gating

The saliency map (edge detection + color contrast, temporally smoothed; pop-out reference) tells the LGN what deserves processing bandwidth. This is the same problem biology solves: the retina captures ~10⁷ bits/sec but the optic nerve transmits ~10⁶. The visual system doesn't "degrade then un-degrade" — it selectively allocates limited processing resources to high-value input. Our simulation mirrors this: salient regions receive up to 70% more of the rendering budget (reduced peripheral filtering), while low-saliency regions are processed cheaply.

fovea para periph

Dashboard — center focus. The fovea reveals full detail at gaze point. Peripheral regions are filtered through the full LGN→V1→V4 pipeline.

Saliency map. Bright = high saliency = more processing bandwidth allocated. Text blocks, faces, and colored status badges demand the most resources.

Stage 2: V1 Feature Extraction

After the LGN gate, the signal that survives enters V1 — and here the nature of the representation changes fundamentally. Primary visual cortex decomposes the signal into features: orientation, spatial frequency, motion. In the periphery, these features are extracted at progressively lower resolution — receptive fields grow larger, features crowd into each other, and positional certainty degrades.

Scrutinizer's V1 stage implements three key peripheral phenomena: positional uncertainty (domain warping), crowding (feature scrambling), and spatial pooling (resolution loss). The LGN's suppression factor modulates all of them — up to 25% attenuation for salient content.

Domain Warping: Positional Uncertainty

In the periphery, you can detect that something is there but not where exactly it is. Scrutinizer simulates this with fractal noise displacement — two octaves of Simplex noise at 150x and 300x frequency, with an anisotropic 2x horizontal bias (horizontal crowding is more severe than vertical in human vision).

Crowding: The Fundamental Limit

"Crowding is not blur." — Denis Pelli (2008). You can detect a letter in your periphery. You cannot identify it when flanked by other letters. The bottleneck isn't resolution — it's obligatory feature integration across the receptive field.

This is arguably the most important phenomenon in peripheral vision — and it's an information-theoretic one (crowding reference stimuli, Bouma spacing). Crowding determines not what you can't see, but what you can't separate. Adjacent features are compulsorily pooled — averaged, texture-ified — once they share a receptive field. The information isn't gone; it's been irreversibly mixed. The individual signals are still in there, but the representation can no longer distinguish them. Bouma's law quantifies the critical spacing:

The shader implements two crowding modes:

• Fractal Noise (Mode 0) — Continuous warp via Simplex noise, increasing amplitude with eccentricity. Preserves overall structure but scrambles local features.
• Discrete Scramble (Mode 1) — Grid-based cell shredding. Each cell gets a random displacement vector, creating a "shattered glass" effect. Saliency guides where the grid breaks.

// V1 Discrete Scramble — lines 403–433 of peripheral2.frag
float scrambleZone = smoothstep(parafovea * 1.0, parafovea * 1.5, dist);
vec2 cellID = floor(uv * cellFreq);
vec2 jitter = hash22(cellID) - 0.5;
vec2 throwDist = vec2(0.008, 0.0016);  // 5:1 H:V ratio
discreteScramble = jitter * throwDist * scrambleZone²;

fovea para periph

Article — center focus. V1 crowding filters paragraph text in the periphery while headlines receive more bandwidth (larger receptive fields).

fovea para periph

E-commerce — product focus. When you fixate on the product image, the price tag, CTA button, and reviews are filtered through V1 scrambling — a visualization of the feature pooling that constrains peripheral object recognition.

Saliency-Guided Pixelation (Mode 3)

A more utilitarian V1 mode: content is quantized into blocks whose size depends on both eccentricity and the LGN's saliency signal. High-saliency regions get smaller blocks (more detail preserved). The saliency metric is discretized into 4 steps to prevent gradient artifacts:

float steppedMetric = floor(saliency * 4.0) / 4.0;
float blockSize = mix(maxBlock, minBlock, steppedMetric);

Toward Mongrel Textures: DoG Band Decomposition

By the time the signal reaches the periphery, what does the representation actually contain? Rosenholtz et al. (2012) answered this precisely: summary statistics — local texture-like representations that preserve aggregate properties (mean color, dominant orientation, spatial frequency distribution) while discarding individual feature identity. Think of it as a lossy compression where the encoder is the visual system itself. Their mongrel images, synthesized to match these statistics, are perceptually indistinguishable from the original in the periphery. The catch: synthesizing mongrels takes >60 seconds per frame, even with 2026 prosumer hardware.

The shader's DoG band decomposition is a real-time approximation of this idea. Rather than computing full texture statistics, it decomposes the image into spatial frequency bands — each corresponding to a different scale of visual structure — and attenuates them independently based on eccentricity. This mirrors how retinal ganglion cells act as bandpass filters with center-surround receptive fields whose size scales with eccentricity (Curcio et al., 1990). Uniform MIP blurring treats all frequencies equally; the DoG decomposition lets fine detail drop out before coarse structure, matching the biological attenuation profile.

The Approximate Laplacian Pyramid⚙

The shader decomposes the hardware MIP chain into an approximate Laplacian pyramid — each band captures a spatial frequency range:

// 8 Half-Octave DoG Bands — peripheral2.frag
// 9 MIP levels at LOD 0.0, 0.5, 1.0, ... 4.0 (half-integer = hardware trilinear)
vec4 mip[9];
mip[0] = textureLod(tex, uv, 0.0);
mip[1] = textureLod(tex, uv, 0.5);
// ... mip[2]–mip[7] at LOD 1.0, 1.5, 2.0, 2.5, 3.0, 3.5
mip[8] = textureLod(tex, uv, 4.0);

vec4 band[8];
band[0] = mip[0] - mip[1];  // ~5.66 cpd: serifs, fine detail
band[1] = mip[1] - mip[2];  // ~4.0 cpd:  thin strokes
// ... band[2]–band[7]: √2-spaced down to ~0.5 cpd (layout blocks)
// residual = mip[8]         // ~0.35 cpd: DC, always preserved

M-Scaling: Cortical Magnification per Band

Each band has its own cutoff eccentricity — the distance from the fovea where that frequency band is fully attenuated. With 8 half-octave bands, the cutoffs follow $\sqrt{2}$ spacing, matching the $E_2$ scaling of cortical magnification (validated against published contrast sensitivity data — see report):

Each band's weight is computed via smoothstep rolloff, with a transition width controlled by u_dog_sharpness. At sharpness=0, the transitions are wide and gradual (biological). At sharpness=1, they're narrow and crisp (engineering utility mode).

// Per-band weights via smoothstep rolloff
float w[8];
for (int k = 0; k < 8; k++) {
    w[k] = 1.0 - smoothstep(c[k] - c[k]*trans, c[k] + c[k]*trans, normEcc);
}

// Reconstruction: residual + weighted bands
result = mip[8];
for (int k = 0; k < 8; k++) { result += band[k] * w[k]; }
result = clamp(result, 0.0, 1.0);

What This Means in Practice

Close to the fovea, all 8 bands survive — full detail. In the parafovea, the finest bands (serifs, thin strokes) drop out first. You can still read words but individual letterforms lose their character. Further out, letter-body bands fade — shapes become blobs. In the far periphery, only the coarsest bands and the DC residual remain: you see buttons and layout blocks but not their labels. With half-octave spacing, each transition step is smaller than the old octave-spaced system — the degradation gradient is smoother.

This cascade — fine detail filtered first, coarse structure passing through longest — is how the biological system manages its bandwidth, and it's how the shader manages it too. The signal isn't uniformly blurred; it's selectively attenuated by frequency band, approximating the frequency-selective loss that Rosenholtz's summary statistics model predicts, at real-time rates.

Structure map — article. The content analysis feeds the LGN. Red channel = text rhythm, Green = density, Blue = element type. DoG band weights are modulated by this structure.

fovea para periph

Techmeme — center focus. Dense news layout. DoG preserves headline structure while filtering individual link text. The data table density is exactly the kind of content where DoG outperforms uniform blur.

Fidelity Gap: What DoG Misses

The DoG decomposition captures frequency-selective attenuation but not the full mongrel texture model. True summary statistics include orientation distributions, phase correlations, and cross-scale feature conjunctions that our isotropic bands discard. Adding oriented DoG filters — even just horizontal/vertical — would double the texture lookups but could capture the anisotropy of real V1 receptive fields (Gabor-like, phase-sensitive). The gap between our real-time approximation and a full Rosenholtz-style synthesis is where future work lives.

Stage 3: V4 Color & Aesthetics

Visual area V4 processes color, shape, and object recognition. In the periphery, color perception shifts dramatically — the transition from cone-dominated (foveal) to rod-dominated (peripheral) processing alters hue sensitivity, saturation, and spectral response (per-channel chromatic decay validated — see report). The shader's V4 stage simulates these shifts using Oklab color space, rod-spectrum tinting, and selectable aesthetic rendering modes.

Oklab: Perceptually Uniform Desaturation⚙

A critical design choice. If you desaturate in RGB or HSL, hue shifts as saturation drops — warm colors skew yellow, cool colors skew cyan. Oklab (Ottosson, 2020) separates lightness (L) from chrominance (a, b) in a perceptually uniform space. Killing a and b channels produces clean grayscale without hue contamination.

The Five Aesthetic Modes

Magnocellular Pathway: Luminance Contrast Preservation🔬

The magnocellular (M) pathway — fast, achromatic, motion-sensitive — preserves luminance contrast even as the parvocellular (P) pathway loses color and detail. The shader samples the clean (undistorted) image, computes a luminance ratio, and blends it back into the simulated output. This is why you can still detect edges and motion in your far periphery even when you can't see color or detail.

// Magnocellular Pathway — lines 555–570 of peripheral2.frag
float cleanLuma = dot(cleanSample, vec3(0.299, 0.587, 0.114));
float distortedLuma = dot(col, vec3(0.299, 0.587, 0.114));
float lumaRatio = cleanLuma / max(distortedLuma, 0.01);
float preservation = mix(0.6, 0.1,
    smoothstep(0.0, parafovea - fovea, eccentricity));
col *= mix(1.0, lumaRatio, preservation * contrastRamp);

Structure map — Techmeme. The structure analysis that feeds the LGN gate. Dense news sites reveal the rhythm/density/type decomposition that drives the entire pipeline.

Saliency map — Techmeme. Edge detection + color contrast in Oklab space. Bright regions receive more processing bandwidth through the LGN gate.

The Full Mapping: Biology → GLSL

Each row below maps a biological phenomenon to its shader implementation, with the controlling parameter. For full citations and methodology, see the scientific literature review.

Mobile: The Smaller Fovea

On mobile, the fovea subtends the same visual angle, but the screen is held closer and the viewport is denser. The simulation reveals something counterintuitive: much of the content on a phone screen sits in the parafoveal zone — not the far periphery, but the transitional region where feature integration begins to degrade. This is where crowding first bites.

fovea para periph

iPhone 14 — article. The same article page at mobile scale. Much of the content sits in the parafoveal zone, where feature integration starts to blur individual elements.

fovea para periph

iPhone 14 — dashboard. Data-dense UIs on mobile. The fovea reveals one card; most content falls into parafoveal territory.

fovea para periph

iPad Air — article. Tablet's larger viewport provides a middle ground. More foveal real estate, but still significant parafoveal load.

fovea para periph

iPad Air — dashboard. The data table density makes this an ideal test case for the DoG band decomposition — rows fade by spatial frequency, not uniformly.

Select References

• Bouma, H. (1970). Interaction effects in parafoveal letter recognition. Nature, 226, 177–178.
• Curcio, C. A., et al. (1990). Human photoreceptor topography. J. Comparative Neurology, 292(4), 497–523.
• Hubel, D. H. & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiology, 160(1), 106–154.
• Ottosson, B. (2020). A perceptual color space for image processing. (Oklab specification).
• Pelli, D. G. (2008). Crowding: a cortical constraint on object recognition. Current Opinion in Neurobiology, 18(4), 445–451.
• Rosenholtz, R., et al. (2012). A summary statistic representation in peripheral vision explains visual search. Journal of Vision, 12(4):14.
• Sherman, S. M. & Guillery, R. W. (2002). The role of the thalamus in the flow of information to the cortex. Phil. Trans. R. Soc. B, 357(1428), 1695–1708.

Further Reading

• Annotated Bibliography — each cited paper mapped to its role in the shader implementation
• Foveated Vision Model — detailed theory document for the simulation
• Project Trajectory — 98-day development arc, commit analytics, the revert day
• Visual Cognition — curated talks on peripheral vision, crowding, and visual attention
• Vision Demonstrations — demos, walkthroughs, and design rationale

About the Author

Andy Edmonds studied cognitive science at Carnegie Mellon, Georgia Tech, and Clemson, then left academia in 1995 for what became two decades of machine learning at scale — shipping recommendation systems, search relevance, and information retrieval products at eBay, Adobe, Microsoft, and Meta. The through-line across all of it is the same question that drew him to cognitive science in the first place: how does information flow through a system, and what happens to the signal at each stage?

Scrutinizer is that question applied to the visual system. The retina decomposes. The LGN gates. V1 pools. V4 recolors. Each stage transforms the signal in specific, modelable ways — and those transformations map cleanly onto GPU operations. The shader isn't a metaphor for the biology. It's an implementation of the same information-processing architecture, running on different hardware.