2026-03-21

Isotropic Cortical Sampling

In 2006, Matt Queen asked a question in Boxes & Arrows that stuck: icon designers work at the pixel level, but users encounter icons at the periphery of their attention, where spatial resolution has already collapsed. What would designers do differently if they could see their work the way the visual system processes it?

Scrutinizer is one answer to that question. It simulates peripheral vision in real time, degrading a live browser frame based on gaze position. The degradation comes from a pipeline modeled on the visual system: spatial frequency decomposition (12 bands of Difference-of-Gaussians filtering, each tuned to a different detail level), differential chromatic decay (color sensitivity drops with eccentricity, red-green channels faster than blue-yellow), and feature displacement that produces crowding without fog.

v2.6 derives the displacement geometry from the cortical magnification function, the mapping between how far something is from where you’re looking and how much cortical tissue is devoted to processing it. Where the pipeline’s noise frequencies and scramble cell sizes were previously fixed constants, they now scale with cortical sector extent as defined by Blauch, Alvarez & Konkle (2026).

Original

Through Scrutinizer

The cortical grid

Your brain dedicates most of its visual processing power to the center of gaze and progressively less to the periphery. Cortical distance grows with the logarithm of eccentricity, w = log(r + a), and this mapping sets the spatial scale at which peripheral features get pooled together.

The GPU’s built-in texture sampling uses a rectangular grid of uniform square tiles across the whole image. That doesn’t match how resolution falls off. The isotropic cortical grid grows cell size with eccentricity and adjusts the number of angular slices at each ring to keep cells roughly square, approximating how receptive fields tile the visual field. Resolution degrades equally in all directions.

Rectangular MIP grid (left) vs. isotropic cortical grid (right). View on CodePen

What the grid parameterizes

Scrutinizer’s MIP level selection already used the same logarithmic curve, log(1 + r/a). What v2.6 adds is a second use of the same geometry: parameterizing displacement. Two mechanisms degrade peripheral content: a noise warp (“Bender”) that shifts features and a discrete scramble (“Cutter”) that breaks within-feature coherence. Both now scale with the local cortical sector extent:

Fixed parameters

Sector-parameterized

Same page, same gaze. The two look similar, as intended. 38% of pixels differ, but the visual character is the same: text destroys, layout survives. The change is in the spatial profile (4-8% lower peripheral variance, smoother degradation gradient). v2.6 changes how the parameters are derived and keeps the same degradation mechanism.

Two mechanisms work together to break letter identity in the periphery. Neither alone is enough: smooth warping preserves structure, and cell scrambling alone leaves fragments recognizable. Combined, they destroy identity while preserving texture:

The displacement pipeline in isolation and in sequence. The Bender shifts features smoothly; the Cutter breaks them into displaced cells. Together they destroy letter identity while preserving texture and layout.

• Bender (noise warp): we scale frequency inversely with sector extent. Near the fovea (~7px sectors), the warp is fine-grained. At 15° (~52px sectors), warp wavelength grows to feature scale. Features shift position smoothly, like looking through textured glass.
• Cutter (discrete scramble): we divide the image into small cells and displace each by a random offset. Cell size tracks sector extent (floored at 8px, capped at 16px). Throw distance (how far each cell moves from its original position) is bounded by one sector width, with 2:1 radial bias (crowding is stronger along the fovea-to-periphery axis than tangentially; Toet & Levi 1992). Features break apart at sub-letter scale.

In practice, the delta is quiet (mostly a reduction in implausible long-range scatters), but it aligns the MIP chain and displacement pipeline under a single cortical geometry.

Working with the hardware

When a texture covers fewer screen pixels (because it’s far away, at a steep angle, or warped), sampling its full detail would create shimmer and noise. The GPU avoids this by choosing a pre-blurred version (MIP level) that matches how densely you’re sampling. It measures this with dFdx: the rate of change of the texture coordinate between neighboring screen pixels. Fast-changing coordinates mean you’re skipping texels, so the hardware picks a blurrier level. Slow-changing means you’re sampling densely, so it picks a sharp one.

Scrutinizer gets its blur by telling the GPU the texture is moving faster than it is.

When the V1 displacement stage warps the sampling UV, the original derivatives become wrong: they describe the undistorted coordinate field. Passing dFdx(v1.distortedUV), the derivative of the warped coordinate, makes the hardware treat the texture as changing faster than it is in the periphery, which selects higher (blurrier) MIP levels automatically. The texture sampling unit computes the eccentricity-scaled blur.

The hardware keeps the tiles square, but the level selection now tracks the cortical warp. Scrutinizer’s 12-band DoG decomposition turns the MIP chain’s blunt power-of-2 steps into graded frequency loss: serifs drop while panel layout survives. (Details in the MIP chain tech brief.)

The real-time constraint forces specific tradeoffs: displacement instead of summary-statistic pooling, hardware MIP instead of continuous receptive fields, box-filtered frequency bands instead of Gaussian convolution. The spec documents these approximations and the implementation journal records the 8 rendering approaches attempted.

The next step: pooling

The isotropic grid gives the correct pooling regions. What remains is implementing the correct pooling computation within them.

The gap is architectural. Pooling-based synthesis needs a WebGPU compute pipeline (pyramid decomposition, cross-scale statistics, multi-band texture matching) within a 3-4ms real-time budget. We’re building that now, with the isotropic sectors providing the pooling geometry and Brown et al.’s metamer implementation as a reference for calibrating information loss.