Having turned the round Waveshare ESP32-S3-Touch-AMOLED-1.75C into a clock, I wanted to push it harder, so I wrote a ray tracer for it: four spheres on a checker floor, with shadows and mirror reflections, running in real time and orbiting under your finger. The scene at the top of this page is that program, compiled to run in the browser; on the device it is the same code driving the 466×466 AMOLED.
Ray tracing is the obvious thing you cannot afford on a microcontroller. The trick is to do almost none of it, and to be careful about which pixels you spend real rays on. Nearly every pixel on screen is predictable from its neighbours; only a few are not, and the renderer is built around finding those.
The scene
There are four spheres of different colours and sizes, one of them a mirror, sitting on an infinite checkerboard under a graded sky. Each pixel can see the floor, a sphere or the sky; a sphere pixel can be lit or in shadow, and a mirror pixel also shows whatever its reflected ray hits. That is the whole model, and it is enough to look like a real ray tracer while staying cheap enough to trace sixty times from a chip that runs software floating point.
The budget
The round face is 466 by 466, about 170,000 visible pixels. A full ray trace would fire a primary ray at every one, test it against four spheres and the plane, then fire a shadow ray and, for the mirror, a reflection ray with its own shadow ray. That is millions of ray-sphere tests per frame, each with a square root, on a chip with no ray-tracing hardware. It is nowhere near real time.
So the renderer never traces at full resolution. It traces a coarse grid, reuses that result across whole blocks of pixels, and only fires more rays where the coarse result is not good enough. Everything below is a variation on that one idea, applied separately to the surface, the shadows and the reflections.
Coarse first, then refine
The screen is divided into a grid of 8 by 8 cells. One primary ray is fired at each cell corner, and what it found — which object, the hit point, the reflected ray, a shadow bit — is cached at that corner. A cell is then filled from its four corners.
If all four corners hit the same object, the whole 8×8 block is interpolated from them with no further rays: the surface point and colour are just blended across the cell. Only when the corners disagree — the edge of a sphere, where some corners hit it and some miss — does the renderer split the cell into four, trace the new mid-corners, and recurse, down to a single pixel if it has to. So the expensive work happens only along the silhouettes, which are a tiny fraction of the screen:
static void fill_quad(raytrace2_t *t, const scene_t *s,
int x0, int y0, int x1, int y1,
const corner_t *c00, const corner_t *c10,
const corner_t *c01, const corner_t *c11)
{
int same_hit = all_eq4(c00->hit, c10->hit, c01->hit, c11->hit);
if (same_hit && c00->hit == RT2_HIT_NONE) return; // all sky: nothing to do
if (same_hit && c00->hit >= RT2_HIT_SPHERE) { // all the same sphere:
fill_sphere(t, s, x0, y0, x1, y1, c00, c10, c01, c11); // interpolate, no new rays
return;
}
if (x1 - x0 == 1 && y1 - y0 == 1) return; // down to one pixel
// corners disagree - split into four, trace the new mid-corners, recurse
int mx = x0 + (x1 - x0) / 2, my = y0 + (y1 - y0) / 2;
corner_t cM0, cM1, c0M, c1M, cMM;
sample_mid(s, mx, y0, c00, c10, c01, c11, /* ... */ &cM0);
/* ...sample the other mid-corners... */
fill_quad(t, s, x0, y0, mx, my, c00, &cM0, &c0M, &cMM);
fill_quad(t, s, mx, y0, x1, my, &cM0, c10, &cMM, &c1M);
fill_quad(t, s, x0, my, mx, y1, &c0M, &cMM, c01, &cM1);
fill_quad(t, s, mx, my, x1, y1, &cMM, &c1M, &cM1, c11);
}When a mid-corner is sampled inside a block whose corners all hit the same sphere, it does not trace at all — it bilinearly interpolates the hit point from the corners and snaps it back onto the sphere. A new ray is only fired when the corners genuinely disagree about what is there.
Reflections, carried along
The mirror sphere needs a reflected ray per pixel, and reflected rays are where a naive tracer spends a second fortune. Here the reflection is treated as just another thing cached at the corners and interpolated across the cell. The reflection direction is stored deliberately un-normalised: the usual reflect formula produces a vector you would normalise with a square root, but if you skip the normalisation the vectors still interpolate correctly across the cell, and the reflected environment lookup does not care about length. So the whole cell's worth of reflected rays costs four reflect computations at the corners and no square roots in between:
static void set_refl_unnorm(const scene_t *s, corner_t *c, vec3 view)
{
vec3 n = normal_of(s, c->hit, c->p);
/* I - 2(I·N)N, left un-normalised so interpolation needs no sqrt. */
c->rdir = vsub(view, vmul(n, 2.0f * vdot(view, n)));
c->rorig = vadd(c->p, vmul(n, RT_EPS * 4.0f));
}Shadows, one bit at a time
Whether a point is in shadow is a single bit, so it is cached as a single bit at each corner. If the four corners agree — all lit, or all shadowed — the whole cell takes that answer for free and never fires a shadow ray. Only when the corners disagree, along the edge of a shadow, does the renderer fire shadow rays into the cell as it subdivides. Shadows are refined on their own schedule, independently of the surface and the reflection: a flat lit sphere over a shadow boundary still subdivides for the shadow even though its surface corners all agree.
The floor's shadows are cheaper still. A sphere's shadow on the ground plane is an ellipse, and testing whether a floor point is inside it is the same maths as a ray-sphere test — but because the light direction is fixed and the test only needs a yes/no, it reduces to a couple of dot products and one comparison, with the square root designed out entirely:
/* Is floor point p in sphere C,r's shadow? Same as a unit-direction
* ray-sphere test, but we only need the sign, so there is no sqrt. */
static int ellipse_shadows_floor(vec3 C, float r, vec3 L, vec3 p)
{
float ocx = p.x - C.x, ocy = -C.y, ocz = p.z - C.z;
float b = ocx * L.x + ocy * L.y + ocz * L.z;
float c = ocx * ocx + ocy * ocy + ocz * ocz - r * r;
if (c <= 0.0f) return 1; // point is inside the sphere
return b < 0.0f && (b * b >= c); // b*b >= c avoids the square root
}Square roots are slow in software floating point, and the renderer avoids them wherever an answer only needs a comparison. The ray-sphere test that the spheres themselves need does keep its square root, but it runs on the coarse grid and the silhouettes, not on every pixel.
Only the cells that matter
Even the coarse grid is too much to redo everywhere, so each frame starts by working out where things can have changed. Each sphere is projected to a rectangle of cells on screen; so is each sphere's reflection and each sphere's floor shadow. Only cells that fall inside one of those rectangles are traced; the rest of the screen is the floor and sky, which are pre-rendered once into a background buffer and copied back. When the camera holds still, only the small boxes around the spheres, their shadows and their reflections are redrawn, and only those rows are sent to the panel.
Both cores
The ESP32-S3 has two cores, and the renderer uses both. Each phase of a frame — projecting, tracing corners, filling, drawing the background — is split in half, with a worker task on the second core doing one half while the main task does the other, and the two meeting at a notification before the next phase. It is a simple split, but it roughly doubles the frame rate for free.
Running it here
The ray tracer at the top of the page is the board's own raytrace2.c, compiled to WebAssembly and drawing to a canvas instead of the panel. The rendering code is unchanged; the only things swapped out are the microsecond timer and the second-core worker. In the browser there is one thread, so the worker is run inline by a tiny cooperative shim — each time the renderer hands the second core a half-phase and waits, that half runs on the spot. Because the two halves are independent, running them one after another gives exactly the same picture, just without the second core's speed-up. If you want to see the chip itself run a binary, there is an emulator for the whole board.