How a pose model sees you

Every second that the camera is on, the browser receives about thirty pictures. To a computer, each picture is only a grid of numbers: how much red, green and blue there is at every point. A pose model turns that grid into something a game can use — a list of thirty-three body points, each with a position and a confidence score that says how sure the model is. Everything FitForm does afterwards, from counting a squat to warning about a bent back, is arithmetic on those numbers.

The model itself is a neural network: millions of small multiplications arranged in layers. It was not programmed with rules such as "a knee is below a hip". Instead it was trained on a very large collection of photographs in which people had already marked where the joints were. During training the network made a guess for each photograph, the guess was compared with the human label, and every multiplication was nudged slightly in the direction that reduced the error. Repeated for days on powerful machines, that slow nudging produces a network that generalises: it can place the joints of a student in a Hanoi classroom it has never seen.

Running such a network on a cheap tablet is the real engineering problem. The version FitForm loads is deliberately the smallest of three — "lite" — and its numbers are stored in half the usual precision, a technique called quantisation. The file is therefore a few megabytes rather than a few dozen, and it is cached after the first visit. The network runs on the graphics processor, which is far better than the main processor at doing thousands of identical multiplications at once. On a four-year-old device the whole pipeline takes roughly twenty-five milliseconds per frame, which leaves enough time to draw the skeleton and still feel instant.

Accuracy and speed pull in opposite directions, and every choice is a compromise. The lite model confuses left and right wrists when the arms cross; it loses the ankles when a student stands too close; it reports low confidence in poor light. The application cannot remove those weaknesses, so it works around them. Points with a confidence below a threshold are ignored rather than trusted. Angles are smoothed over several frames, because a single bad frame should never add a repetition. A repetition is only counted when the body passes through a low position and then returns to a high one, in that order, with a minimum time between the two — a rule that tolerates noise but refuses to reward shaking the camera.

The most important design decision has nothing to do with accuracy. All of this happens inside the browser tab. The video never leaves the device, is never uploaded and is never stored; what travels to the server is a short summary — the movement, the number of repetitions, the duration, the score. That choice costs the team something: they cannot review a disputed session, and they cannot collect their own training data to improve the model. In exchange, a teacher can answer a parent's first question honestly in one sentence, and the school has nothing to lose if the server is ever breached.

It is worth noticing what the model does not understand. It has no idea that a squat is exercise, that a streak matters, or that the person in front of the camera is tired. It produces coordinates. Everything that makes those coordinates meaningful — the rules, the feedback, the league table, the encouragement — was written by people who thought carefully about what a thirteen-year-old would actually do in a noisy school hall.