Trail Intelligence

Trail difficulty is a star rating someone else gave, and many riders don’t trust it. In a blind survey of thirteen cyclists, ten had felt a trail’s rating was wrong for how it rode, and ten had hit a buddy mismatch: a trail that suited a riding partner and was over or under their own level. Some experienced riders already work around this by hand, scrubbing other people’s POV footage on YouTube instead of trusting the stars. Trail Intelligence automates that habit: a per-frame read of what a trail demands, built from the footage and sensor data riders already record, so one rider’s run can become the next rider’s preview.

It started as an experiment: could NVIDIA’s Cosmos Reason 2, a vision model trained on robotics and driving footage, read mountain-bike video from pixels alone? I ran the 2B model locally and the 8B on a rented A100 through Vast.ai. It handled terrain and obstacles and failed on state: direction of travel, conditions, how steep a slope really was. A later probe of seven other vision models, frontier and open from 7B to 72B parameters, found the same failure: on descent frames, the best reached 38% only by counting hedged answers, and most defaulted to uphill. A chest camera pitches with the rider, so the slope on screen is not the slope underfoot. That null result is what the product is built on. The archived experiment is on GitHub.

To check that this wasn’t a small-model problem, I withheld the GPS grade and asked seven vision models, frontier and open, to read direction from the frame alone. Most defaulted to uphill, and the best was still wrong on most descents. This was an early, exploratory probe, and the raw outputs weren’t kept, so treat the table as the shape of the failure rather than a benchmark.

The fix is to stop asking the camera questions a sensor already answers. On a 473-frame bench run locally on Gemma 4 31B, the model read direction (descent, climb or flat) right 64% of the time from pixels alone, and its errors ran both ways: 29 of 279 descents read as climbs, 43 of 134 climbs read as descents. With the GPS grade stated beside each frame, the same model on the same frames reached 95.6%, and descents read as climbs went to zero.

The labels cost nothing. The 473 frames come from five clips. Descent and flat labels come from the sign of the synchronized GPS gradient; the 134 climb frames come from two clips that climb throughout. Nothing was hand-labeled, and every later change could be scored against the same set.

It is important to note what this measures. Once the grade is in the prompt, the test is whether the model trusts a stated number over a misleading camera, not whether it sees better. That is why production takes direction straight from GPS and asks the vision model only what a frame can show.

Align. Raw video is matched to the GPS track so every frame carries true-time telemetry. The sync falls back through several methods (the camera’s embedded GPS, a cross-correlation of turning motion between video and track, the file’s clock, a visual landmark match), because no single one survived every clip.

Gate and select. Frames where the rider stopped are skipped, so a pause isn’t read as a trail feature. Frames where the GPS froze while the rider kept moving are kept, marked ungrounded, and the model is told the telemetry is unavailable; deleting real riding would be worse. Frames are then picked at points the telemetry marks as meaningful, or at a fixed interval, as in the demo.

Split the questions by instrument. GPS owns direction and speed; the accelerometer owns airtime. The vision model answers only what a single frame can show (surface, maintenance, visible hazards, the spot where a rider has a decision to make) through constrained checklists with an explicit “cannot determine.” Per-frame prompts never name the trail, so the model can’t lean on what it may know about a named place instead of looking at the frame.

Synthesize. A second pass reads across frames for what stays consistent, what evolves and where the peaks are, then writes briefings by skill level.

Grounding left a stubborn residual: 12 of 134 climbs still read as downhill, with a +11% grade stated in the prompt. The model had the answer and overrode it with what it saw. That is an instruction problem, not a telemetry one, and it became the first real job outside Rubrica for Ratchet, my gate for prompt changes. A hill-climb scored candidate clauses against the GPS labels and kept only what survived a held-out split. The winner was one line: read the gradient’s sign first, then answer to match it. A general version, take the gradient’s sign as the direction even when the camera’s tilt says otherwise, went into the production prompt.

Ported into the production prompt and re-run on the full bench: 95.6% to 97.7%, climbs read as downhill 12 to 0, uphill recall 91% to 100%. The cost: flat recall dipped from 85% to 82%, because near-zero grades now get pushed to commit to a sign.

The model is still wrong in known ways, so the page says where. Left and right is the weak axis: the GPS grade grounds which way the trail falls ahead, not which side drops away. When a frame’s side call disagrees with its neighbors, it is re-asked, with telemetry withheld, on the original and a mirrored copy. A claim read from the pixels should flip with them; one that doesn’t can’t be trusted as a reading of the image, and its side claim is withheld on screen. Re-asking until the neighbors agree would have been easier, and it would only have measured the model agreeing with itself.

When instruments disagree, the page shows both and resolves neither. Where the accelerometer measured the rider leaving the ground and the vision pass saw no drop, the frame carries the measurement and marks the contradiction. Distance comes from the bike computer’s odometer, and the frame track agrees with it within 0.24%.

A small version of the demo, from the same run. Move along the descent to see each frame the model was given, beside what the instruments measured around that moment.

high risk, modelairtime, IMU

time

2:40

grade

−18.2%

speed

19.2 mph

elevation

510 m

model risk

moderate

imu

0.08 s unweighted · 3.6 g landing

The demo is one real descent: Luge, in the Cleveland National Forest, 0.98 miles and 118 analyzed frames. Gemma 4 31B on a laptop made the per-frame reads; Claude Opus 4.8 wrote the segment summary, and the left/right mirror check ran on Claude Sonnet 4.6. Drag along the trail outline to scrub. Each frame shows the exact image the model saw, the telemetry strip (speed, grade, elevation) and the model’s read, with risk marked on the map and the airtime the accelerometer measured (one drop, three jumps) listed as notable points. It is a static export with no backend, so what you scrub is exactly what the pipeline produced.

The demo borrows the look of the trail apps riders already use. It is an independent prototype, not affiliated with Strava or any other platform.

This is a proof of concept. The footage is one rider’s, and the discovery behind the problem is a small sample. Direction is the only read with a large scored bench, and production hands it to GPS; the reads the product rests on, such as surface, hazards and risk, have no comparable accuracy number yet. The lateral axis is flagged rather than solved. The next fix follows the same thesis: travel heading plus a public elevation model gives which side of the trail falls away, per frame, for free, and that becomes the next Ratchet target.

The assistant can go further into why a chest camera misreads slopes, how the video is synced to the GPS, or what the mirror probe withholds.