# 11 — Stage 7 lane geometry

Date: 16 July 2026

Project: `classical-lidar-lane-detection`

Plan: [`07-Plan-From-Here-On.md`](07-Plan-From-Here-On.md)

## Executive summary

Stage 6 gave us a cleaner set of lane-like dots. Stage 7 now tries to turn
those dots into actual lane curves.

- **What I built:** Stage 7 finds groups of dots that could belong to one lane.
  It then fits three shapes to every group: a straight line, a quadratic, and a
  cubic. A quadratic can make one smooth bend. A cubic can describe a more
  complicated bend, including an S-like change in direction. The code also
  produces an adaptive result, where each point group chooses the simplest
  stable shape that fits it well enough.

- **How the grouping works:** I used RANSAC. In plain language, RANSAC tries
  many small combinations of dots, builds a possible line or curve from each
  combination, and keeps the one supported by the most other dots. I added
  road-level checks so the lanes stay roughly parallel, do not sit on top of
  each other, and do not cross. A fitted lane is drawn only where supporting
  dots exist; gaps longer than 4 m remain blank.

- **What was compared:** Every accepted point group is fitted with all three
  shapes. They are compared using distance from the dots, behavior on
  alternating distance bands, physical slope and bend limits, final lane-mask
  scores, and runtime. The K-Lane label is loaded only after fitting and never
  changes which model is chosen.

- **The held-out result:** On 40 frames that were not used while adjusting the
  fitter, Stage-6 input has **19.23% tolerant F1**. Straight fitting gives
  **18.66%**, quadratic gives **19.37%**, cubic gives **19.60%**, and adaptive
  gives **19.44%**. The change is modest in the tolerant score, but exact-cell
  F1 jumps from **5.01% at Stage 6 to 12.22% with adaptive fitting**. That means
  the fitted lanes land much more precisely even though upstream evidence is
  still missing over large parts of the road.

- **Did cubic win?** It wins the headline mask score, but only narrowly. It
  also has the weakest stability evidence: many cubic groups do not contain
  enough independent distance bands to fit the cubic twice and compare the
  answers. Cubic is plausible on 154 of 162 held-out groups, compared with 160
  for quadratic and 161 for a straight line. I would not force cubic onto every
  lane for a 0.16-point F1 gain over adaptive.

- **My recommendation:** use the **adaptive per-group output for the MVP**.
  Most groups stay straight, some use a quadratic, and cubic is chosen only
  when it provides enough improvement while passing the physical and stability
  checks. On held-out frames it selected 110 straight models, 44 quadratics,
  and 8 cubics. Keep the forced cubic output as a comparison, not as the
  default.

- **The important limitation:** Stage 7 still depends on good dots arriving
  from Stages 3–6. It can connect a supported lane segment; it cannot safely
  invent a lane through a region with no evidence. Only 30.33% of held-out
  Stage-6 cells were assigned to a trustworthy lane group. Adaptive fitting
  improved 18 frames, was unchanged on 1, and became worse on 21. The total
  micro-averaged score improves because the gains occur on more heavily
  supported frames, but the failure rate is real and should not be hidden.

- **What to inspect:** Start with the held-out artifact index, then look at the
  best and worst panels linked below. Panel 2 shows which Stage-6 dots were
  trusted as groups. Panels 3–5 let you compare the three shapes directly.
  Panel 6 shows where the adaptive answer matches or misses the label.

Checkpoint 10 was explicitly skipped by user direction, and its number was not
reused. Checkpoint 11 stops here for review.

## What you can check

Primary held-out evidence:

- [held-out artifact index](../artifacts/11_stage7_geometry_heldout/README.md);
- [held-out benchmark CSV](../artifacts/11_stage7_geometry_heldout/benchmark.csv);
- [held-out aggregate JSON](../artifacts/11_stage7_geometry_heldout/benchmark.json);
- [best adaptive improvement](../artifacts/11_stage7_geometry_heldout/images/seq_02_000940693000510_stage7.png),
  from 16.03% to 30.84% tolerant F1; and
- [largest adaptive regression](../artifacts/11_stage7_geometry_heldout/images/seq_02_000818804882560_stage7.png),
  from 25.02% to 11.25%.

Development evidence and source:

- [development artifact index](../artifacts/11_stage7_geometry/README.md);
- [development aggregate JSON](../artifacts/11_stage7_geometry/benchmark.json);
- [model implementation](../src/lidar_lane_detection/lane_geometry.py);
- [benchmark and held-out selection](../src/lidar_lane_detection/lane_geometry_benchmark_cli.py);
- [six-panel renderer and legends](../src/lidar_lane_detection/lane_geometry_diagnostics.py); and
- [controlled model tests](../tests/test_lane_geometry.py).

## How to read the six-panel plots

1. **Stage-6 cleaned evidence:** green dots are the exact input to Stage 7.
2. **Discovered groups:** each bright color is one lane-sized group found by
   robust fitting. Gray dots were not trusted enough to join a group.
3. **Straight fits:** blue shows what happens when every group is forced to use
   a straight line.
4. **Quadratic fits:** blue shows the same groups with one smooth bend allowed.
5. **Cubic fits:** blue shows the same groups with a more flexible curve.
6. **Adaptive strict error map:** white is an exact same-cell match, blue is a
   fitted cell with no label in that exact cell, and magenta is a labeled cell
   the fit missed.

The vehicle is below every plot. Forward is up, and positive lateral `y` is to
the left. The header gives the Stage-6, straight, quadratic, cubic, and adaptive
tolerant F1 values for that frame. It also lists how many point groups were
found and which model degrees adaptive fitting selected.

## Terms used in the tables

- **Strict score:** the fitted output and label must occupy exactly the same
  grid cell.
- **Tolerant score:** a one-cell offset is accepted, matching K-Lane's scoring
  convention.
- **Residual:** the sideways distance between an input dot and its fitted
  curve. Smaller is better.
- **Stability:** fit the same model using alternating forward-distance bands.
  If the two answers disagree, the model is sensitive to which dots it saw.
- **Plausible fit:** the curve stays inside the road view and passes fixed
  slope, bend, and bend-change limits.
- **Micro average:** add all true positives, false positives, and false
  negatives across the frames before calculating the score. Frames with more
  labeled cells therefore contribute more than nearly empty frames.

## Held-out model comparison

These 40 frames are disjoint from the 40 development timestamps. The Stage-7
configuration was frozen before this run and was not changed afterward.

### K-Lane one-cell-tolerant result

| Output | TP | FP | FN | Precision | Recall | F1 |
|---|---:|---:|---:|---:|---:|---:|
| Stage-6 input | 3,402 | 9,513 | 19,063 | 26.34% | 15.14% | 19.23% |
| Straight | 3,212 | 8,756 | 19,253 | 26.84% | 14.30% | 18.66% |
| Quadratic | 3,330 | 8,588 | 19,135 | 27.94% | 14.82% | 19.37% |
| Cubic | 3,338 | 8,266 | 19,127 | 28.77% | 14.86% | 19.60% |
| Adaptive | 3,349 | 8,643 | 19,116 | 27.93% | 14.91% | 19.44% |

Fitting mainly improves precision. The adaptive output removes 870 tolerant
false-positive cells while losing 53 tolerant true-positive hits relative to
Stage 6.

### Strict same-cell result

| Output | TP | FP | FN | Precision | Recall | F1 |
|---|---:|---:|---:|---:|---:|---:|
| Stage-6 input | 867 | 11,001 | 21,877 | 7.31% | 3.81% | 5.01% |
| Straight | 2,150 | 12,078 | 20,594 | 15.11% | 9.45% | 11.63% |
| Quadratic | 2,246 | 11,985 | 20,498 | 15.78% | 9.88% | 12.15% |
| Cubic | 2,273 | 11,699 | 20,471 | 16.27% | 9.99% | 12.38% |
| Adaptive | 2,268 | 12,093 | 20,476 | 15.79% | 9.97% | 12.22% |

The strict table is the clearest evidence that geometry fitting is doing real
work. Adaptive produces 2.62 times as many exact true-positive cells as the
scattered Stage-6 mask.

## Development-versus-held-out check

| Output | Development tolerant F1 | Held-out tolerant F1 |
|---|---:|---:|
| Stage-6 input | 17.83% | 19.23% |
| Straight | 17.40% | 18.66% |
| Quadratic | 17.80% | 19.37% |
| Cubic | 17.91% | 19.60% |
| Adaptive | 17.76% | 19.44% |

The ordering is consistent: cubic has the highest aggregate mask score,
quadratic and adaptive are close behind, and forcing every group to remain
straight is worst. The held-out check does not support a claim that every
individual frame improves.

## Straight and curved subsets

A frame is tagged `curved` only for evaluation when any labeled lane's
quadratic shape departs from its straight approximation by at least `0.30 m`.
Labels never affect fitting or model choice.

| Held-out label shape | Frames | Stage-6 F1 | Straight F1 | Quadratic F1 | Cubic F1 | Adaptive F1 |
|---|---:|---:|---:|---:|---:|---:|
| Straight | 30 | 19.72% | 19.77% | 20.59% | 20.73% | 20.57% |
| Curved | 10 | 17.61% | 14.67% | 14.98% | 15.48% | 15.36% |

This is an important warning. The more flexible models beat a straight line on
the curved subset, but all fitted outputs remain below the Stage-6 input there.
The fitter often sees only short pieces of a curve, so its road-wide grouping
can still choose the wrong parallel family. Curves need more work in a later
failure-audit checkpoint; higher polynomial degree alone does not solve them.

## Per-sequence held-out result

| Sequence | Stage-6 F1 | Straight F1 | Quadratic F1 | Cubic F1 | Adaptive F1 |
|---|---:|---:|---:|---:|---:|
| `seq_01` | 24.97% | 20.68% | 22.68% | 22.51% | 22.51% |
| `seq_02` | 18.89% | 18.25% | 18.15% | 18.35% | 18.11% |
| `seq_03` | 19.82% | 21.29% | 22.15% | 22.57% | 22.25% |
| `seq_04` | 11.93% | 12.52% | 12.52% | 13.01% | 13.03% |

The gains come from sequences 3 and 4, while sequences 1 and 2 regress. This
matches the frame-level split: adaptive improves 18 frames, is unchanged on 1,
and worsens on 21.

## How Stage 7 works

### 1. Find the road's shared direction

The code tests a fixed range of physically possible lane slopes and asks which
direction supports the strongest set of separated, roughly parallel lanes.
This prevents the first large scan arc from deciding the direction of the
whole road.

### 2. Discover one point group at a time

For straight, quadratic, and cubic hypotheses, deterministic RANSAC performs
60 trials per degree. A group needs at least seven points, five distinct
forward-distance bins, and 4 m of forward span. Points must lie within `0.24 m`
of the proposed curve.

After accepting one group, its points are removed and the search repeats for
up to six lanes. Overlapping fitted lanes must remain at least `1.20 m` apart,
and their direction must remain compatible with the road's shared direction.

### 3. Fit and compare all degrees

All three models are fitted to the same accepted group. Forward distance is
centered and scaled before fitting so large raw `x`, `x^2`, and `x^3` values do
not make the calculation numerically fragile.

Adaptive selection combines:

- median sideways fitting error;
- disagreement between alternating distance-band fits;
- a small cost for each extra polynomial degree; and
- a required minimum improvement before moving to a more complex model.

### 4. Apply physical limits

Every fit is sampled across its support and rejected if slope, curvature, or
change in curvature exceeds the fixed limits. Models that leave the accepted
K-Lane lateral area are also rejected.

### 5. Draw only supported segments

The output is rasterized at `288 x 288`, then reduced to `144 x 144` for K-Lane
evaluation. Each curve is drawn only between observed forward positions. A gap
over 4 m splits the curve into separate rendered pieces.

## Fit error and stability

| Model | Held-out fits | Plausible | Median residual | Median stability difference |
|---|---:|---:|---:|---:|
| Straight | 162 | 161 | 0.097 m | 0.088 m |
| Quadratic | 162 | 160 | 0.088 m | 0.132 m |
| Cubic | 162 | 154 | 0.084 m | 0.300 m* |

`*` The 0.300 m cubic value is often the configured penalty for insufficient
independent distance bands, not a measured 0.300 m disagreement. That is still
useful information: the available points often do not support a trustworthy
second cubic fit.

Cubic lowers fitting error by only 4 mm relative to quadratic while failing
the plausibility/support checks six additional times. This is the main reason
not to force cubic globally.

## Runtime

On the held-out sample, Stage 7 alone takes:

| Statistic | CPU time |
|---|---:|
| Median | 1,245.86 ms |
| Mean | 1,485.79 ms |
| 95th percentile | 2,775.17 ms |

This is acceptable for an unoptimized MVP audit but not yet a real-time
implementation. The main cost is repeated Python/NumPy RANSAC evaluation.

## Tests and verification

All **40** repository tests pass. Six Stage-7 tests verify that:

- straight evidence selects the straight model;
- clear one-bend evidence selects quadratic;
- clear S-like evidence can select cubic;
- a long unsupported gap remains blank;
- transverse evidence without forward support is rejected; and
- invalid masks and invalid configuration are rejected.

Both the source and tests pass Python bytecode compilation. Artifact checks
confirm 39 development panels/reports plus one recorded malformed source frame,
and 40 disjoint held-out panels/reports with no failures.

Reproduce the two runs with:

```bash
uv run klane-stage7 /mnt/arnavs_hdd/data/02-lane-detection/K-Lane \
  --sequence 1,2,3,4 --sample-count 10 \
  --output-dir artifacts/11_stage7_geometry

uv run klane-stage7 /mnt/arnavs_hdd/data/02-lane-detection/K-Lane \
  --sequence 1,2,3,4 --sample-count 10 --selection heldout \
  --output-dir artifacts/11_stage7_geometry_heldout
```

## Recommendation and stop gate

Use adaptive per-group geometry for the MVP. It captures most of cubic's mask
gain while refusing cubic when support or stability is weak. Keep quadratic as
the expected curved-lane workhorse and linear as the normal answer for simple
groups. Do not make forced cubic the default based on its small aggregate lead.

The implementation, tests, development artifacts, frozen held-out artifacts,
model comparison, stability analysis, failure cases, plot legends, and report
are complete. Stop here for review before assigning another checkpoint.