# 08 — Stage 3 reevaluation and spatial-coverage audit

Date: 15 July 2026

Project: `classical-lidar-lane-detection`

Plan: [`07-Plan-From-Here-On.md`](07-Plan-From-Here-On.md)

## Executive summary

Here is what we were trying to understand: the Stage-5 visuals suggested that
Stage 3 sometimes throws away useful road and lane-marking points, but the old
Stage-3 report only gave us a total inlier count. A large count near the car
could make the result look healthy even if the detector had lost the complete
far field. This reevaluation checks where the retained points actually are.

- **The setup:** I kept the existing Stage-2 crop and Stage-3 RANSAC plane
  exactly as they were and ran them on 40 evenly spaced labeled frames across
  all four available sequences. Thirty-nine frames were readable. One known
  malformed K-Lane PCD failed and was recorded rather than stopping the run.

- **The problem we tested:** Stage 3 uses one global plane and one fixed
  `+/-0.12 m` acceptance band. We needed to know whether it preserves useful
  anchor points across the road, especially at distance, rather than merely
  producing a respectable total point count.

- **The method:** I mapped the Stage-2 and Stage-3 points onto the same BEV
  grid, divided the road into eight near-to-far bands, measured the residuals
  to the fitted plane, and checked how many lane-label cells with an available
  Stage-2 point still had a point after Stage 3. Labels were used only for this
  after-the-fact measurement; they did not influence the plane.

- **The outcome:** Most frames are not catastrophic. Median Stage-3 spatial
  retention is 72.25%, and Stage 3 keeps 77.17% of the labeled cells that had
  at least one Stage-2 point available. However, three frames clearly select a
  bad global surface: their far-field retention falls to 0%, 0%, and 3.48%.
  Sequence 4 has another problem—its total road coverage can look reasonable
  while the retained points miss the actual lane corridors.

- **How to interpret the scary 78.7% number:** it is not a 78.7% Stage-2
  failure. K-Lane supplies dense, continuous lane labels, while one LiDAR frame
  gives us sparse samples. A later line, quadratic, or cubic fit is supposed
  to connect many empty cells between reliable observations. The number tells
  us how sparse the anchors are; it does not tell us the final achievable lane
  recall.

- **Why Stage 3 still matters:** geometry fitting can bridge gaps when it has
  a few well-distributed anchors. It cannot safely reconstruct the road when
  Stage 3 keeps only near-field points or removes almost every point along a
  lane corridor. The three collapsed frames and the sequence-4 corridor losses
  are therefore worth fixing before asking Stage 7 to draw complete lanes.

- **Recommended path:** keep the Stage-3.1 idea as a post-MVP refinement. For
  the MVP, continue to Stage-6 cleanup with the current Stage 3. When Stage 3
  is revisited later, first check whether the RANSAC seed has believable
  support across distance, then test the planned smooth surface
  `z = a + bx + cy + dx^2`.

The rest of this report contains the exact measurements, thresholds, failure
examples, implementation details, and reproduction commands behind that
summary.

## What was added

### Occupied-cell coverage

Stage-2 and Stage-3 points are independently rasterized into the accepted
K-Lane `144 x 144` geometry. Coverage is measured using occupied cells rather
than point counts so a dense near-field scan does not hide an empty far field.

The principal data-only measures are:

- Stage-3 occupied cells divided by Stage-2 occupied cells;
- the same ratio in eight equal forward-distance bands;
- the same ratio in eight equal lateral bands;
- far-field retention for `x >= 23.04 m`; and
- signed and absolute plane residual statistics within every band.

### Conditional label support

The label audit asks a narrow question: when a K-Lane lane cell contains at
least one Stage-2 ROI point, does Stage 3 retain at least one point in that
cell? The resulting conditional retention isolates Stage-3 loss from label
cells that had no LiDAR support beforehand.

Labels are loaded only after RANSAC fitting and never change its plane or
inlier mask.

### Explicit warning flags

The audit records fixed diagnostic warnings:

- overall occupied-cell retention below 35%;
- more than one populated forward band below 25% retention;
- far-field retention below 25%; and
- evaluation-only conditional label support below 75%.

The first three determine the data-only `insufficient_spatial_coverage` flag.
The label warning remains separate. These thresholds are inspection aids, not
detector parameters or accuracy claims.

### Four-view visual artifact

Every successful frame has a panel showing:

1. Stage-2 occupied cells and the subset retained by Stage 3;
2. label cells supported, lost at Stage 3, or missing before Stage 3;
3. occupied-cell retention in eight near-to-far bands; and
4. signed point-to-plane residual versus forward distance.

The complete artifact index is
[`../artifacts/08_stage3_reevaluation/README.md`](../artifacts/08_stage3_reevaluation/README.md).

## Audit population

The command selected ten evenly spaced labeled frames from each of the four
locally available sequences. It attempted 40 frames and completed 39.

`seq_04/001680832844300` failed because its PCD declares 786,432 values but
contains 160,567. The audit recorded the failure and continued. This is the
same malformed source frame observed during checkpoint 06.

The 39 successful frames are a development audit, not a held-out result.

## Aggregate results

| Scope | Frames | Median overall cell retention | Median far retention | Conditional label retention (micro) | Spatial warnings | Label warnings | Median plane tilt |
|---|---:|---:|---:|---:|---:|---:|---:|
| **Overall** | 39 | 72.25% | 50.19% | 77.17% | 12 | 17 | 1.24 deg |
| `seq_01` | 10 | 78.37% | 66.16% | 84.35% | 3 | 2 | 1.04 deg |
| `seq_02` | 10 | 83.18% | 68.65% | 82.24% | 2 | 2 | 1.00 deg |
| `seq_03` | 10 | 72.54% | 51.79% | 78.00% | 3 | 5 | 1.01 deg |
| `seq_04` | 9 | 57.24% | 44.69% | 46.68% | 4 | 8 | 2.91 deg |

Across the 39 frames:

- 9 frames triggered both the data-only and label-support warnings;
- 3 triggered only the data-only warning;
- 8 triggered only the label-support warning; and
- 19 triggered neither warning.

That separation is important. Overall road coverage can appear acceptable
while the retained surface misses the particular corridors containing lane
labels.

## Label-support accounting

| Quantity | Cells | Share of all labeled cells |
|---|---:|---:|
| All labeled lane cells | 22,101 | 100.00% |
| Supported by at least one Stage-2 point | 4,708 | 21.30% |
| Supported after Stage 3 | 3,633 | 16.44% |
| Lost specifically at Stage 3 | 1,075 | 4.86% |
| Missing before Stage 3 | 17,393 | 78.70% |

Stage 3 retains 77.17% of the label cells that Stage 2 could potentially
support. The remaining 78.70% without a Stage-2 point should be read as sparse
single-frame anchor coverage, not as a requirement for Stage 2 to populate
every labeled cell. Later geometry fitting is expected to interpolate between
well-supported anchors.

## Main findings

### 1. Three frames suffer catastrophic global-plane collapse

The worst frames are:

| Frame | Overall retention | Far retention | Conditional label retention | Plane tilt |
|---|---:|---:|---:|---:|
| `seq_01/001466408273720` | 19.33% | 0.00% | 18.95% | 7.20 deg |
| `seq_02/001207470978270` | 20.51% | 0.00% | 27.44% | 5.09 deg |
| `seq_01/001351019145550` | 25.92% | 3.48% | 15.38% | 3.89 deg |

In `seq_01/001466408273720`, Stage 3 retains no point beyond 11.52 m even
though Stage 2 contains occupied cells throughout the forward ROI. Median
signed residual rises from `+0.05 m` in the nearest band to `+4.72 m` in the
farthest band. This smooth, large forward drift is not explained by sparse
sampling alone. The globally dominant plane is not a credible model of the
complete visible road surface.

Representative panel:
[`seq_01_001466408273720_stage3_coverage.png`](../artifacts/08_stage3_reevaluation/images/seq_01_001466408273720_stage3_coverage.png)

### 2. The 20-degree tilt ceiling is only a wall guard

The median selected tilt is 1.24 degrees and the 95th percentile is 4.44
degrees, but the configured maximum remains 20 degrees. The three catastrophic
frames occur among the higher-tilt fits. Tilt alone is not a sufficient
decision rule, but the current limit permits surfaces that are implausible
when their support is concentrated near the sensor and their residual grows
strongly with distance.

A future Stage-3 refinement should not merely feed every accepted RANSAC plane
into a smooth surface fit. It should first require spatially distributed
support and a plausible residual trend.

### 3. Sequence 4 has lane-corridor loss beyond global coverage

Sequence 4 has 57.24% median overall retention but only 46.68% micro
conditional label retention. Eight of nine successful samples trigger the
label-support warning.

`seq_04/001611741463910` is the clearest example: overall occupied-cell
retention is 55.90% and no data-only spatial warning fires, yet Stage 3 keeps
only 1 of 94 potentially supported lane cells. The plane supports other road
regions while missing the label corridors.

This means that total road-point count or total BEV occupancy cannot be the
only acceptance measure for a future Stage-3 refinement.

### 4. The fixed band contributes to moderate losses

Several non-catastrophic frames retain broad spatial support but show median
absolute residuals around or above the fixed `0.12 m` boundary in individual
forward bands. A smoothly varying surface could recenter these bands and
recover support without globally widening the threshold.

This is evidence for the planned low-order surface refinement. It is not
evidence for simply increasing the fixed band, which would also retain more
low obstacles.

### 5. Sparse input support is mainly a later geometry problem

Most labeled cells contain no Stage-2 point in a single frame. That is expected
from LiDAR sparsity, dashed paint, range, and occlusion, while the labels draw
continuous lane geometry. Stage 7 is where supported fragments should be
connected into a line or curve. A future Stage-3 refinement should therefore
report conditional anchor retention separately from absolute label coverage:
its job is to preserve useful evidence, not fill the final lane mask by itself.

## Scope boundary

Checkpoint 08 adds measurement and reporting only. It does not change the
Stage-3 plane or road-point decision. The Stage-3.1 surface is deferred until
after the MVP, and the later BEV cleanup and geometry fitting remain separate
checkpoints.

## Implementation and tests

The checkpoint adds:

- `src/lidar_lane_detection/road_evaluation.py` for deterministic coverage and
  warning calculations;
- `src/lidar_lane_detection/road_coverage_diagnostics.py` for the four-view
  panels;
- `src/lidar_lane_detection/road_audit_cli.py` for multi-sequence execution,
  failure preservation, summaries, CSV, and artifact indexing;
- `tests/test_road_evaluation.py` for exact occupied-cell loss, label
  separation, mask alignment, band output, and invalid configuration; and
- the `klane-stage3-audit` command.

The full suite passes:

```text
Ran 28 tests
OK
```

Source and test bytecode compilation also passes. The final full audit run
reported a Stage-3 fit-time median of 71.78 ms and p95 of 93.01 ms. Timing is
diagnostic only and varies with cache and machine load.

## Reproduction

```bash
env UV_CACHE_DIR=/tmp/uv-cache uv run klane-stage3-audit \
  /mnt/arnavs_hdd/data/02-lane-detection/K-Lane \
  --sequence all \
  --sample-count 10 \
  --output-dir artifacts/08_stage3_reevaluation

env UV_CACHE_DIR=/tmp/uv-cache uv run python -m unittest discover -s tests -v
env UV_CACHE_DIR=/tmp/uv-cache uv run python -m compileall -q src tests
```

Machine-readable outputs are
[`audit.json`](../artifacts/08_stage3_reevaluation/audit.json) and
[`audit.csv`](../artifacts/08_stage3_reevaluation/audit.csv).

## Deferred Stage-3 refinement recommendation

Retain the following design for a future post-MVP Stage-3 refinement:

1. retain the existing RANSAC plane as a seed candidate;
2. reject or flag seeds with concentrated spatial support or implausible
   forward residual drift;
3. robustly fit `z = a + bx + cy + dx^2` from plausible ground candidates;
4. constrain slope and curvature explicitly;
5. mark unsupported regions rather than extrapolating confidently; and
6. compare conditional lane-cell support, far-field coverage, retained-point
   growth, and downstream raw Stage-5 behavior against the unchanged Stage 3.

The active checkpoint 09 is Stage-6 BEV cleanup, not this deferred refinement.