# Stage 5.1 findings: candidate quality audit and one minimal refinement Date: 14 July 2026 Project: `classical-lidar-lane-detection` Architecture plan: ## Short outcome The original five Stage-5 images were correctly exposing the raw global-Otsu decision, but they were too few to judge it. The visible horizontal structures are mostly LiDAR scan arcs, not longitudinal lane markings. On the expanded 39-frame successful benchmark, raw Otsu occupancy reaches only **17.16%** K-Lane one-cell-tolerant F1. Stage 5.1 adds exactly three things: a multi-sequence benchmark, quantitative candidate metrics with clearer plots, and one small row-wise candidate proposal. The proposal raises tolerant F1 to **21.02%** on this exploratory sample, a relative increase of **22.47%**, while remaining deterministic, CPU-only, and free of ML. This is a useful improvement, not a good detector yet. ## What I observed ### 1. The dominant false-positive shape is transverse The raw candidate plots repeatedly show curved horizontal bands across the road. In the ground-truth BEV, lane cells are primarily longitudinal. The strict error maps confirm that most of these scan-arc cells are false positives. This is not a coordinate flip: the ground-truth orientation remains top-forward and left-positive-`y`, and occasional candidate streaks do overlap the lanes. ### 2. Per-frame Otsu is unstable as a lane-candidate budget Global Otsu reacts to the full reflectivity distribution. In the expanded sample, it sometimes keeps almost nothing and sometimes keeps a very dense bright tail. Examples include 6 candidate points in `seq_01/001351019145550` and 1,620 in `seq_01/001466408273720`. A correct Otsu split is therefore not necessarily a useful lane split. ### 3. Some failures are inherited from Stage 3 Several frames contain road support only in a limited near-field region. No Stage-5 threshold can recover lane evidence that Stage 3 did not retain. The failure is especially visible in `seq_01/001466408273720`, where the Stage-5.1 proposal makes the score worse because it removes evidence from an already poor spatial support set. ### 4. Sequence 4 is a qualitatively different failure Sequence 4 has curved ground truth in several sampled frames, while much of the bright retained evidence follows unrelated road or scene structure. Its raw and Stage-5.1 tolerant F1 values are only 8.10% and 9.35%. A row budget cannot solve missing signal, incorrect road support, or curve geometry. ### 5. One sampled source frame is malformed `seq_04/001680832844300` declares 786,432 PCD values but contains 160,567. Stage 5.1 records this as a failed frame and continues, rather than losing the entire benchmark. The run therefore attempted 40 frames and produced 39 panels. ## What I proposed and tested I kept the exploration deliberately small. 1. **Measure more than five frames.** Use ten evenly spaced labeled samples from each of the four local sequences, preserve failures, and keep a clean PNG-only directory for visual review. 2. **Score raw candidate occupancy.** Report exact-cell and official K-Lane-style one-cell-tolerant precision, recall, and F1. Show raw occupancy, ground truth, and TP/FP/FN colors in the same artifact. 3. **Test one bounded transverse-noise rule.** Use the top reflectivity decile, take the maximum reflectivity per 144 x 144 cell, and keep at most the six strongest cells in each longitudinal row. Six comes from K-Lane's six lane class slots; it is not estimated from an individual frame's ground truth. I also tested a longitudinal-neighbor support filter because it seemed like the most obvious response to the scan arcs. It reduced recall enough to lower F1, so it was rejected and is not part of the implementation. ## What I implemented, and why ### 1. Expanded benchmark and clean artifact layout `klane-stage5-1` accepts one sequence, a comma-separated list, or `all`. The benchmark continues past a malformed sampled frame and reports the failure. Its artifact layout is: ```text artifacts/stage5_1/ images/ # PNG files only reports/ # per-frame JSON benchmark.csv # flat quantitative table benchmark.json # complete run record README.md # visual and summary index ``` This addresses the immediate review problem without building a general experiment framework. ### 2. Strict and K-Lane-tolerant candidate metrics The tolerant implementation reproduces the confidence behavior in the [official K-Lane metric source at commit 82c613a](https://github.com/kaist-avelab/K-Lane/blob/82c613aaa886954b5f6f31abe314b97a29f6d350/baseline/utils/metric_utils.py): a prediction within one neighboring cell can match, and the outermost border is not scored. Metrics are micro-aggregated from TP, FP, and FN counts. Each six-panel image now contains raw Otsu points, its histogram, raw 144 x 144 occupancy, Stage-5.1 occupancy, the ground-truth BEV, and a strict error map. The ground truth is evaluation-only and never changes a frame's candidates. ### 3. Top-decile plus six-cells-per-row proposal The new proposal is intentionally one function with two fixed constants. It does not add morphology, connected components, curve fitting, tracking, or a second signal channel. Compared with raw Otsu, the top decile raises recall; the six-cell row budget stops a single transverse scan arc from filling most of a row. The original Stage-5 Otsu implementation and `klane-stage5` command remain available unchanged. Stage 5.1 compares against that baseline rather than silently replacing it. ## Quantitative results ### Overall 39 successful frames | Candidate output | Precision | Recall | F1 | |---|---:|---:|---:| | Raw Otsu, strict same-cell | 6.36% | 3.60% | 4.60% | | Stage 5.1, strict same-cell | 7.25% | 4.72% | 5.72% | | Raw Otsu, K-Lane one-cell tolerant | 22.59% | 13.84% | 17.16% | | **Stage 5.1, K-Lane one-cell tolerant** | **25.24%** | **18.01%** | **21.02%** | Stage 5.1 improved tolerant per-frame F1 on 35 of 39 successful samples and worsened it on 4. The median absolute per-frame improvement was 3.75 percentage points. Candidate processing through Stage 5.1, including PCD loading and Stages 2-5 but excluding label loading, metrics, and PNG rendering, had a 205.1 ms median and 276.4 ms p95 on this run. ### By sequence | Sequence | Conditions | Frames | Raw tolerant F1 | Stage 5.1 tolerant F1 | |---|---|---:|---:|---:| | `seq_01` | urban, night, uncrowded | 10 | 24.71% | 29.21% | | `seq_02` | urban, day, crowded | 10 | 15.42% | 21.12% | | `seq_03` | urban, day, uncrowded | 10 | 18.42% | 21.11% | | `seq_04` | urban, day, uncrowded | 9 | 8.10% | 9.35% | The full per-frame table is in [`../artifacts/stage5_1/benchmark.csv`](../artifacts/stage5_1/benchmark.csv), and the visual index is [`../artifacts/stage5_1/README.md`](../artifacts/stage5_1/README.md). ## What the results do and do not mean The improvement is real on this sampled development audit, but it is not a held-out benchmark claim. The same sample was used to compare simple proposals, and the policy still fails badly on curves and poor upstream road support. The raw and refined masks are candidate occupancy, not fitted lane geometry. The most justified next improvements are not more Stage-5 constants: 1. add a visible Stage-3 spatial-coverage/failure flag before trusting a threshold score; 2. proceed to the planned Stage-6 connected-component/orientation cleanup and reject short transverse structures there; 3. freeze parameters and evaluate on a separate, larger held-out set before treating the 21.02% result as generalizable. None of those is implemented here. That keeps Stage 5.1 within the requested three additions and preserves YAGNI. ## Reproduction ```bash env UV_CACHE_DIR=/tmp/uv-cache uv run klane-stage5-1 \ /mnt/arnavs_hdd/data/02-lane-detection/K-Lane \ --sequence all \ --sample-count 10 \ --output-dir artifacts/stage5_1 env UV_CACHE_DIR=/tmp/uv-cache uv run python -m unittest discover -s tests -v env UV_CACHE_DIR=/tmp/uv-cache uv run python -m compileall -q src tests ```