# Stage 4 report: unthresholded road-point reflectivity Date: 14 July 2026 Project: `classical-lidar-lane-detection` Architecture plan: ## 1. Outcome Stage 4 is complete. The package now selects K-Lane's `reflectivity` field from the complete PCD rows retained by Stage 3 and exposes those values as a one-dimensional signal aligned to the road points. It applies no threshold, normalization, clipping, smoothing, or label-driven decision. The real-data workflow produces one human-readable PNG and one JSON report per frame, a run manifest, and an artifact landing page. Each PNG shows the reflectivity spatially in the accepted K-Lane top-down orientation and as a raw-value histogram. This stage exposes the marking signal. It does not yet decide which values are lane-marking candidates; global Otsu is Stage 5. ## 2. Fixed Stage-4 contract The implementation follows the architecture plan's single fixed choice: | Contract | Stage-4 behavior | |---|---| | Input rows | Stage-3 road-plane inliers, with complete PCD rows preserved | | Signal | K-Lane PCD `reflectivity` field | | Signal processing | None | | Alignment | One reflectivity value for every road-point row, in the same order | | Validation | At least one value; every selected value must be finite | | Output | Raw `float32` values, distribution report, and diagnostic PNG | The PCD reader stores all fields in its existing `float32` row array. Stage 4 selects the named reflectivity column from that array. It does not use `intensity`, labels, alternate channels, or a missing-ring fallback. ## 3. How to read the PNG The **left top-down panel** contains only Stage-3 road points. Top is forward, bottom is near, left is positive lateral `y`, and right is negative lateral `y`. Dark blue represents lower reflectivity; green and yellow represent higher reflectivity. The processing result remains raw. For display only, points that land on the same image pixel use their maximum value, values above the frame's 99.5th percentile share the brightest color, and a square-root color scale keeps ordinary road returns visible beside the bright tail. The exact display cap is printed in every PNG. The **right panel** is a 64-bin histogram of the same raw road-point values. Its horizontal axis is linear reflectivity from zero to the 99.5th-percentile display cap; the last bin includes higher values. Bar height uses `log(1 + count)` so low-count bright returns remain visible. These two display choices do not feed the detector. ## 4. Code and interfaces added - `src/lidar_lane_detection/reflectivity.py` contains the aligned result, direct signal selection, validation, and distribution report. - `src/lidar_lane_detection/reflectivity_diagnostics.py` creates the two-view PNG using the existing NumPy and Pillow dependencies. - `src/lidar_lane_detection/reflectivity_cli.py` repeats Stages 1-3, selects reflectivity, and writes the per-frame and run-level artifacts. - `tests/test_reflectivity.py` verifies direct value preservation, row alignment, no input mutation, report bounds, and non-finite rejection. - `pyproject.toml` exposes the workflow as `klane-stage4`. `RoadReflectivityResult` deliberately contains only the raw aligned values and their report. There is no Stage-4 configuration object because there is no processing parameter to configure. ### Why reflectivity, not intensity | Field | Meaning | Main problem | |---|---|---| | **Intensity** | Raw returned laser strength—how much signal reached the receiver. | Strongly changes with range, incidence angle, beam/channel sensitivity, and sensor settings. | | **Reflectivity** | A sensor-derived estimate of the target’s inherent near-infrared reflectance, using intensity plus calibration/range compensation. | More comparable, but still not a perfect material constant. | On consistency: reflectivity is more consistent, not absolutely consistent. If the same material appears in two locations, its measured reflectivity can still differ because of: - different laser channel/ring calibration; - different range; - different incidence angle—the beam hitting paint/asphalt more obliquely; - surface state: wetness, dirt, paint wear; - saturation, multipath, and sampling/spot-size effects. ## 5. Five-frame real-data artifacts The artifact landing page is [`../artifacts/stage4/README.md`](../artifacts/stage4/README.md). One representative panel is [`../artifacts/stage4/seq_01_001270427447090_reflectivity.png`](../artifacts/stage4/seq_01_001270427447090_reflectivity.png). The command reused the five evenly spaced sequence-1 frames from Stages 2 and 3: | Timestamp | Road points | Minimum | Median | P95 | Maximum | |---|---:|---:|---:|---:|---:| | `001270427447090` | 13,714 | 97 | 1,077 | 12,196 | 63,835 | | `001334420901190` | 14,739 | 91 | 1,198 | 14,926 | 65,063 | | `001412513600980` | 13,263 | 79 | 1,211 | 13,565 | 65,314 | | `001455209231720` | 13,324 | 69 | 767 | 12,981 | 65,084 | | `001501304913440` | 8,442 | 83 | 1,049 | 9,799 | 65,068 | The top-down panels show a broad background of lower-reflectivity road scan lines plus a much smaller set of brighter streaks and compact returns. Those bright structures are the evidence Stage 5 will threshold, but Stage 4 does not call them lanes. The distributions are strongly long-tailed in every frame, which makes a per-frame global threshold plausible enough to test next without adding a second method. The fifth frame remains visibly sparse because its Stage-3 plane retained only 39.1% of the ROI points. Stage 4 intentionally carries that upstream limitation forward rather than filling or replacing missing support. ## 6. Automated behavior verified The complete project suite result is: ```text Ran 14 tests in 0.265s OK ``` The complete source and test trees also pass Python bytecode compilation. The five-frame Stage-4 command completed and every generated PNG was visually inspected. ## 7. Explicitly not done To preserve the YAGNI boundary, this stage did not add: - Otsu or any other candidate threshold; - ring-wise processing, range correction, intensity fusion, or normalization; - a BEV detector raster, morphology, connected components, or line fitting; - label overlays, accuracy metrics, or parameter tuning; - a plotting dependency, interactive UI, ML, GPU, ROS, or temporal support. The label is intentionally absent from Stage-4 artifacts: these panels answer what reflectivity is visible on extracted road points, not whether a candidate decision agrees with ground truth. ## 8. Reproduction Run from the project root: ```bash env UV_CACHE_DIR=/tmp/uv-cache uv sync env UV_CACHE_DIR=/tmp/uv-cache uv run klane-stage4 \ /mnt/arnavs_hdd/data/02-lane-detection/K-Lane \ --sequence 1 \ --sample-count 5 \ --output-dir artifacts/stage4 env UV_CACHE_DIR=/tmp/uv-cache uv run python -m unittest discover -s tests -v env UV_CACHE_DIR=/tmp/uv-cache uv run python -m compileall -q src tests ``` ## 9. Next boundary Stage 5 may apply one global Otsu threshold per frame to these road-point values and visualize retained versus rejected candidates. No thresholding work has been folded into Stage 4.