feat(stage1): audit survey input structures

2026-09-20 07:18:51 +12:00
parent f34add281f
commit 5c31417935
6 changed files with 861 additions and 3 deletions
@@ -2,6 +2,64 @@

 Command failures and integration errors.

+---
+## [ERR-20260920-028] python-assignment-expression-comprehension
+
+**Logged**: 2026-09-20
+**Priority**: low
+**Status**: resolved
+**Area**: tests
+
+### Summary
+An inline audit command used an assignment expression in a comprehension iterable, which Python rejects.
+
+### Error
+`SyntaxError: assignment expression cannot be used in a comprehension iterable expression`.
+
+### Context
+The command was a read-only attempt to aggregate GSHS design-field coverage; no data or project file changed.
+
+### Suggested Fix
+Use an ordinary loop in a maintained audit script instead of compressing stateful aggregation into a one-liner.
+
+### Metadata
+- Reproducible: yes
+- Related Files: `research/stage1/audit_input_sources.py`
+- Pattern-Key: runtime.python-syntax
+
+### Resolution
+- **Resolved**: 2026-09-20
+- **Notes**: Replaced the inline expression with a reusable streaming audit script.
+
+---
+## [ERR-20260920-029] pandas-csv-memory-exhaustion
+
+**Logged**: 2026-09-20
+**Priority**: medium
+**Status**: resolved
+**Area**: tests
+
+### Summary
+Pandas exhausted memory while parsing the 0.77 GB, 637-column GSHS CSV even though only 12 columns were requested.
+
+### Error
+`pandas.errors.ParserError: Error tokenizing data. C error: out of memory`.
+
+### Context
+The failed read was non-mutating. The GSHS file is wide enough that the C parser's tokenization buffers remain material despite `usecols`.
+
+### Suggested Fix
+Stream rows with the standard CSV reader, map only required column indexes, and retain aggregate counters rather than a DataFrame.
+
+### Metadata
+- Reproducible: yes
+- Related Files: `research/stage1/audit_input_sources.py`
+- Pattern-Key: runtime.memory-exhaustion
+
+### Resolution
+- **Resolved**: 2026-09-20
+- **Notes**: The streaming implementation completed the full 673,499-row audit without loading the dataset into memory.
+
 ---
 ## [ERR-20260920-024] git-push-repository-not-found