import(stage1): validate YRBS 2021 and 2023 layouts

.learnings/ERRORS.md

@@ -2,6 +2,35 @@
 
 Command failures and integration errors.
 
+---
+## [ERR-20260920-026] sas-input-block-comment-collision
+
+**Logged**: 2026-09-20
+**Priority**: low
+**Status**: resolved
+**Area**: data
+
+### Summary
+The first YRBS fixed-width layout parser matched the words “Input Program” in the SAS header comment instead of the executable `Input` statement.
+
+### Error
+`ValueError: No fields parsed` for the official 2021 SAS input program.
+
+### Context
+The broad first-occurrence regular expression selected a comment fragment. No Parquet or audit output was written.
+
+### Suggested Fix
+Anchor the parser to a line containing only the SAS `Input` keyword, then parse the following field declarations.
+
+### Metadata
+- Reproducible: yes
+- Related Files: `research/stage1/import_yrbs_2021_2023.py`
+- Pattern-Key: parser.comment-keyword-collision
+
+### Resolution
+- **Resolved**: 2026-09-20
+- **Notes**: The input-block expression is line-anchored and the import is rerun from the official files.
+
 ---
 ## [ERR-20260920-036] pdf-text-console-encoding
 
@@ -1055,7 +1084,7 @@ Remove the override and tolerate the harmless warning, or use an actual readable
 - **Notes**: The override is removed; no index changes occurred in the failed attempt.
 
 ---
-## [ERR-20260920-024] parquet-field-name-assumption
+## [ERR-20260920-037] parquet-field-name-assumption
 
 **Logged**: 2026-09-20
 **Priority**: low
@@ -1084,7 +1113,7 @@ Inspect the authoritative schema before column projection and use the source-spe
 - **Notes**: Re-ran the count query against `YEAR` and obtained annual distributions for `YUSUITHK` and `YUSUIPLN`.
 
 ---
-## [ERR-20260920-025] yrbs-attempt-response-shape
+## [ERR-20260920-038] yrbs-attempt-response-shape
 
 **Logged**: 2026-09-20
 **Priority**: medium
@@ -1113,3 +1142,32 @@ Apply the shared five-level frequency mapping to both GSHS and YRBS attempt item
 - **Notes**: Canonical attempt categories are now `zero`, `one`, `two_or_three`, `four_or_five`, and `six_or_more` in both surveys.
 
 ---
+## [ERR-20260920-039] sas-input-parser-edge-cases
+
+**Logged**: 2026-09-20
+**Priority**: low
+**Status**: resolved
+**Area**: data
+
+### Summary
+The first two YRBS fixed-width parser attempts encountered a SAS header-comment keyword collision and the SAS numeric missing token `.`.
+
+### Error
+The first attempt found no layout fields; the second could not convert `.` to a float.
+
+### Context
+Both failures occurred before a complete Parquet/audit output was accepted.
+
+### Suggested Fix
+Anchor the layout parser to a line containing only `Input` and map the SAS numeric missing token `.` to null.
+
+### Metadata
+- Reproducible: yes
+- Related Files: `research/stage1/import_yrbs_2021_2023.py`
+- Pattern-Key: parser.sas-fixed-width-edge-cases
+
+### Resolution
+- **Resolved**: 2026-09-20
+- **Notes**: Both cases are handled explicitly and the official-layout import is rerun from source.
+
+---

Progression.md

@@ -1,9 +1,9 @@
 # 项目进度档案
 
-版本：1.13
+版本：1.14
 建立日期：2026-09-20  
 当前总状态：`in_progress`  
-当前阶段：阶段 1 `in_progress`；84 个主要结局题目实例、396 条回答编码、28 条设计登记和 84 条人工双审队列已形成；`responses_long.parquet`、YRBS 2021/2023 导入、设计复核和实际双人审核尚未完成
+当前阶段：阶段 1 `in_progress`；84 个主要结局题目实例与编码/双审包已形成，YRBS 1991–2023 共 17 年完整表均通过来源与值域检查；`responses_long.parquet`、设计复核和实际双人审核尚未完成
 
 ## 1. 管理规则
 
@@ -28,7 +28,7 @@
 | 阶段 | 名称 | 状态 | 当前判定 |
 |---|---|---|---|
 | 0 | 冻结研究问题、范围与版本 | `passed` | 四项交付物齐备，Gate 0 已通过；研究工具链补强与锁定完成 |
-| 1 | 有来源证据的题库和回答层 | `in_progress` | 主要结局题库、回答字典、构念本体、候选连接、设计登记、冲突表和人工双审队列已生成；回答长表、2021/2023 导入、设计复核和实际双重审核尚未完成 |
+| 1 | 有来源证据的题库和回答层 | `in_progress` | 主要结局结构化包已生成，YRBS 2021/2023 已按官方 SAS 布局解除导入隔离；回答长表、设计复核和实际双重审核尚未完成 |
 | 2 | 可行性、可识别性和连接审计 | `not_started` | 等待 Gate 1 |
 | 3 | 冻结验证协议与已知真值模拟 | `not_started` | 等待 Gate 2 |
 | 4 | 直接协调和传统基线 | `not_started` | 等待 Gate 3 |
@@ -87,7 +87,7 @@ Gate 0 当前判定：`passed`。四项交付物齐备；目标总体政策、
 | P1 | R、PyMC/ArviZ、心理测量包与 CUDA Toolkit 曾缺失 | 已解决：项目 Python/R 环境和 CUDA 端到端验收通过 | 后续仅按锁定文件维护；任何升级必须重跑三套烟雾测试 |
 | P1 | 文献初筛尚未形成可审计矩阵 | 研究定位和“贡献”表述仍不稳定 | 建立 literature matrix，记录 DOI、来源、证据层级和项目关系 |
 | P0 | NSDUH 的 `YSUI03/YUSUICTRY` 已确认施测，但尝试及严重度字段不在当前本地公开使用数据中；2024 指南明确记录派生尝试变量因披露风险排除 | 当前微数据只能外部验证意念/计划，不能验证尝试 | 不从其他题推导尝试；如核心设计必须使用尝试，评估受限数据申请或预先设计的聚合结果方案 |
-| P1 | Solomon Islands 2023 已有 WHO 官方目录的逐题来源与总体资料，但未取得国家问卷文件/实际施测语言证据；YRBS 2021/2023 仍处于导入隔离 | 不影响该组件的题文/编码来源，但限制语言字段批准；后两年不能进入回答层 | 保持语言 provisional；按官方 SAS 布局验证 YRBS 2021/2023 导入 |
+| P1 | Solomon Islands 2023 已有 WHO 官方目录的逐题来源与总体资料，但未取得国家问卷文件/实际施测语言证据 | 不影响该组件的题文/编码来源，但限制语言字段批准 | 保持语言 provisional；在语言 DIF 分析前取得施测语言证据或排除相应主张 |
 
 ## 8. 行动日志
 
@@ -757,11 +757,24 @@ Gate 0 当前判定：`passed`。四项交付物齐备；目标总体政策、
 - 编码决策：GSHS/YRBS 尝试次数保留五级 `zero/one/two_or_three/four_or_five/six_or_more`，不在原始字典中静默二分；NSDUH 的“不确定”保留为观测类别，拒答、不知道、数据错误、空白和合法跳题分别保留为缺失类型；任何未知缺失均不得自动转为阴性。
 - 冲突登记：21 条 YRBS 旧映射作为已由直接来源解决的冲突保留审计轨迹；7 个 GSHS 组件的实际施测语言证据缺口和 4 个 NSDUH 尝试题的公用微数据访问缺口保持开放。
 - 设计边界：GSHS 使用规范化权重/PSU/stratum 候选字段；YRBS 2017/2019 设计字段缺口及 2021/2023 导入隔离被显式标记；NSDUH 的 `ANALWT2_C1`–`C4` 年度适用映射未凭猜测选择。所有设计记录仍为 provisional。
-- 工程记录：一次 NSDUH Parquet 字段名假设错误与一次 YRBS 尝试题响应形态假设错误均无数据改动，并分别记录为已解决的 `ERR-20260920-024`、`ERR-20260920-025`；后者由 fail-closed 检查在生成文件前中止，修正后重建通过。
+- 工程记录：一次 NSDUH Parquet 字段名假设错误与一次 YRBS 尝试题响应形态假设错误均无数据改动，并分别记录为已解决的 `ERR-20260920-037`、`ERR-20260920-038`；后者由 fail-closed 检查在生成文件前中止，修正后重建通过。
 - 质量边界：本行动没有生成 `responses_long.parquet`，没有解除 YRBS 2021/2023 隔离，没有解决 NSDUH 尝试题的数据访问，也没有将 AI 检查登记为人工审核。Gate 1 保持 `in_progress`。
 - 推送状态：`pushed`；研究增量提交 `f7e0dbb` 已推送至 `origin/main`。
 - 下一步：按官方 SAS 程序与布局验证并导入 YRBS 2021/2023；随后从已通过来源和编码检查的可用年度生成 `responses_long.parquet`，并安排两名实际审核者处理队列。
 
+### A-20260920-060：按官方 SAS 布局导入并解除 YRBS 2021/2023 隔离
+
+- 时间：2026-09-20
+- 阶段：1（有来源证据的题库和回答层）
+- 动作：行动前再次完整读取工作流与进度档案并核对干净 Git 状态；解析 CDC 官方 2021/2023 SAS Input 程序的定长字段位置，以源文件 SHA-256、逐行长度、主结局值域、设计字段完整性和 Parquet 往返读取作为解除隔离条件。
+- 输入：2021/2023 官方 SAS Input/Formats 程序、CDC 下载归档中的 `.dat`、本地 raw `.dat`、年度问卷与用户指南。
+- 输出：新增 `import_yrbs_2021_2023.py`、`yrbs_2021_2023_import_audit.json`、`yrbs_2021_2023_import_audit.md`；生成忽略于 Git 的 `yrbs_2021_full.parquet` 与 `yrbs_2023_full.parquet`；重建 YRBS 来源审计、主要结局登记和结构化包。
+- 检查：本地 raw 与 CDC 归档副本逐字节一致；两年每行均为 421 字节。2021 解析 17,232×239，2023 解析 20,103×252；两年 weight/stratum/PSU 均全量非空；Q26–Q28（2021）及 Q27–Q29（2023）观测值全部位于官方 1–2/1–5 值域；Parquet 写入后往返形状一致。YRBS 来源审计现为 17/17 年可用、隔离 0 年、值域失败 0。
+- 状态变更：仅解除“导入布局隔离”；题目仍为 `provisional`，不等于设计方差方法已批准或人类双审已完成。
+- 工程记录：SAS 标题注释中的 `Input` 误匹配和数值缺失符号 `.` 两个解析边界均在接受输出前失败关闭，修正并记录为 `ERR-20260920-039`。
+- 推送状态：`push_pending`；完成差异检查、提交和远端推送后更新。
+- 下一步：用当前题库/回答字典生成跨调查 `responses_long.parquet`，保留 eligibility、observed status 与各类缺失；同时准备实际双人审核安排。
+
 ## 9. 剩余工作量估算（2026-09-20 基线）
 
 | 阶段 | 主要工作 | 预计人周 | 主要波动来源 |
@@ -784,6 +797,6 @@ Gate 0 当前判定：`passed`。四项交付物齐备；目标总体政策、
 
 阶段 1 下一行动：
 
-1. 按官方 SAS 程序和机器可读布局验证 YRBS 2021/2023 导入，核对行数、值域、题号映射和设计字段；验证前继续隔离；
+1. 以题库和回答字典为唯一映射源，为当前可用且通过检查的数据生成 `responses_long.parquet`，显式保存 eligibility、observed status 与缺失类型；
-2. 以题库和回答字典为唯一映射源，为当前可用且通过检查的数据生成 `responses_long.parquet`，显式保存 eligibility 与 observed status；
+2. 复核 GSHS、YRBS、NSDUH 的年度/组件设计字段与官方方差说明，禁止凭字段名推断方差方法；
 3. 安排两名实际审核者逐项处理 `primary_outcome_human_review_queue.csv`；审核完成前保持全部主要结局为 `provisional`。

research/stage1/audit_yrbs_primary_outcome_sources.py

@@ -106,7 +106,7 @@ def build_audit() -> dict:
             source_files.setdefault(relative, {"role": role, "sha256": sha256(path)})
 
         full_path = PACKAGE / "full_by_year" / f"yrbs_{year}_full.parquet"
-        full_schema = set(pq.ParquetFile(full_path).schema_arrow.names) if year <= 2019 else set()
+        full_schema = set(pq.ParquetFile(full_path).schema_arrow.names) if full_path.exists() else set()
         for offset, construct in enumerate(CONSTRUCTS):
             official_variable = f"q{data_start + offset}"
             questionnaire_number = questionnaire_start + offset
@@ -118,7 +118,7 @@ def build_audit() -> dict:
             if data_start != questionnaire_start:
                 questionnaire_data_number_differences.add(year)
 
-            if year <= 2019:
+            if full_path.exists():
                 if official_variable not in full_schema:
                     raise AssertionError(f"{year} full table lacks {official_variable}")
                 official_counts = value_counts(full_path, official_variable)
@@ -130,8 +130,8 @@ def build_audit() -> dict:
             else:
                 official_counts = None
                 previous_counts = None
-                code_domain_status = "not_tested_quarantined"
+                code_domain_status = "not_tested_missing_import"
-                data_status = "quarantined_pending_import_layout_validation"
+                data_status = "missing_import"
 
             items.append(
                 {
@@ -177,8 +177,8 @@ def build_audit() -> dict:
         "summary": {
             "years_reviewed": 17,
             "items_reviewed": len(items),
-            "years_available_in_full_tables": 15,
+            "years_available_in_full_tables": len({item["year"] for item in items if item["data_status"] == "available_full_table"}),
-            "years_quarantined": 2,
+            "years_quarantined": len({item["year"] for item in items if item["data_status"] != "available_full_table"}),
             "previous_mapping_mismatch_years": sorted(mismatch_years),
             "previous_mapping_mismatch_items": sum(item["mapping_status"] != "verified" for item in items),
             "questionnaire_data_number_difference_years": sorted(questionnaire_data_number_differences),
@@ -188,7 +188,7 @@ def build_audit() -> dict:
         "decision": {
             "registry_action": "replace prior YRBS primary-outcome variable mappings with this direct-source crosswalk",
             "data_action": "use verified data-user-guide variables from annual full tables; do not infer data columns from printed questionnaire numbers",
-            "quarantine": "retain 2021 and 2023 as metadata-only until their imports and layouts are independently validated",
+            "quarantine": "2021 and 2023 released from layout quarantine after independent official-SAS import validation; item review remains provisional",
             "gate_1": "in_progress",
         },
     }
@@ -232,11 +232,11 @@ def render_markdown(audit: dict) -> str:
         "",
         "## Gate 1 decision",
         "",
-        "Gate 1 remains **in progress**. The 51 YRBS primary-outcome source records now have a direct-source crosswalk and clean response options, but 2021/2023 respondent imports remain quarantined, missing reasons require a conservative rule, and two actual human reviewers have not yet signed the primary outcomes.",
+        "Gate 1 remains **in progress**. The 51 YRBS primary-outcome source records have a direct-source crosswalk, clean response options, and validated 1991-2023 full tables. Missing reasons still require the conservative rule, and two actual human reviewers have not yet signed the primary outcomes.",
         "",
         "## Reproduction",
         "",
-        "Run `research/stage1/audit_yrbs_primary_outcome_sources.py` in the project Python environment. It verifies source hashes, checks the 1991-2019 annual Parquet schemas, and compares official versus previous mappings.",
+        "Run `research/stage1/import_yrbs_2021_2023.py` first, then `research/stage1/audit_yrbs_primary_outcome_sources.py`. The scripts validate official layouts, verify source hashes, check all 1991-2023 annual Parquet schemas, and compare official versus previous mappings.",
         "",
     ]
     return "\n".join(lines)

research/stage1/build_primary_outcome_package.py

@@ -219,7 +219,7 @@ def build() -> dict[str, int]:
             "response_type": source["response_type"], "language": source["language"],
             "questionnaire_version": f"YRBS national questionnaire {year}",
             "data_availability": source["data_status"],
-            "analysis_eligibility": "quarantined_pending_import_layout_validation" if year in {"2021", "2023"} else "provisional_pipeline_development_only",
+            "analysis_eligibility": "provisional_pipeline_development_only",
             "source_file": source["questionnaire_source_file"], "source_url": "",
             "page_or_section": f"{source['questionnaire_page_or_section']}; data guide {source['data_guide_page_or_section']}",
             "reviewer_1": "", "reviewer_2": "", "review_status": "provisional",
@@ -238,9 +238,7 @@ def build() -> dict[str, int]:
             })
 
     for year in sorted({str(r["year"]) for r in yrbs_rows}):
-        if year in {"2021", "2023"}:
+        if year in {"2017", "2019"}:
-            status = "quarantined_pending_import_layout_validation"
-        elif year in {"2017", "2019"}:
             status = "provisional_design_incomplete"
         else:
             status = "provisional"
@@ -253,7 +251,7 @@ def build() -> dict[str, int]:
             "namespace_rule": "prefix PSU and stratum with survey year before pooling",
             "design_source": next(r["data_guide_source_file"] for r in yrbs_rows if str(r["year"]) == year),
             "design_status": status,
-            "notes": "2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.",
+            "notes": "2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.",
         })
 
     # NSDUH: four youth-module years. Attempt is documented but unavailable locally.
@@ -432,7 +430,7 @@ Generated from the direct-source registry and audits by `build_primary_outcome_p
 - source conflicts or evidence gaps: {conflicts}
 - human reviewers recorded: 0
 
-The package does not satisfy Gate 1 by itself. Two actual human reviewers must independently sign every primary-outcome item; `responses_long.parquet` is not yet built; YRBS 2021/2023 remain quarantined; NSDUH youth attempt is absent from local public-use microdata; and several design and administered-language fields remain provisional.
+The package does not satisfy Gate 1 by itself. Two actual human reviewers must independently sign every primary-outcome item; `responses_long.parquet` is not yet built; NSDUH youth attempt is absent from local public-use microdata; and several design and administered-language fields remain provisional.
 """
     (STAGE1 / "primary_outcome_package.md").write_text(text, encoding="utf-8")
 

research/stage1/import_yrbs_2021_2023.py

@@ -0,0 +1,152 @@
+"""Validate CDC fixed-width layouts and import quarantined YRBS 2021/2023 files."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from collections import Counter
+from pathlib import Path
+
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+
+ROOT = Path(__file__).resolve().parents[2]
+SOURCE = ROOT / "Dataset" / "YRBS_National_1991_2023"
+OUT = ROOT / "Dataset" / "可直接分析数据包_NSDUH_YRBS" / "YRBS" / "full_by_year"
+STAGE1 = ROOT / "research" / "stage1"
+YEARS = {
+    2021: {
+        "raw": SOURCE / "raw" / "YRBS_2021_National.dat",
+        "archive": SOURCE / "source_archive" / "2021_cdc_download" / "XXH2021_YRBS_Data.dat",
+        "sas": SOURCE / "source_archive" / "2021_cdc_download" / "2021XXH-SAS-Input-Program.sas",
+        "items": ("Q26", "Q27", "Q28"),
+    },
+    2023: {
+        "raw": SOURCE / "raw" / "XXH2023_YRBS_Data.dat",
+        "archive": SOURCE / "source_archive" / "2023_cdc_download" / "XXH2023_YRBS_Data.dat",
+        "sas": SOURCE / "source_archive" / "2023_cdc_download" / "2023XXH-SAS-Input-Program.sas",
+        "items": ("Q27", "Q28", "Q29"),
+    },
+}
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def parse_layout(path: Path) -> list[dict[str, object]]:
+    text = path.read_text(encoding="utf-8-sig")
+    block = re.search(r"^\s*Input\s*$\s*(.*?);", text, flags=re.I | re.M | re.S)
+    if not block:
+        raise ValueError(f"No SAS Input block in {path}")
+    fields = []
+    pattern = re.compile(r"@(\d+)\s+([A-Za-z][A-Za-z0-9_]*)\s+(\$?)(\d+)(?:\.(\d+))?\.")
+    for start, name, character, width, decimals in pattern.findall(block.group(1)):
+        fields.append({
+            "start": int(start), "name": name.lower(), "width": int(width),
+            "character": bool(character), "decimals": int(decimals or 0),
+        })
+    if not fields:
+        raise ValueError(f"No fields parsed from {path}")
+    names = [str(field["name"]) for field in fields]
+    if len(names) != len(set(names)):
+        raise ValueError(f"Duplicate field names in {path}")
+    return fields
+
+
+def convert(raw: str, character: bool, decimals: int):
+    value = raw.strip()
+    if not value or value == ".":
+        return None
+    if character:
+        return value
+    if decimals or "." in value:
+        return float(value)
+    return int(value)
+
+
+def import_year(year: int, spec: dict[str, object]) -> dict[str, object]:
+    raw = spec["raw"]
+    archive = spec["archive"]
+    raw_hash = sha256(raw)
+    archive_hash = sha256(archive)
+    if raw_hash != archive_hash:
+        raise ValueError(f"{year}: raw file differs from CDC archive copy")
+    layout = parse_layout(spec["sas"])
+    columns = {str(field["name"]): [] for field in layout}
+    line_lengths = Counter()
+    with raw.open("r", encoding="ascii", newline="") as handle:
+        for source_row, line in enumerate(handle, start=1):
+            line = line.rstrip("\r\n")
+            line_lengths[len(line)] += 1
+            for field in layout:
+                start = int(field["start"]) - 1
+                width = int(field["width"])
+                columns[str(field["name"])].append(convert(line[start : start + width], bool(field["character"]), int(field["decimals"])))
+    if set(line_lengths) != {421}:
+        raise ValueError(f"{year}: unexpected record lengths {dict(line_lengths)}")
+    count = sum(line_lengths.values())
+    table_data = {"survey_year": [year] * count, "source_row": list(range(1, count + 1))}
+    table_data.update(columns)
+    table = pa.table(table_data)
+    output = OUT / f"yrbs_{year}_full.parquet"
+    pq.write_table(table, output, compression="zstd")
+    reread = pq.read_table(output)
+    if reread.num_rows != count or reread.num_columns != len(table_data):
+        raise ValueError(f"{year}: Parquet round-trip shape mismatch")
+    item_counts = {}
+    domains = {}
+    for item in spec["items"]:
+        values = columns[item.lower()]
+        counts = Counter("<missing>" if value is None else str(value) for value in values)
+        item_counts[item] = dict(sorted(counts.items()))
+        observed = {value for value in values if value is not None}
+        allowed = {"1", "2"} if item != spec["items"][2] else {"1", "2", "3", "4", "5"}
+        domains[item] = "passed" if {str(v) for v in observed} <= allowed else "failed"
+    design_nonmissing = {name: sum(value is not None for value in columns[name]) for name in ("weight", "stratum", "psu")}
+    return {
+        "year": year, "source_file": str(raw.relative_to(ROOT)).replace("\\", "/"),
+        "sas_layout": str(Path(spec["sas"]).relative_to(ROOT)).replace("\\", "/"),
+        "sha256": raw_hash, "archive_copy_identical": True, "record_length": 421,
+        "rows": count, "columns": len(table_data), "layout_fields": len(layout),
+        "output_file": str(output.relative_to(ROOT)).replace("\\", "/"),
+        "parquet_roundtrip": "passed", "primary_outcome_value_counts": item_counts,
+        "primary_outcome_domains": domains, "design_nonmissing": design_nonmissing,
+        "status": "import_validated",
+    }
+
+
+def main() -> None:
+    results = [import_year(year, spec) for year, spec in YEARS.items()]
+    audit = {
+        "schema_version": "1.0", "generated_on": "2026-09-20",
+        "status": "passed_for_import; still provisional for human item review",
+        "method": "Official CDC SAS fixed-width Input positions; raw/archive SHA-256 identity; record-length, domain, design-field, and Parquet round-trip checks.",
+        "years": results,
+        "decision": "YRBS 2021 and 2023 respondent files are released from import-layout quarantine. Primary-outcome items remain provisional pending two human reviewers.",
+    }
+    (STAGE1 / "yrbs_2021_2023_import_audit.json").write_text(json.dumps(audit, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+    lines = ["# YRBS 2021/2023 official-layout import audit", "", audit["decision"], ""]
+    for result in results:
+        lines.extend([
+            f"## {result['year']}", "",
+            f"- rows × columns: {result['rows']:,} × {result['columns']}",
+            f"- raw file identical to archived CDC download: yes (`{result['sha256']}`)",
+            f"- fixed record length: {result['record_length']} bytes for every row",
+            f"- design completeness: `{result['design_nonmissing']}`",
+            f"- primary outcome domains: `{result['primary_outcome_domains']}`",
+            f"- output: `{result['output_file']}`", "",
+        ])
+    lines.extend(["## Boundary", "", "This resolves the import-layout quarantine only. It does not substitute for two actual human item reviewers or approve design-based variance estimation.", ""])
+    (STAGE1 / "yrbs_2021_2023_import_audit.md").write_text("\n".join(lines), encoding="utf-8")
+    print(json.dumps(audit, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

research/stage1/item_bank.csv

@@ -65,12 +65,12 @@ YRBS::2017::Q28,YRBS::suicide_attempt::past_12_months,YRBS::2017,YRBS_NATIONAL::
 YRBS::2019::Q26,YRBS::suicide_ideation::past_12_months,YRBS::2019,YRBS_NATIONAL::2019,YRBS,United States,2019,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q26,"During the past 12 months, did you ever seriously consider attempting suicide?",suicide_ideation,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2019,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2019_YRBS-National-HS-Questionnaire.pdf,,page 8; data guide pages 27-28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
 YRBS::2019::Q27,YRBS::suicide_plan::past_12_months,YRBS::2019,YRBS_NATIONAL::2019,YRBS,United States,2019,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q27,"During the past 12 months, did you make a plan about how you would attempt suicide?",suicide_plan,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2019,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2019_YRBS-National-HS-Questionnaire.pdf,,page 8; data guide pages 27-28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
 YRBS::2019::Q28,YRBS::suicide_attempt::past_12_months,YRBS::2019,YRBS_NATIONAL::2019,YRBS,United States,2019,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q28,"During the past 12 months, how many times did you actually attempt suicide?",suicide_attempt,past 12 months,ordinal_frequency,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2019,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2019_YRBS-National-HS-Questionnaire.pdf,,page 8; data guide pages 27-28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2021::Q26,YRBS::suicide_ideation::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q26,"During the past 12 months, did you ever seriously consider attempting suicide?",suicide_ideation,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2021::Q26,YRBS::suicide_ideation::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q26,"During the past 12 months, did you ever seriously consider attempting suicide?",suicide_ideation,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2021::Q27,YRBS::suicide_plan::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q27,"During the past 12 months, did you make a plan about how you would attempt suicide?",suicide_plan,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2021::Q27,YRBS::suicide_plan::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q27,"During the past 12 months, did you make a plan about how you would attempt suicide?",suicide_plan,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2021::Q28,YRBS::suicide_attempt::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q28,"During the past 12 months, how many times did you actually attempt suicide?",suicide_attempt,past 12 months,ordinal_frequency,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2021::Q28,YRBS::suicide_attempt::past_12_months,YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q28,"During the past 12 months, how many times did you actually attempt suicide?",suicide_attempt,past 12 months,ordinal_frequency,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2021,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2021-YRBS-National-HS-Questionnaire.pdf,,page 8; data guide page 28,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2023::Q27,YRBS::suicide_ideation::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q27,"During the past 12 months, did you ever seriously consider attempting suicide?",suicide_ideation,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2023::Q27,YRBS::suicide_ideation::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q27,"During the past 12 months, did you ever seriously consider attempting suicide?",suicide_ideation,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2023::Q28,YRBS::suicide_plan::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q28,"During the past 12 months, did you make a plan about how you would attempt suicide?",suicide_plan,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2023::Q28,YRBS::suicide_plan::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q28,"During the past 12 months, did you make a plan about how you would attempt suicide?",suicide_plan,past 12 months,binary,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
-YRBS::2023::Q29,YRBS::suicide_attempt::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q29,"During the past 12 months, how many times did you actually attempt suicide?",suicide_attempt,past 12 months,ordinal_frequency,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,quarantined_pending_import_layout_validation,quarantined_pending_import_layout_validation,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
+YRBS::2023::Q29,YRBS::suicide_attempt::past_12_months,YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,national,students in grades 9-12 attending public and private schools in the United States; detailed eligibility remains subject to annual design review,Q29,"During the past 12 months, how many times did you actually attempt suicide?",suicide_attempt,past 12 months,ordinal_frequency,English questionnaire source; actual accommodation/language coverage not established by this audit,YRBS national questionnaire 2023,available_full_table,provisional_pipeline_development_only,Dataset/YRBS_National_1991_2023/documentation/Questionnaire/2023_YRBS_National_HS_Questionnaire.pdf,,page 9; data guide page 29,,,provisional,blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason
 NSDUH::2021::YUSUITHK,NSDUH::suicide_ideation::past_12_months,NSDUH::2021,NSDUH_YOUTH::2021,NSDUH,United States,2021,national household survey youth module,adolescents aged 12-17; final eligibility and skip logic require annual guide sign-off,YUSUITHK,"The next few questions are about thoughts of suicide. At any time in the past 12 months, that is from [DATEFILL] up to and including today, did you seriously think about trying to kill yourself ?",suicide_ideation,past 12 months,categorical_binary_with_explicit_uncertainty,English source metadata; administered-language/accommodation coverage not established,NSDUH 2021 youth module,available_in_local_public_use_table,provisional_pipeline_development_only,Dataset/可直接分析数据包_NSDUH_YRBS/NSDUH/nsduh_complete_variable_dictionary.csv,https://datatools.samhsa.gov/api/surveys/NSDUH-2021-DS0001/?format=json,YUSUITHK,,,provisional,Keep 'not sure' and refusal/nonresponse distinct until skip and missing rules are verified.
 NSDUH::2021::YUSUIPLN,NSDUH::suicide_plan::past_12_months,NSDUH::2021,NSDUH_YOUTH::2021,NSDUH,United States,2021,national household survey youth module,adolescents aged 12-17; final eligibility and skip logic require annual guide sign-off,YUSUIPLN,"During the past 12 months, did you make any plans to kill yourself?",suicide_plan,past 12 months,categorical_binary_with_explicit_uncertainty,English source metadata; administered-language/accommodation coverage not established,NSDUH 2021 youth module,available_in_local_public_use_table,provisional_pipeline_development_only,Dataset/可直接分析数据包_NSDUH_YRBS/NSDUH/nsduh_complete_variable_dictionary.csv,https://datatools.samhsa.gov/api/surveys/NSDUH-2021-DS0001/?format=json,YUSUIPLN,,,provisional,Keep 'not sure' and refusal/nonresponse distinct until skip and missing rules are verified.
 NSDUH::2022::YUSUITHK,NSDUH::suicide_ideation::past_12_months,NSDUH::2022,NSDUH_YOUTH::2022,NSDUH,United States,2022,national household survey youth module,adolescents aged 12-17; final eligibility and skip logic require annual guide sign-off,YUSUITHK,"Note: Beginning in 2022, questions YSUI01, YCOV9, YSUI02, YCOV10, YSUI03, YCOV11, YSUI04, and YSUI05 were moved from the youth mental health service utilization section to the youth experiences section. The next few questions are about thoughts of suicide. You can answer “I’m not sure” or “I don’t want to answer” to any question. At any time in the past 12 months, that is from [DATEFILL] up to and including today, did you seriously think about trying to kill yourself?",suicide_ideation,past 12 months,categorical_binary_with_explicit_uncertainty,English source metadata; administered-language/accommodation coverage not established,NSDUH 2022 youth module,available_in_local_public_use_table,provisional_pipeline_development_only,Dataset/可直接分析数据包_NSDUH_YRBS/NSDUH/nsduh_complete_variable_dictionary.csv,https://datatools.samhsa.gov/api/surveys/NSDUH-2022-DS0001/?format=json,YUSUITHK,,,provisional,Keep 'not sure' and refusal/nonresponse distinct until skip and missing rules are verified.

research/stage1/primary_outcome_package.md

@@ -10,4 +10,4 @@ Generated from the direct-source registry and audits by `build_primary_outcome_p
 - source conflicts or evidence gaps: 32
 - human reviewers recorded: 0
 
-The package does not satisfy Gate 1 by itself. Two actual human reviewers must independently sign every primary-outcome item; `responses_long.parquet` is not yet built; YRBS 2021/2023 remain quarantined; NSDUH youth attempt is absent from local public-use microdata; and several design and administered-language fields remain provisional.
+The package does not satisfy Gate 1 by itself. Two actual human reviewers must independently sign every primary-outcome item; `responses_long.parquet` is not yet built; NSDUH youth attempt is absent from local public-use microdata; and several design and administered-language fields remain provisional.

research/stage1/primary_outcome_registry.json

@@ -1513,7 +1513,7 @@
       "previous_registry_variable": "Q28",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -1543,7 +1543,7 @@
       "previous_registry_variable": "Q26",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -1573,7 +1573,7 @@
       "previous_registry_variable": "Q27",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -1606,7 +1606,7 @@
       "previous_registry_variable": "Q29",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -1636,7 +1636,7 @@
       "previous_registry_variable": "Q27",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -1666,7 +1666,7 @@
       "previous_registry_variable": "Q28",
       "mapping_status": "verified",
       "present_in_local_data": false,
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
       "source_verification_status": "direct_questionnaire_and_data_user_guide_reviewed",
       "review_status": "provisional",
       "reviewer_1": null,

research/stage1/survey_design.csv

@@ -6,23 +6,23 @@ GSHS::JAM_2023_GSHS_v01,JAM_2023_GSHS_v01::jam_st_cath_2023,GSHS,Jamaica,2023,sc
 GSHS::MNG_2023_GSHS_v01,MNG_2023_GSHS_v01::mng2023,GSHS,Mongolia,2023,school-attending students; exact sampled ages/grades remain pending component source review,sample_weight,sampling_psu,sampling_stratum,pending official component design review,prefix PSU and stratum with component_id before pooling,Dataset/GSHS-全球学生健康调查数据/GSHS/Questionnaire/GSHS Per Country/Mongolia 2023/Mongolia-GSHS-Questionnaire-ENGLISH.docx,provisional,Canonical fields are present; observed full GSHS table has 20 missing values in each design field overall. Component-specific design validity remains to be reviewed.
 GSHS::SLB_2023_GSHS_v01,SLB_2023_GSHS_v01::slb_2023,GSHS,Solomon Islands,2023,school-going adolescents aged 13-17 years; Form 1-Form 6; all students in selected classes eligible,sample_weight,sampling_psu,sampling_stratum,pending official component design review,prefix PSU and stratum with component_id before pooling,https://extranet.who.int/ncdsmicrodata/index.php/catalog/992,provisional,Canonical fields are present; observed full GSHS table has 20 missing values in each design field overall. Component-specific design validity remains to be reviewed.
 GSHS::WLF_2023_GSHS_v01,WLF_2023_GSHS_v01::wlf_2023,GSHS,Wallis and Futuna,2023,school-attending students; exact sampled ages/grades remain pending component source review,sample_weight,sampling_psu,sampling_stratum,pending official component design review,prefix PSU and stratum with component_id before pooling,Dataset/GSHS-全球学生健康调查数据/GSHS/Questionnaire/GSHS Per Country/Wallis and Futuna 2023/Wallis and Futuna - GSHS Questionnaire - Final - French.docx,provisional,Canonical fields are present; observed full GSHS table has 20 missing values in each design field overall. Component-specific design validity remains to be reviewed.
-YRBS::1991,YRBS_NATIONAL::1991,YRBS,United States,1991,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1991_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::1991,YRBS_NATIONAL::1991,YRBS,United States,1991,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1991_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::1993,YRBS_NATIONAL::1993,YRBS,United States,1993,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1993_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::1993,YRBS_NATIONAL::1993,YRBS,United States,1993,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1993_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::1995,YRBS_NATIONAL::1995,YRBS,United States,1995,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1995_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::1995,YRBS_NATIONAL::1995,YRBS,United States,1995,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1995_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::1997,YRBS_NATIONAL::1997,YRBS,United States,1997,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1997_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::1997,YRBS_NATIONAL::1997,YRBS,United States,1997,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1997_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::1999,YRBS_NATIONAL::1999,YRBS,United States,1999,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1999_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::1999,YRBS_NATIONAL::1999,YRBS,United States,1999,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_1999_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2001,YRBS_NATIONAL::2001,YRBS,United States,2001,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2001_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2001,YRBS_NATIONAL::2001,YRBS,United States,2001,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2001_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2003,YRBS_NATIONAL::2003,YRBS,United States,2003,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2003_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2003,YRBS_NATIONAL::2003,YRBS,United States,2003,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2003_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2005,YRBS_NATIONAL::2005,YRBS,United States,2005,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2005_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2005,YRBS_NATIONAL::2005,YRBS,United States,2005,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2005_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2007,YRBS_NATIONAL::2007,YRBS,United States,2007,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2007_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2007,YRBS_NATIONAL::2007,YRBS,United States,2007,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2007_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2009,YRBS_NATIONAL::2009,YRBS,United States,2009,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2009_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2009,YRBS_NATIONAL::2009,YRBS,United States,2009,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2009_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2011,YRBS_NATIONAL::2011,YRBS,United States,2011,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2011_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2011,YRBS_NATIONAL::2011,YRBS,United States,2011,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2011_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2013,YRBS_NATIONAL::2013,YRBS,United States,2013,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2013_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2013,YRBS_NATIONAL::2013,YRBS,United States,2013,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/YRBS_2013_National_User_Guide.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2015,YRBS_NATIONAL::2015,YRBS,United States,2015,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2015_yrbs-data-users_guide_smy_combined.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2015,YRBS_NATIONAL::2015,YRBS,United States,2015,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2015_yrbs-data-users_guide_smy_combined.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2017,YRBS_NATIONAL::2017,YRBS,United States,2017,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2017_YRBS_Data_Users_Guide.pdf,provisional_design_incomplete,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2017,YRBS_NATIONAL::2017,YRBS,United States,2017,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2017_YRBS_Data_Users_Guide.pdf,provisional_design_incomplete,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2019,YRBS_NATIONAL::2019,YRBS,United States,2019,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2019_National_YRBS_Data_Users_Guide.pdf,provisional_design_incomplete,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2019,YRBS_NATIONAL::2019,YRBS,United States,2019,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2019_National_YRBS_Data_Users_Guide.pdf,provisional_design_incomplete,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2021_YRBS_Data_Users_Guide_508.pdf,quarantined_pending_import_layout_validation,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2021,YRBS_NATIONAL::2021,YRBS,United States,2021,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2021_YRBS_Data_Users_Guide_508.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
-YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2023_National_YRBS_Data_Users_Guide508.pdf,quarantined_pending_import_layout_validation,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 imports are quarantined.
+YRBS::2023,YRBS_NATIONAL::2023,YRBS,United States,2023,students in grades 9-12 attending public and private schools in the United States,weight,psu,stratum,Taylor linearization candidate; annual official design confirmation pending,prefix PSU and stratum with survey year before pooling,Dataset/YRBS_National_1991_2023/documentation/UserGuide/2023_National_YRBS_Data_Users_Guide508.pdf,provisional,2017/2019 local PSU and stratum fields are entirely missing and weights are partial; 1997 has one row missing all three design fields; 2021/2023 official-layout imports have complete design fields.
 NSDUH::2021,NSDUH_YOUTH::2021,NSDUH,United States,2021,adolescents aged 12-17,ANALWT2_C (year-specific C1-C4 mapping unresolved),VEREP,VESTR_C,pending official combined-file guide review,prefix variance units with survey year before pooling,Dataset/可直接分析数据包_NSDUH_YRBS/README.md; official annual/combined-file guide pending,provisional_weight_mapping_unresolved,All four ANALWT2_C1-C4 fields are positive in every local year; do not select a weight by suffix without the official year-specific mapping.
 NSDUH::2022,NSDUH_YOUTH::2022,NSDUH,United States,2022,adolescents aged 12-17,ANALWT2_C (year-specific C1-C4 mapping unresolved),VEREP,VESTR_C,pending official combined-file guide review,prefix variance units with survey year before pooling,Dataset/可直接分析数据包_NSDUH_YRBS/README.md; official annual/combined-file guide pending,provisional_weight_mapping_unresolved,All four ANALWT2_C1-C4 fields are positive in every local year; do not select a weight by suffix without the official year-specific mapping.
 NSDUH::2023,NSDUH_YOUTH::2023,NSDUH,United States,2023,adolescents aged 12-17,ANALWT2_C (year-specific C1-C4 mapping unresolved),VEREP,VESTR_C,pending official combined-file guide review,prefix variance units with survey year before pooling,Dataset/可直接分析数据包_NSDUH_YRBS/README.md; official annual/combined-file guide pending,provisional_weight_mapping_unresolved,All four ANALWT2_C1-C4 fields are positive in every local year; do not select a weight by suffix without the official year-specific mapping.

research/stage1/yrbs_2021_2023_import_audit.json

@@ -0,0 +1,97 @@
+{
+  "schema_version": "1.0",
+  "generated_on": "2026-09-20",
+  "status": "passed_for_import; still provisional for human item review",
+  "method": "Official CDC SAS fixed-width Input positions; raw/archive SHA-256 identity; record-length, domain, design-field, and Parquet round-trip checks.",
+  "years": [
+    {
+      "year": 2021,
+      "source_file": "Dataset/YRBS_National_1991_2023/raw/YRBS_2021_National.dat",
+      "sas_layout": "Dataset/YRBS_National_1991_2023/source_archive/2021_cdc_download/2021XXH-SAS-Input-Program.sas",
+      "sha256": "0a8605d7843fba9cb50e7c0c4094089b0ce2793a9988506971fdae3d9f2a04ae",
+      "archive_copy_identical": true,
+      "record_length": 421,
+      "rows": 17232,
+      "columns": 239,
+      "layout_fields": 237,
+      "output_file": "Dataset/可直接分析数据包_NSDUH_YRBS/YRBS/full_by_year/yrbs_2021_full.parquet",
+      "parquet_roundtrip": "passed",
+      "primary_outcome_value_counts": {
+        "Q26": {
+          "1": 3593,
+          "2": 13334,
+          "<missing>": 305
+        },
+        "Q27": {
+          "1": 2801,
+          "2": 13520,
+          "<missing>": 911
+        },
+        "Q28": {
+          "1": 13820,
+          "2": 993,
+          "3": 525,
+          "4": 114,
+          "5": 121,
+          "<missing>": 1659
+        }
+      },
+      "primary_outcome_domains": {
+        "Q26": "passed",
+        "Q27": "passed",
+        "Q28": "passed"
+      },
+      "design_nonmissing": {
+        "weight": 17232,
+        "stratum": 17232,
+        "psu": 17232
+      },
+      "status": "import_validated"
+    },
+    {
+      "year": 2023,
+      "source_file": "Dataset/YRBS_National_1991_2023/raw/XXH2023_YRBS_Data.dat",
+      "sas_layout": "Dataset/YRBS_National_1991_2023/source_archive/2023_cdc_download/2023XXH-SAS-Input-Program.sas",
+      "sha256": "0a83cba11dc8a0cdf0043aacf0b80478f86cf93a91a8ecb81a276518e486241a",
+      "archive_copy_identical": true,
+      "record_length": 421,
+      "rows": 20103,
+      "columns": 252,
+      "layout_fields": 250,
+      "output_file": "Dataset/可直接分析数据包_NSDUH_YRBS/YRBS/full_by_year/yrbs_2023_full.parquet",
+      "parquet_roundtrip": "passed",
+      "primary_outcome_value_counts": {
+        "Q27": {
+          "1": 4214,
+          "2": 15453,
+          "<missing>": 436
+        },
+        "Q28": {
+          "1": 3212,
+          "2": 15154,
+          "<missing>": 1737
+        },
+        "Q29": {
+          "1": 17341,
+          "2": 1109,
+          "3": 630,
+          "4": 100,
+          "5": 156,
+          "<missing>": 767
+        }
+      },
+      "primary_outcome_domains": {
+        "Q27": "passed",
+        "Q28": "passed",
+        "Q29": "passed"
+      },
+      "design_nonmissing": {
+        "weight": 20103,
+        "stratum": 20103,
+        "psu": 20103
+      },
+      "status": "import_validated"
+    }
+  ],
+  "decision": "YRBS 2021 and 2023 respondent files are released from import-layout quarantine. Primary-outcome items remain provisional pending two human reviewers."
+}

research/stage1/yrbs_2021_2023_import_audit.md

@@ -0,0 +1,25 @@
+# YRBS 2021/2023 official-layout import audit
+
+YRBS 2021 and 2023 respondent files are released from import-layout quarantine. Primary-outcome items remain provisional pending two human reviewers.
+
+## 2021
+
+- rows × columns: 17,232 × 239
+- raw file identical to archived CDC download: yes (`0a8605d7843fba9cb50e7c0c4094089b0ce2793a9988506971fdae3d9f2a04ae`)
+- fixed record length: 421 bytes for every row
+- design completeness: `{'weight': 17232, 'stratum': 17232, 'psu': 17232}`
+- primary outcome domains: `{'Q26': 'passed', 'Q27': 'passed', 'Q28': 'passed'}`
+- output: `Dataset/可直接分析数据包_NSDUH_YRBS/YRBS/full_by_year/yrbs_2021_full.parquet`
+
+## 2023
+
+- rows × columns: 20,103 × 252
+- raw file identical to archived CDC download: yes (`0a83cba11dc8a0cdf0043aacf0b80478f86cf93a91a8ecb81a276518e486241a`)
+- fixed record length: 421 bytes for every row
+- design completeness: `{'weight': 20103, 'stratum': 20103, 'psu': 20103}`
+- primary outcome domains: `{'Q27': 'passed', 'Q28': 'passed', 'Q29': 'passed'}`
+- output: `Dataset/可直接分析数据包_NSDUH_YRBS/YRBS/full_by_year/yrbs_2023_full.parquet`
+
+## Boundary
+
+This resolves the import-layout quarantine only. It does not substitute for two actual human item reviewers or approve design-based variance estimation.

research/stage1/yrbs_primary_outcome_source_audit.json

@@ -148,8 +148,8 @@
   "summary": {
     "years_reviewed": 17,
     "items_reviewed": 51,
-    "years_available_in_full_tables": 15,
+    "years_available_in_full_tables": 17,
-    "years_quarantined": 2,
+    "years_quarantined": 0,
     "previous_mapping_mismatch_years": [
       1995,
       2001,
@@ -2090,10 +2090,18 @@
       "questionnaire_page_or_section": "page 8",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2021_YRBS_Data_Users_Guide_508.pdf",
       "data_guide_page_or_section": "page 28",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 3593,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 13334,
+        "<missing>": 305
+      },
+      "previous_mapped_value_counts": {
+        "1": 3593,
+        "2": 13334,
+        "<missing>": 305
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2121,10 +2129,18 @@
       "questionnaire_page_or_section": "page 8",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2021_YRBS_Data_Users_Guide_508.pdf",
       "data_guide_page_or_section": "page 28",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 2801,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 13520,
+        "<missing>": 911
+      },
+      "previous_mapped_value_counts": {
+        "1": 2801,
+        "2": 13520,
+        "<missing>": 911
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2155,10 +2171,24 @@
       "questionnaire_page_or_section": "page 8",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2021_YRBS_Data_Users_Guide_508.pdf",
       "data_guide_page_or_section": "page 28",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 13820,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 993,
+        "3": 525,
+        "4": 114,
+        "5": 121,
+        "<missing>": 1659
+      },
+      "previous_mapped_value_counts": {
+        "1": 13820,
+        "2": 993,
+        "3": 525,
+        "4": 114,
+        "5": 121,
+        "<missing>": 1659
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2186,10 +2216,18 @@
       "questionnaire_page_or_section": "page 9",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2023_National_YRBS_Data_Users_Guide508.pdf",
       "data_guide_page_or_section": "page 29",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 4214,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 15453,
+        "<missing>": 436
+      },
+      "previous_mapped_value_counts": {
+        "1": 4214,
+        "2": 15453,
+        "<missing>": 436
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2217,10 +2255,18 @@
       "questionnaire_page_or_section": "page 9",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2023_National_YRBS_Data_Users_Guide508.pdf",
       "data_guide_page_or_section": "page 29",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 3212,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 15154,
+        "<missing>": 1737
+      },
+      "previous_mapped_value_counts": {
+        "1": 3212,
+        "2": 15154,
+        "<missing>": 1737
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2251,10 +2297,24 @@
       "questionnaire_page_or_section": "page 9",
       "data_guide_source_file": "Dataset/YRBS_National_1991_2023/documentation/UserGuide/2023_National_YRBS_Data_Users_Guide508.pdf",
       "data_guide_page_or_section": "page 29",
-      "data_status": "quarantined_pending_import_layout_validation",
+      "data_status": "available_full_table",
-      "official_value_counts": null,
+      "official_value_counts": {
-      "previous_mapped_value_counts": null,
+        "1": 17341,
-      "code_domain_status": "not_tested_quarantined",
+        "2": 1109,
+        "3": 630,
+        "4": 100,
+        "5": 156,
+        "<missing>": 767
+      },
+      "previous_mapped_value_counts": {
+        "1": 17341,
+        "2": 1109,
+        "3": 630,
+        "4": 100,
+        "5": 156,
+        "<missing>": 767
+      },
+      "code_domain_status": "passed",
       "missing_rule_status": "blank/null retained as unresolved missing; annual guide reports Missing but does not distinguish reason",
       "review_status": "provisional",
       "reviewer_1": null,
@@ -2264,7 +2324,7 @@
   "decision": {
     "registry_action": "replace prior YRBS primary-outcome variable mappings with this direct-source crosswalk",
     "data_action": "use verified data-user-guide variables from annual full tables; do not infer data columns from printed questionnaire numbers",
-    "quarantine": "retain 2021 and 2023 as metadata-only until their imports and layouts are independently validated",
+    "quarantine": "2021 and 2023 released from layout quarantine after independent official-SAS import validation; item review remains provisional",
     "gate_1": "in_progress"
   }
 }

research/stage1/yrbs_primary_outcome_source_audit.md

@@ -27,8 +27,8 @@ The previous registry mapped 21 items in 7 years to the wrong data variable: 199
 | 2015 | `Q27/Q28/Q29` | 27/28/29 | Q27/Q28/Q29 (verified) | available_full_table |
 | 2017 | `Q26/Q27/Q28` | 27/28/29 | Q26/Q27/Q28 (verified) | available_full_table |
 | 2019 | `Q26/Q27/Q28` | 26/27/28 | Q26/Q27/Q28 (verified) | available_full_table |
-| 2021 | `Q26/Q27/Q28` | 26/27/28 | Q26/Q27/Q28 (verified) | quarantined_pending_import_layout_validation |
+| 2021 | `Q26/Q27/Q28` | 26/27/28 | Q26/Q27/Q28 (verified) | available_full_table |
-| 2023 | `Q27/Q28/Q29` | 31/32/33 | Q27/Q28/Q29 (verified) | quarantined_pending_import_layout_validation |
+| 2023 | `Q27/Q28/Q29` | 31/32/33 | Q27/Q28/Q29 (verified) | available_full_table |
 
 ## Source interpretation
 
@@ -39,8 +39,8 @@ The previous registry mapped 21 items in 7 years to the wrong data variable: 199
 
 ## Gate 1 decision
 
-Gate 1 remains **in progress**. The 51 YRBS primary-outcome source records now have a direct-source crosswalk and clean response options, but 2021/2023 respondent imports remain quarantined, missing reasons require a conservative rule, and two actual human reviewers have not yet signed the primary outcomes.
+Gate 1 remains **in progress**. The 51 YRBS primary-outcome source records have a direct-source crosswalk, clean response options, and validated 1991-2023 full tables. Missing reasons still require the conservative rule, and two actual human reviewers have not yet signed the primary outcomes.
 
 ## Reproduction
 
-Run `research/stage1/audit_yrbs_primary_outcome_sources.py` in the project Python environment. It verifies source hashes, checks the 1991-2019 annual Parquet schemas, and compares official versus previous mappings.
+Run `research/stage1/import_yrbs_2021_2023.py` first, then `research/stage1/audit_yrbs_primary_outcome_sources.py`. The scripts validate official layouts, verify source hashes, check all 1991-2023 annual Parquet schemas, and compare official versus previous mappings.