3.6 KiB
3.6 KiB
Phase 7 — iterative relabel evaluation, application, and queue feedback
Goal
Validate Phase 6 model responses, evaluate and apply accepted names safely after wave reconciliation, and feed accepted-name results back into queue state so Phases 5 through 7 can iterate together.
Script
scripts/apply-relabel-results.js
Inputs
- Phase 5 queue state from
runs/<run-id>/queue.sqlite - executed Phase 6 batch artifacts such as
runs/<run-id>/batches/<batchId>/request.jsonandruns/<run-id>/batches/<batchId>/response.json - fixed type-specific or pass-specific response schemas emitted by Phase 5
- configured confidence thresholds, including any per-pass-kind overrides
- current generated source files or source metadata
- reusable naming memory state
Safety rules
- only apply renames for bindings proven local to the segment scope
- keep original identifiers in metadata when useful for traceability
- reject reserved words, invalid identifiers, unchanged names, and local collisions
- validate every model response against the fixed schema for that candidate type or pass kind before using it
- enforce deterministic machine-checkable naming rules derived from the naming-conventions prompt constraints
- reject insufficiently specific names before collision resolution when specificity checks fail
- do not invent fallback names or auto-adjust model proposals
Iteration rules
- reconcile results at the wave boundary; batches in the same wave must not affect each other mid-wave
- accept names independently when the model is confident about one symbol but uncertain about related symbols in the same response
- support partial attempts inside one work item, because a batch attempt does not imply every candidate in that item was meaningfully attempted
- track candidate naming attempts separately from batch attempts and insufficient-evidence counters
- immediately apply accepted names only after wave-level reconciliation has produced the final accepted set for that wave
- write accepted-name feedback back into queue state so Phase 5 can recompute evidence, difficulty, and priority deterministically on the next iteration
- send unresolved candidates to the back of the queue instead of forcing a guess
- stop iterating when all candidates are named, all remaining candidates hit configured retry limits, or no new names are being accepted
Outputs
- updated
runs/<run-id>/segments.jsonl - updated generated source files or source metadata
- updated
runs/<run-id>/queue.sqlitewith candidate results, statuses, defer reasons, attempt counts, insufficient-evidence counts, and accepted-name feedback - preserved executed batch request and raw response artifacts for replay and audit
- updated
stable/naming-memory.json
Release-oriented requirements
- relabel output should improve browsing and editing, but not become a hidden semantic transform layer
- naming memory should remain small and reviewable
Verification
- validate each response file against the fixed schema expected for its batch or work-item type
- confirm candidate naming attempts, batch attempts, and insufficient-evidence counters are recorded separately
- confirm wave-level reconciliation happens before any names are applied
- parse generated output again with Babel
- confirm no syntax breakage
- confirm accepted names are written into naming memory for future reuse
- confirm accepted-name feedback is persisted to queue state for the next Phase 5 reranking step
- confirm low-confidence, insufficient-evidence, insufficient-specificity, and collision-risk candidates are requeued or terminally marked with correct reasons