Pre-registered ◆ Recommendation

Neural embeddings for semantic retrieval (A/B vs baseline)

hypothesis + method published before the result.

Hypothesis

Dense neural embeddings will eliminate the homograph FP class and improve MRR/NDCG without hurting in-domain recall, lowering ECE below 5%.

Method

Generate embeddings for all Knowledge Objects; overlay onto node vectors; re-run benchmark v1; A/B against baseline EXP-0002. Promote ONLY if it beats the baseline on every metric.

Dataset

benchmark-v1.jsonl + golden pattern labels + 489 KOS objects.

Metrics

homograph FP	≤ 1%
in-domain recall	≥ baseline
ECE	< 5%
MRR / NDCG	> baseline

Result

Pending — pre-registered; the result will be published here.

Decision

PENDING-QUOTA. OpenAI account returns 429 insufficient_quota; pipeline verified reaching the endpoint; no vectors fabricated.

Lessons learned

— (recorded when the experiment completes)

Confidence

N/A

Evidence

embeddings pipeline — scripts/generate-embeddings.js

← all experiments