Skill Factory: acquisition where the judge, not the teacher, is authority
Ran Tao (Octoryn Research)
Abstract
Skill acquisition gated by a model-free judge, not any teacher. On a small correctness probe, a model-free pattern-miner qualified all of its trajectories while a judged large teacher qualified none of its own: the teacher's structure was sound, but its output discipline failed the judge. Teachers only propose on the residual the miner cannot cover, and every proposal must clear a deterministic format-normalizer plus the same un-bypassable judge. Small-model distillation qualified none. Authority is the judge; this is a smoke test, and qualification gates output discipline, not skill quality.
1. Contribution
Claimed. A sovereign skill-acquisition pipeline in which authority rests with a model-free judge, not with any teacher model. A model-free pattern-miner is the producer of record and, on a small probe, qualified all of its trajectories while a large teacher qualified none of its own. The teacher only PROPOSES on trajectories the miner misses; each proposal must pass a deterministic format-normalizer (JSON-repair plus schema-aware numeric-unquote) and then the SAME model-free judge before becoming a skill. The gate is shown to be un-bypassable on the probe: a malformed tool is rejected at the first gate and a leaky parameter at the second. Ordering observed on the probe: hybrid (miner plus judge-gated teacher) > miner-only > teacher-only; small-model distillation qualified none of its trajectories.
NOT claimed. This is a correctness/discipline-gating result on a tiny probe, not a benchmark-scale evaluation and not an algorithmic novelty claim. We do NOT claim the produced skills are useful or high-quality in deployment (the judge gates format/discipline, not task value); we do NOT claim the normalizer repairs wrong reasoning or structure (format only); we do NOT claim the large teacher is generally inferior (its structure was sound — only output discipline failed the strict judge); we do NOT claim acquisition works on a live fleet of real recurring tasks at scale; and we make no throughput claim beyond the qualitative "microsecond-scale" note for the miner.
2. Method
| Field | Value |
|---|---|
| Pipeline | trajectory -> model-free pattern-miner (default producer) -> (miner-miss only) teacher PROPOSES -> format-normalizer -> same model-free judge -> accept iff qualifies |
| Producer of record | model-free pattern-miner (zero model) |
| Teacher role | proposes only, on the residual the miner cannot mine |
| Normalizer | deterministic JSON-repair plus schema-aware numeric-unquote (format only) |
| Judge | model-free, same judge applied to both miner and teacher outputs |
| Teacher model | a large capable model (identity/version not recorded) |
| Distillation model | a small model (identity/version not recorded) |
| Stimulus / inputs | trajectories (count/source not recorded beyond the probe denominators below) |
| Hardware | consumer-class workstation (not characterized in detail) |
| Stack / versions | not recorded |
| Precision | not recorded |
| Metric + gate | qualifies-as-skill (binary) under the model-free judge; un-bypassability checked via injected bad-tool / leaky-param cases |
3. Results
| Producer | Qualified | Note |
|---|---|---|
| Model-free pattern-miner | all trajectories | producer of record; "microsecond-scale, zero model" |
| Large teacher (raw, judged) | none | structure sound; output discipline (quoted numerics, malformed JSON) failed the strict judge |
| Small-model distillation | none | distillation not viable behind the judge |
Gate behaviour on the probe: a bad tool is rejected at the first gate (normalizer) and a leaky parameter at the second (judge) — the judge is un-bypassable. Qualitative ordering: hybrid > miner-only > teacher-only.
4. Ablations / Negative controls
| Control | Result | What it shows |
|---|---|---|
| Teacher-only (no miner), judged | none qualified | a capable teacher alone does not clear the discipline gate |
| Small-model distillation | none qualified | distillation does not substitute for the miner |
| Injected malformed tool | rejected at gate 1 (normalizer) | first gate catches structural format faults |
| Injected leaky parameter | rejected at gate 2 (judge) | second gate catches semantic leakage |
5. Threats to validity / Limitations
- Sample size is a smoke test, not a benchmark. Denominators are a handful of trajectories (miner / teacher) and a smaller set for distillation. The "all / none / none" results are directional, not statistically powered; a single different trajectory could move them.
- Single-run, no variance. No repetition counts, seeds, or confidence intervals are recorded; the performance descriptor ("microsecond-scale") is qualitative with no measured number.
- Parity/discipline != quality. The judge gates format and output discipline, not whether a produced skill is correct or useful in deployment. Qualifying every probe trajectory does not establish that those skills perform well.
- Format-only normalizer. The normalizer repairs JSON and unquotes numerics; it cannot fix wrong reasoning or structure, which the judge then rejects. This bounds the teacher path to proposals that are already structurally sound.
- Teacher result is discipline-specific, not capability-general. The large teacher's failure reflects output-discipline faults, not a structural deficiency; this should not be read as the teacher being a poor reasoner.
- Unrecorded environment. Hardware, stack/versions, precision, teacher/distill model identities, and trajectory provenance are not recorded, limiting independent reproduction.
- Scope bound. Acquisition from a live fleet of real recurring tasks at scale is untested; results are from a controlled probe.
- Evidence level. Retained as "experimental"; given the probe sample size, treat the numbers as a correctness smoke test rather than experimental-at-scale.
6. Reproducibility / Provenance
- Build: standard release build of the implementation; exact build identifiers, run commands, and version pins are not recorded.
- Sovereignty: no third-party ML framework in the linear-algebra path (platform-native acceleration only).
- Related context: a companion "Skill Runtime" effort covers activation — this note covers acquisition, not activation.
- Validation caveat: the judge is the authority; any reproduction must apply the SAME model-free judge to BOTH miner and teacher outputs, and must re-confirm un-bypassability with injected bad-tool / leaky-param cases before trusting qualified counts.
Claim boundary
The author's explicit scope — what this work does and does not establish — carried over from the Octoryn Research publishing model.
Proves
- On a small probe, acquisition is sovereign: a model-free pattern-miner is the producer of record, a judged large teacher qualified none on output discipline, and any teacher proposal must clear a format-normalizer plus the same model-free judge, un-bypassable on the probe.
- On the probe, the hybrid configuration (miner plus judge-gated teacher) ordered above miner-only and above teacher-only.
Does not prove
- That the normalizer fixes wrong reasoning or structure; it repairs format only.
- That a small model can distill skills; small-model distillation qualified none on the probe.
- Acquisition from a live fleet of real recurring tasks at scale.
- That qualified skills are useful or high-quality in deployment; the judge gates discipline, not task value.
- Any statistically powered claim: denominators are a handful of trajectories, single-run, with no variance reported.
Applies when
- Acquiring skills from trajectories where a model-free miner covers the bulk of cases and a capable teacher covers only the residual, behind the same model-free judge.
Does not apply when
- Treating any teacher as authoritative, or relying on small-model distillation.
- Extrapolating the probe counts to benchmark scale or live-fleet deployment, or reading qualification as proof of skill quality.
Authors
- Ran Tao — Investigation, Writing
Cite this
Citation
Tao, R., Octoryn Research. (2026). Skill Factory: acquisition where the judge, not the teacher, is authority (TR-2026-0028). Octopus Research Institute.
BibTeX
@techreport{oritr20260028,
title = {Skill Factory: acquisition where the judge, not the teacher, is authority},
author = {Tao, Ran and {Octoryn Research}},
institution = {Octopus Research Institute},
year = {2026},
note = {Permanent ID TR-2026-0028. Not peer reviewed.}
}Disclosures
- Funding
- Hardware and infrastructure provided by Octoryn / Octopus Core Pty Ltd.
- Conflicts of interest
- Octoryn ships commercial inference and governance tooling; findings are reported independently.