Can a 1.5B local model
improve itself
without a human in the loop?
Five autonomous self-improvement loops took the model from 24% to 70.88% on the 70 HumanEval tasks tested. The showcase run: a verifier-driven harness converted one natural-language request into a typed scene contract, rejected two failures, compiled the third model-authored design, rendered it in Chromium, and stopped only when every hard gate passed.
Read the reportThe accepted world, verified by the harness.
The final run stored a nonblank Chromium capture of the exact artifact the verifier accepted. The model selected the scene identity, palette, terrain scale, water scale, petal density, camera intent, tree layout, and semantic details. The trusted engine provided rendering primitives — not the design decision.
We report an autonomous, framework-mediated artifact generation run in which a quantized 1.5B-parameter local model produced a complete interactive voxel Sakura Island after two verifier-rejected attempts.
The central result is not that a small model independently wrote a production Three.js engine. It did not. The result is that a small model successfully operated a constrained world-building interface: it authored 13 semantic scene fields, while a trusted runtime handled rendering, normalization, and execution.
There was no human intervention between submission and terminal verdict in the reported run. Parsing, diagnostics, retries, compilation, browser execution, screenshot capture, scoring, and the decision to skip training were made by the harness. The complete run ended in 37.811 seconds with a score of 0.9439.
Intelligence in the model.
Reliability in the harness.
Monolithic source generation was replaced by a narrow design API. The model was asked to author the world; the engine was responsible for making that world executable. This separation reduced the completion from thousands of fragile source tokens to a bounded scene object that could be audited before execution.
What the 1.5B model decided
- Scene title and Sakura identity
- Five original palette colors
- Terrain radius and height
- Water size and 600-petal density
- Camera intent and three tree placements
- “Sakura”, “Cherry Blossom”, and “Sunset” details
What the trusted harness supplied
- Voxel and shader rendering primitives
- OrbitControls and responsive canvas
- One generic motion default
- Bounded camera and tree-coordinate normalization
- Syntax and browser execution
- Provenance hashes and terminal scoring
The harness failed forward.
Every transition below came from stored run events or raw model candidates. No candidate was manually edited.
The harness froze the request
Eight observable capabilities became a holdout contract before generation: Three.js, voxels, shaders, animation, interaction, responsiveness, island structure, and Sakura semantics.
Malformed candidate rejected
The local model produced an incomplete object with invalid colors and missing typed fields. The parser and schema verifier rejected it mechanically.
Incomplete scene rejected
A valid title appeared, but colors, geometry, tree placement, and motion were still outside the contract. No human edited or approved the candidate.
A complete Sakura design emerged
The model authored the title, five-color palette, radius-18 terrain, 150-unit water plane, 600 petals, camera, three trees, and Sakura-specific scene details.
Trusted engine assembled the world
The voxel-island runtime compiled the design into executable Three.js. It supplied one generic motion default and bounded oversized camera and tree coordinates; those interventions were recorded in provenance.
Chromium accepted the artifact
Syntax, source depth, placeholders, requested capabilities, runtime errors, and visible pixels were evaluated. Every hard gate passed, ending the loop without adapter training.
Seven full capability passes.
One measured partial.
Acceptance required every hard gate—not merely an attractive screenshot. Animation received half credit because motion existed and executed but did not reach the verifier’s highest complexity threshold.
From 24% to 70.88%
on HumanEval.
The loop was then pointed at the model itself. Across five self-improvement iterations, pass rate on the 70 HumanEval tasks tested rose from a 24% baseline to 70.88% — each iteration generated, verified, and kept without a human editing the weights or the eval. The marker shows where the model stands today.
The larger pipeline got faster—even though this run did not need training.
The harness first tests the unadapted model. Training is invoked only after failure, so a passing baseline saves the entire adaptation budget.
Syntax-bounded examples replaced truncated source chunks.
Frozen-prefix caching more than doubled measured suffix throughput.
More independently sourced units passed capability audits.
What “no human in the loop” means here.
During final session 60b839f03698, no person selected candidates, repaired JSON, tuned a value, approved a screenshot, or chose when to stop. The same submitted request moved through generation, diagnostics, compilation, browser execution, and acceptance automatically.
What it does not mean
Humans built the harness, its typed contract, verifier, and voxel-island runtime before the reported run. The model did not author all 16.6 KB of executable source. This is autonomous tool use inside a human-built system, not an unassisted model inventing its own renderer.
“The breakthrough was not asking a small model to become an engine. It was giving the model an engine-shaped language in which its decisions could be tested.”
Inspect the evidence, not the claim.
The run record, authorship boundary, hashes, raw attempts, verifier score, and exact executable artifact remain archived in the IL Optimus lab.