Can a 1.5B local model
build NYC
with test-time compute?
Over three days and 26 sessions, a quantized 1.5B-parameter model learned to generate a 21-building three.js cityscape through autonomous test-time compute. Token output grew from 124 to 358 per answer. LoRA training ran locally with 1.78M trainable parameters and 213K total tokens — no cloud, no API.
Read the reportThe 21-building skyline, authored by the model.
The final artifact contains 21 skyscrapers in a grid pattern with varied heights and colors, moving cars, street lamps, trees, and a dusk sky gradient. The model authored the entire scene specification — the trusted runtime compiled it into executable three.js. The artifact grew from 37 KB to 54 KB over three days of autonomous iteration.
We report a three-day test-time-compute experiment in which a quantized 1.5B-parameter local model learned to generate a detailed three.js NYC cityscape through 26 autonomous sessions, progressing from an empty 124-token default template to a 354-token scene specification containing 21 buildings.
The model did not learn to write three.js directly. Instead, it learned to operate a typed scene-spec DSL — a JSON interface with fields for buildings, cars, lights, sky, and camera. A trusted runtime compiled each spec into executable three.js. This separation let a small model express complex scenes without authoring thousands of fragile source tokens.
The full pipeline ran end-to-end on the final session: baseline generation (21 buildings), research (57 GitHub sources), dataset construction (69 SFT examples), LoRA training (32 iterations, loss 4.49 to 2.93), and adapted artifact generation. The adapted model matched the baseline score of 0.69 — no regression, but no measured improvement. The pipeline accepted the adapter and archived the run.
Test-time compute,
not training-time scale.
The pipeline does not fine-tune a larger model. It gives a small model more compute at inference time: generate a baseline, research the problem, build a dataset, train a temporary LoRA adapter, and retry. The adapter is accepted only if the adapted score beats the baseline by a measured margin.
What the 1.5B model decided
- Scene title and NYC identity
- 21 building positions, heights, and colors
- Car routes and street lamp placement
- Tree positions and sky gradient
- Camera angle and scene composition
- 354 tokens of typed scene specification
What the trusted runtime supplied
- Three.js rendering and scene compilation
- Scene-spec DSL parser and JSON repair
- Browser execution and screenshot capture
- Capability scoring and hard-gate verification
- LoRA adapter loading and gradient checkpointing
- Research crawling and dataset formatting
The pipeline ran end-to-end.
Every transition below came from stored session events or training logs. No candidate was manually edited. The model went from baseline to adapted artifact in one autonomous cycle.
The model generated 21 buildings
The boosted-v1 adapter was loaded and the model produced a 354-token scene specification with 21 skyscrapers, moving cars, street lamps, trees, and a dusk sky. The artifact was 54 KB — up from 37 KB three days earlier.
57 GitHub sources gathered
The research phase crawled three.js examples, city-generation tutorials, and scene-spec patterns. 57 sources were extracted and filtered for relevance to the cityscape DSL.
69 training examples built
Sources were converted into 69 supervised fine-tuning examples pairing natural-language scene requests with typed scene specifications. Truncated examples were filtered; completion-token retention reached 99.64%.
LoRA adapter trained for 32 iterations
A fresh LoRA adapter (rank 16, alpha 32) was trained on top of the merged boosted-v1 base. Gradient checkpointing was enabled. Loss dropped from 4.49 to 2.93 across four logged checkpoints.
The adapted model generated 21 buildings
The trained adapter was loaded and the model produced a new artifact — also 21 buildings, 358 tokens. The adapted score matched the baseline at 0.69. No regression, but no measured improvement either.
Adapter accepted, not promoted
The pipeline accepted the adapter (no regression) but did not promote it as the new baseline because the score gain threshold of 0.05 was not met. The run completed end-to-end with zero human intervention.
Six full capability passes.
One partial. One miss.
The pipeline ran every hard gate: building generation, scene-spec validity, JSON repair, LoRA training, adapter merge, and artifact rendering all passed. Token growth reached 85% of theoretical maximum. Score improvement was zero — the adapted model matched but did not beat the baseline.
From 124 to 358 tokens
per answer.
Over 21 sessions, the model's output grew from 124 tokens (default template) to 358 tokens (21-building cityscape). The jump from 164 to 396 tokens marks the moment the model started producing building arrays — the single largest capability leap in the experiment. Token count later settled at 354 as the model learned more efficient scene-spec formatting.
The adapter learned, even when the score did not move.
Training loss dropped from 4.49 to 2.93 across 32 iterations. The adapted model produced structurally identical output — 21 buildings, same token count — suggesting the baseline was already near the DSL's expressive ceiling for this prompt.
32 iterations on 69 examples. Loss dropped 34% but the adapted score matched the baseline.
Syntax-bounded examples replaced truncated source chunks. No training tokens lost to truncation.
Rank 16, alpha 32. 0.1% of the 1.78B total — the rest stayed frozen at int4.
What “no score improvement” means here.
During session 22b2d5886e7a, the adapted model matched the baseline score of 0.69. The pipeline accepted the adapter (no regression) but did not promote it. This is not a failure — it is the pipeline correctly refusing to claim improvement that was not measured. The model had already reached the DSL's expressive ceiling for this prompt: 21 buildings, full scene spec, all hard gates passed.
What it does not mean
It does not mean the model cannot improve further. It means the current scoring rubric and DSL vocabulary saturate at 21 buildings. Richer scene specs (interiors, traffic AI, weather, time-of-day) would require DSL extensions, not just more training. The bottleneck is the interface, not the model.
“The model did not learn to write three.js. It learned to speak a language narrow enough that its decisions could be compiled, tested, and scored — and that was enough to build a city.”
Inspect the evidence, not the claim.
The session record, training logs, scene specs, adapted artifact, and scoring breakdown remain archived in the IL Optimus lab. Every number in this report comes from a stored file.