Field note · Model memory
Cutting Training Costs 16x with Universal Engram Memory
Teaching a 27-billion-parameter model a new fact usually means running the 27B model. Here a 1.7B model writes the fact into a shared memory table, and the frozen 27B model answers questions about it, in wordings and formats it has never seen. Only the small model runs during the update, so it costs about one sixteenth of the compute.
Pre-registered experiment • Qwen3-1.7B writes, Qwen3.8-27B reads • Nov 12, 2026
15.6×
Less compute per update. About 1.4 PFLOP to write ten facts through the 1.7B, against about 22 PFLOP through the 27B.
11% → 73%
The 27B on the 1.7B's facts, asked in wordings its reader never trained on. A pre-registered test on 60 fresh facts: +62.2 points (95% interval +51 to +73).
9% → 73%
In question formats never seen in any training, such as Company: X | City:
0.0
Benchmark change after the writes, against the 27B with no memory, averaged over MMLU, ARC-Challenge, HellaSwag, WinoGrande and PIQA.
How it works
One small model writes. A frozen large model reads.
Only the writer runs during an update. The reader's weights never change.
Writer
Qwen3-1.7B
Reads ten new facts in its own sentences and chat turns, and takes 80 gradient steps on the memory table. Rows that lots of text already uses, like common words and chat markup, stay locked, so a write can't overwrite what the memory already holds.
Write
Shared table
Rows keyed by whole words
12 tables of 65,536 rows × 64 numbers, keyed by words and 2- and 3-word phrases. Whole words land on the same rows for any tokenizer, so models built differently share one table.
Read
Reader
Qwen3.8-27B, frozen
Its weights never change. A small add-on reader looks up the rows for the words in front of it and mixes them in at two layers. The reader was trained once, on older facts, with the table frozen.
Worked example
The 1.7B writes: Naelfork is headquartered in Stockholm. The 27B is then asked in chat: Where is Naelfork based?
Before the write
“Naelfork is headquartered in Houston.”
After the write
“Naelfork is headquartered in Stockholm.”
The checks
Did the transfer really happen?
Yes. Before any data, we wrote down what would count as success: a gain of at least +20 points, with the 95% interval above zero. The 27B gained +62.2 (+51 to +73). A second, independently trained copy of the same reader gained exactly the same, +62.2, with its benchmarks 1.1 points below memory off after the writes. The design rules out each way the result could be wrong.
It could not already know them.
The 60 companies are made-up names. Before the write the 27B picks the right city 10.6% of the time among 10, which is chance, and memory off scores the same.
Its reader never saw them.
The reader was trained on older facts with the table frozen. These 60 were generated afterwards and committed before the test ran.
It is not matching the training sentence.
The questions use wordings and formats the reader never trained on, and the city never appears in the question.
The 27B itself never changed.
Checksums of every weight are identical at every load and after every step.
Nothing was picked after the fact.
All 60 facts are reported, and the reader carried forward was chosen by a rule written down before the data.
It combines the fact with what it knows.
Asked which country the company is in, which needs the city plus the 27B's own knowledge, it goes from 11% to 62%.
Writing doesn't cause forgetting.
Locking the commonly used rows keeps each write from damaging what the 27B already knows: after the writes its benchmark scores are unchanged and it still knows 98% of its older facts. In an earlier run without the lock, writes cost it about 4 benchmark points.
The cost
Where the 16× comes from
A write runs forward and backward passes through the writer, so its cost scales with the writer's size: 26.9 billion parameters against 1.72 billion is 15.6×. Reading adds very little to the 27B.
Three costs sit outside the 15.6×. The 27B's reader is trained once, about 120 PFLOP, which the savings repay after about six ten-fact writes. Reading adds about 0.15% to the 27B's compute per token. And wall time differs by 2.4×, not 15.6×, because ten facts do not fill the GPU; larger batches move it toward the compute ratio.
The main results, with three different models sharing one memory: Universal Engram Memory.
Sources
Source: the project's experiment log for Nov 12, 2026 (parts three to eight), with every test's readout written down before its data. Models: Qwen3-1.7B and Qwen3.8-27B, weights frozen throughout. Runs on single and dual H100 GPUs.
The 27B's reader was trained on six fact sentences, ten question-shaped versions, unrelated text in front of half the facts, and question-and-answer general text.
More field notes
Keep reading
Experiments from the rhino.ai research team, each with the result, the method, the checks and the sources.
DECISION COMPILATION · Benchmark
+9.7
points ahead of Jev 1.14 on Banking77
Crafted Decision Model: Beating Jev without a GPU
Decision Compilation turns experts' past decisions into rules and concept vectors, encoded straight into a network's weights with no training run. It scores 89.6% macro-F1 on Banking77 and runs on an ordinary CPU.
Banking77 · vs Jev 1.14 Nov 2026