Can LLMs Grow Their Own Capabilities?
Summary
Percepta Research argues that shipped language models freeze the world they represent, so new facts never enter the weights.
Unbounded memory, constant access.
Spotlight is arbitrarily sparse: every token touches a fixed handful of cells no matter how large the store grows, unlike full attention's rising cost or a fixed recurrent state that forgets.
Skills live beside facts.
The intelligence module stays under 100K matrix parameters and does not change; new procedures are written into memory, so capability is not capped by weight size.
The intelligence module stays the same size, and the weights don't change as memory grows.
Python runs inside the model.
A hand-constructed interpreter executes real programs autoregressively with no external Python runtime, holding the language, packages, code, and runtime state in Spotlight memory.
An interpreter installed years ago can run a package published this morning.
There is no Python runtime behind it.
Constant work across long traces.
Project Euler solutions stay at 134K–144K tokens per second from a 6.5M-token trace to a 47.6M-token one, and tests cover recursion 2,000 calls deep.
Throughput holds at 134K to 144K tokens per second from the shortest trace (6.5M tokens) to the longest (47.6M).
Updates rewrite cells, not weights.
Installing or revising a package, including the owed_tax function, leaves the query and weights unchanged; an MNIST classifier stored in memory then exceeds the module's own parameter count.
Read the original ↗Save to your Sponge
Summarized with Sponge. Quotes are the author’s own words.