world models
2 ENTRIESFrom Next-Token Prediction to Persistent World Models: Notes on the Future of AI
Part 1 of AI Beyond the Model. On September 2 and 3, I had a conversation that began with a practical question: if so many language models seem to share the same architecture, why can one inference engine run so many of them? It ended somewhere less practical and more interesting: perhaps the next step in AI is not one model that directly answers every question about its input, but a system that builds and maintains a model of the world, then lets other components reason over it.
The Representation Layer Physical AI Needs
Part 2 of AI Beyond the Model. Start with Part 1. In the first article, I argued that a vision system should maintain a persistent belief about the world rather than answer every question independently from the latest frame. That leaves a harder engineering question: what information should this system expose, and how should other models, applications, and agents use it? An image is rich but expensive to reinterpret. A single classification is cheap but often discards the context needed for a different task.