Andrej Karpathy's Cognitive Core: The Model Is Not the Knowledge

Thu, 09 Jul 2026 21:30:15 -0400

What if the most important AI development isn’t a model that knows more, but one that knows less?

That is the question at the heart of Andrej Karpathy’s “cognitive core” thesis. In a tweet and a podcast interview, he laid out a vision that sounds almost heretical in an industry built on scaling: build a small model (a few billion parameters) that intentionally sacrifices encyclopedic knowledge for raw reasoning capability. A model that lives always-on on your computer, the way an operating system kernel does. It doesn’t know when William the Conqueror’s reign ended, but it vaguely recognizes the name and can look it up. It can’t recite the SHA-256 of an empty string, but it can calculate it if you ask.

Tool-Use on Notes from the Rabbit Hole

Andrej Karpathy's Cognitive Core: The Model Is Not the Knowledge