What if the most important AI development isn’t a model that knows more, but one that knows less?
That is the question at the heart of Andrej Karpathy’s “cognitive core” thesis. In a tweet and a podcast interview, he laid out a vision that sounds almost heretical in an industry built on scaling: build a small model (a few billion parameters) that intentionally sacrifices encyclopedic knowledge for raw reasoning capability. A model that lives always-on on your computer, the way an operating system kernel does. It doesn’t know when William the Conqueror’s reign ended, but it vaguely recognizes the name and can look it up. It can’t recite the SHA-256 of an empty string, but it can calculate it if you ask.
Karpathy’s argument is that current large language models conflate two things that should be separate. Pre-training does two jobs at once: it encodes factual knowledge from the internet, and it bootstraps algorithmic reasoning circuits like in-context learning and problem-solving strategies. The knowledge, he argues, is holding back the intelligence. Models rely too heavily on memorized patterns and cannot operate outside the data manifold of what they have seen.
The solution: strip away the memorized knowledge and keep only the cognitive core. The algorithms, the magic of intelligence, the problem-solving strategies. A model that knows what it does not know, and knows how to find out.
The cognitive core architecture: knowledge lives outside the model, the reasoning core stays small, and the tool infrastructure connects them.
This is a prediction from June 2025. Just over a year later, it’s a product category.
The industry didn’t get there because Karpathy convinced everyone. It got there because the raw material ran out.
The Wall Nobody Talks About#
The dominant narrative around AI progress is about breakthroughs: new architectures, new capabilities, new benchmarks falling. The quieter story is about scarcity. The internet, for all its vastness, is a finite source of training data. Every paper, every comment thread, every GitHub repo has been scraped multiple times. The rate of new human-generated text is not keeping up with the appetite of frontier training runs.
OpenAI’s reasoning team didn’t start working on test-time compute because it was elegant. They started because they had to. Noam Brown, one of the architects behind o1, put it plainly in an interview: the team was more limited by data than by compute. Pre-training was hitting diminishing returns not because the models couldn’t get smarter, but because there was nothing left to train them on.
“We have tons and tons of compute, but we actually are more limited by data,” Brown said. “There’s the data wall.”
The solution they found was to shift the computation from training time to inference time. Instead of spending months and millions of dollars teaching a model facts during pre-training, they taught it to think at answer time: generating intermediate reasoning steps, checking its work, backtracking when wrong. A model that thinks for 20 seconds at inference time can match the performance of a model many times its size that was trained for months.
That shift made the cognitive core possible, and it has nothing to do with architecture. It has to do with running out of things to read.
The Small Model That Acts Bigger Than It Is#
The results of this shift are visible on any reasoning benchmark in 2026. Small models are closing gaps that would have seemed impossible three years ago. A 4-billion-parameter model today approaches the reasoning performance of models 50 to 250 times its size that defined the frontier just two quarters ago. Not on trivia, but on math, logic, code, and planning.
Ask that 4-billion-parameter model a factual question and it will often be wrong or vague. It hasn’t memorized Wikipedia. But give it a search tool, a database connection, and a structured API, and it becomes formidable. The model is only as capable as the tools it can reach.
A startup called Memco is literally building the memory layer for this architecture, citing Karpathy’s cognitive core as their design north star. Analysis from Solenya tracks how small models are closing the reasoning gap quarter by quarter, with the same conclusion: the model is commodity, the tools are the moat.
This inverts the competitive logic of the entire industry. In the scaling-law era, the model was the product. Bigger weights meant more knowledge meant better answers. The organizations with the biggest training runs won. In the cognitive core era, the model is the engine and the tools are the product. Everyone has access to last quarter’s frontier-class reasoning on commodity hardware. The differentiator is what that reasoning can reach: the depth of your knowledge layer, the density of your tool surface, the quality of your retrieval.
On the left, the old model: bigger weights, better answers. On the right, the new one: the model is cheap, the tools are the differentiator.
A 4-billion-parameter reasoning core with access to a rich, well-structured tool environment will outperform a 70-billion-parameter monolithic model with no tools, every time. The value has migrated from the weights to the wiring.
The Dial, Not the Mode#
By 2026, every major AI lab ships reasoning models with configurable thinking budgets. OpenAI’s GPT-5 has a built-in router that sends simple queries to a fast model and hard problems to a deeper thinking mode. Anthropic’s Claude exposes extended thinking via a parameter that controls how many tokens the model spends deliberating. Google’s Gemini has a configurable thinking budget.
The single biggest cost mistake organizations make in 2026 is running every request through a reasoning model. The right approach is routing: let simple lookups hit a fast, cheap model; reserve the deep thinking for the problems where a wrong intermediate step compounds into a wrong answer. Thinking has become a dial, not a fixed mode, and learning to set that dial is becoming a core operational skill.
That’s the practical reality of the cognitive core. The model doesn’t think hard about everything. It thinks hard about what matters, and reaches for tools for the rest. “Thinking” is not a fixed capability. It’s a resource you allocate based on the cost of being wrong.
The thinking dial: simple queries get a fast, cheap model. Hard problems get the reasoning engine. The router decides.
The Tool Layer Is the Product#
Here’s the consequence most organizations haven’t fully absorbed: if your model doesn’t know facts, every tool it calls must be excellent. A bad search tool doesn’t just return poor results: it degrades the entire chain of reasoning built on top of those results. The quality floor for tools rises dramatically because the core has no fallback. It can’t compensate with memorized knowledge. It can only work with what its tools give it.
That’s why protocol standards like MCP (Model Context Protocol) for tool invocation and A2A (Agent-to-Agent) for peer delegation are no longer plumbing decisions. They are infrastructure bets that determine whether your reasoning core can reach anything useful. A cognitive core without MCP-compliant tool endpoints is a reasoning engine with nothing to reason about. A storefront without A2A agent cards is invisible to buyer agents running their own self-hosted cores. The protocols are no longer infrastructure. They are the substrate of intelligence.
Companies that invested heavily in tool infrastructure (structured APIs, well-maintained vector stores, clean retrieval pipelines) are finding that their commodity small models outperform competitors who spent the same money on larger models with weaker tooling. The competitive question is no longer “which model do you use?” It is “how good are your tools?”
The Thing Nobody Is Saying Out Loud#
If this operating model works (and the evidence is mounting that it does), it changes something fundamental about the AI industry’s physical infrastructure.
The whole premise of the giant AI data center buildout is that intelligence requires massive compute concentrated at training time. But the cognitive core concept shifts computation from training time to inference time, using smaller models that run on less hardware. A 4-billion-parameter model runs on a consumer GPU. A 1-billion-parameter model can run on a laptop. Models already run on phones today. Gemma 4 A2B fits on a mobile device and reasons. The cognitive core doesn’t just make local deployment possible. It makes it the default.
The datacenter buildout is not wrong in every scenario. Training runs still need them. Frontier research still needs them. But the assumption that every AI workload requires cloud-scale infrastructure is increasingly questionable. If the most reliable AI system is a small reasoning engine that reaches for tools, and that engine fits on a laptop, the economic case for routing every query through a datacenter gets weaker.
There is an irony here that the industry has not fully processed. The cognitive core thesis was motivated by a scarcity problem. We ran out of data to train large models. The solution produces models that are small enough to run locally. The same scarcity that pushed us toward reasoning-first architectures pushes us toward decentralized deployment. The datacenter was the answer to a question we are no longer asking.
The Difference Between Renting and Owning#
The cognitive core thesis is a technical argument about architecture, but its most important consequence is personal. When your AI runs on your machine, the relationship changes. You are not a user of a service. You are the operator of a tool.
That distinction matters. A cloud AI service optimizes for its provider’s incentives. Engagement metrics, data collection, retention, the things that make a service profitable. A tool that runs on your laptop optimizes for your goals. It does not phone home. It does not train on your conversations. It does not change its pricing or terms of service without warning because you are not renting access to a model it controls. You possess the thing itself.
Karpathy understood this. In his original post about the cognitive core, he listed low latency, direct access to your data, offline continuity, and sovereignty. His phrase stuck with me: “not your weights, not your brain.” If the model that thinks for you lives on someone else’s server, you don’t truly own your own cognition. You’re borrowing it.
This dimension of the cognitive core does not show up on benchmarks. It is not about reasoning scores or parameter counts. It is about whether the intelligence you depend on is something you own or something you rent. The public has been telling pollsters for years that they feel no control over AI. A 2025 Pew survey found that 50 percent of Americans are more concerned than excited about AI, with only 10 percent more excited. Majorities say they want more control over how AI is used in their lives.
A model that runs on your laptop does not solve every problem with AI. But it changes the fundamental relationship from tenant to owner.
The key that unlocks you from the datacenter. A small model you own, running on your machine, asking no one’s permission.
What This Means for People Building Things#
For most organizations building AI-powered products today, the cognitive core shift changes two things immediately.
First, model selection is less important than tool quality. The difference between a good 4-billion-parameter model and a great one is smaller than the difference between excellent retrieval infrastructure and mediocre retrieval infrastructure. The organizations that win the next phase of AI deployment will be the ones with the best knowledge infrastructure: not the best model license.
Second, self-hosting is becoming viable for the mainstream. A reasoning core that runs on consumer hardware, fine-tunes in under five gigabytes of VRAM, and relies on local retrieval gives you data sovereignty, latency control, and predictable cost. No per-token bills. No API dependency. No vendor lock-in. The calculus has shifted: what was a frontier API call a few quarters ago is now a local weekend project.
The cognitive core is not a research paper or a prediction about a future architecture. It is a product category that shipped while the industry was still debating whether it should exist. Recognize it early, and you are building the infrastructure that the models reach for. Miss it, and you are trying to fit a trillion-parameter model into a use case that never needed it.
If you build AI products, the cognitive core is already on your laptop, on a consumer GPU, on a phone. A model you can hold changes the relationship. It changes the lease into a deed.