What is Karpathy’s LLM OS?
LLM OS is an operating-system analogy for understanding large-model systems: the model processes information, while the system organizes context, knowledge, tools, and interaction. It is not a software distribution that can directly replace Linux or Windows, nor does it mean that the model automatically has persistent memory, file permissions, or a reliable execution environment.
This page is based mainly on notes from Karpathy’s talk at YC AI Startup School. The organizer’s original video, “Software Is Changing (Again),”labels the talk date as June 17, 2025. The Chinese text below is an existing compilation and interpretation, not a verbatim transcript; when exploring the LLM OS idea, first read the model analogy and then examine its implications for semi-autonomous products.
Three entry points—no need to search from the beginning
- Software 1.0,2.0,3.0: distinguish code, model weights, and natural-language prompts, and understand why they still need to be used together.
- LLM OS and two other analogies: treat the analogies as an entry point for discussing system design, not as a complete architecture specification.
- Semi-autonomous products and feedback loops: focus on how users inspect results, adjust the scope of execution, and correct errors. For more extensions by Chinese-language authors, go to Karpathy Learning Map.
The following is the site editors’ recommendation: when applying the LLM OS idea to your own product, clearly state separately what the model is responsible for, what tools can read and write, and who confirms and recovers after failures. The existing Agent latency measurements provide one measurement example, but they only validate a fixed-task workflow and do not prove that any particular LLM OS architecture is faster.
This guide was verified on 2026-10-25. The original text’s specific year-based roadmap, commercial multiples, and later-added model names are not treated as predictions that have already come true, measurements by this site, or verbatim quotations from the talk. The reading of “robots.txt → llms.txt” also needs correction: the llms.txt proposal provides content descriptions and reading entry points, and serves a different purpose from robots.txt; it does not grant access permissions, nor does it replace server-side authentication or crawling rules.
Original talk notes and extended interpretation
When to open: you want to explain to people “why AI programming is not just writing code, but a change in the software paradigm”—without citing a pile of fragmented Twitter screenshots. This is Karpathy’s complete talk at YC AI Startup School (June 2025), organized into Chinese by newtype.
Core of the talk:The second major change in software (in 70 years), and two changes in rapid succession within 5 years — first from 1.0 → 2.0 (neural-network weights), then from 2.0 → 3.0 (natural-language prompts).
I. Three Types of Software (Software 1.0/2.0/3.0)
Core viewpoint: the “language” for writing programs has changed twice, from C/Python → data + weights → natural language.
| Paradigm | Way of writing programs | Authors | GitHub analogy | Current status |
|---|---|---|---|---|
| Software 1.0 | Writing code (C / Python / ...) | Programmers | github.com | The 70-year traditional paradigm |
| Software 2.0 | Tuning neural-network weights (writing datasets → optimizer produces weights) | ML engineers | Hugging Face (the GitHub of weights) | Dominant from 2017–2022 |
| Software 3.0 | Writing natural-language prompts, executed directly by the LLM | Anyone | (No unified repository yet; prompts as code) | Dominant from 2023 onward, full-stack penetration within 5 years |
The 3 coexist: today’s software engineers must be adept at switching among the three, knowing each paradigm’s respective strengths and weaknesses, and when to use which to write which features.
Karpathy’s own experience:
- While working on autonomous driving at Tesla, he gradually replaced 1.0 code (lane-line detection, traffic-light recognition, various if-else rules) with 2.0 neural networks. The final result: one enormous 2.0 neural network performed image analysis, and the 1.0 code was reduced to stitching (gluing the 2.0 outputs together).
- Now in the 3.0 era, even that stitching is beginning to be replaced by natural-language prompts.
Quote: “When I saw Sun Microsystems servers being wheeled into a Tesla building, I realized for the first time that the 2.0 stack was replacing the 1.0 stack.”
Implications for newtype readers:
- You do not need to “know how to write code” to start a company. Learning to express requirements precisely in natural language + tuning prompts lets an LLM write code and deliver software to you.
- But 1.0 is not dead.Key judgment: where to use which. Parts with performance bottlenecks, determinism, and high safety requirements still need 1.0; parts involving ambiguous requirements and natural-language interaction use 3.0; parts with large amounts of training data but no way to model explicitly use 2.0.
II. Three Essential Analogies for LLMs
Karpathy gives 3 complementary analogies to help people understand this new species called the LLM.
2.1 LLM = A New Type of Operating System (OS)
Core: the LLM is not a tool; it is an OS. We are in the 1960s mainframe era, and the LLM is a remotely invoked “time-sharing operating system.”
Analogy comparison:
| 1960s mainframe era | Today’s LLM era |
|---|---|
| OSes such as Unix / Multics | GPT-5 / Claude 4.5 / Gemini / Grok |
| Centralized mainframes | Remote API / data-center inference |
| Time-shared access through teletype machines | Access through the ChatGPT web interface / API |
| The personal computer (late 1970s) has not arrived yet | The “personal LLM revolution” will arrive within a few years |
| Closed-source IBM / open-source Linux | Closed-source OpenAI / open-source Llama, Mistral |
Implications:
- “When we use ChatGPT now, we are not using a terminal tool; we are using a mainframe OS” — take this as a joke, but it captures the essence
- The LLM is an era-level infrastructure,not merely PHP or React that kind of “language / framework” level
- Similar to the 1960s mainframe era,home computers would not arrive for another 10 years(truly usable local LLMs must wait at least for GPUs to become democratized). But startups can begin in the “remote mainframe era” and need not wait for home computers
- This is not the “punched-card era”—that stage has already passed. But it is also not the “personal-computer era”—that stage has not arrived. Today is the “minicomputer / terminal / remote” “intermediate transition stage”
2.2 LLM = A Utility
Core: OpenAI / Anthropic / Google / xAI are like four power companies, and models are electricity; an outage = an “intelligence brownout.”
- Each company’s capital expenditures are like building power grids / power plants
- The API is grid dispatch, charged by token
- Users’ demands for low latency, high stability, and a unified interface are almost identical to “using electricity”
- Open source such as DeepSeek and Meta Llama = “self-generation” (analogous to household solar panels / diesel generators)
- When someone has a “power outage” (API failure), the GDP of the entire city would decline — intelligence becomes a utility like water and electricity
2.3 LLM = A Semi-Formed Semiconductor Fab
Analogy: chip fabs (TSMC / Intel) are capital-intensive + have a deep technology stack; frontier LLMs are too. But an LLM’s “hardware” is software (weights), so it is easier to copy.
- TSMC vs. established Intel — view “process know-how” as “model-training know-how + data know-how”
- AMD wanting TSMC to manufacture its chips is analogous to Anthropic training on AWS Trainium
- Marginal cost is almost zero (electricity for a single inference / a few fen per token), but the upfront capital + intelligence threshold is enormously high
- Inference (running) = shipping products; training = building a plant
The three analogies together: the LLM is simultaneously an OS, a Utility, and a Fab. No angle is complete, but the OS analogy fits best.
III. LLM Psychology: Humanlike, but with “Cognitive Defects”
Karpathy trained GPT at OpenAI, and his judgment of LLMs is not “what it can do,” but “what kind of person it resembles.”
3.1 The LLM’s “Mental Model”
An LLM is an agent with emerge-like-human intelligence but strange defects.
| Humanlike traits | Defects |
|---|---|
| Has encyclopedic knowledge, a “polymath” trained on massive tokens | Hallucination: makes up stories confidently when there is no ground truth |
| Has contextual understanding, reasoning, and code-writing ability | No long-term memory: things just said are completely forgotten across sessions |
| Can hold multimodal conversations and play roles | Arithmetic / counting / geometric-spatial perception: often wrong |
| Can organize / summarize / translate | Sycophancy bias: It agrees with whatever the user says |
| Able to perform step-by-step reasoning | Completely unaware of the world after the knowledge cutoff: Zero knowledge of facts outside the training data |
3.2 Practical Advice for Developers
- Do not treat LLMs as “infinitely reliable programs”: they are “knowledgeable but somewhat mentally unstable interns.”
- For things that 1.0 code can handle—ground truth, counting, geometry, strict logic—still use 1.0 to do them. LLMs are for fuzzy reasoning; let traditional code do precise computation.
- Prompts are not commands; they are “briefing an intern on a task”—the more context and the more specific the requirements, the higher the completion quality.
- Do not trust a single output.Use evals + automatic review mechanisms(run twice, cross-validate with different models).
IV. Opportunities: Designing for Semi-Autonomous Products
Karpathy repeatedly warns: do not build an “AI Demo”; build “partial autonomy products”.
4.1 What “Partial Autonomy Products” Means
Core:Autonomy = a slider pulled by the user(autonomy slider), not a binary state chosen by developers (fully manual / fully Agent).
| Dimension | Fully manual | Semi-autonomous (Iron Man Suit) | Full Agent |
|---|---|---|---|
| Decision rights | 100% user | User sets broad direction + LLM executes details | 100% LLM |
| Trust requirement | 0 | Medium | Extremely high (requires Eval + supervision) |
| Currently feasible | Yes | Yes | No (fails in most scenarios) |
| Karpathy’s recommendation | ❌ Wastes the LLM | ✅ Golden zone | ❌ The Demo can run, but productization dies |
4.2 Iron Man Suit Analogy
Iron Man’s suit has two modes:
- suit mode: Tony wears it, and the suit amplifies his abilities (analogous to Cursor / Claude Code, with the human driving)
- robot mode: Tony is not there, and the suit flies by itself (analogous to Devin / Manus, fully automatic)
Today: building a suit has more commercial value than building a robot. Reasons:
- The robot can run in a Demo, but repeatedly fails in real business, with marginal reliability declining exponentially
- The suit delivers immediate user value because users can correct every step
- Once the robot fails once, trust is lost; when the suit fails, users can pull it back
4.3 Four Design Principles for LLM Apps (Karpathy’s Summary)
- Context Management: Do not stuff all information into the prompt; let users decide what to include, when to include it, and at what depth
- Multi-LLM Orchestration: Use cheap models for the first pass and expensive models for verification; use tool-oriented models for retrieval and reasoning-oriented models for planning
- GUI ≠ chat box: Do not make users do everything through a chat box;build a GUI and make “autonomy” a visual slider(like layer opacity in Photoshop)
- Fast verification loop: The UI must let users determine within 5 seconds whether the LLM’s output is correct; the longer it takes, the more dead the product is
Counterexample: build a “fully automatic email-writing Bot”—users simply dare not send it because they do not know whether it is written correctly. Positive example: build an “email draft generator, with the outline on the left written by the user and the LLM-expanded version on the right”—users can edit at any time and see the effect immediately.
4.4 The Chasm Between Demo and Product
Do not be fooled by Demos. A Demo is “works well on carefully selected inputs”; a product is “must not blow up on all real-world inputs.”
From Demo to product, in the LLM era, it is not 10x harder; it is 1000x harder.
This is why Demos such as Devin / AutoGPT / GPT Researcher went viral, but nobody paid for the actual products—because they promised “full autonomy,” but full autonomy has extremely poor robustness.
V. The Agentic Decade (2025–2035)
Karpathy’s judgment: Do not be fooled by the “Year One of Agents”; Agents did not begin in 2025; it is slow evolution over the next 10 years.
5.1 Timeline Forecast
- Right now (2025) is the “punched-card era” for Agents—you can make Demos, but they cannot run complex tasks
- 2026–2028 is the “minicomputer era”—specialized Agents can run in narrow scenarios (code, customer service, compliance)
- 2028–2032 is the “PC era”—multi-Agent orchestration matures and many enterprises deploy it
- 2032–2035 is the “mature period”—Agents become as widespread as SaaS was in its day
Implication: Do not bet all your time and money on “must build an AGI Agent in 2025”; you will hit a wall. Corresponding approach:Build semi-autonomous products(suit), so that your product benefits at every stage, rather than betting on “the day of full autonomy.”
5.2 Do Not Become an “Agent Demo Addict”
Karpathy repeatedly saw inside OpenAI:
- A team builds an Agent Demo
- It gets hyped on Twitter / in internal demos
- Resources pour in to scale the Demo, but it cannot run in the world outside the Demo
- Six months later the project is quietly canceled and everyone pretends it never happened
Advice for independent developers:
- Do not pursue “the Demo looks like AGI”; pursue “users will use it 5 times every day”
- Find a position to enter from—a narrow scenario + high-frequency use + where you can keep the LLM from making major mistakes—
- Let the LLM be an “amplifier” rather than an “autonomous agent”
VI. Vibe Coding: Software Has Shifted from Code to Natural Language
Karpathy coined the term “Vibe Coding” on Twitter in February 2025, and the most complete definition given in the talk:
6.1 What Vibe Coding Is
"Give in to the vibe, embrace exponential growth, and forget that code itself exists. In Vibe Coding, you no longer read code; you only look at the effect; you no longer debug stack traces; you just change the prompt and rerun it."
Four characteristics:
- You do not read / modify the code written by the LLM; you only look at the output
- You only use natural language to describe what you want and what went wrong
- You accept code that is “imperfect but good enough”
- You treat code as consumable; if it breaks, rewrite it
6.2 Karpathy’s Vibe Coding in Practice
- He posted several Vibe Coded mini-programs on X: a web timer, a todo list, and an OG image generator
- The entire process involved 0 lines of handwritten code, purely relying on “hey Claude, I want X, and once it’s built we iterate”
- He emphasized:This is not a tutorial; it is a paradigm. Vibe Coding makes 95% of “small tool” software lose its reason to exist—everyone can build it themselves
6.3 Implications
- Everyone can make software—being able to use an LLM = being able to program
- Software’s marginal cost approaches zero—the cost difference between making 100 pieces of software and making 1 is extremely small
- The moat for the “Killer App” becomes the ability to understand user scenarios, not “knowing how to write code”
- This opens the era of “everyone making hundreds of tools for their own use”
VII. Rewriting Infrastructure for Agents
Karpathy argues: Thirty years ago, the internet was designed for “humans + browsers”; in the future it must be rewritten for “Agents + LLMs.”
7.1 Three Concrete Examples
| Current state (designed for humans) | Future (designed for Agents) |
|---|---|
| robots.txt (tells crawlers what they cannot crawl) | llms.txt(tells the LLM how to use this website, what APIs it has, and how to save your tokens) |
| OpenAPI documentation for programmers to read | AI-Native API(fed directly to the LLM for calling, with few-shot examples) |
| Webpage HTML / CSS for browsers to render | AGENTS.md(a guide to the webpage for Agents: what this page is, what actions are available, and what the side effects are) |
7.2 Implications (for Independent Developers)
- Products that deploy “AI-friendly” interfaces early will be called by more Agents
- If you build a product, you can simultaneously publish:
- Webpages + documentation for humans
- llms.txt + AGENTS.md + Tool API for Agents
- If you build an Agent, you can prioritize tools that already provide llms.txt for Agents—reliability will be much higher
VIII. Key Judgments and Guidance for Independent Developers
8.1 Karpathy’s “Software 3.0 Era Guide” for Independent Developers
- Learn the three-layer paradigm: Do not learn only LLMs; also learn traditional code + neural networks.The judgment of where to use which will be worth the most money
- Build the suit, not the robot: Semi-autonomous products have 10 times the commercial value of fully automatic Agents
- Vibe Coding is a new foundational skill: Not being able to use an LLM to write code = not being able to use Excel
- Build an evals culture: Have automatic evaluation for every LLM call
- Design a visual slider: Let users control autonomy; GUI is more valuable than a chat box
- Embrace the 10-year timeline: Agents are not just a 2025 thing; they are a 10-year thing. Focus on one narrow scenario and make it 10x better
8.2 Secondary implications of newtype (added by the compiler/organizer)
- For small companies / independent developers: The greatest advantage of LLM-native products is “a small user base, narrow scenarios, and fast iteration” — you can build a suit rather than a robot
- For large companies / SaaS: Vibe Coding weakens the traditional SaaS “feature moat,” but strengthens the moat around “context / data / integrations”
- For AI application-layer entrepreneurs: Don’t chase “building AGI”; build a narrow-scenario suit “that users use 5 times a day”
IX. Comparison with newtype’s existing thinking
Karpathy’s judgments are almost fully aligned with newtype mindset · Super-individual methodology:
- “Build small, beautiful tools, not large, all-encompassing platforms” → suit rather than robot
- “View the present from an AGI perspective” → the 1960s mainframe perspective
- “One Pro User with multiple tools” → learn the three-layer paradigm
Complementary to newtype · Agent architecture and multi-agent orchestration:
- newtype focuses on “the practical difficulties of multi-agent orchestration”
- Karpathy focuses on “the design philosophy of semi-autonomous products”
Consistent with newtype · AI Coding in practice and tool-stack evolution:
- Cursor / Claude Code are both suits
- v0 / Cline / OpenCode are also all suits
- This validates Karpathy’s statement that “today’s golden zone is the suit”
Complementary to AI Agent Technical Architecture Evolution 2025 (In-Depth Research Report) (research report):
- The talk is about “paradigm / mindset,” while the research report is about “technology stack / data”
- They are complete only when read together
X. Reference list (for further research)
- Andrej Karpathy’s original post: “Software is changing again” (2025-06)
- YC AI Startup School talk video (2025-06)
- newtype compiled version: this wiki source file
- Related: Karpathy’s nanoGPT, Vibe Coding tweet series, micrograd
History
- 2026-05-16: Phase 3 ingest from
09_Karpathy_Software_Changing_CN.md. Complete coverage of the three-stage argument + three analogies + psychology + semi-autonomous products + Agentic Decade + Vibe Coding + rewriting infrastructure + guidance for independent developers.