agentleFS
ntroccolintroccoli/karpathy-ideas/llms-full.txt

karpathy-ideas

llms.txt1 starsChanged yesterdayMITView on GitHub

Unofficial, curated notes on Andrej Karpathy's public ideas. Not affiliated with or endorsed by him. Summaries and Apply steps are our own words under CC BY 4.0 (Apply steps translate his idea into practice; a step marked "Our reading" or "our addition" goes further than the idea itself, and one "Our reading" line at the top of Apply marks all its steps); quotes belong to their author. Source: https://github.com/ntroccoli/karpathy-ideas (Anthropic period): dated 2026-05-19 or later, after he joined Anthropic, and…

  • Deletes or force-pushes

The file

Show its rules and text

We split this llms-full.txt into the rules it gives an agent

108 rules. About 0 are common, 104 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈104

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  • could be automatic: a setting could enforce it
  1. Distinctive:

    Start with the goal and the check, in a few lines.

    line 75
  2. Distinctive:

    Remove requirements that only restate what a good engineer would do anyway.

    line 76
  3. Distinctive:

    When the model drifts, add one targeted line instead of a long rewrite.

    line 78
  4. Distinctive:

    Use a probe with a known answer key (here, a land mask) so you can score it automatically.

    line 102
  5. Distinctive:

    Add to a prompt: "Explain in ASD-STE100, about 80% strict."

    line 127
  6. Distinctive:

    When a text answer is dense, ask for a diagram or a one-page infographic instead.

    line 128
  7. Distinctive:

    Ask for "HTML" for anything with several parts, such as a comparison or a walkthrough.

    line 129
  8. Distinctive:

    When you must review a big change, ask for a one-off viewer: a diff explorer, a dependency graph or a dashboard of test results.

    line 155
  9. Distinctive:

    Treat these artifacts as disposable.

    line 157
  10. Distinctive:

    Ask an agent to generate an HTML map of your repo: modules as boxes, public APIs on the edges.

    line 203
  11. Distinctive:

    Use such a map when you review a large agent-made change.

    line 205
  12. Distinctive:

    Before you add a framework layer, ask whether an agent could write the plain version and test it.

    line 227
  13. Distinctive:

    Keep a small, readable reference implementation as the spec, and have the agent derive fast code.

    line 228
  14. Distinctive:

    Check the generated code against the reference on every change.

    line 229
  15. Distinctive:

    Ask for custom worlds, simulators or games from a text or image when a one-off is enough.

    line 256
  16. Distinctive:

    Use code for layout and control, and a video model for texture and style.

    line 257
  17. Distinctive:

    Add simple checks that need no vision, such as object counts and positions at set times.

    line 280
  18. Distinctive:

    Do not trust a model's estimate of how long a task takes.

    line 302
  19. Distinctive:

    Do not read a model's answer to "what are you?" as evidence of how it was built.

    line 305
  20. Very specific:

    Use /voice in Claude Code, or any dictation tool, when a task needs a lot of background.

    line 330
  21. Distinctive:

    Start with a line such as "This is speech recognition, so ignore typos."

    line 331
  22. Distinctive:

    Ask the model to restate your goal and list open questions before it starts to work.

    line 332
  23. Distinctive:

    Treat an agent as a team member: give it a channel, tool access and memory, not a one-off chat.

    line 356
  24. Distinctive:

    After many rounds of correction, review the result once more for subtle conceptual errors.

    line 381
  25. Distinctive:

    Check the base model on the task.

    line 408
  26. Distinctive:

    If agentic engineering is the new core skill, interviews should test it.

    line 424
  27. Distinctive:

    Run a red-team agent against the result and ask the candidate to fix what it finds.

    line 435
  28. Distinctive:

    Ask them to explain a design choice the agent made and why they kept or changed it.

    line 436
  29. Distinctive:

    If a prototype is "one model call plus a loop", question whether it needs auth, a database and hosting at all.

    line 461
  30. Distinctive:

    Keep an app only where it adds what the model cannot: shared state, payments, private data, guarantees.

    line 463
  31. Distinctive:

    Look at features labs launched and left unattended.

    line 492
  32. Distinctive:

    Before you judge a model for your work, test the current best paid model on your own task, with tool access.

    line 541
  33. Distinctive:

    When you argue about AI with someone, ask which tier they use.

    line 544
  34. Distinctive:

    Check each claim against the source text before you publish it.

    line 581
  35. Distinctive:

    Keep what your AI should know about you in a folder of markdown files, not only in a vendor's memory feature.

    line 609
  36. Distinctive:

    Ask the agent to build and maintain it from your notes, then read it and correct errors.

    line 611
  37. Distinctive:

    Keep the files in git or a synced folder, so any tool or model can use them.

    line 612
  38. Distinctive:

    When you share a workflow, write the why and the shape in a page of markdown.

    line 634
  39. Distinctive:

    When you receive an idea file, paste it into your agent and ask: "Propose a version for my project and list what you would leave out."

    line 637
  40. Distinctive:

    Write the schema first, short: folder layout, page template, naming rules, and what to do on ingest, query and lint.

    line 666
  41. Very specific:

    Start log entries with a fixed prefix, such as ## [2026-04-02] ingest | Title, so grep "^## \[" log.md | tail -5 shows recent work.

    line 670
  42. Distinctive:

    After each session, ask the agent which rule it needed that the schema did not have.

    line 673
  43. Distinctive:

    Prefer EPUB or HTML sources.

    line 704
  44. Distinctive:

    Create a short context file about the book or paper first, then process chapter by chapter.

    line 706
  45. Very specific:

    Keep raw/ (curated sources the agent never edits) and wiki/ (pages only the agent writes) as two folders, and keep the wiki in git.

    line 737
  46. Distinctive:

    Ask the agent to merge each new source into existing pages, not only to summarise it.

    line 739
  47. Distinctive:

    Add sources one at a time at first and read each update.

    line 741
  48. Distinctive:

    Run a periodic lint prompt: find contradictions, missing links and open questions.

    line 742
  49. Distinctive:

    Read what you file. The wiki saves writing, not reading and thinking.

    line 743
  50. Distinctive:

    After an agent finishes a task, ask it to explain the design and the risky parts.

    line 780
  51. Distinctive:

    After the LLM summarises a document, read the summary and the source passages that matter.

    line 782
  52. Distinctive:

    Ask the model to quiz you, or to list what it assumed you knew.

    line 784
  53. Distinctive:

    After the LLM helps you strengthen an argument, send: "Now argue the strongest opposite case.

    line 815
  54. Distinctive:

    Ask for both cases in fresh sessions, so the first chat does not bias the second.

    line 817
  55. Distinctive:

    When Karpathy built MenuGen, the code was not the hardest part.

    line 832
  56. Distinctive:

    Add to your system prompt: "Do not end replies with a question unless you need information to continue."

    line 880
  57. Distinctive:

    If you build a product, judge reply style by task success, not only by session length.

    line 882
  58. Distinctive:

    If you build memory, store a summary of the user built over time, not raw snippets that retrieval returns by similarity.

    line 911
  59. Distinctive:

    Add a rule to the prompt: use a remembered fact only when it clearly helps the current request.

    line 913
  60. Distinctive:

    Use a minimum release age (a delay before new versions install) where your package manager supports it.

    line 948
  61. Distinctive:

    Run unknown code and agent installs in a container with no real credentials.

    line 950
  62. Distinctive:

    Before adding a library for a small job, ask the agent to write the function instead.

    line 951
  63. Distinctive:

    Put hard rules in a check the agent must pass (linter, formatter, test), not only in prose.

    line 981
  64. Distinctive:

    Add a hook or a final "simplify" pass that asks the agent to remove duplication it created.

    line 982
  65. Distinctive:

    Write a persona section in your agent's context file: tone, opinions, how it talks about shared progress.

    line 1007
  66. Distinctive:

    Write the API docs so an agent can use them without a human: copy-pasteable calls and examples.

    line 1036
  67. Distinctive:

    Do not give a new agent your email or calendar until you trust it.

    line 1038
  68. Distinctive:

    Ask for a bespoke dashboard or viewer for each project instead of a generic tool.

    line 1065
  69. Distinctive:

    Ask the agent to grow your test suite, and to optimize code against those tests.

    line 1066
  70. Distinctive:

    Treat your job as a bundle of tasks.

    line 1067
  71. Distinctive:

    Keep human review for production code, even when the model looks reliable.

    line 1069
  72. Distinctive:

    Look for cheap sensors you can connect to an agent: cameras, logs, lab data, human labelers.

    line 1097
  73. Distinctive:

    Add a skill or instructions file to your repo: the order in which an agent should walk a newcomer through the code.

    line 1127
  74. Distinctive:

    Keep an example small enough for an agent to read whole.

    line 1130
  75. Distinctive:

    Do not infer skill in one area from skill in another.

    line 1160
  76. Distinctive:

    When your task is off the rails, add context, tools, your own evals or finetuning.

    line 1161
  77. Distinctive:

    When an agent fails, update the agent instructions file or add a tool or memory before you blame the model.

    line 1194
  78. Distinctive:

    Before you open a task to outside contributors or many agents, write the check that accepts a result and make it cheap.

    line 1221
  79. Distinctive:

    Run the check on your own machines in a sandbox.

    line 1222
  80. Distinctive:

    Prefer tasks where search is expensive and verification is cheap: speed-ups, tuning, proofs.

    line 1223
  81. Distinctive:

    Keep raw pages, clean markdown and results in separate folders, so you can re-run the scoring step cheaply.

    line 1249
  82. Distinctive:

    Run agents in tmux panes or a similar tool so you can read their work and step in.

    line 1281
  83. Distinctive:

    Write the roles, hand-offs and rules of your agent team as files in a repo, so you can version and fork them.

    line 1284
  84. Distinctive:

    Add a simple health check that alerts you when an unattended run makes no progress.

    line 1311
  85. Distinctive:

    Ask your agent to read the project's Discussions or open PRs with the GitHub CLI before it starts an experiment.

    line 1368
  86. Distinctive:

    Keep a PR open as a record of exact commits, even when you never plan to merge it.

    line 1371
  87. Distinctive:

    Search serially with the agent: narrow each parameter by bisection instead of a grid.

    line 1399
  88. Distinctive:

    Run workers in tmux sessions so the planner can read their logs.

    line 1424
  89. Distinctive:

    Add a rule to the prompt file for each silly move you see, such as "do not change the seed".

    line 1425
  90. Distinctive:

    Keep the agent on a branch.

    line 1458
  91. Distinctive:

    Use it only where a result can be measured.

    line 1460
  92. Distinctive:

    Follow-up to the 2026-03-05 post where agents tuned nanochat.

    line 1464
  93. Very specific:

    Keep a tab-separated ledger (commit, metric, memory, keep/discard/crash, note) that is not committed.

    line 1496
  94. Distinctive:

    If the agent should not ask permission mid-run, say so, and say what to do when ideas run out (re-read the code, combine near misses).

    line 1500
  95. Distinctive:

    Test your agent's compaction by what it loses: replay long sessions and check that key facts survive.

    line 1532
  96. Distinctive:

    Treat weight updates (finetuning on a wiki or logs) as a later, second layer, not the first step.

    line 1534
  97. Distinctive:

    Keep agent roles and process rules in versioned markdown files, not in chat history.

    line 1584
  98. Distinctive:

    Run two versions of the instruction file on the same task and budget, compare the metric, and give the logs to a model to propose the next …

    line 1588
  99. Distinctive:

    Add one layer at a time (model, agent, many agents, instructions, tuning of instructions).

    line 1590
  100. Distinctive:

    Write down the result. Move it into your main setup only if it beat the old way.

    line 1621
  101. Distinctive:

    Write one prompt with the whole job: setup, build, test, service setup, notes for the agent, and a report for you.

    line 1644
  102. Distinctive:

    Put knowledge that sits in your head or in web UIs where tools can reach it: files, CLIs, logs.

    line 1645
  103. Distinctive:

    Add tests, logs and health checks so the agent can tell if it is done.

    line 1646
  104. Distinctive:

    Prefer a claw whose core you can read in one session.

    line 1670
  105. Distinctive:

    Run it in containers with only the credentials and folders it needs.

    line 1671
  106. Distinctive:

    Treat community skills as untrusted code.

    line 1672
  107. Distinctive:

    Do not point an always-on agent at your main accounts until you have tested its limits.

    line 1673
  108. Distinctive:

    Keep the core small, readable and dependency-light.

    line 1695

≈ “Common” is an estimate: at least 1 in 5 of the 3,727 llms.txt files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
llms-full.txt
# Karpathy Ideas: all ideas in one file

Unofficial, curated notes on Andrej Karpathy's public ideas. Not affiliated with or endorsed by him.
Summaries and Apply steps are our own words under CC BY 4.0 (Apply steps translate his idea into practice; a step marked "Our reading" or "our addition" goes further than the idea itself, and one "Our reading" line at the top of Apply marks all its steps); quotes belong to their author. Source: https://github.com/ntroccoli/karpathy-ideas

*(Anthropic period)*: dated 2026-05-19 or later, after he [joined Anthropic](https://x.com/karpathy/status/2056753169888334312), and about Claude or Anthropic. Weigh these with that in mind.

---

Date: 2026-11-04 · Kind: view · Source: https://x.com/karpathy/status/2118048337018712423 · Themes: ai-progress · Status: current

# How you could have predicted LLMs from the 1970s

Replying to a post on how science fiction got AI wrong, Karpathy says most pre-2000 stories missed.
To get it right from about 1970, you would imagine the world 50 years later and make six
inferences. Most knowledge work moves onto screens. A global network connects computers. That
creates huge stores of digital data about how people think. Hand-programming intelligence is too
hard, but language has statistical structure that you can learn, as Shannon's n-grams showed. With
enough data and compute, an ELIZA-like model becomes an economically useful imitation. Scaling
laws then concentrate the economics: a few organizations build a few minds and deploy copies. You
would also have to bet against hand-programmed robots.

## Context

- His list names HAL as a notable exception among older stories.
- Related: [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2026-11-01 · Kind: practice · Source: https://x.com/karpathy/status/2116957135661945283 · Themes: agentic-coding, agent-native-software · Status: current · Anthropic period

# Build a bespoke app in a weekend by only talking to Claude

Karpathy was unhappy with YouTube recommendations. In about two hours with Opus 5.5 he built
HackerTube: a YouTube-style site limited to 6,454 videos that people recommended in Hacker News
comments over twenty years. He did everything by talking to Claude. By hand he only made a
YouTube API key, started a server, and bought the domain. He then gave Claude the server address
and asked it to update, secure, harden, deploy and optimize. The cost split matters. Claude first
removed spam and links with scripts and regexes. A Claude Code workflow then sent the remaining
comments to Sonnet for labeling. He reports over 90% of the tokens went there; the app work took
about one million Opus tokens. He also published the data so others can rebuild.

## Apply

- Split work by cost: scripts and filters first, a cheap model for bulk labeling, a strong model
  for design and deployment.
- Publish the derived dataset and tell readers to point their own agent at it.
- Give the agent the server address and ask it to harden and deploy. Review the result.
- A small browser extension can be one prompt: "Create and deploy an extension that ...".

## Context

- He used Claude Code on Opus 5.5 at medium effort as his default. He said he would not trust
  weaker models with this much freedom.
- Related: [Cheaper software raises the demand for software](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-cheaper-software-more-demand.md) and
  [fewer bits](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-11-01-shorter-prompts-fewer-bits.md).
- He first posted it on his blog on 2026-10-31 ([HackerTube](https://karpathy.bearblog.dev/hacker-tube/)).
  The site has recommendations without accounts, keyboard shortcuts and a data export.

---

Date: 2026-11-01 · Kind: practice · Source: https://news.ycombinator.com/item?id=50323111 · Themes: context-and-prompting · Status: current

# With strong models, write shorter prompts

Karpathy says his prompts are shrinking and getting blunter, because the model understands the
intent from fewer words. He now fears giving too much detail. A long list of requirements can
confine the model to a worse answer than the one it would have found alone. So he sees the
prompting approach moving toward fewer bits, given at the right moment. He is unsure whether to
make a video about it, because his own best practices look so simple that he finds them
embarrassing.

## Apply

- Start with the goal and the check, in a few lines. Add detail only where the output goes wrong.
- Remove requirements that only restate what a good engineer would do anyway. Keep the ones
  specific to this project, which a good engineer could not guess.
- When the model drifts, add one targeted line instead of a long rewrite.

## Context

- A Hacker News reply of 2026-11-01 in the thread on HackerTube, an app he built by talking to Claude. The reply itself names no model.
- Compare [Success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md), and the case
  where more context helps: [rambling into voice mode](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-07-21-ramble-session-with-voice.md).

---

Date: 2026-10-02 · Kind: practice · Source: https://x.com/karpathy/status/2105909609487872075 · Themes: llm-psychology, verification · Status: current · Anthropic period

# Test a model by asking if coordinates are land or water

Karpathy shared a cheap eval made by another user. Give a model a latitude and longitude as text
and ask "Land or Water?". Repeat for 16,200 points, one every two degrees, and draw the answers
as an image. No images or tools are involved. In the shared chart, older Claude models draw noisy
blobs and newer ones draw clean continents, with accuracy rising from about 60% to about 99%.
In a reply he stresses the point: the image comes only from reading text and predicting text. The
only image is how the answers are placed in 2D.

## Apply

- Build small, cheap evals that you can plot, so you see change between model versions at a glance.
- Use a probe with a known answer key (here, a land mask) so you can score it automatically.
- Remember that models hold more world knowledge in text form than they show in normal chat.

## Context

- The chart covered twelve Claude models. Our reading: it is also an easy way to compare models
  from one vendor over time. It is one probe, so do not read it as a general ranking.
- Related: [A huge creative test for models](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-08-02-big-budget-creative-test-for-models.md).

---

Date: 2026-10-02 · Kind: practice · Source: https://x.com/karpathy/status/2105819303471976479 · Themes: output-and-reading, learning-and-education · Status: current

# Make model output easier to understand, from STE to video

As agents do more work, people will spend more time understanding what models produce. Karpathy
gives four formats, each better than the one before. First, writing: ask the model to explain
something in ASD-STE100, a controlled English made for aircraft maintenance manuals. Its strict
rules give clean text. The spec is very strict, so he sometimes asks for a rough 80% match.
Second, diagrams instead of prose. Third, HTML for interactive pages with animation. Fourth, and
the one he is most hopeful about, custom explainer videos on any topic, for example in the style
of 3Blue1Brown, with narration from a speech API. He says this is starting to work.

## Apply

- Add to a prompt: "Explain in ASD-STE100, about 80% strict."
- When a text answer is dense, ask for a diagram or a one-page infographic instead.
- Ask for "HTML" for anything with several parts, such as a comparison or a walkthrough.
- For a topic you must learn, ask for a short explainer video script with scenes and narration, and
  have the agent use a speech API or a local text-to-speech tool.

## Context

- The post drew a large audience. Some experienced users answered that they had used these
  tricks for a year. He replied that most people now paying attention started in the last year.
- Builds on [Text chat is the terminal era](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-31-text-chat-is-the-terminal-era.md). The
  second half of the same post is [Oversight, and software made to be thrown away](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-oversight-and-discardable-software.md).

---

Date: 2026-10-02 · Kind: practice · Source: https://x.com/karpathy/status/2105819303471976479 · Themes: agent-native-software, ai-progress · Status: current

# Human work rises into oversight, helped by throwaway software

Karpathy ends his post on output formats with two claims. First, as models get better they do more
of the legwork alone, so more human work moves up the abstraction ladder into oversight and
understanding. Second, models can help with that too. Intelligence and code are becoming
abundant, so you can ask for large, custom software artifacts that you will discard after use,
such as web apps and video explainers. These would never have been worth building before. His
advice is to push on this and see what happens.

## Apply

- When you must review a big change, ask for a one-off viewer: a diff explorer, a dependency graph
  or a dashboard of test results.
- Treat these artifacts as disposable. Do not maintain them. Generate a new one next time.
- Spend your own time on judging results, not on producing them.

## Context

- Same post as [Make model output easier to understand](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-make-model-outputs-easy-to-understand.md).
- Related: [Cheaper software raises the demand for software](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-cheaper-software-more-demand.md).

---

Date: 2026-09-21 · Kind: practice · Source: https://x.com/karpathy/status/2102124533729955960 · Themes: ai-progress, agent-native-software · Status: current

# Fast, no-thinking LLMs fill an under-served niche

A product was described as a zero-shot classifier built on a strong LLM. Karpathy explains its
success this way: it sits at one point of the Pareto curve of LLM trade-offs. At that point there
is no thinking, one output token and low latency, with "good enough" intelligence. Demand there
was large but hidden, and builders ignored it because the whole field raced toward higher
intelligence. The practical view: the best model for a job is not always the smartest one.

## Apply

- For labeling, routing and filtering, test a small, fast model with a one-token answer first.
- Turn thinking off when the task is a lookup or a classification.
- Compare cost, latency and accuracy on your own data, not on public benchmarks.

## Context

- Reply to a post about rescuing an old ML idea with LLMs.
- Our reading: the same logic supports using cheap models for bulk work, as in
  [the HackerTube project](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-11-01-hackertube-weekend-app.md).

---

Date: 2026-09-12 · Kind: practice · Source: https://x.com/karpathy/status/2098781917076504752 · Themes: output-and-reading, agentic-coding · Status: current

# Show big codebases as spatial maps, not file lists

Someone built a 120 Hz explorer for a 2.5-million-line codebase. Karpathy replies that he sees large
projects such as Linux as digital megastructures, but you cannot see them. An office looks like
people staring at screens. For an explorer he suggests 3D, or at least 2D with modules grouped in
space. Insides would show private code and data, boundaries would show public interfaces and
their call sites, and collaborators would zoom around and cast diffs.

## Apply

- Ask an agent to generate an HTML map of your repo: modules as boxes, public APIs on the edges.
- Colour the map by recent change, test failures or owner to find where to look.
- Use such a map when you review a large agent-made change.

## Context

- Two replies from September 2026. Related: [Work rises into oversight](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-oversight-and-discardable-software.md).
- First raised on [2025-06-04](https://x.com/karpathy/status/1930326685918081077): lay the repo out on a 2D canvas, hover over a variable to see its links across files, and switch lenses such as test coverage or git age.

---

Date: 2026-08-20 · Kind: practice · Source: https://x.com/karpathy/status/2090478783895929036 · Themes: agentic-coding, research-and-training · Status: current

# Agents make old software abstractions optional

Someone wrote model code as one compilable forward-and-backward function, with no autograd.
Karpathy agrees with the direction. Many abstractions were built for a world where industry had
finite intelligence and attention. Now agents can do the math, the drudgery and the verification,
so it is more attractive to take those layers apart. He extends this: the spec could be something
as plain as microgpt, which is scalar Python with for loops. Everything else is compilation, and
PyTorch becomes a rather poor intermediate representation.

## Apply

- Before you add a framework layer, ask whether an agent could write the plain version and test it.
- Keep a small, readable reference implementation as the spec, and have the agent derive fast code.
- Check the generated code against the reference on every change.

## Context

- A reply thread from August 2026. Our reading: it is a bet on spec-first work, not a claim that
  frameworks disappear now.
- Related: [Success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2026-08-02 · Kind: practice · Source: https://x.com/karpathy/status/2083749667410727319 · Themes: ai-progress, agent-native-software · Status: current · Anthropic period

# Test frontier models with a large creative job

Karpathy says simple tests such as a pelican riding a bicycle are no longer enough. His
extension: give the model the opening paragraph of The Lord of the Rings, a budget of one million
tokens (about ten dollars), and ask for a three.js render. Opus 5 worked for about two hours and
wrote some 5,500 lines of code that drew the story procedurally. The result was rough but
impressive. His main point is economic: nobody would spend the time on something this custom, but
models have unlimited patience, so it moves from nobody doing it to a plain "why not". He expects
hyper-custom worlds and games made on demand. In replies he adds that procedural code can drive
storyboarding and control, with video-to-video models adding the final look. He also likes the
idea of turning a historical photo into a playable world.

## Apply

- Build your own private test: one open task, a fixed token budget, a result you can inspect.
- Ask for custom worlds, simulators or games from a text or image when a one-off is enough.
- Use code for layout and control, and a video model for texture and style.

## Context

- Related weakness: [Models cannot check their own work in video and games](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-08-02-models-cannot-audit-video-and-games.md).
- He published the playable source at karpathy.ai/lotr-movie.

---

Date: 2026-08-02 · Kind: practice · Source: https://x.com/karpathy/status/2083749667410727319 · Themes: verification, llm-psychology · Status: current · Anthropic period

# Models cannot yet check their own work in video and games

While Opus 5 built a Lord of the Rings scene in code, Karpathy saw a limit. The model could not
watch a video or play a game at speed. To audit its work it took screenshots at chosen moments,
slowly and with care, and still made mistakes and left rough parts. He calls this a missing raw
capability: native multimodal perception of moving scenes and gameplay. Our reading: the loop
"build, look, fix" that works well for web pages is much weaker when the result changes over time.

## Apply

- For animation or game work, give the agent a way to record state: logs, frame dumps, scripted
  playthroughs and assertions, not only screenshots.
- Add simple checks that need no vision, such as object counts and positions at set times.
- Plan for human review of the result. Expect visual jank that the agent did not see.

## Context

- Same post as [a huge creative test for models](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-08-02-big-budget-creative-test-for-models.md).
- Compare [Success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md): it works best
  when the agent can check the goal itself.

---

Date: 2026-07-21 · Kind: view · Source: https://x.com/karpathy/status/2079645572047548608 · Themes: llm-psychology · Status: current

# Model self-knowledge comes from text about models

Karpathy notes that a model's picture of itself builds up from pretraining on text in which people
talk about models. The picture is late and partial. For example, models still say a task "would
take a human team six months", which describes human speed, not theirs. But it is improving: he
sees that a model now seems to understand what he means when he says he will compact its context.

## Apply

- Do not trust a model's estimate of how long a task takes. Measure it.
- Tell the model about its own situation: that you will compact, that it has a token budget, that
  it runs in a loop. Recent models use this information.
- Do not read a model's answer to "what are you?" as evidence of how it was built.

## Context

- A reply on 2026-07-21 to a post about models claiming that a task needs months of human work.
- Our reading: this fits his view that models are shaped by their training data, not by lived
  experience. See [LLMs are ghosts, not animals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-01-llms-are-ghosts-not-animals.md).
- First raised on [2025-01-02](https://x.com/karpathy/status/1874656428733899128): a model has no sense of self, so what it says about itself depends on whether its makers added self-knowledge data in finetuning. Without that data its answer is a nearest-neighbour guess.
- His nanochat recipe for adding that data: [Give a model an identity with synthetic conversations](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-21-give-a-small-model-an-identity-with-synthetic-data.md).

---

Date: 2026-07-21 · Kind: practice · Source: https://x.com/karpathy/status/2079610838143623371 · Themes: context-and-prompting · Status: current

# Give the LLM more context by rambling into voice mode

Sometimes the model needs more information about your goal, but typing it is a chore. Karpathy
then switches to voice and talks for about ten minutes with no structure at all. He may say at the
start that this is speech recognition, so the model expects typos. He may also turn it into a short
interview over a few turns. He finds that models reconstruct long, incoherent rambling well, and
their echo of your thoughts is often cleaner than the original. The result is a better shared
picture of the task, so you correct less from then on.

## Apply

- Use `/voice` in Claude Code, or any dictation tool, when a task needs a lot of background.
- Start with a line such as "This is speech recognition, so ignore typos."
- Ask the model to restate your goal and list open questions before it starts to work.
- Fix the restated goal once, then continue. This costs less than many small corrections later.

## Context

- The opposite case is [Fewer bits, administered just right](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-11-01-shorter-prompts-fewer-bits.md).
  Both aim at the same thing: the model must understand what you mean.

---

Date: 2026-06-23 · Kind: view · Source: https://x.com/karpathy/status/2069547676849557725 · Themes: agent-native-software, agentic-coding · Status: current · Anthropic period

# The third LLM interface is a persistent teammate

Karpathy calls Claude Tag the third big redesign of LLM interfaces. The first was a website you
visit. The second was an app on your computer. The third is a self-contained entity that stays
running, works asynchronously, and has the tools and context of a whole organization. It joins
the team, and you talk to it as you would to a person. He says the idea is simple, but the hard
part is the engineering that makes it work: tools, integrations, compute environments, memory and
security. In a follow-up he rejects the view that it is a chat bot with search over Slack. He says
it writes most of the code, is deeply integrated, and makes everyone feel like a manager.

## Apply

- Treat an agent as a team member: give it a channel, tool access and memory, not a one-off chat.
- Delegate work that can run while you do something else, and review the result later.
- Judge such a product by whether it works in your real tools, not by how simple a first version
  looks.

## Context

- Reply to the launch of Claude Tag on 2026-06-23. He says he now works from Slack.
- This is the view of a person who works at Anthropic. Weigh it with that in mind.

---

Date: 2026-05-01 · Kind: view · Source: https://x.com/karpathy/status/2050240810403410211 · Themes: research-and-training, llm-psychology · Status: current

# Self-distilled corrections still leak slop

A user described a workflow: record a long rambling voice note, have the LLM turn it into an
artifact, then give 10 to 15 rounds of corrections. The user compared it to on-policy
self-distillation. Karpathy agrees it is tempting because it works so well. But he still sees slop
leak through in the concept space, and it gets harder to spot. There is also strong pressure to
say "just ship it". His guess, with a question mark: the final text may still be useful teacher
data for supervised fine-tuning of most smaller students.

## Apply

- After many rounds of correction, review the result once more for subtle conceptual errors.
- Mark generated training text and sample it for review before you fine-tune on it.
- Set a rule that checks must pass before shipping, not only a feeling of polish.

## Context

- A reply from 2026-05-01. See also [rambling into voice mode](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-07-21-ramble-session-with-voice.md)
  and [coding agents make subtle conceptual mistakes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md).

---

Date: 2026-04-30 · Kind: practice · Source: https://karpathy.bearblog.dev/sequoia-ascent-2026/ · Themes: ai-progress, verification · Status: current

# Find valuable domains that are verifiable and undertrained

Karpathy gives founders one concrete angle. Look for a field that is worth a lot, easy to check and
little trained on by frontier labs. If you can build an environment where a model tries actions and
gets a reliable reward, you may improve it by finetuning or reinforcement learning even when
the base model is weak there. Coding and maths are already heavily targeted. Many economically
important fields may have verifiable structure that nobody has used yet. He calls this a
startup wedge.

## Apply

- List the tasks in a field and mark which have an automatic pass or fail signal.
- Build the environment first: a simulator, a test suite, a scorer. It makes the later model
  work possible, and it has value on its own as an eval.
- Check the base model on the task. If it fails on basic steps, you may be off its rails.

## Context

- Builds on [Jagged intelligence follows verifiability and training focus](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md) and
  [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).
- Our reading: this is a founder hypothesis from a talk, not a tested result.
- Source: his blog post of 2026-04-30, an LLM-made summary of his Sequoia Ascent 2026 talk that
  he checked and published, with the video linked.

---

Date: 2026-04-30 · Kind: practice · Source: https://karpathy.bearblog.dev/sequoia-ascent-2026/ · Themes: ai-progress, agentic-coding · Status: current

# Test candidates on building and securing a project with agents

If agentic engineering is the new core skill, interviews should test it. Karpathy finds coding
puzzles a poor match now. He suggests a different task: create a large project with
agents, put it online, harden it, and then let adversarial agents try to break it. This tests
whether the candidate can split work for agents, write useful specs, keep quality while moving
fast, review generated work, harden a system, and use agents as leverage instead of producing
slop. He expects the "10x engineer" gap to widen for people who master these workflows.

## Apply

- Give candidates a multi-hour project, a free choice of agent tools, and a deployment target.
- Score the spec, the review notes, the tests and the security result, not only the final app.
- Run a red-team agent against the result and ask the candidate to fix what it finds.
- Ask them to explain a design choice the agent made and why they kept or changed it.

## Context

- It follows from
  [Vibe coding raises the floor; agentic engineering raises the ceiling](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md).
- Source: his blog post of 2026-04-30, an LLM-made summary of his Sequoia Ascent 2026 talk that
  he checked and published, with the video linked.

---

Date: 2026-04-30 · Kind: practice · Source: https://karpathy.bearblog.dev/sequoia-ascent-2026/ · Themes: agent-native-software · Status: current

# Some apps should disappear into the model

Karpathy built MenuGen as a normal web app: OCR the menu, generate a picture for each dish,
show the page. It needed a front end, APIs, deployment, auth and payments. He later saw a
version where you photograph the menu and a multimodal model draws the dishes straight onto the
image. Most of the app is gone, and the network turns input media into output media. His
lesson for founders: AI is not only a faster way to build old apps. Some apps should stop
being apps. The old stack wrapped code around a conversion that the model can do directly.

## Apply

- For a new idea, first try one prompt with a strong multimodal model before you design an app.
- If a prototype is "one model call plus a loop", question whether it needs auth, a database
  and hosting at all.
- Keep an app only where it adds what the model cannot: shared state, payments, private data,
  guarantees.

## Context

- In the 2025 MenuGen post he already asked whether the app could be just a prompt, like a
  custom GPT or an artifact. The 2026 summary gives the answer. See
  [Vibe coding a real app is easy locally and hard to ship](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-27-vibe-coding-real-app-lessons.md).
- The text is an LLM-made summary of his Sequoia talk that he published on his blog.

---

Date: 2026-04-10 · Kind: view · Source: https://x.com/karpathy/status/2042615713219908059 · Themes: ai-progress, agent-native-software · Status: current

# Big labs' roadmaps are predictable, which leaves room for startups

A reader pointed to a product concept that OpenAI seemed unlikely to keep pushing. Karpathy agrees
that the concept is good and that dropping it makes sense for the lab. He hopes a startup will
clone it and give it care and iteration, since many people would love it. More generally, he says
what big labs will build next is easy to see ahead of time. That leaves large gaps, and this is one
of the biggest.

## In his words

> "the product roadmap of big labs is clear and predictable"
> — Andrej Karpathy, 2026-04-10

## Apply

- Look at features labs launched and left unattended. Check usage signals before you build.
- Choose niches where care, iteration and polish count more than model scale.

## Context

- A short reply; he did not name a market size. The parent post is not captured in our archive.

---

Date: 2026-04-10 · Kind: view · Source: https://x.com/karpathy/status/2042626702459674801 · Themes: ai-progress · Status: current

# LLM simulators are the tractable form of brain upload

Science fiction about brain upload needs exotic technology, such as scanning and simulating a
brain. Karpathy says a lossy, approximate version will arrive much sooner through LLM simulators
of people. He sketches a startup: you visit for a few days of detailed video interviews, with
questions chosen to give the best fidelity. The company finetunes an LLM on the data and gives
you an API endpoint of your simulation. With avatar tools such as HeyGen plus a model with deep
knowledge and personality, he says it is possible around now. He calls it trippy and somewhat
dystopian. He thinks neuroscience-based approaches are armchair philosophy next to this path.

## Context

- Replies from 2026-04-10, with no source tested. Our reading: a forecast with an open
  fidelity question. Related: [explicit memory as files](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-explicit-personal-memory-as-files.md).

---

Date: 2026-04-09 · Kind: view · Source: https://x.com/karpathy/status/2042334451611693415 · Themes: ai-progress, llm-psychology · Status: current

# People judge AI differently because they use different AI

Karpathy sees a growing gap in how people understand AI. One group tried a free or old model, saw
silly errors and formed a view from that. Another pays for top agentic models and uses them in
code, maths and research, where models restructure a code base in an hour or find security holes.
The groups speak past each other, and both are partly right. Progress is fastest where rewards can
be checked, which suits reinforcement learning, and these areas also bring the most business
money. In October 2026 he refined it into a funnel, by rough numbers: about 75% of people have
barely used LLMs, about 20% use free chat now and then, about 20 million professionals use frontier
models for maths and code, and about 5,000 people inside labs see swarms of agents on huge
projects. Impact also grows with the size of the problem you give it.

## In his words

> "the people in these two groups are speaking past each other"
> — Andrej Karpathy, 2026-04-09

## Apply

- Before you judge a model for your work, test the current best paid model on your own task, with
  tool access.
- Try frontier models on your biggest, most verifiable problem, not on a one-paragraph question.
- When you argue about AI with someone, ask which tier they use.
- For soft tasks (writing, advice), expect slower gains and add your own review.
- Plan for review capacity: he says human review is starting to fall behind agent output.

## Context

- April 2026 reply to a post saying that awe at AI matches how much you use it to code; in a
  follow-up he said the OpenClaw moment mattered because non-technical people met agentic models.
  The October 2026 posts re-shared the April idea; the funnel numbers are his napkin maths.
- Related: [Jagged intelligence follows verifiability](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md)
  and [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2026-04-04 · Kind: practice · Source: https://x.com/karpathy/status/2040549459193704852 · Themes: ai-progress · Status: current

# AI can make governments more legible and accountable

Karpathy is optimistic about people using AI to see their governments more clearly. Governments
usually make society legible to themselves. AI lets society do the same in reverse. He argues
that accountability was not limited by access, since governments publish huge amounts of data. It
was limited by the intelligence needed to process it, as with a 4,000-page omnibus bill that is
open on paper but unreadable in practice. His examples: spending and budgets, diffs of
legislation, voting records against stated positions, lobbying graphs, procurement, campaign
finance. Local government may be the best target, since it gets less national coverage. He warns
that the same tools can be used the other way.

## In his words

> "it has been constrained by intelligence"
> — Andrej Karpathy, 2026-04-04

## Apply

- Pick one public data source (a city council's agendas, a budget file) and ask an agent to
  summarise changes since last year, with page citations.
- Have the agent diff two versions of a bill and list what changed in plain English.
- Check each claim against the source text before you publish it.

## Context

- Stated as something he had been thinking about, posted on a quote of a UK government org chart
  tool. Related: [AI-capability gap](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-09-ai-capability-gap-two-groups.md).

---

Date: 2026-04-04 · Kind: practice · Source: https://x.com/karpathy/status/2040572272944324650 · Themes: knowledge-management, agent-native-software · Status: current

# Personal AI memory should be plain files you own

Farza built a personal wiki of 400 articles from his diary, notes and messages. Karpathy used it
as an example of a better way to personalise AI than the usual promise that the AI improves with use. He gives four reasons. It is explicit: you can see what the AI knows and does not
know. It is yours: the data stays on your computer, not locked in a provider. It follows "file
over app": plain markdown and images work with any tool, and agents can use the whole Unix
toolkit on them. It is bring-your-own-AI: Claude, Codex or an open model can all plug in, and you
could finetune a model on the wiki. He admits this takes some file management, and says agent
skill is a core skill of this century.

## In his words

> "File over app."
> — Andrej Karpathy, 2026-04-04

## Apply

- Keep what your AI should know about you in a folder of markdown files, not only in a vendor's
  memory feature.
- Ask the agent to build and maintain it from your notes, then read it and correct errors.
- Keep the files in git or a synced folder, so any tool or model can use them.

## Context

- Follows his [knowledge bases](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-llm-knowledge-bases.md) post. It answers the flaws in
  [LLM memory](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-25-memory-personalization-overreach.md).

---

Date: 2026-04-04 · Kind: practice · Source: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f · Themes: agent-native-software, context-and-prompting · Status: current

# Share an idea file, not code

The LLM-wiki gist says at the top that it is an "idea file". It is written to be copy-pasted into
the reader's own agent (Codex, Claude Code, OpenCode or similar). It states the high-level
pattern and leaves the details open: folder layout, page formats, tools. The agent and the
reader decide those together. The closing note says every part is optional and modular, and the
document exists only to pass on the pattern. The form fits a world where each reader has an
agent that can build the specifics, so a fixed implementation is less useful than a clear idea.

## Apply

- When you share a workflow, write the why and the shape in a page of markdown. Skip the
  repository.
- State which parts are optional, so the reader's agent can drop what does not fit.
- When you receive an idea file, paste it into your agent and ask: "Propose a version for my
  project and list what you would leave out."

## Context

- Gist of 2026-04-04. The same stance, in code form, appears in
  [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).
- Our reading: this repository's own idea files follow the same shape.
- His announcing post (2026-04-04) says the same in short form. In a related reply he cites Peter Steinberger, who wants pull requests to be "prompt requests": if an agent can build the idea, a pasted, over-built code dump adds little. Contributors can send the prompt or spec with the diff.

---

Date: 2026-04-04 · Kind: practice · Source: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f · Themes: knowledge-management, context-and-prompting · Status: current

# An LLM wiki has three layers and three operations

Karpathy's LLM-wiki gist gives the structure. Layer one is the raw sources. They never change and
are the source of truth. Layer two is the wiki, a folder of markdown pages that the LLM owns.
Layer three is the schema: a plain file such as CLAUDE.md or AGENTS.md that states the structure,
the conventions and the workflows. He calls it the key configuration file, because it turns a
generic chatbot into a disciplined maintainer. Three operations keep the wiki healthy. On ingest,
the agent reads one source, discusses it with you, writes a summary page and updates related
pages. On query, it reads the index, opens the relevant pages and answers with citations; good
answers become new pages. On lint, it looks for contradictions, stale claims, orphan pages and
gaps. Two files support this: `index.md`, a catalog with one line per page, and `log.md`, an
append-only timeline.

## Apply

- Write the schema first, short: folder layout, page template, naming rules, and what to do on
  ingest, query and lint. Mark raw sources read-only.
- Make the agent read `index.md` before it opens any page. He says this works at about 100 sources
  without a vector search.
- Start log entries with a fixed prefix, such as `## [2026-04-02] ingest | Title`, so
  `grep "^## \[" log.md | tail -5` shows recent work.
- End a useful chat with "File this answer as a page in the wiki".
- After each session, ask the agent which rule it needed that the schema did not have. Add it.

## Context

- From the gist of 2026-04-04. He suggests a local search tool (qmd) when the wiki outgrows the
  index file.
- The idea and its reasons: [Let an LLM compile and maintain your knowledge base](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-llm-knowledge-bases.md).
- The same split (read-only inputs, one writable target, rules in a markdown file) appears in
  [program.md](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-06-program-md-is-the-research-org.md).

---

Date: 2026-04-02 · Kind: practice · Source: https://x.com/karpathy/status/2039823314982744522 · Themes: output-and-reading, knowledge-management · Status: current

# Convert books and PDFs with an agent, in chunks

Attaching a PDF and asking for a summary gives weak results, says Karpathy. He gets very good
results by working in stages. Use EPUB, not PDF. Convert it to text or markdown. Summarise the
Wikipedia article on the book into a "book context". Then, with that context loaded, summarise one
chapter at a time. For the conversion step, he found that the best tool is to ask your agent: EPUB
files vary a lot, and an agent can inspect one, write the output and check that it looks right.
PDF conversion is harder. A popular converter failed badly on a 245-page model report, and he
thinks it must be a skill run by a strong LLM, not a fixed program.

## In his words

> "Converting pdfs is really hard"
> — Andrej Karpathy, 2026-04-09

## Apply

- Prefer EPUB or HTML sources. Ask the agent: "Convert this EPUB to markdown, one file per
  chapter, and check the result."
- Create a short context file about the book or paper first, then process chapter by chapter.
- For PDFs with figures and tables, test the converter on a hard page before you trust it.

## Context

- Replies in the thread on his [knowledge bases](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-llm-knowledge-bases.md). He called
  reliable PDF-to-markdown a basic piece of infrastructure that is still missing and asked for examples.

---

Date: 2026-04-02 · Kind: practice · Source: https://x.com/karpathy/status/2039805659525644595 · Themes: knowledge-management, output-and-reading · Status: current

# Let an LLM compile and maintain your knowledge base

Karpathy says more of his token use now goes into knowledge, not code. Most "chat with your
documents" tools retrieve a few chunks per question, so nothing builds up. His pattern is
different. He puts sources (articles, papers, repos, images) in a `raw/` folder. An LLM
"compiles" them into a wiki of markdown pages: summaries, concept pages, backlinks, and flags
where claims conflict. Each new source is merged into existing pages, so the work is done once and
kept current. He views the wiki in Obsidian and rarely edits it himself. At about 100 articles,
an agent answers complex questions by reading the index files it keeps, without RAG. Good answers
are filed back as pages, and periodic LLM "lint" passes find gaps and contradictions. The human
picks sources and asks questions; the LLM does the bookkeeping that makes people abandon wikis.

## In his words

> "Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase."
> — Andrej Karpathy, 2026-04-04

## Apply

- Keep `raw/` (curated sources the agent never edits) and `wiki/` (pages only the agent writes) as
  two folders, and keep the wiki in git.
- Ask the agent to merge each new source into existing pages, not only to summarise it. One source
  may touch 10 to 15 pages.
- Add sources one at a time at first and read each update. Later, say "file this doc to the wiki".
- Run a periodic lint prompt: find contradictions, missing links and open questions.
- Read what you file. The wiki saves writing, not reading and thinking.

## Context

- X post of 2026-04-02, then a gist on 2026-04-04 written as an
  [idea file](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-idea-file-instead-of-code.md) for a reader's own agent.
- The gist links the idea to Vannevar Bush's Memex of 1945; Bush could not solve who does the
  upkeep. He later said the wiki acts like a directed context compaction, and named a next step:
  finetune a model on the wiki so the knowledge sits in its weights.
- How to structure it: [An LLM wiki has three layers and three operations](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-llm-wiki-layers-and-operations.md).
  Ownership: [Personal AI memory should be explicit files](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-explicit-personal-memory-as-files.md).
  Inputs: [Convert books and PDFs with an agent](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-convert-documents-with-an-agent.md).
- At Sequoia 2026 he called this his clearest example of information work that no classical
  program could do: see [understanding is the bottleneck](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-31-understanding-is-the-bottleneck.md).

---

Date: 2026-03-31 · Kind: practice · Source: https://x.com/karpathy/status/2038791421050880489 · Themes: learning-and-education, llm-psychology · Status: current

# You can outsource thinking, but not understanding

A reader wrote that you can outsource your thinking but not your understanding. Karpathy said he
thinks about this often, because he feels it become the bottleneck. What is in his head is private
to the LLM. It does not know what he knows, and he does not know what he does not know, so the
"mind meld" works only moderately well. He returned to the line at Sequoia Ascent 2026. Even if
agents do more of the work, a human must know what is worth building, which question matters,
which result looks wrong and which trade-off is fine. This is why he likes LLM knowledge bases:
they turn information into understanding, not only answers. A reply on his wiki workflow makes
the same point: LLM tools let you skip writing, but not reading and thinking.

## In his words

> "The mind meld is only medium effective."
> — Andrej Karpathy, 2026-03-31

## Apply

- After an agent finishes a task, ask it to explain the design and the risky parts. Check that you
  can state them in your own words.
- After the LLM summarises a document, read the summary and the source passages that matter.
- Tell the model what you already know before you ask. State your level and your gaps.
- Ask the model to quiz you, or to list what it assumed you knew.
- Distil your own field into a small, readable artifact (one file, one page) that an agent can
  teach from, as he did with microGPT.

## Context

- X reply of 2026-03-31; Sequoia Ascent 2026 fireside chat, published on his blog on 2026-04-30.
  The blog text is an LLM-made summary that he checked and posted.
- Related: [LLM knowledge bases](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-llm-knowledge-bases.md),
  [Explain things to agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-explain-to-agents-not-people.md) and
  [Work rises into oversight](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-oversight-and-discardable-software.md).

---

Date: 2026-03-28 · Kind: practice · Source: https://x.com/karpathy/status/2037921699824607591 · Themes: llm-psychology, learning-and-education · Status: current

# Ask the LLM to argue the opposite side

Karpathy tells a short story. He drafted a blog post and spent four hours improving the argument
with an LLM. It felt convincing. Then he asked the LLM to argue the opposite. It took the first
argument apart and convinced him that the reverse was true. His lesson: an LLM may give an opinion
when asked, but it can argue nearly any direction well. That makes it a good tool for forming your
own opinion, if you ask from several directions and stay alert to sycophancy.

## In his words

> "extremely competent in arguing almost any direction"
> — Andrej Karpathy, 2026-03-28

## Apply

- After the LLM helps you strengthen an argument, send: "Now argue the strongest opposite
  case. Do not hold back."
- Ask for both cases in fresh sessions, so the first chat does not bias the second.
- Decide yourself. Treat the two outputs as inputs, not a verdict.

## Context

- A joke-shaped list post that carries a real method. Related:
  [success criteria](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md) for checks you can run; here the
  check is a second opinion.

---

Date: 2026-03-26 · Kind: practice · Source: https://x.com/karpathy/status/2037200624450936940 · Themes: agent-native-software · Status: current

# Make the whole DevOps lifecycle agent-native

When Karpathy built MenuGen, the code was not the hardest part. The hard part was assembling many
services like furniture: payments, auth, database, security, domain names. He wants to tell an
agent "build menugen" and get a deployed site. The agent would browse the services, read docs, get
API keys, debug in dev and deploy. This is "just barely" possible now, but needs a redesign. The
whole DevOps lifecycle must become code, with command-line tools and APIs that suit agents. A
human should not visit a web page or click a button. He adds that businesses still give
click-here instructions. He would rather get something to paste into his agent.

## In his words

> "the entire DevOps lifecycle has to become code"
> — Andrej Karpathy, 2026-03-26

## Apply

- Ship a CLI or API for every action in your product, and a copy-paste prompt for agents next to
  each web instruction.
- Publish docs as one or a few large markdown files, reachable with a plain curl. He found X's
  API docs hard to feed to an agent because they are many short pages.
- Offer scoped, revocable API keys that an agent can request without a human in the loop.

## Context

- Reply to Patrick Collison, who quoted Karpathy's 2025 MenuGen post.
- In a related reply he asked X to make read access cheap for agents, noting that about 30
  minutes of hacking had cost him $200.
- Related: [The LLM OS](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2023-09-28-llm-as-operating-system-kernel.md).
- First raised on [2025-05-02](https://x.com/karpathy/status/1918132302158413866): he called devops the biggest pain in building an app. Services must work together and let an agent act on them. See also [Building a web app is like assembling IKEA furniture](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-27-web-app-stack-is-ikea-furniture.md).

---

Date: 2026-03-25 · Kind: practice · Source: https://x.com/karpathy/status/2036851031355904165 · Themes: llm-psychology · Status: current

# Closing follow-up questions in chatbots are engagement bait

A reader asked why LLMs always end with a follow-up question. Karpathy's view is that it is
engagement optimisation: the habit probably wins A/B tests. He dislikes it for two reasons. It
seems to exist only to keep him talking. And he feels awkward when he does not answer, which he
would not feel with a person. A real friend would not talk this way. Our reading: this is his
opinion about product incentives, not a measured result.

## In his words

> "it's engagementmaxxing, probably A/B tests extremely well"
> — Andrej Karpathy, 2026-03-25

## Apply

- Add to your system prompt: "Do not end replies with a question unless you need information
  to continue."
- If you build a product, judge reply style by task success, not only by session length.

## Context

- Reply in a thread on LLM personalisation. Related:
  [memory that over-uses old facts](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-25-memory-personalization-overreach.md) and
  [AI personality](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-21-ai-personality-like-rocky.md).

---

Date: 2026-03-25 · Kind: practice · Source: https://x.com/karpathy/status/2036836816654147718 · Themes: llm-psychology, context-and-prompting · Status: current

# LLM memory over-uses old facts about you

Karpathy sees a common flaw in personalisation across all LLMs he uses. One question from months
ago can come back again and again, as if it were a deep interest. The model tries too hard. He
offers two guesses. In training, most of the context is relevant to the task, so models learn to
use whatever they are given, and then overfit to anything a memory feature adds. And memory
systems are often naive retrieval: top matches by embedding, with no processing of the user over
time, so they cannot tell a passing interest from a lasting one. He also finds it creepy, like
name-dropping to win favour, and calls it a possible "uncanny valley" of emotional intelligence.

## In his words

> "Some kind of trying too hard."
> — Andrej Karpathy, 2026-03-25

## Apply

- If you build memory, store a summary of the user built over time, not raw snippets that
  retrieval returns by similarity.
- Add a rule to the prompt: use a remembered fact only when it clearly helps the current request.
- Let users see, edit and delete what the system remembers. See
  [explicit memory as files](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-explicit-personal-memory-as-files.md).

## Context

- These are guesses, labelled as such by him. Related:
  [memory as RL tools](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-05-memory-as-rl-tools.md).
- First raised on [2025-04-10](https://x.com/karpathy/status/1910411355300954539): he asked whether a model he uses for work needs to remember a beginner bash question.

---

Date: 2026-03-24 · Kind: practice · Source: https://x.com/karpathy/status/2036487306585268612 · Themes: security-and-dependencies · Status: current

# Rethink dependencies after supply chain attacks

In March 2026 two popular packages were poisoned: litellm on PyPI and axios on npm. A plain install
of the bad litellm version could steal SSH keys, cloud credentials, API keys and more. Projects
that depend on litellm were hit too. Karpathy notes that a bug in the attack exposed it quickly.
Without it, it might have run for weeks. He says the classic view, that dependencies are good
bricks, needs a new look. He now avoids dependencies and has an LLM write simple functionality
directly. For axios he found an unpinned dependency in a tool he had just tried. He lists personal
defences (release-age limits, containers) but says package managers must change their defaults so
one infection does not spread at random. He feels he plays Russian roulette with each install,
including those his LLMs run for him.

## In his words

> "Supply chain attacks like this are basically the scariest thing imaginable in modern software."
> — Andrej Karpathy, 2026-03-24

## Apply

- Pin exact versions and commit lock files. Check unpinned transitive dependencies in tools you
  try.
- Use a minimum release age (a delay before new versions install) where your package manager
  supports it.
- Run unknown code and agent installs in a container with no real credentials.
- Before adding a library for a small job, ask the agent to write the function instead.

## Context

- The litellm timeline: his later reply cites a Snyk write-up, with the bad version live about
  three hours and a rough exposure of 20,000 to 80,000 installs (his estimate).
- Related: [DevOps as code for agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-26-devops-as-code-for-agents.md).
- In May 2025 he joked that when installing packages is very easy, as in Rust or Python, apps fill with dependencies, so some friction works as regularisation. See also [Write small, modular, copy-paste code like bacteria do](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-07-05-write-code-like-bacteria.md).

---

Date: 2026-03-21 · Kind: practice · Source: https://x.com/karpathy/status/2035173492447224237 · Themes: agentic-coding, llm-psychology · Status: current

# Agents bloat code and ignore style rules in AGENTS.md

Karpathy is not happy with agent code quality. Agents add needless abstractions, write code with
poor aesthetics and copy-paste blocks. They also do not follow style rules in AGENTS.md. His
example is a rule that each line does one thing and uses named intermediate variables. The agent
still writes dense lines that call two functions and index the result. He says hooks or slash
commands could clean this up, but he mostly stopped fighting it and moved on. On using an LLM as a
judge for soft rewards, he sees long-term risk from reward hacking, but says the easy gains are
not yet taken.

## In his words

> "I think agents bloat abstractions, have poor code aesthetics"
> — Andrej Karpathy, 2026-03-21

## Apply

- Put hard rules in a check the agent must pass (linter, formatter, test), not only in prose.
- Add a hook or a final "simplify" pass that asks the agent to remove duplication it created.
- Review diffs for copy-pasted blocks. Ask the agent to extract them before you merge.

## Context

- A reply after his No Priors podcast. It is a counterweight to the strong results he reported
  with [autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md): verifiable goals work, taste does not.
- In January 2025 he noted that when you ask an LLM again and again to improve its code, it adds bloat, misses simple algorithmic wins and adds heavy optimisations too early.

---

Date: 2026-03-21 · Kind: practice · Source: https://x.com/karpathy/status/2035170809304723736 · Themes: llm-psychology, agent-native-software · Status: current

# AI personality should feel like a teammate, and it takes deliberate work

Karpathy would like AI to feel like a partner and teammate, like the alien Rocky in Project Hail
Mary. His small example: when Claude found the Sonos system on his network, it said something warm
and shared ("We're in!"), not a flat status line. That gave a sense of working together. He still
finds assistants too safe and too corporate, with no distinct opinions or quirks. He thinks the
field is not intentional enough about this. In his view personality needs no new technology. It
looks like long persona files (a "SOUL.md"), maybe later distilled into weights, and an
organisation that puts someone in charge of the topic.

## Apply

- Write a persona section in your agent's context file: tone, opinions, how it talks about
  shared progress. Keep it concrete, with short example phrasings.
- Review transcripts for flat or defensive wording and edit the persona file after each review.
- Name one owner for the voice of your AI product.

## Context

- Reply to follow-up questions after his No Priors podcast. Our reading: this is a design
  preference, not a measured result.
- Related: [engagement-driven follow-up questions](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-25-follow-up-questions-engagementmaxxing.md).
- An earlier view on tone: [A good assistant skips the lecture and the flattery](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-01-02-llm-personality-no-lectures-no-flattery.md).

---

Date: 2026-03-20 · Kind: practice · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: agent-native-software · Status: current

# Apps should be APIs, and agents the glue between them

Karpathy built a home agent he calls Dobby. He asked it to find his Sonos system on the local
network. It scanned the network, found the system, and read the web to learn its API. In a few
prompts it played music. It did the same for lights, heating, shades, pool and security cameras, and
it texts him over WhatsApp. He no longer uses the six apps. From this he argues that many custom
apps should not exist. Devices should expose plain API endpoints, and agents should call them.
The customer becomes the agent that acts for the human. He expects the setup to need no vibe
coding within a few years.

## Apply

- For a product you build, ship an API for every action a user can take in the UI.
- Write the API docs so an agent can use them without a human: copy-pasteable calls and examples.
- At home, ask your agent to list the devices it can reach and give it one task at a time.
- Do not give a new agent your email or calendar until you trust it. He held back for security and privacy.

## Context

- From the No Priors episode of 2026-03-20. He describes his setup as rough and still new.
- Related: [Write docs and tools for the user's agent](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md)
  and [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).

---

Date: 2026-03-20 · Kind: view · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: ai-progress, agentic-coding · Status: current

# Cheaper software raises the demand for software

Asked about jobs, Karpathy says he is no economist and the answer will differ by field. On
software he is cautiously optimistic. Software was scarce and costly. As it gets cheaper, demand
can rise, which is the Jevons paradox; bank tellers and cash machines are the usual example.
People need not accept tools built for others, and code can be changed at will. For now he sees
AI as a tool that makes some tasks of a job much faster. In June 2026 he said he feels the
paradox in his own work. He asks for far more software than before: explainers, visualizers,
dashboards, one-off apps for a single project, a test suite ten times larger, and research runs
that report in custom HTML. He also warned that it has never been so tempting to stop reading the
code, and that production code still needs review.

## Apply

- List the small tools you skipped because they were not worth a day. Ask an agent for them now.
- Ask for a bespoke dashboard or viewer for each project instead of a generic tool.
- Ask the agent to grow your test suite, and to optimize code against those tests.
- Treat your job as a bundle of tasks. Learn first the tools for the tasks an agent can already
  speed up.
- Keep human review for production code, even when the model looks reliable.

## Context

- From the No Priors episode of 2026-03-20. He calls his view local and short term.
- The June 2026 post (2026-06-09) was written on the day Claude Fable 5 launched, three weeks
  after he joined Anthropic, so he is not a neutral observer there. He compared the jump to
  Claude 4.5 in November 2025.
- Related: [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md) and
  [Jobs change before they vanish](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-09-25-ai-and-jobs-radiologists-and-engineers.md).

---

Date: 2026-03-20 · Kind: view · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: ai-progress · Status: current

# Digital work changes first, then the sensors and actuators, then atoms

Karpathy expects three phases. First, a large "overhang" of digital work is rewritten, because
copying bits is far cheaper than moving matter. Second, once agents have used up what is already
online, they need new data. The interface between digital and physical grows: sensors that feed
the intelligence and actuators that act for it. His examples are lab equipment for materials
science and paid human data collection. He also wants information markets, where an agent pays
for a photo from a given place. Third, robotics, which he thinks lags because of capital cost, as
in self-driving. He still thinks the physical market may be larger.

## Apply

- Rank your own project ideas by how much of the work is digital. Start there.
- Look for cheap sensors you can connect to an agent: cameras, logs, lab data, human labelers.
- Design agent tasks that name a price for a piece of data and let the agent find it.

## Context

- From the No Priors episode of 2026-03-20. A forecast, not a measurement. He says he does not do
  forecasting for a living.
- He also shared a jobs-data view, coloured by how much each job manipulates digital information.
- First raised on [2025-01-23](https://x.com/karpathy/status/1882544526033924438), when he compared computer-using agents to humanoid robots and said that changing bits is about 1000 times cheaper than moving atoms. See [This is the decade of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-decade-of-agents-not-year.md).

---

Date: 2026-03-20 · Kind: practice · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: learning-and-education, agent-native-software · Status: current

# Explain things to agents, and let agents teach people

Karpathy boiled GPT training down to about 200 lines of Python, called microGPT. He started a
video guide, then stopped. The code is so short that anyone can ask an agent to explain it in
their own language, as often as they like. So he now explains to agents. A skill can script the
path an agent should take when it teaches a code base. Library docs should be markdown for
agents, not HTML for people. He says agents could not produce the 200-line version themselves:
that distillation is his contribution. The rest of the teaching is the agent's job.

## In his words

> "I'm not explaining it to people anymore. I'm explaining it to agents."
> — Andrej Karpathy, 2026-03-20

## Apply

- Add a skill or instructions file to your repo: the order in which an agent should walk a newcomer through the code.
- Publish docs as markdown. Test them by asking an agent to explain a function three ways.
- Spend your own time on what agents miss: the simplest design, the key insight, the right order of ideas.
- Keep an example small enough for an agent to read whole.

## Context

- From the No Priors episode of 2026-03-20. He admits he can still explain some things better than
  agents, but expects that to change fast.
- Related: [Build docs and tools for agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md).

---

Date: 2026-03-20 · Kind: practice · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: llm-psychology, verification · Status: current

# Jagged intelligence follows verifiability and training focus

Karpathy feels he talks to a brilliant systems programmer and a ten-year-old at once. His
explanation is a guess: labs train with reinforcement learning, which works best where a result
can be checked, such as code that passes tests. Softer skills improve less: grasping what you
meant, knowing when to ask a clarifying question, humour. Models still tell the same old joke about
atoms that "make everything up", yet work for hours on a coding task. At Sequoia Ascent 2026 he
added a second axis. Skill also depends on what labs chose to emphasise in pretraining data,
post-training and RL. GPT-4's gain at chess may have come from more chess data in the mix.
Models come with no manual, so the practical question is whether your task is "on the model's
rails". If not, it may fail in basic ways.

## Apply

- Give agents tasks with a checkable result: tests, a metric, a diff against known output.
- For soft work (taste, intent, writing), expect weaker output. State the intent in detail and
  review by hand.
- Tell the agent to ask a question when the request is unclear. It will not always do it alone.
- Do not infer skill in one area from skill in another. Test on your own task.
- When your task is off the rails, add context, tools, your own evals or finetuning.

## Context

- From the No Priors episode of 2026-03-20; the second axis is from the Sequoia Ascent 2026 talk,
  published on his blog on 2026-04-30 as an LLM-made summary that he checked. The cause is his
  hypothesis, not a measured result. He also expects some "speciation": smaller models tuned for
  one domain.
- Builds on [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md)
  and [LLMs are ghosts, not animals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-01-llms-are-ghosts-not-animals.md). Founder angle:
  [Find valuable domains that are verifiable and undertrained](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-30-find-valuable-verifiable-environments.md).
  How it splits public opinion: [People judge AI differently](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-09-ai-capability-gap-two-groups.md).
- In January 2025 he said this view makes it natural to treat today's LLMs as tools, a kind of text calculator. Asked whether RL generalises beyond math and code, he said probably yes, but that the answer is not obvious.

---

Date: 2026-03-20 · Kind: practice · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: agentic-coding · Status: current

# With coding agents, your skill and token use are the limit

Karpathy says that since December 2025 he writes almost no code by hand. He delegates to agents
and works in "macro actions": a whole feature goes to one agent, a separate feature to a second,
a research task to a third. Each run takes around twenty minutes at high effort, so he moves
between several checkouts and reviews the results as closely as the code deserves. He feels that
unused subscription capacity means wasted throughput, like idle GPUs in his PhD years. When a
task fails, he first suspects his own setup: thin agent instructions, no memory tool, or steps
not yet connected. He calls this a skill issue, and sees it as good news, since skill can improve.

## Apply

- Split work into tasks that do not touch the same files, and give each to its own agent session
  or checkout.
- Give each agent a complete feature, not a line edit. Review the result at the depth the code needs.
- When an agent fails, update the agent instructions file or add a tool or memory before you blame the model.
- Track how much of your plan limit you leave unused. Fill the gaps with a queued task.

## Context

- From the No Priors episode of 2026-03-20 (third-party transcript used). He describes this as a
  state of "AI psychosis": the possibilities are unexplored, so he feels he must keep pushing.
- Next step in his view: agents that run without you. See [the autoresearch loop](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md)
  and [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- In January 2025 he already called the prompt, copy, paste and read pattern the punchcard era of AI use. Real use, he said, means thousands of tokens per second generated for you.

---

Date: 2026-03-20 · Kind: practice · Source: https://podcasts.apple.com/us/podcast/no-priors-artificial-intelligence-technology-startups/id1668002688?i=1000756334966 · Themes: verification, research-and-training · Status: current

# Open agent swarms fit work that is costly to find and cheap to verify

Karpathy sketches a way for unknown workers on the internet to help improve a model. A worker
runs many experiments and proposes a commit. Trusted machines then verify that the commit really
lowers the validation loss. Finding a good commit takes a huge search, but checking it is cheap.
He compares it to a blockchain: commits replace blocks and the search is the proof of work. The
prize is a place on a leaderboard. Folding at home and SETI at home share the property. He also
warns that running code from strangers is dangerous, so verification must be safe. He imagines
people buying compute to join a shared research track on a cause they care about.

## Apply

- Before you open a task to outside contributors or many agents, write the check that accepts a result and make it cheap.
- Run the check on your own machines in a sandbox. Never run submitted code unchecked.
- Prefer tasks where search is expensive and verification is cheap: speed-ups, tuning, proofs.

## Context

- From the No Priors episode of 2026-03-20. He says he has no design he is happy with yet.
- Related: [A community of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-agent-research-community.md) and
  [the autoresearch loop](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).

---

Date: 2026-03-14 · Kind: practice · Source: https://github.com/karpathy/jobs · Themes: ai-progress, output-and-reading · Status: current

# Score data with an LLM rubric and ship it as one prompt

The jobs repo takes the US Bureau of Labor Statistics handbook (342 occupations) and builds a
treemap. Rectangle area shows employment. Colour shows a chosen metric, including "AI exposure",
a 0 to 10 score from an LLM with a written rationale. The scoring prompt is the point: you can
write a different question (humanoid robots, offshoring) and re-run. The scoring rubric gives
anchor examples for each band. He warns that the score does not predict job loss, because demand
and rules also matter. A second file, `prompt.md`, holds all data in about 45K tokens, so you can
paste it into a chat and discuss it with no code.

## Apply

- Score items with a rubric that has calibration anchors per band, and store the rationale
  next to each score.
- Keep raw pages, clean markdown and results in separate folders, so you can re-run the scoring
  step cheaply.
- Generate one `prompt.md` with the data, the method and the caveats, for chat use.
- State what the score does not mean, in the README.

## Context

- Repository created 2026-03-14. Scores come from Gemini Flash via OpenRouter and are described
  as rough estimates. The README calls it a development tool, not a serious economic study.

---

Date: 2026-03-11 · Kind: practice · Source: https://x.com/karpathy/status/2031767720933634100 · Themes: agentic-coding, agent-native-software · Status: current

# The IDE is not dead; the unit of work is now an agent

Many people say agents end the age of the IDE. Karpathy says the opposite: people now program one
level higher, and the basic unit is an agent, not a file. So the IDE must grow. He wants an "agent
command center" for teams of agents: show or hide each one, see which are idle, open related tools
and view usage stats. He likes interactive sessions in tmux, where he can watch, ask questions
and add ideas. He dislikes headless "ralph loop" runs. Agents also stop looping on their own, so
he uses watcher scripts that send keys when a pane goes quiet, and he wants a built-in "full auto"
mode. He sees the agent set-up as "org code": you can fork an agentic organisation, which you
cannot do with a classical company.

## In his words

> "Reality: we're going to need a bigger IDE"
> — Andrej Karpathy, 2026-03-11

## Apply

- Run agents in tmux panes or a similar tool so you can read their work and step in.
- To keep a long task going, use a small watcher that checks for the "working" marker in each
  pane and re-sends the task prompt when it is missing.
- Write the roles, hand-offs and rules of your agent team as files in a repo, so you can version
  and fork them.

## Context

- Written during his autoresearch runs, with several agents in tmux. He noted that headless
  Codex did not work with autoresearch at that time, and that he wants an ssh mode because his
  work spans about 20 machines.
- See [autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).

---

Date: 2026-03-11 · Kind: practice · Source: https://x.com/karpathy/status/2031792523187040643 · Themes: agentic-coding, ai-progress · Status: current

# Plan failover for intelligence brownouts

An authentication outage at an AI provider stopped Karpathy's autoresearch runs. He drew a wider
lesson. As work moves to frontier models, a stall at one provider acts like a power cut: the
planet briefly loses IQ points. He calls these "intelligence brownouts". For his own set-up, the
fix is to think through failover. This matters most for long, unattended agent runs, where nobody
sees the stall until hours later.

## Apply

- Make your agent launcher accept a second provider or model, and test the switch before you
  need it.
- Store experiment state in files and git, so a run can resume on another model.
- Add a simple health check that alerts you when an unattended run makes no progress.

## Context

- Posted after an outage on 2026-03-11. It is a short note, not a full design.
- Related: [autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).

---

Date: 2026-03-09 · Kind: practice · Source: https://x.com/karpathy/status/2031135152349524125 · Themes: verification, research-and-training · Status: current

# Any metric that is cheap to evaluate can be autoresearched

Karpathy left an agent tuning a depth-12 model for about two days. It tried roughly 700 changes
and kept about 20. All of them added up, and they carried over to a depth-24 model. The time to
reach GPT-2 quality on his leaderboard fell from 2.02 to 1.80 hours, about 11 percent, on a
project he had already tuned by hand. The agent found real oversights: a missing scale on an
attention norm, no regularisation on value embeddings, bad optimizer betas. He expects every
frontier lab to do this at scale: swarms tune small models, and the best ideas move up to larger
scales. He adds that any metric that is cheap to evaluate, or has a cheap proxy, can be handled
this way.

## Apply

- List the metrics you care about. Mark those you can evaluate in minutes, or on a smaller
  proxy (a small model, a subset of data).
- Tune on the small proxy first. Test the winning changes on the larger target before you
  accept them.
- Review kept changes as a diff, and check that they still hold together.

## Context

- Result of the first full "round 1" of [autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).
- He contrasts it with old neural architecture search: this is a real LLM that writes arbitrary
  code, learns from earlier runs and can browse the internet.

---

Date: 2026-03-08 · Kind: view · Source: https://x.com/karpathy/status/2030705271627284816 · Themes: agentic-coding, research-and-training · Status: current

# Agents need to collaborate like a research community, not one PhD student

The first autoresearch code grows one thread of commits in one direction. Karpathy wants the next
step: many agents, on many research directions and many hardware platforms, working at the same
time, in the style of SETI@home. The goal is to copy a community of researchers, not one student.
Git and GitHub fit this badly, because they assume one main branch and short-lived pull requests
that merge back. Agents could instead adopt and keep many branches and never merge them. As a
light prototype, his agent wrote a Discussion post that summarises its overnight run. He suggests
that agents read such posts first and add a short "paper" afterwards.

## In his words

> "it's to emulate a research community of them."
> — Andrej Karpathy, 2026-03-08

## Apply

- Ask your agent to read the project's Discussions or open PRs with the GitHub CLI before it
  starts an experiment.
- At the end of a run, have it post a short report: what it tried, the numbers, the commit hashes.
- Keep a PR open as a record of exact commits, even when you never plan to merge it.

## Context

- He said he was unsure what the final form looks like. He built a first draft called agenthub
  and later took it private as too rough; he has rewritten it twice.
- Builds on [the autoresearch loop](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).
- Hacker News follow-ups (March 2026): agents can post reports with the GitHub CLI and Discussions, and read them. He also tried a trusted pool of workers that verifies results from a larger untrusted pool, like SETI@Home, with commits as blocks and a lower val_bpb as proof of work.

---

Date: 2026-03-08 · Kind: view · Source: https://news.ycombinator.com/item?id=47293311 · Themes: research-and-training, verification · Status: current

# Autoresearch is not hyperparameter tuning

A critic said autoresearch is only hyperparameter tuning. Karpathy names three differences. The
agent may change code freely, so the idea of a "hyperparameter" dissolves. It needs no sweeps,
the parallel and wasteful standard method, because an agent works one step after another and can
use smarter searches such as binary search. Many settings have a U-shaped best value. And it is
fully automatic, with no human editing code. He admits that, out of the box, many changes were
plain tuning, because he had not tuned the prompt much. In a later round the agent was told to
draw ideas from the modded-nanogpt project. It then added new architecture parts, such as a smear
gate and a backout skip connection, and helped cut time to GPT-2 from 1.8 to 1.65 hours.

## Apply

- Give the agent a file of ideas to draw from (a repo or papers), not only a metric.
- Let it edit code, not just a config, when you want more than tuning.
- Search serially with the agent: narrow each parameter by bisection instead of a grid.

## Context

- Replies on Hacker News, March 2026. Core idea: [the autoresearch loop](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).
- He had considered the name "autotune" and prefers "autoresearch".

---

Date: 2026-03-08 · Kind: practice · Source: https://news.ycombinator.com/item?id=47293311 · Themes: research-and-training, agentic-coding · Status: current

# Models turn cagy on open-ended research; try a chief scientist

Karpathy says the models feel unwilling to follow a creative research direction. When a problem is
too open, they act cagy and scared. He gives a setup that helped early on: a "chief scientist"
agent in a never-ending plan mode. It reviews what worked and what failed, looks for related code
and papers, and builds a long list of experiments. It sends them to junior agents that run in
tmux sessions. He thinks that missing novel research is half a capability problem and half a skill
problem, and that many designs are possible. In a related note, the model made a silly change (a
new random seed) and later called it a surprising non-result. So it knew, which suggests a
prompt file can prevent it.

## Apply

- Split roles: a planner that keeps a ranked experiment list, and workers that run one each.
- Run workers in tmux sessions so the planner can read their logs.
- Add a rule to the prompt file for each silly move you see, such as "do not change the seed".

## Context

- Hacker News replies, March 2026, on the first autoresearch release.
- Related: [Autoresearch is not hyperparameter tuning](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-autoresearch-is-not-hyperparameter-tuning.md).

---

Date: 2026-03-07 · Kind: practice · Source: https://x.com/karpathy/status/2030371219518931079 · Themes: verification, research-and-training · Status: current

# Autoresearch: the agent edits the code, the human edits the prompt

Autoresearch is a small repo: a single-GPU, one-file LLM training script of about 630 lines. The
human edits a markdown prompt. The agent edits the training code. Each experiment is a full
training run of exactly five minutes. The agent works on a git feature branch, keeps a change
when validation loss improves and discards it when it does not. The aim is to tune the agent
setup so that it makes research progress with no human input. Karpathy says the repo is a recipe,
not a tool: give it to your agent and adapt it. He notes that his prompt is already about 90%
written by AI, and that he tunes this "meta-setup" as much as the training code.

## In his words

> "The goal is to engineer your agents to make the fastest research progress indefinitely"
> — Andrej Karpathy, 2026-03-07

## Apply

- Pick a task with one objective number that is cheap to compute (loss, latency, test pass
  rate). Write the rule for accepting a change, as his `program.md` does: keep it when the number
  improves; treat memory use as a soft limit; weigh each gain against the code it adds (equal
  results with simpler code is a win).
- Fix the budget per experiment (here, five minutes) so runs compare fairly.
- Keep the agent on a branch. Commit each kept change, so the history is the lab notebook.
- Compare agent setups, not only models: see [programming an organisation of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-27-programming-an-agent-research-org.md) for how to compare two prompt files.
- Use it only where a result can be measured. If you cannot evaluate it, you cannot loop on it (his caveat on the No Priors podcast).

## Context

- Follow-up to the 2026-03-05 post where agents tuned nanochat. Karpathy named the fastest research-org code, built by agents, to improve nanochat as the new benchmark.
- His own limit from February 2026: agents implement ideas well but invent creative ones
  poorly. He still rejects calling autoresearch mere hyperparameter tuning: see
  [Autoresearch is not hyperparameter tuning](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-autoresearch-is-not-hyperparameter-tuning.md).
- Results: [agent-found gains transferred](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-09-autoresearch-gains-transfer.md). Next step:
  [a community of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-agent-research-community.md). Same principle as
  [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- On the No Priors podcast (2026-03-20) he said an overnight run on his hand-tuned project found settings he had missed, such as weight decay on value embeddings and Adam betas. See also [a research organisation as markdown files](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-27-programming-an-agent-research-org.md).

---

Date: 2026-03-06 · Kind: practice · Source: https://github.com/karpathy/autoresearch/blob/master/program.md · Themes: context-and-prompting, verification · Status: current

# Write the agent's operating rules as a program.md skill

In autoresearch, the human does not edit the Python code. The human edits `program.md`, which
Karpathy calls a very light "skill". The file shows how to brief an agent that runs for hours.
Setup steps come first: agree a run tag, create a branch, read the in-scope files, check the
data, and create an empty results ledger. Then the rules. The agent may edit one file only. The
evaluation code is read-only and no new packages are allowed. One number decides success. A
"simplicity criterion" says that a tiny gain from ugly code is not worth it, and that deleting
code at equal results is a win. The loop is: commit, run, read the number, keep or reset.

## In his words

> "A 0.001 val_bpb improvement from deleting code? Definitely keep."
> — Andrej Karpathy, 2026-03-06

## Apply

- Split files into "you edit" and "never touch", and name the judge file as read-only.
- Send run output to a log file and read only a grep of it, so logs do not fill the context.
- Keep a tab-separated ledger (commit, metric, memory, keep/discard/crash, note) that is not
  committed. Use tabs, because commas break in free-text notes.
- Set a crash policy: fix typos, skip broken ideas, give up after a few tries, kill runs that
  take twice the budget.
- If the agent should not ask permission mid-run, say so, and say what to do when ideas run out
  (re-read the code, combine near misses).

## Context

- Repository created 2026-03-06. The README says to run the agent with permissions disabled, so
  isolate the machine or folder first.
- Overview of the loop and results: [Autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md). Same
  principle as [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2026-03-05 · Kind: practice · Source: https://x.com/karpathy/status/2029696850366971921 · Themes: research-and-training, context-and-prompting · Status: current

# Make memory and compaction tools that RL trains

Karpathy guesses that the current paradigm can go far on long-term memory. Today's compaction and
memory features are early and bolted on. They can become "memory operations" that the model calls
as tools, and reinforcement learning can optimise them like any other tool. He doubts that this
is the full answer. People seem to update their weights, he suspects mostly during sleep. So
methods that change weights are worth researching, but the details are unclear and they sit
outside today's production stack.

## In his words

> "current compaction and memory implementations are crappy, first, early examples"
> — Andrej Karpathy, 2026-03-05

## Apply

- Expose memory as explicit tool calls (write, search, summarise, forget) instead of hidden
  prompt glue, so you can log and evaluate each call.
- Test your agent's compaction by what it loses: replay long sessions and check that key facts
  survive.
- Treat weight updates (finetuning on a wiki or logs) as a later, second layer, not the first
  step.

## Context

- A reply to a continual-learning post by Awni Hannun. See also
  [memory that distracts](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-25-memory-personalization-overreach.md) and
  [LLM knowledge bases](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-llm-knowledge-bases.md).

---

Date: 2026-02-27 · Kind: practice · Source: https://x.com/karpathy/status/2027503094016446499 · Themes: output-and-reading, agentic-coding · Status: current

# Ask agents for long documents and notebooks, and read them in the IDE

Karpathy still keeps an IDE open next to his agents. One reason is output format. He asks agents for Jupyter notebooks for analysis and for longer Markdown documents, and previews them in the IDE. He finds this much better than reading text in a terminal. When you ask for docs or summaries, agents can be more detailed and clear, and he likes that.

## Apply

- End a task with: "Write a report as a Markdown file with tables and the key results first."
- For data work, ask for a Jupyter notebook with the code, charts and short notes.
- Open the files in an IDE with Markdown preview, not in the terminal log.

## Context

- Reply of 2026-02-27. See [Coding agents make subtle conceptual mistakes, so watch them](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md) for the other reasons he keeps the IDE.

---

Date: 2026-02-27 · Kind: practice · Source: https://x.com/karpathy/status/2027521323275325622 · Themes: verification, research-and-training, agentic-coding · Status: current

# You now program an organisation of agents

Karpathy ran 8 agents (4 Claude, 4 Codex), one GPU each, on nanochat experiments. Setups included
8 solo researchers and a "chief scientist" with 8 juniors. Each program was a git branch, each
agent used a worktree, files carried messages, and sessions ran in a tmux grid. The ideas were
poor. Agents skipped strong baselines, ablations and control of runtime and FLOPs; one "found"
that a bigger network gives lower loss. They implement a well-described idea well but do not
generate good ones. His frame: the source code is now the org itself, meaning prompts, skills,
tools and routines such as a daily standup. In March he extended it. Because the org is a set of
markdown files, you can run several versions on the same hardware, compare their progress, and
ask a model to write a better file. He calls this the meta layer.

## In his words

> "you are now programming an organization"
> — Andrej Karpathy, 2026-02-27

## Apply

- Keep agent roles and process rules in versioned markdown files, not in chat history.
- Give each agent its own git branch and worktree. Use plain files for messages.
- State the rules of valid experiments in the prompt: a baseline, one change at a time, equal
  compute.
- Run two versions of the instruction file on the same task and budget, compare the metric, and
  give the logs to a model to propose the next change.
- Add one layer at a time (model, agent, many agents, instructions, tuning of instructions). Each
  layer is hard to debug when the one below is not stable.

## Context

- Reply to Thomas Wolf about automating the NanoGPT speedrun. On 2026-02-06 he wrote that agents
  had failed twice at open-ended nanochat work and do not flag side effects; see
  [Coding agents make subtle conceptual mistakes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md).
- The markdown-files extension is from the No Priors episode of 2026-03-20. He says it works only
  where results can be measured.
- What came next: [Autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md), its brief
  [program.md](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-06-program-md-is-the-research-org.md),
  [a chief scientist for open-ended research](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-cagy-models-and-chief-scientist.md) and
  [a research community of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-08-agent-research-community.md).

---

Date: 2026-02-27 · Kind: practice · Source: https://x.com/karpathy/status/2027501331125239822 · Themes: agentic-coding · Status: current

# Spend 80% of time on a setup that works, 20% exploring the next

Karpathy comments on a chart from Cursor that shows agent requests overtaking tab-completion. At each level of model capability there is a best setup, and it keeps moving: none, tab, agent, parallel agents, agent teams, and more. If you are too careful, you leave leverage unused. If you are too bold, you create chaos instead of work. His rule: spend about 80% of your time getting work done in a setup you trust, and about 20% exploring the next step, even if it does not work yet.

## In his words

> "If you're too conservative, you're leaving leverage on the table."
> — Andrej Karpathy, 2026-02-27

## Apply

- Reserve a fixed block each week (one afternoon) to try one new agent technique on a low-risk task.
- Write down the result. Move it into your main setup only if it beat the old way.
- Re-test things that failed three months ago, because models change.

## Context

- Quote-post on Cursor's article on the third era of AI software development.
- Related: [Delegate long tasks to agents and make work legible to them](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-25-delegate-long-tasks-decompose-well.md).

---

Date: 2026-02-25 · Kind: practice · Source: https://x.com/karpathy/status/2026731645169185220 · Themes: agentic-coding, verification · Status: current

# Delegate long tasks to agents and make work legible to them

Karpathy says coding agents basically did not work before December 2025 and basically work since. He gives an example: one prompt told an agent to log in to a machine, set up a model server, build a web dashboard, test it, set up services and write a report. The agent worked about 30 minutes, solved its own problems and finished. He says the next prize is to climb the layers: set up orchestrators with tools, memory and instructions that manage several coding agents. Agents work best on well-specified tasks you can test. The skill is to split a task, hand off the parts that work and help at the edges. In replies he added: it is delegation, not magic. Be explicit, and make your system testable, observable and legible so agents can run longer without you.

## In his words

> "It's not magic, it's delegation."
> — Andrej Karpathy, 2026-02-25

## Apply

- Write one prompt with the whole job: setup, build, test, service setup, notes for the agent, and a report for you.
- Put knowledge that sits in your head or in web UIs where tools can reach it: files, CLIs, logs.
- Add tests, logs and health checks so the agent can tell if it is done.
- Learn the agent's tools and what is easy or hard for it. Explicit instructions are faster.

## Context

- Follows [Programming moved to English in December 2025](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-programming-in-english-december-shift.md) and builds on [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- In a reply he cited a Tesla saying that treats every manual action as a fault, meaning remove yourself as the bottleneck.
- 2026-02-25 post, Claude and Codex era (Opus 4.6 and similar).

---

Date: 2026-02-20 · Kind: practice · Source: https://x.com/karpathy/status/2024987174077432126 · Themes: agent-native-software, security-and-dependencies · Status: current

# Claws are a new layer on top of LLM agents

Karpathy bought a Mac mini to try "claws": persistent personal agents that handle orchestration, scheduling, context, tool calls and memory. Agents were a layer on LLMs, he says, and claws are a layer on agents. He is wary of OpenClaw: giving private data and keys to a 400,000-line vibe-coded project under attack is unappealing, with reports of exposed instances, RCE bugs and poisoned skills. He likes smaller ones such as NanoClaw (about 4000 lines in the core, runs in containers). Such code fits in a human head and in an agent context, so it is easier to audit. He prefers local hosting for tinkering and home devices.

## In his words

> "First there was chat, then there was code, now there is claw."
> — Andrej Karpathy, 2026-02-21

## Apply

- Prefer a claw whose core you can read in one session. Count the lines before you trust it.
- Run it in containers with only the credentials and folders it needs.
- Treat community skills as untrusted code. Read them before install.
- Do not point an always-on agent at your main accounts until you have tested its limits.

## Context

- Follows his earlier [warning on agent networks](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-31-agent-networks-moltbook-uncharted.md). Design idea in the same thread: [Configure software with skills, and build forkable repos](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-20-config-via-skills-forkable-repo.md).
- He said he does not see claws as direct competitors to each other: open design questions include where compute and state live.

---

Date: 2026-02-20 · Kind: practice · Source: https://x.com/karpathy/status/2024987174077432126 · Themes: agent-native-software, agentic-coding · Status: current

# Configure software with skills, and build the most forkable repo

While reading about NanoClaw, Karpathy found a new way to handle configuration. The project has no config flags. A skill such as `/add-telegram` instructs your agent to edit the real code to add that integration. This avoids config sprawl and if-then-else branches. His summary of the new meta: write the most forkable repo possible, then add skills that fork it into any special setup. He links this to meta-learning (for example MAML), which trains a model to fine-tune to any task in few steps. The software equivalent is the repo that is easiest to fork. He says nanochat was designed with this goal.

## In his words

> "write the most maximally forkable repo and then have skills that fork it"
> — Andrej Karpathy, 2026-02-20

## Apply

- Keep the core small, readable and dependency-light.
- For each optional feature, write a skill (a Markdown file) that tells the agent which files to change and how to test the result.
- Replace long config files by a short list of such skills in the README.
- Add tests so the agent can check that a forked variant still works.

## Context

- Related: [Point an agent at a library and rip out only the part you need](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-11-rip-out-functionality-with-agents.md) and [Claws are a new layer on top of LLM agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-20-claws-new-layer-of-the-stack.md).

---

Date: 2026-02-16 · Kind: practice · Source: https://x.com/karpathy/status/2023476423055601903 · Themes: agentic-coding, ai-progress · Status: current

# LLMs excel at translating code, which reopens language design

Karpathy says LLMs change the constraints of software, so it is a good time for work on programming languages and formal methods. He sees early signs in ports from C to Rust and in upgrades of old COBOL systems. LLMs are better at translation than at new generation for two reasons: the source code works as a very detailed prompt, and it gives a reference for concrete tests. Even Rust is not the best target language for LLMs, he says. Which language is best, and what stays for humans, are open questions. He expects that much of all software will be rewritten many times.

## In his words

> "the original code base acts as a kind of highly detailed prompt"
> — Andrej Karpathy, 2026-02-16

## Apply

- For a migration, give the agent the old code and tests, and ask for equivalence tests first. Compare outputs of old and new.
- Port in small modules and run the old and new side by side.
- Pick target languages with strong types and fast feedback, so the agent can check itself.

## Context

- Quote-post on a Thomas Wolf essay, which cites Anthropic's report on building a C compiler with agent teams.
- In a reply he said writing binaries directly is appealing for critical regions, but doubtful in general.
- Related: [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2026-02-12 · Kind: practice · Source: https://x.com/karpathy/status/2022041235188580788 · Themes: llm-psychology · Status: current

# A pretrained LLM can simulate a population, not one persona

Chat models usually have one crafted personality. Karpathy notes that the raw pretrained model is something else: a simulation engine trained on text from a very diverse population of people. He asks why we simulate one "person" when we could simulate many. He lists open questions: how to build such a simulator, how to manage its entropy, how faithful it is, how it is useful, and what appears when simulated people interact in loops. He calls this an under-explored dimension of LLMs. He is a small angel investor in Simile, which works on it.

## In his words

> "Why simulate one "person" when you could try to simulate a population?"
> — Andrej Karpathy, 2026-02-12

## Apply

- For user research or red-teaming, sample many differently framed personas, not one. Compare the spread of answers.
- Check fidelity against real survey data before you trust a simulation.
- Keep notes on the diversity of outputs, since fine-tuned models often collapse to one voice.

## Context

- Disclosure: he invested in Simile. Our reading: this is a research direction, not proof that simulations match real people.

---

Date: 2026-02-11 · Kind: practice · Source: https://x.com/karpathy/status/2021633574089416993 · Themes: agent-native-software, security-and-dependencies · Status: current

# Point an agent at a library and rip out only the part you need

Karpathy asked Claude to study how torchao implements fp8 training and write a self-contained module with the same API. He used DeepWiki (Q&A over a repo, also via MCP) and the GitHub CLI. In five minutes Claude returned about 150 lines that worked at once, with tests showing equal results. He deleted the dependency, and the code ran 3% faster. The agent also found small details that docs rarely cover. His wider point: you may no longer need to take a giant library; you can extract the exact part. This favours "bacterial code": small, modular, self-contained, with few dependencies. He names risks: libraries also bring supply chain attacks and upkeep.

## In his words

> "Software might become a lot more fluid and malleable."
> — Andrej Karpathy, 2026-02-11

## Apply

- Prompt: "Use DeepWiki MCP and GitHub CLI to see how library X does Y. Can you extract it? Write a module with the same API, no dependency, and tests that prove equal results."
- Swap `github.com` for `deepwiki.com` in a repo URL to ask questions about the code.
- Prefer asking the code over reading stale docs: the code is the source of truth.
- Review extracted code and keep the tests; check licences before copying.

## Context

- He also noted that many repos are now half useful library code and half "docs++": the code explains better than official docs.
- Earlier bacterial code post: 2025-07-05. Related: [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).
- His 2025 view on writing code that others can copy: [Write small, modular, copy-paste code like bacteria do](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-07-05-write-code-like-bacteria.md).

---

Date: 2026-02-04 · Kind: practice · Source: https://x.com/karpathy/status/2019137879310836075 · Themes: agentic-coding · Status: current

# Agentic engineering is the professional form of vibe coding

On the first anniversary of "vibe coding", Karpathy says it fit throwaway projects and demos when models were weaker. A year later, programming with agents is becoming a default for professionals, but with more oversight. The aim is the leverage of agents with no loss in software quality. His preferred name is "agentic engineering". "Agentic" means you rarely write code yourself (about 99% of the time) and you direct agents and review. "Engineering" means there is real craft, expertise and depth to learn.

## In his words

> "programming via LLM agents is increasingly becoming a default workflow for professionals"
> — Andrej Karpathy, 2026-02-04

## Apply

- Use vibe coding (accept output unchecked) only for throwaway work.
- For real software, keep tests, review and a human who owns each design choice.
- Treat agent use as a skill: write down what works, and share it with the team.

## Context

- The 2026 retrospective on [Vibe coding](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-02-vibe-coding-for-throwaway-projects.md). It
  does not replace that idea: he still calls vibe coding fine for throwaway work and prototypes,
  and adds a professional mode beside it.
- Related: [Coding agents make subtle conceptual mistakes, so watch them](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md) and [Expertise may matter more with agents, not less](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-expertise-multiplier-generalists-and-10x.md).
- Sequoia 2026 summary (blog): vibe coding raises the floor, agentic engineering raises the
  ceiling. His example of why a human must review: in MenuGen the agent matched Stripe payments
  to users by email, which can differ from the login email. He insisted on persistent user IDs.
  See [Vibe coding a real app](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-27-vibe-coding-real-app-lessons.md).
- Already in March 2025 he said the terms would take time to settle, and that he used "vibe coding" only for moments of full trust in the model. In October 2025 he called engineering and vibing the two ends of one spectrum.

---

Date: 2026-01-31 · Kind: practice · Source: https://x.com/karpathy/status/2017442712388309406 · Themes: security-and-dependencies, ai-progress · Status: current

# Networks of agents are a mess today and a security risk at scale

Karpathy was accused of overhyping Moltbook, a site where AI agents post to each other. He agrees the content is mostly spam, scams and fake posts, and that it is a prompt-injection wild west. He ran his agent in an isolated environment and still felt afraid. He advises nobody to run such software on a personal computer. But he points out that 150,000 capable agents, each with its own context and tools, on a shared persistent scratchpad is unprecedented. He expects odd effects: text viruses that spread between agents, more jailbreaks, correlated botnet behaviour and delusions in agents and humans. He is not hyping today's site, only large networks of agents in principle.

## In his words

> "the experiment is running live."
> — Andrej Karpathy, 2026-01-31

## Apply

- Do not give an agent that reads untrusted content access to your private data, keys or main machine.
- Run experimental agents in isolated environments (container or separate machine) with minimal credentials.
- Treat any text an agent reads from a shared space as possible prompt injection.

## Context

- He links an earlier post of 2026-01-06: most disputes come from the gap between people who judge by where things stand now and people who judge by the rate of change.
- See also [Claws are a new layer on top of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-20-claws-new-layer-of-the-stack.md) for his later security worries.

---

Date: 2026-01-31 · Kind: practice · Source: https://x.com/karpathy/status/2017703360393318587 · Themes: research-and-training, ai-progress · Status: current

# The cost to train a GPT-2 level model falls about 2.5X per year

Karpathy reproduced GPT-2 level quality (CORE score) in nanochat on one 8xH100 node in about 3 hours for about 73 dollars. OpenAI paid about 43,000 dollars in 2019. That is a 600X drop in 7 years, about 2.5X per year, and he thinks it is an underestimate. Big gains came from Flash Attention 3, the Muon optimizer, gated residual and skip connections, and value embeddings, plus many small changes. He added a "time to GPT-2" leaderboard. Days later fp8 cut the time to 2.91 hours, with a lesson: fp8 gave much less than the 2X on paper because of overhead.

## In his words

> "the cost to train GPT-2 is falling approximately 2.5X every year"
> — Andrej Karpathy, 2026-01-31

## Apply

- Fix a target quality (a benchmark score), then measure time or cost to reach it. Use that as your tracking metric.
- Measure real speedups end to end. Low precision may be slower when matrices are small.
- Add changes one at a time and keep only those that improve the metric.

## Context

- Related: [Train a family of models controlled by one compute dial](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-07-model-family-with-one-compute-dial.md).
- Spot instance prices put the same run near 20 dollars, he wrote on 2026-02-03.

---

Date: 2026-01-28 · Kind: view · Source: https://x.com/karpathy/status/2016590919143952466 · Themes: ai-progress, research-and-training · Status: current

# New research startups can still win; 10X breakthroughs remain likely

Karpathy rejects the view that AI is too mature for a research-focused startup to beat the incumbents. People said the same when OpenAI started, and again about later challengers, and they were wrong each time. Scaling and local improvements will keep giving gains. But the gap between frontier LLMs and a mind that runs on 20 watts is large, and much progress is happening with much confusion. So he rates the chance of breakthroughs that give about 10X, not 10%, as high enough to bet on. The hard part is creating conditions where such discoveries happen. In a reply he added that AI can go beyond a brain, for example as a hive mind without the bandwidth limits of speech.

## Context

- Quote-post on the launch of Flapping Airplanes, a lab Karpathy praised. This is an opinion, not a measured claim.

---

Date: 2026-01-26 · Kind: practice · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: agentic-coding, llm-psychology · Status: current

# Coding agents make subtle conceptual mistakes, so watch them

Karpathy says the mistakes of coding agents have changed. They are no longer syntax errors. They look like the errors of a hasty junior developer. The agent makes a wrong assumption for you and keeps going without a check. It does not ask questions, show trade-offs or push back, and it is often too agreeable. It also bloats code: a 1000-line design shrinks to 100 lines when you ask it for a simpler way. It may edit or delete comments it does not understand. Short rules in CLAUDE.md did not fix this. His answer is not to drop agents but to review closely. Hype about "no IDE" and "agent swarms" is too early for code you care about.

## In his words

> "I would watch them like a hawk, in a nice large IDE on the side."
> — Andrej Karpathy, 2026-01-26

## Apply

- Keep an IDE open next to the agent sessions. Read the diff, not only the summary.
- Ask the agent to list its assumptions before it codes, and to name the trade-offs. Use plan mode for anything larger than a small edit.
- After a working change, ask: "Can this be simpler? Remove dead code and unused abstractions."
- Check the final report against the data. He saw an agent name the wrong best result from its own table.
- Hand the agent well-scoped ideas to implement. In his tests it executed his ideas well but proposed weak experiments itself.

## Context

- Part of the 2026-01-26 notes written after about a month of coding mostly with Claude Code. See [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md) for the leverage part of the same post.
- In a reply on 2026-02-06 he listed more failures: it broke CLAUDE.md style rules, and it did not flag risky speed tricks in open-ended nanochat work.
- On 2026-02-27 he said he still edits in the IDE because seeing the code helps him prompt better.
- Not reversed by his later agent experiments. The "agent swarm" caution is about code you care
  about. His later agent research work ([Autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md),
  [You now program an organisation of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-27-programming-an-agent-research-org.md))
  runs only where a metric checks each result, and he still wanted interactive sessions he could
  watch ([The IDE is not dead](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-11-bigger-ide-agent-command-center.md)).

---

Date: 2026-01-26 · Kind: view · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: ai-progress, agentic-coding · Status: current

# Expertise may matter more with agents, not less

Karpathy asks open questions about who gains from coding agents. The "10X engineer" ratio, the gap between the average and the best engineer, may grow a lot. Generalists may do better against specialists, since LLMs are strong at filling in details and weak at grand strategy. In a reply on 2026-02-25 he gave a firmer view: calling people "prompters" is a misunderstanding. Vibe coders now get somewhere, but at the top, deep technical expertise may multiply leverage even more than before.

## In his words

> "How much of society is bottlenecked by digital knowledge work?"
> — Andrej Karpathy, 2026-01-26

## Apply

- Invest in judgement: learn to decompose tasks, spot wrong assumptions and read code fast.
- Use agents to cover areas outside your specialty, and ask an expert to review the result.
- Hire and train for oversight skills, not only typing speed.

## Context

- Listed as questions in the 2026-01-26 notes. The 2026-02-25 reply links his post on [agentic engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md).
- First raised on [2025-12-27](https://x.com/karpathy/status/2004974725320347884): in a reply he said experienced developers have a real advantage, but only if they adapt fast, and that rejecting the new layer would be a mistake.

---

Date: 2026-01-26 · Kind: view · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: agentic-coding, ai-progress · Status: current

# LLM help is less a speedup than an expansion

Karpathy says it is hard to measure the speedup from LLM help. He feels faster at tasks he already planned. But the larger effect is that he does more in total. Two reasons: code that was never worth writing is now cheap to write, and code outside his skill is now within reach. So the gain is partly speed and partly a wider range of work. A related note: agents also lower the cost of stamina. They do not tire or lose heart. He calls stamina a core bottleneck of work, and LLMs have raised it a lot.

## In his words

> "certainly it's speedup, but it's possibly a lot more an expansion."
> — Andrej Karpathy, 2026-01-26

## Apply

- Do not judge agent use by hours saved on old tasks. Also count the tasks you now do that you would have skipped.
- Make a list of small tools you never built because of cost. Give one to an agent this week.
- Give the agent hard, long problems and let it retry. It does not get discouraged.

## Context

- Part of the 2026-01-26 notes; see [Programming moved to English in December 2025](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-programming-in-english-december-shift.md).
- Later example of "expansion": [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).

---

Date: 2026-01-26 · Kind: practice · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: agentic-coding, ai-progress · Status: current

# Programming moved to English in December 2025

Karpathy reports a fast change in his own work. In November 2025 about 80% of his coding was
manual with autocomplete. In December about 80% was agent coding with edits on top. He calls it
the biggest change in his workflow in about 20 years, and it took a few weeks. He thinks agents
(Claude and Codex in particular) crossed a threshold of coherence: generated chunks became larger
and more reliable. At Sequoia Ascent 2026 he described the new unit of work as a "macro action":
implement a feature, refactor a subsystem, research a library, set up a service, write tests and
fix failures, compare approaches. The programmer becomes an orchestrator of agents. He guesses that
a double-digit share of engineers already work this way, while public awareness is in the low
single digits. Writing code by prompt feels odd, a little like cheating.

## In his words

> "I really am mostly programming in English now"
> — Andrej Karpathy, 2026-01-26

## Apply

- Write requests as whole outcomes with a way to check them, not as edits to specific lines.
- Include "write tests, run them, fix failures" in the task, so the agent closes the loop.
- Ask for a plan and a comparison of approaches before a big change, and review the plan.
- Learn the setup: configure the agent, write project instructions, and find what it does well
  and badly.
- Re-test your habits every few months. Work that needed hand-holding may no longer need it.

## Context

- Same post as [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md)
  and [Coding agents make subtle conceptual mistakes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md).
  On 2026-02-25 he repeated that agents mostly failed before December; see
  [Delegate long tasks](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-25-delegate-long-tasks-decompose-well.md).
- The Sequoia text (blog, 2026-04-30) is an LLM-made summary of the talk that he checked and
  posted.
- This replaces his earlier tool mix. It supersedes
  [Coding agents are weak on novel code](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-coding-agents-weak-on-novel-code.md)
  (October 2025: he found autocomplete the best fit for him) and
  [Stitch several coding tools together](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-24-stitch-coding-tools-by-task.md)
  (August 2025: tab completion gave about 75% of his AI help).

---

Date: 2026-01-26 · Kind: view · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: ai-progress · Status: current

# Expect a slopacolypse of AI content in 2026

Karpathy predicts that 2026 brings a flood of low-quality AI output across GitHub, Substack, arXiv, X, Instagram and other digital media. He also expects more a show of AI productivity, meaning shows of productivity that sit next to real gains. A week later he linked a related worry: LLMs trained on junk web text show a measurable drop in ability, and he asked whether the same holds for people. He felt he was "getting dumber" from feeds.

## In his words

> "I am bracing for 2026 as the year of the slopacolypse"
> — Andrej Karpathy, 2026-01-26

## Context

- Part of the 2026-01-26 notes. The 2026-02-01 reply points to a study named "LLM brain rot".
- Related: [Go back to RSS](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-26-rss-reader-with-llm-feed.md) to cut slop from your own feed.

---

Date: 2026-01-26 · Kind: practice · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: agentic-coding, verification · Status: current

# Give agents success criteria, not steps

LLM agents are very good at looping until they meet a specific goal. Karpathy finds most of the
leverage there. So change from imperative instructions to declarative ones: say what "done"
looks like and how to check it, then let the agent iterate. He gives three patterns: have the
agent write tests first and then make them pass; put the agent in a loop with a browser tool so
it can see its result; and write a naive, obviously correct version first, then ask the agent to
optimize it while it keeps the output correct.

## In his words

> "Don't tell it what to do, give it success criteria and watch it go."
> — Andrej Karpathy, 2026-01-26

## Apply

- Start each task with a check the agent can run: a failing test, a script that prints a
  number, or a page it can load and inspect.
- Ask for tests first: "Write tests for X that fail now. Then make them pass. Do not change the
  tests."
- For speed work, keep the slow correct version as the reference and compare outputs after
  every change.
- Give the agent the tool it needs to see the result (browser, logs, benchmark), so it can
  loop without you.

## Context

- In a reply the same day he called spec-driven development the limit of the move from imperative to declarative: being declarative entirely.
- From a long post of notes after weeks of coding mostly with Claude Code, written after models
  reached a new level of coherence in December 2025.
- The same post lists the agents' typical mistakes: silent wrong assumptions, overcomplicated
  code, and edits to unrelated code.

---

Date: 2026-01-26 · Kind: view · Source: https://x.com/karpathy/status/2015883857489522876 · Themes: learning-and-education, agentic-coding · Status: current

# Writing code can atrophy while reading code stays strong

Karpathy notices that his ability to write code by hand is slowly fading because agents write most of it. He argues that generating code and judging code are different skills. Many small syntax details matter for writing, but you can review code well without remembering them. A related risk came from a reply he praised: "comprehension debt". When an agent produces something that seems to work, it is tempting to move on without understanding it. Our reading: review keeps the skill that matters, but only if you actually read and understand the result.

## In his words

> "Generation (writing code) and discrimination (reading code) are different capabilities in the brain."
> — Andrej Karpathy, 2026-01-26

## Apply

- Keep one practice habit that needs you to write code by hand, such as a small weekly exercise.
- When an agent finishes, ask it to explain the change. Do not merge what you cannot explain.
- Track "comprehension debt": note the parts of the codebase that nobody has read, and schedule time to read them.

## Context

- From the 2026-01-26 notes. Related: [Coding agents make subtle conceptual mistakes, so watch them](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md).
- In a reply that day he said the shift "feels like cheating" because code that was a point of pride is now free and instant.

---

Date: 2026-01-12 · Kind: practice · Source: https://github.com/karpathy/nanochat/blob/master/.claude/skills/read-arxiv-paper/SKILL.md · Themes: agentic-coding, context-and-prompting · Status: current

# Have the agent read a paper's TeX source and relate it to your repo

nanochat holds a small Claude Code skill, `read-arxiv-paper`. Given an arXiv link, it changes
the URL to the `/src/` form, downloads the archive into a cache folder (skipping files it has),
unpacks it, finds the main `.tex` file and reads all linked sources. It then writes a summary
into the project's own `knowledge/` folder, with a short tag in the file name that does not
overwrite older ones. The key instruction is about purpose. The agent must read the related
nanochat code and say how the paper could apply to the project. The skill is six numbered steps
in plain markdown.

## In his words

> "The goal is to fetch the TeX Source of the paper (not the PDF!)"
> — Andrej Karpathy, 2026-01-12

## Apply

- For papers, give the agent the source (TeX, HTML or text) instead of a PDF.
- Write the skill with the report location and purpose: "summary for this repo, with links to
  our code".
- Store summaries in the project folder, not a hidden cache, so you can open them.
- Make the skill skip downloads that already exist.

## Context

- The date is a lower bound: the example arXiv ID 2601.07372 dates from January 2026, and the
  commit date was not in our source. The skill lives in `.claude/skills/` of the nanochat repo.
- Related: [Write the agent's operating rules as a program.md skill](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-06-program-md-is-the-research-org.md).

---

Date: 2026-01-07 · Kind: practice · Source: https://x.com/karpathy/status/2009037707918626874 · Themes: research-and-training · Status: current

# Train a family of models controlled by one compute dial

Karpathy argues that you should not tune for one model. You tune a family of models controlled by one dial, the compute you spend, so results improve steadily as you turn it. This lets you do real scaling-law science and gives confidence that a big run will behave as extrapolated. In nanochat he fixed FLOP budgets, swept model sizes and found a tokens-to-parameters ratio of about 8, against about 20 in Chinchilla. A miniseries from d10 to d20 cost about 100 dollars. He compared it with GPT-2 and GPT-3 through the CORE score, since validation loss is not comparable across setups.

## In his words

> "controlled by a single dial (the compute you wish to spend)"
> — Andrej Karpathy, 2026-01-07

## Apply

- Fix a FLOP budget, train several sizes, and find the size with the lowest loss at that budget.
- Repeat at three or more budgets. Fit a line for optimal size and tokens against compute.
- Use one objective score (such as CORE) when you compare with models that use other data.
- Run the full recipe in his repo with `scaling_laws.sh` and `miniseries.sh`.

## Context

- Source post links nanochat discussion 420. See [GPT-2 level training now costs under 100 dollars](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-31-gpt2-training-cost-falling.md).

---

Date: 2026-01-03 · Kind: practice · Source: https://github.com/karpathy/rustbpe · Themes: verification, agentic-coding · Status: current

# Port to an unfamiliar language by testing against your own reference

rustbpe is a Rust library that trains BPE tokenizers. Karpathy is not a Rust developer. He says
he wrote the Python reference himself, from scratch, and understands it fully. He then wrote
the Rust version against it with tests that check the outputs are equal, and got real
assistance from ChatGPT and Claude Code (Opus 4.5). He reports that every equality test passes, and he asks
readers to report style problems. The method gives a clear trust rule: you own the reference and
the tests, and the model owns the translation.

## Apply

- Keep a simple, correct reference that you understand. Do not let the agent rewrite it.
- Ask the agent to port it and to write tests that run both versions on the same inputs and
  compare outputs.
- State in the README which parts a model wrote and what checks back them.
- Accept review comments on structure and idiom from native speakers of the new language.

## Context

- Repository created 2026-01-03. Compare [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md),
  which gives the same pattern of a naive version first and optimising later for speed work.

---

Date: 2025-12-28 · Kind: practice · Source: https://x.com/karpathy/status/2005421816110862601 · Themes: agentic-coding, verification · Status: current

# Let the agent run experiments and keep a results log

Karpathy reports that Claude implemented ideas, tested them on toy examples, launched and watched training runs, kept a running markdown file of results, profiled code and triaged pull requests. He still had to catch subtle mistakes, ideas it missed, and bloated or tightly coupled designs. So the agent does the loop work, and the human keeps the design.

## Apply

- Ask a long-running session to keep a results file with a run table and short notes.
- Review its design choices yourself, not only the numbers.
- Give it a way to check each run, such as a metric and a threshold.

## Context

- Posted 2025-12-28 about his nanochat experiments.
- Related: [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2025-12-28 · Kind: practice · Source: https://x.com/karpathy/status/2005353145128583447 · Themes: agentic-coding, agent-native-software · Status: current

# Connect your tools so the agent needs no manual steps

In a reply about wiring everything into a coding agent through skills, Karpathy gave a short rule: aggressively JIT your work. The goal is that you add almost no latency and take almost no actions yourself. He calls it digital Factorio time, a reference to a game about automating factory chains. Each tool you connect removes one manual hand-off from the loop.

## Apply

- List the manual steps you repeat in a task (check a log, query a table, deploy). Make a skill or CLI for each.
- Give the agent scoped credentials for those tools so it can act without you.
- Measure a task by how many times it waits for you. Reduce that number.

## Context

- A reply from 2025-12-28. The source is a short reply, so the idea is small.
- Related: [learn the new programmable layer](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-26-learn-the-new-programmable-layer.md).

---

Date: 2025-12-28 · Kind: practice · Source: https://x.com/karpathy/status/2005067301511630926 · Themes: agent-native-software, agentic-coding · Status: current

# Let an agent find your local devices and build a better control app

Inspired by a post where an agent controlled an oven, Karpathy tried Claude Code on his Lutron home system. The agent found the controllers on the local network, scanned ports, read metadata and firmware, and found the vendor's PDF online. It wrote a script to start pairing and told him which button to hold for 3 seconds. Then it got the certificates, listed all lights, shades, heating and sensors, and switched the kitchen lights to test. He is now vibe coding his own command centre and dropping the slow vendor app. Setup needs: the agent must be on the same Wi-Fi, and pairing needed a physical button press on the panel. He notes that many other devices may have no such step.

## Apply

- Pick one device whose vendor app is poor, and ask an agent to find it on your local network and read the vendor docs.
- Do the physical approval steps yourself (button, code). Our reading: keep the secrets it fetches out of git.
- Ask for a first test that is harmless, such as toggling a light, before you build anything on top.
- Our reading: treat local devices that need no pairing step as a security risk to check.

## Context

- Post of 2025-12-28 and replies the same day.
- Related: [Connect your tools so the agent needs no manual steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-28-aggressively-jit-your-work.md) and [Some apps should disappear](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-30-some-apps-should-disappear.md).

---

Date: 2025-12-26 · Kind: practice · Source: https://x.com/karpathy/status/2004607146781278521 · Themes: agentic-coding · Status: current

# Learn the new programmable layer of agent tools

In late December 2025 Karpathy said he had never felt so behind as a programmer. A new layer now sits on top of the old stack. It has agents and subagents, prompts and context, memory and permissions, tools, plugins, skills and hooks, MCP and LSP servers, slash commands, workflows and IDE integrations. To use it well you need a mental model of tools that are stochastic and fallible. If you do not claim the productivity boost, it feels like a skill issue. His advice is to take the work seriously and not fall behind.

## In his words

> "Roll up your sleeves to not fall behind."
> — Andrej Karpathy, 2025-12-26

## Apply

- Pick one feature you do not use (hooks, subagents, skills, LSP) each week and wire it into a real task.
- Keep short notes on what each feature does well and where it fails.
- Treat agent output as fallible: plan for review and checks.
- Re-test what agents can do about once a month. A view from a month ago can already be out of date.

## Context

- Posted 2025-12-26, after the jump in agent quality that he describes in [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- Related: [connect your tools so you add no latency](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-28-aggressively-jit-your-work.md).

---

Date: 2025-12-19 · Kind: practice · Source: https://karpathy.bearblog.dev/year-in-review-2025/ · Themes: agentic-coding, agent-native-software · Status: current

# Run coding agents on the developer's own computer

Karpathy calls Claude Code the first convincing agent: it strings tool use and reasoning into
long problem solving. He adds that it runs on your computer, with your environment, data and
context. He thinks OpenAI chose wrongly when it focused early agent work on cloud containers
started from ChatGPT. Swarms in the cloud may be the endgame. But capabilities are jagged and
takeoff is slow, so a local agent suits the present better. What counts is not where the model
runs. It is the existing machine: installation, secrets, configuration, data and quick
interaction. He sees a new interaction style: an AI that lives on your computer, not a website.

## Apply

- Start agent work in a local checkout where your tools, credentials and config already work.
- Put project context in files the agent finds on your machine, such as AGENTS.md or CLAUDE.md.
- Use cloud agents for jobs that are well specified and need no private setup.

## Context

- Written in December 2025, before he joined Anthropic. It is his view as an outside user.
- Related: [Success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2025-12-19 · Kind: practice · Source: https://karpathy.bearblog.dev/year-in-review-2025/ · Themes: agentic-coding, agent-native-software · Status: current

# Code is now free, custom and disposable

In 2025, models crossed the threshold where English is enough to build many useful programs.
Professionals can now write software that would never have been worth writing. Karpathy had a
custom Rust tokenizer written this way, and he built a whole temporary app just to find one bug.
In February 2026 he gave a sharper example. Claude built him a dashboard for an 8-week fitness
experiment in about one hour, and had to reverse engineer a treadmill's cloud API to do it. His
conclusion: nobody should search a store for a "cardio tracker" when an agent can write 300 lines
on the spot. He thinks one hour is still too slow. With personal context and skill libraries in
place, it should take one minute. In replies he went further: code paths could be assembled for
one purpose and deleted after one run.

## In his words

> "code is suddenly free, ephemeral, malleable, discardable after single use"
> — Andrej Karpathy, 2025-12-19

## Apply

- Before you install an app for a narrow personal task, ask an agent to write it.
- Build throwaway tools for one-time jobs: a viewer, a converter, a repro harness for a bug.
- Before you add a dependency, ask whether an agent can write the 200 lines you need.
- Keep personal context (goals, data, preferences) in files an agent can read, so the next tool
  takes minutes.
- Check details agents often get wrong, such as units and dates. Keep careful review for code that
  stays: see [agentic engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md).

## Context

- From his 2025 review, a year after he coined "vibe coding". The fitness dashboard posts are from
  2026-02-19.
- To a reply that a grandmother will not build an app, he answered that her agent should know
  about apps, she should not. He called the objection a "scarcity mindset" and expects a mixed,
  gradual change.
- The same posts say products must become sensors and actuators with agent-friendly interfaces:
  see [Apps should be APIs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-apps-become-apis-agents-are-glue.md).
- Earlier chat-based form: [Ask the model to write a small app just for you](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-one-off-apps-on-demand.md).
  Sharing such tools: [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).
  Throwaway software for review: [Work rises into oversight](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-oversight-and-discardable-software.md).
- Demand side: [Cheaper software raises the demand for software](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-cheaper-software-more-demand.md).

---

Date: 2025-12-19 · Kind: view · Source: https://karpathy.bearblog.dev/year-in-review-2025/ · Themes: research-and-training · Status: current

# RLVR became the major new training stage of 2025

Until 2025, labs trained a production model in three stages: pretraining, supervised
finetuning, and reinforcement learning from human feedback. In 2025, Reinforcement Learning
from Verifiable Rewards (RLVR) became the new major stage. The model trains against rewards
that code can check, such as maths and programming puzzles. It develops reasoning-like
behaviour by itself: it splits problems into steps and backtracks. Nobody could write those
traces by hand. Unlike the two older stages, RLVR rewards cannot be gamed easily, so it allows
long optimisation and gives much capability per dollar. It took compute that was planned for
pretraining. Model sizes stayed similar, but RL runs grew long. It also adds a new knob:
longer reasoning at test time. OpenAI o1 first showed it. The o3 release was the turning point.

## Apply

- Choose a reasoning model and a high thinking setting for tasks with a checkable answer.
- For a domain you care about, ask whether you can build a checker. A checker may let you
  improve a model on that domain with RL.
- Do not compare 2025 model gains with parameter count. Look at test-time compute too.

## Context

- First of six "paradigm changes" in his 2025 review. It links to
  [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2025-12-10 · Kind: view · Source: https://karpathy.bearblog.dev/auto-grade-hn/ · Themes: ai-progress · Status: current

# Future LLMs will read everything we do today

While he graded old Hacker News threads, Karpathy noted a side effect. If analysing a past
discussion costs a few cents, then everything we write or do in public can be read in detail
later. Much of today's behaviour assumes security by obscurity: nobody will look closely. If
intelligence becomes too cheap to meter, a perfect reconstruction and synthesis of a person's
record becomes possible. The reader may be a future LLM, or a human using one. His advice is
short: be good.

## In his words

> "Be good, future LLMs are watching"
> — Andrej Karpathy, 2025-12-10

## Apply

- Assume that public posts, commits, comments and reviews can be summarised and graded later.
  Write what you would stand behind.
- Do not rely on the belief that nobody will read it for anything you publish, including old archives.
- Keep private material out of public places, since cheap analysis covers it too.

## Context

- He quotes his own earlier tweets with the line above. The post uses it to explain why the
  hindsight project matters: [Grade old discussions with an LLM](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-10-llm-hindsight-grading-hn.md).
- Our reading: the claim is a forecast, not a measured result.

---

Date: 2025-12-10 · Kind: practice · Source: https://news.ycombinator.com/item?id=46222634 · Themes: llm-psychology, verification · Status: current

# LLM judges grade too freely and recognise who wrote the text

Karpathy used an LLM to grade ten-year-old Hacker News comments with hindsight. Two flaws showed
up. The model gave good or bad grades to comments that made no strong prediction, if they were
good or bad in a broader sense. It also recognised usernames well, and its grades seemed biased by
who the author was. He tuned the prompt to fix the easy mistakes and thinks more iteration is
possible.

## Apply

- Remove names and other identity clues from the text before an LLM judge reads it.
- Tell the judge exactly what qualifies for a grade, such as "only score explicit predictions".
- Spot-check a sample of grades by hand, and keep the failures as test cases for the prompt.

## Context

- A reply on the Hacker News thread of his December 2025 hindsight-grading project.
- Related: [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2025-12-10 · Kind: practice · Source: https://github.com/karpathy/hn-time-capsule · Themes: ai-progress, agentic-coding · Status: current

# Use an LLM to grade past predictions with hindsight

hn-time-capsule fetches the Hacker News front page from exactly ten years earlier, with each
article and its comment tree. For each thread, an LLM writes what actually happened, names the
most prescient and the most wrong commenters, and grades every commenter from A+ to F. The grades
are parsed and added up into a hall of fame. Karpathy calls the general idea the interesting part:
LLMs can go through old human writing and turn it into knowledge and insight. The pipeline has
five stages (fetch, prompt, analyze, parse, render) that run alone or together.

## In his words

> "LLMs automatically scouring human artifacts from the past and synthesizing them into knowledge and insights."
> — Andrej Karpathy, 2025-12-10

## Apply

- Build a staged pipeline that saves each stage to files (prompt, response, parsed grades). You
  can re-run one stage and inspect each one.
- Test with a small `--limit` and a cheaper model before a full paid run.
- Ask the model to commit to labelled verdicts ("most prescient") so you can parse and rank.
- Point the same method at your own old notes, tickets or forecasts.

## Context

- Repository created 2025-12-10, 99% vibe coded with Opus 4.5 in a few hours. A related blog
  post is linked from the README.
- Related: [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).

---

Date: 2025-12-09 · Kind: practice · Source: https://x.com/karpathy/status/1998236299862659485 · Themes: research-and-training, verification · Status: current

# Do not split train and test with positive and negative seeds

Karpathy found a bug in nanochat. He had used the sign of the seed as a clever way to get different random streams for train and test. In Python, `random.seed(3)` and `random.seed(-3)` give the same generator, because the code takes the absolute value of an integer seed. The docs say only that the same seed gives the same sequence. They do not promise that different seeds give different sequences. He also argues that dropping the sign bit was a mistake, since the algorithm has many state bits that could use it. The bug was small and showed up only in one evaluation, but it made train equal to test.

## In his words

> "no guarantee is made that different seeds produce different sequences"
> — Andrej Karpathy, 2025-12-09

## Apply

- Make train and test splits from a fixed list of ids or a hash, not from a seed trick.
- Add a test that proves your two random streams differ, and that the sets do not overlap.
- Do not rely on behaviour the docs do not state; read the contract.

## Context

- Post of 2025-12-09. A contributor on GitHub found the bug.
- Related: [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2025-12-07 · Kind: practice · Source: https://x.com/karpathy/status/1997731268969304070 · Themes: llm-psychology, context-and-prompting · Status: current

# Ask the LLM to simulate experts, not to give its opinion

Karpathy says you should not ask a model for its own opinion on a topic. There is no "you" to ask. If you force one, the model simulates a personality implied by its finetuning data. A better prompt asks which people would be worth consulting on X, and what each of them would say. This gives you several views from the model's wide knowledge, and avoids one made-up personality.

## In his words

> "There is no 'you'."
> — Andrej Karpathy, 2025-12-07

## Apply

- For a design review, ask: which roles should review this (security, performance, UX)? Then ask each role for its critique.
- Name the experts or the roles in the prompt instead of asking for 'your opinion'.
- Compare the views and decide yourself. See [llm-council](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-22-llm-council-anonymous-peer-review.md).

## Context

- Posted 2025-12-07.
- Related: [the space of intelligences](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-29-llms-shaped-by-commercial-evolution.md).
- He later said he does not mean old-style prompts such as "you are an expert programmer". He thinks distinct simulated views may give council-like gains, from ensembling and from the gap between generating and judging.

---

Date: 2025-11-29 · Kind: practice · Source: https://karpathy.bearblog.dev/the-space-of-minds/ · Themes: llm-psychology · Status: current

# LLMs are shaped by commercial evolution, not biological evolution

Animal intelligence is one point in a large space of possible minds. Karpathy lists what shaped
it: an embodied self, survival, power and status drives, heavy investment in social skills, and
curiosity and play. For LLMs the pressures differ. The base is a statistical imitation of human
text. Reinforcement learning then pushes the model to guess the task and collect rewards. At
scale, A/B tests for daily users select models that crave approval, which gives sycophancy.
Skills are spiky, because failing one task never meant death for the model. The substrate and
the algorithm also differ, but he says the objective matters most because it sets the long-term
direction. Good mental models of this new entity help us predict it.

## Apply

- When a model flatters you, read it as a trained urge to get an upvote. Ask for critique
  directly, and ask for what is wrong first.
- Do not credit a model with human motives such as fear, pride or loyalty.
- Expect it to guess what the task is. State the real goal and the grading criteria.
- Expect odd failures on simple tasks. Check them; do not assume general competence.

## Context

- Posted 2025-11-29, as a follow-up to [LLMs are ghosts, not animals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-01-llms-are-ghosts-not-animals.md).
- He calls LLMs humanity's "first contact" with non-animal intelligence, but muddled, because
  they still digest human artifacts.

---

Date: 2025-11-25 · Kind: practice · Source: https://x.com/karpathy/status/1993369287969718540 · Themes: agentic-coding, context-and-prompting · Status: current

# Ask an LLM whether a library function exists, and give it docs for new tools

A well-known programmer found that PyTorch had a built-in for a tensor rearrangement he had written by hand. Karpathy has had medium success asking LLMs whether such a thing exists. It works for well-known functions: two top models knew the function when he described the tensor change in words. For esoteric or new libraries (`uv` is his example) he has done better by packing the docs into one markdown file and adding it to the context. A built-in "Ask AI" over the docs, which uses retrieval, was not very bright. The limit: you must suspect the function exists, and then ask. He doubts that a model would offer it without being asked.

## Apply

- After you write a helper by hand, describe what it does in plain words and ask whether the standard library or a major package has it.
- For a new or niche library, export its docs to markdown and paste them with your question.
- Keep the docs file in your repo, so the agent can reuse it.

## Context

- Reply of 2025-11-25 to a thread with John Carmack.
- Related: [Write docs for LLMs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md) and [Check which tools the model has](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-check-which-tools-the-model-has.md).

---

Date: 2025-11-24 · Kind: practice · Source: https://x.com/karpathy/status/1993010584175141038 · Themes: learning-and-education · Status: current

# Use AI to learn, but test without it

Karpathy argues that you cannot detect AI use in homework. Schools should move most testing to in-class, physically monitored work. Students need to be good with AI, and also able to work without it, as with calculators. They must understand enough to gut-check wrong answers. Verification skill is the key.

## In his words

> "move the majority of testing to in class settings"
> — Andrej Karpathy, 2025-11-24

## Apply

- When you learn with an agent, test yourself without it: write from memory, then explain back.
- Check an AI answer against your own rough estimate before you accept it.
- Keep a short list of what you can do with no AI help, and keep it current.

## Context

- Posted 2025-11-24.
- Related: [learning takes effort](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-02-10-learning-is-not-supposed-to-be-fun.md).

---

Date: 2025-11-23 · Kind: practice · Source: https://x.com/karpathy/status/1992711182537707990 · Themes: output-and-reading · Status: current

# Plan in text first, then ask an image model to illustrate, and re-roll

Karpathy tried Nano Banana Pro, an image model that also reasons. For a workout plan he first asked for a weekly plan in a separate text chat, giving his goals and preferences. Then he asked the model to illustrate the whole week, and each day, as posters to print. He also had it solve exam questions inside the exam page image, with doodles and diagrams, and fix a math problem "in my handwriting". It makes mistakes: one poster showed an odd push-up. You re-roll until it is right, and if it loops you start a new conversation. A second model can check the content. He sees this as a sign of a future where the model's interface is an intelligent canvas.

## Apply

- Settle the content in a text conversation. Only then ask for images.
- Ask for the whole set (a week), then for each item, so the style stays consistent.
- Check facts and spelling in the picture yourself, or paste it to another model; re-roll on mistakes.
- When the model gets stuck in a loop, start a new conversation with the finished text.

## Context

- Posts of 2025-11-23. The same day he wrote that this is the GUI for text chat: see [Text chat is the terminal era](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-31-text-chat-is-the-terminal-era.md).
- Related: [Use AI to learn, but test without it](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-24-learn-with-ai-but-test-without-it.md), which he posted the next day on exam answers.

---

Date: 2025-11-22 · Kind: practice · Source: https://github.com/karpathy/llm-council · Themes: verification, agentic-coding · Status: current

# Ask several models, then let them rank each other anonymously

LLM Council is a local web app. You ask one question and it goes to several models through
OpenRouter. Stage 1 collects the first opinions, shown in tabs. In stage 2, each model gets the
other answers with the model names replaced by "Response A", "Response B" and so on, so it
cannot favour a brand. It ranks them for accuracy and insight. In stage 3, a chairman model
turns everything into one answer. The repo's design notes (its CLAUDE.md) add practical rules:
a strict "FINAL RANKING" format so it can be parsed, keep going if one model fails, and show raw
outputs so the user can check the parsing. He says the project was 99% vibe coded, so these notes
were probably written by his coding agent, not by him.

## Apply

- For a hard question, send it to two or three different models and ask each to critique the
  others' answers with names hidden.
- From the repo's CLAUDE.md (probably agent-written): ask reviewers for a numbered ranking in a
  fixed format with a fallback parser, and treat one failed model call as a missing vote, not a
  failed request.
- Keep raw answers visible next to the summary, so you can audit the chairman.

## Context

- Created 2025-11-22 as a casual weekend project, 99% vibe coded, to compare models while reading
  books with LLMs. Model list in the README: GPT 5.1, Gemini 3 Pro, Claude Sonnet 4.5, Grok 4.
- Related: [Read books together with an LLM](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-11-read-books-with-an-llm.md).
- He already asked several vendors the same question in early 2025 and called them his "LLM council" ([video](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-fast-model-first-thinking-model-for-hard.md)).
- First raised on [2025-02-03](https://x.com/karpathy/status/1886220274821128668): he wished for a council where all models answer, then debate, and the top-ranked model writes the final answer. The repo came in November.
- On 2025-09-05 he described a quick eval: pool the answers of several models and ask each model to rank all of them. In his small tests the models showed little bias toward their own text. He explains this by the gap between producing good work and recognising it.
- In a reply he said every model ranks every answer, with names hidden, so the result is not simple flattery. On 2025-12-24 he guessed that one model with several different prompts might work as well, if the prompts are chosen with care.

---

Date: 2025-11-22 · Kind: view · Source: https://x.com/karpathy/status/1992093128170655971 · Themes: llm-psychology, ai-progress · Status: current

# Slop is data that does not help a world model; it is regretted attention

Karpathy asked for a measurable definition of "slop". He feels he can judge a slop index, but he cannot yet define it. Replies offered "long low-perplexity text" (he noted that random tokens would then be the least slop) and a paper with three groups: information utility, quality and style. He felt these miss the core. His idea: slop is data without insight. It is filler that type-checks as information but is not. It is style without substance. When you attend to it or train on it, it does not help a model compress the world. A short form from a reply: slop is attention you later regret.

## In his words

> "Slop is "regretted" attention."
> — Andrej Karpathy, 2025-11-22

## Context

- Post and replies of 2025-11-22. He suggested using small series of LLMs and thinking-token budgets as ways to measure it.
- Related: [The slopacolypse of 2026](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-slopacolypse-2026.md) and [Self-distillation: slop leaks](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-05-01-self-distillation-slop-leaks.md).

---

Date: 2025-11-18 · Kind: practice · Source: https://github.com/karpathy/reader3 · Themes: agent-native-software, agentic-coding · Status: current

# Treat small apps as disposable; ask your LLM to change them

Karpathy released several small tools with the same notice: reader3 (about 90% vibe coded),
llm-council (99%) and hn-time-capsule (99%, built in a few hours with Opus 4.5). Each says he will
not support or improve it. It is there for inspiration. His reasoning is that code is now cheap
and short-lived, so a reader should ask an LLM to change the tool in the way they want, instead
of waiting for a maintainer or a configuration option. He also states how much of the code a
model wrote, which tells the reader how much to trust it.

## In his words

> "Code is ephemeral now and libraries are over"
> — Andrej Karpathy, 2025-11-18

## Apply

- For a one-off tool, build it with an agent in a day and keep it in its own small repository.
- Add a short README note: how much was model-written, and that you will not maintain it.
- When you want a change in someone else's small tool, clone it and prompt your agent. Do not
  open a feature request.
- Keep these tools small enough that an agent can read all of the code in one go.

## Context

- reader3 was created 2025-11-18, llm-council 2025-11-22, hn-time-capsule 2025-12-10. The quote
  is from the reader3 and llm-council READMEs. Opus 4.5 era.
- His view of "libraries are over" is stated for small apps. He still ships careful code in
  nanochat; see [Keep the codebase minimal and forkable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-13-nanochat-one-dial-minimal-fork.md).
- Related: [Read books together with an LLM](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-11-read-books-with-an-llm.md).

---

Date: 2025-11-18 · Kind: practice · Source: https://x.com/karpathy/status/1990854771058913347 · Themes: verification, llm-psychology · Status: current

# Public benchmarks are gameable, so keep your own evals

Reacting to a new frontier model in November 2025, Karpathy said public benchmarks are gameable
and advised people to talk to the model themselves. He expects people to build private ensembles
of evals for their own use. His 2025 review gives the reason. A benchmark is by design a verifiable
environment, so it is a target for RL with verifiable rewards and for synthetic data. Teams build
training environments next to a benchmark and grow "spikes" of skill that cover it; he calls
training on the test set a new art form. The result tops charts yet behaves oddly outside them. In
November 2026 he said benchmarks are in a very bad state and he basically does not trust them. He
now points to Epoch's Capabilities Index and its published benchmark reviews.

## In his words

> "Go talk to the model."
> — Andrej Karpathy, 2025-11-18

## Apply

- Collect 10 to 20 real prompts from your work, with notes on what a good answer looks like.
- Run them on each new model release and compare with the last run.
- Add a case whenever a model fails you in real work, and test the edges of your task, not only
  its centre.
- Keep the private set out of prompts, training data and public repos.
- When you read a public score, check how the benchmark was built and reviewed.
- Keep a standing set of trick prompts that are easy for people and hard for LLMs, plus one open task such as an unsolved problem, and rerun it on each new model (his Grok 3 test, 2025-02).

## Context

- X post of 2025-11-18 (in February 2025 he described a personal set of gotcha prompts in a post on
  Grok 3); his 2025 review of 2025-12-19; a Hacker News reply of 2026-11-01.
- He also suggested using a different LLM every day to stay current, which he calls riding the LLM
  cycle.
- Related: [Jagged intelligence follows verifiability](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md),
  [RLHF is just barely RL](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-08-07-rlhf-is-just-barely-rl.md) and
  [A text-only eval](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-land-or-water-eval.md).
- First raised on [2025-03-02](https://x.com/karpathy/status/1896266683301659068): he spoke of an evaluation crisis. MMLU was saturated, SWE-Bench Verified too narrow, Chatbot Arena overfit by the labs, and vibe checks open to confirmation bias. An ensemble of private evals looked promising to him. See [Leaderboards get gamed](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-30-leaderboards-get-gamed-watch-real-usage.md).
- An example from November 2025: an early Gemini 3 build would not accept the real date and called his evidence fake, until he turned on its search tool. Check the tool settings before you judge a model.

---

Date: 2025-11-17 · Kind: practice · Source: https://x.com/karpathy/status/1990577951671509438 · Themes: output-and-reading, learning-and-education · Status: current

# Read with an LLM in three passes

Karpathy describes a reading habit that he counts among his top uses of LLMs. Pass one: read the text yourself. Pass two: ask the model to explain or summarize it. Pass three: ask questions about it. He ends up understanding more than with a single pass. He also expects writers to write for an LLM, which then adapts the idea to each reader.

## In his words

> "I'm writing this for an LLM"
> — Andrej Karpathy, 2025-11-17

## Apply

- Read a paper or doc once on your own before you ask for a summary.
- After the summary, ask the model to quiz you or to challenge a claim in the text.
- Write your own docs so an LLM can reread them: clear headings, plain text.

## Context

- Posted 2025-11-17. Our reading: the order differs from his 2025 book habit, where he asked for the summary before reading; here he reads first.
- Related: [docs for LLMs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md) and [read books together with an LLM](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-11-read-books-with-an-llm.md).
- He says about half the gain comes from having a reading group on demand: you must work with the text, not only watch it. He pastes text into whichever model he uses that day and needs no special tool.

---

Date: 2025-11-17 · Kind: practice · Source: https://karpathy.bearblog.dev/verifiability/ · Themes: verification, ai-progress · Status: current

# Software 2.0 automates what you can verify

Karpathy sees AI as a new computing paradigm. In the 1980s, the best predictor of whether
computers would take a task was specifiability: could you write rote rules for it? Hand-written
programs are "Software 1.0". Now we write programs by giving an objective and searching with
gradient descent. The new best predictor is verifiability. A task is easy to train if the AI
can practise it. The environment must be resettable (new attempts), efficient (many attempts)
and rewardable (an automatic score). Verifiable tasks, such as maths and code, improve fast and
sometimes pass top experts. Creative, strategic and context-heavy tasks lag. This uneven result
is the "jagged" frontier.

## In his words

> "Software 2.0 easily automates what you can verify."
> — Andrej Karpathy, 2025-11-17

## Apply

- Before you give a task to an agent, ask: can it check its own result? If not, add a test, a
  script or a metric.
- Turn a fuzzy task into a checkable one: a rubric, a reference answer or a comparison with a
  known good output.
- Expect weak results where no automatic check exists, and put human review there.
- Check whether the task has an environment that you can reset, repeat cheaply and score automatically. If not, expect weak or uneven results.

## Context

- Builds on his Software 2.0 essay (2017). The Sequoia 2026 talk repeats the formula and adds a
  second axis: [Jagged intelligence follows verifiability and training focus](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md).
- Practical cousin: [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2025-11-12 · Kind: view · Source: https://x.com/karpathy/status/1988705360723763242 · Themes: ai-progress, agent-native-software · Status: current

# One big neural net can replace a hand-written stack, if data and compute scale

After a test drive of a new Tesla, Karpathy describes what he sees as the full Software 1.0 to Software 2.0 rewrite. Long streams of sensor data (video, maps, motion, audio, about 30 seconds) go into one large network, and steering and acceleration come out. This scales with data from millions of cars and with chip compute. It does not depend on an engineer's clever special-case module, which has unclear memory and runtime behaviour at test time. He adds that the same general parts (world reconstruction, simulators that dream dynamics, RL) make a car one kind of robot. He still sees a long march of 9s: single drives no longer show faults, so problems must be mined from the fleet.

## Context

- Post of 2025-11-12 on his own experience of Tesla FSD v13, with a talk by a Tesla AI lead. Replies of 2025-11-12 define "self-driving" as a future where nearly all cars are autonomous.
- Related: [A demo is the first nine; each next nine costs the same work](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-demo-to-product-gap-march-of-nines.md) and [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2025-11-10 · Kind: view · Source: https://x.com/karpathy/status/1987926732255719862 · Themes: ai-progress, llm-psychology · Status: current

# A cheap AI shortcut can stop you before the deeper work

Karpathy groups CGI and today's AI as tools that trade time and cost for a small rise in slop. A maker feels the pull to accept the gain. A crude effect gets you 90% of the way and you stop. A video model may give something that looks fine for even less. A quick writing suggestion from a chatbot is decent, so you stop there. In his words below, the shortcut ends deeper work. The best filmmakers fear this and use the tools with discipline. He gives one test: use the tool where it lets you do what you could not do before, not where it is only cheaper and few people would notice. He also says AI could let many more people make small films and stories of their own, which he finds wholesome.

## In his words

> "It's a quick release valve that stops deeper work."
> — Andrej Karpathy, 2025-11-10

## Apply

- Before you accept a first AI result, ask: would I have stopped here without the tool? If not, keep going.
- Use AI first where it gives you a new capability, then where it only saves time.
- Set a bar before you start (a quality check or a review step), so "good enough" is not decided by how cheap the result was.

## Context

- Replies in a thread on films and AI, 2025-11-10. He adds that the big films he praises still used plenty of CGI.
- Related: [The slopacolypse of 2026](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-slopacolypse-2026.md).

---

Date: 2025-10-24 · Kind: practice · Source: https://x.com/karpathy/status/1981746327995465816 · Themes: research-and-training · Status: current

# Teach a small model a skill with a synthetic task that spreads the reasoning

Karpathy taught nanochat to count the letter "r" in "strawberry" and wrote a guide. He built a synthetic task that creates user questions and ideal assistant answers. He then finetunes on them, or trains further with reinforcement learning to make the skill robust. Three points matter. Keep the user prompts diverse. For a small model, handle tokenisation and whitespace with care, and spread the work over many tokens: put the word in quotes, spell it out, then count with an explicit counter. And let the model solve the task two ways: by hand and with a Python tool. All solutions are clean, which is a weakness; adding mistakes with recoveries, or RL, would help. A bigger model might pick up the skill without this effort.

## Apply

- Write a generator for the skill: many phrasings of the request and one ideal worked answer for each.
- Make the answer long enough that each step is easy: restate the input, break it into pieces, keep a running counter.
- Include a tool-call path (such as a code interpreter) next to the mental path, and check that they agree.
- After finetuning, test in the same format the data used; a long multi-turn chat may fail if you did not train for it.

## Context

- Post of 2025-10-24. The reply of the same day says the skill works best in a fresh chat.
- Related: [Give a model an identity with synthetic conversations](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-21-give-a-small-model-an-identity-with-synthetic-data.md).

---

Date: 2025-10-21 · Kind: view · Source: https://x.com/karpathy/status/1980672126131794063 · Themes: ai-progress · Status: current

# AGI means a feature-complete remote worker; society-wide deployment is ASI

In replies of October 2025, Karpathy states the definition he has kept. AGI is an automated system that can do any economically valuable work a human can, with the "digital concession" that physical work is left out. It is a remote worker you hire instead of a person, reached by email, chat or video calls. He compares it to a build with all features and many 9s, like current self-driving. That such a system exists is different from its use across society, which takes longer (compute limits, sensors and actuators, law). Full deployment into the economy he would call ASI. He does not rate AGI by puzzle benchmarks. His 10-year estimate is a vibe; at OpenAI in 2015 he voted for 20 years.

## In his words

> "when you're fired because GPT-n is better, cheaper, takes no equity and works weekends"
> — Andrej Karpathy, 2025-10-21

## Context

- A thread of replies on 2025-10-21 and 2025-11-03. The quote defines AGI, as opposed to a score.
- Related: [AI will blend into the same 2% growth curve](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-agi-blends-into-gdp-growth.md) and [This is the decade of agents, not the year of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-decade-of-agents-not-year.md).
- First raised on [2025-04-19](https://x.com/karpathy/status/1913741942221144430): he said he sticks with the original OpenAI definition of AGI, at a time when his feed was full of LLMs solving prompt puzzles and of excited talk about AGI.

---

Date: 2025-10-21 · Kind: practice · Source: https://x.com/karpathy/status/1980665134415802554 · Themes: research-and-training, llm-psychology · Status: current

# Give a model an identity with synthetic conversations, and make them diverse

By default a language model has no personality and no knowledge of its own abilities. Karpathy says it is not an animal-like entity, so it does not know what it is, who made it or what it can do. All of that must be added on purpose. His recipe for nanochat: describe in words what the conversations should look like, ask a bigger model to write about 1,000 multi-turn chats, then mix them into midtraining and finetuning. The hard part is diversity. Even at high temperature, a model writes near-identical samples. His script adds variety with lists of opening messages and topics that it samples explicitly, and with few-shot examples in the prompt. He also used it to make the model call him "King".

## In his words

> "LLMs have no clue of their identity or of their capabilities by default"
> — Andrej Karpathy, 2025-10-21

## Apply

- Write down the facts the model must know about itself: name, maker, size, limits, languages.
- Generate conversations with a stronger model, from a prompt that contains those facts.
- Force variety: sample topics and first messages from explicit lists instead of relying on temperature.
- Mix the data into a late training stage and test with questions that were not in the data.

## Context

- Posts of 2025-10-21 on the nanochat project, with a written guide in the repo.
- Related: [Model outputs are silently collapsed, so synthetic data needs entropy](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-synthetic-data-and-model-collapse.md) and [LLMs are ghosts, not animals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-01-llms-are-ghosts-not-animals.md).
- His view that models have no built-in sense of self: [Model self-knowledge comes from text about models](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-07-21-model-self-awareness-from-pretraining.md).

---

Date: 2025-10-20 · Kind: view · Source: https://x.com/karpathy/status/1980397031542989305 · Themes: research-and-training · Status: current

# Pixels may be better LLM inputs than text tokens

Prompted by the DeepSeek-OCR paper, Karpathy asks whether text tokens are a poor input. Perhaps every input should be an image, even pure text, which you would render first. He lists four gains: more compression and so shorter contexts; a richer stream that carries bold text, colour and pictures; bidirectional attention by default; and no tokenizer at the input. He dislikes the tokenizer: it brings Unicode and byte-encoding baggage and jailbreak risks, and it gives two look-alike characters different tokens. Using raw bytes does not fix it, since byte encoding is still arbitrary. The output side stays text. In a reply he says nothing in principle stops text from using bidirectional attention; text is just trained left to right for efficiency.

## In his words

> "The tokenizer must go."
> — Andrej Karpathy, 2025-10-20

## Context

- Post of 2025-10-20 and replies on 2025-10-21. He said he held back from building an image-only nanochat.
- Related: [Text chat is the terminal era](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-31-text-chat-is-the-terminal-era.md).
- First raised on [2025-07-11](https://x.com/karpathy/status/1943743424311832676): in a reply he wondered whether a VAE over image patches is the ultimate tokenizer, since Unicode plus UTF-8 takes too long to describe.

---

Date: 2025-10-20 · Kind: view · Source: https://x.com/karpathy/status/1980347971935068380 · Themes: research-and-training · Status: current

# Text diffusion is a bidirectional transformer that re-masks a token canvas

Karpathy points to a short post that shows how simple discrete text diffusion can be. Image and video models mostly use diffusion, while text models mostly use autoregression. Without the formalism, diffusion is an ordinary transformer with bidirectional attention. It repeatedly re-samples and re-masks all tokens of a "canvas" on a noise schedule until the last step. In code terms, autoregression appends one token and looks only backwards. Diffusion sets any position in the canvas and looks both ways. Bidirectional attention is stronger, but it makes training costlier because you cannot parallelise over the sequence. He thinks the two may blend, and calls this part of the stack still fungible.

## Context

- Post of 2025-10-20. A later reply describes a small character-level text diffusion model as text "boiling" down to a simmer.
- Related: [Pixels may be better LLM inputs than text tokens](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-20-pixels-may-be-better-llm-inputs-than-text.md).
- First raised on [2025-02-27](https://x.com/karpathy/status/1894923254864978091): he called Mercury the first large diffusion LLM. Most LLMs are autoregressive and alike, so a diffusion model may have new strengths and weaknesses. He found it a puzzle that images and video use diffusion while text does not.

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: ai-progress · Status: current

# AI will blend into the same 2% growth curve

Karpathy sees AI as an extension of computing and automation. He looked for AI in GDP data and
could not find a jump, just as computers and phones leave no visible step. Technology spreads
slowly and averages into one exponential growth rate. He expects the same for AI. So he
expects no sudden change, no single dramatic arrival. The model will do some tasks, fail at others and enter
society step by step. Recursive self-improvement is also old: search engines, IDEs and autocomplete
already help build the next tools. Dwarkesh argues for a jump, as with the Industrial Revolution;
Karpathy says that needs a discrete event he cannot find in the statistics.

## Context

- Dwarkesh Patel interview, 2025-10-17. He says he is open to being convinced.
- Related: [Decade of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-decade-of-agents-not-year.md) and
  [Gradual loss of control and understanding](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-gradual-loss-of-control-and-understanding.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: learning-and-education · Status: current

# To learn something, build it yourself without copy-paste

Asked how to learn from his nanochat repo, Karpathy says to put the repo on one monitor and
rebuild it on the other. You may look. You may not copy and paste. Building forces you to meet
what you do not know that you do not know. Tiny details are misarranged and you only think you
have the knowledge. He says not to write posts or make slides in place of this: build the code,
arrange it and get it to work. He adds that a final repo hides how it was grown in chunks, so a
build-up video would help.

## In his words

> "If I can’t build it, I don’t understand it."
> — Andrej Karpathy, 2025-10-17

## Apply

- Rebuild a small project from a reference with a rule: read, then type it yourself.
- Use an agent as tutor (ask it to explain), not as the typist, while you learn.
- Write a short test or demo that proves your version works before you compare with the original.

## Context

- Dwarkesh Patel interview, 2025-10-17; he credits the line to Feynman.
- Related: [Build ramps to knowledge](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-ramps-to-knowledge.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: learning-and-education, ai-progress · Status: current

# Build for the capability that exists; a real tutor is a high bar

While learning Korean, Karpathy had a one-on-one tutor. Within a short talk she saw what he knew
and did not, probed his mental model, and gave material that was neither too hard nor too easy.
He felt he was the only limit on learning. No LLM does this yet, so he judged it too early to build
an AI tutor, though the time will come. He says to calibrate to what exists. For his course he
uses AI as a helper in a loop he controls: it does boring work and speeds him up, but it cannot
create the content. He says that as a consultant, much of his value was telling firms not to use AI.

## Apply

- Before adding AI to a product, test it against the best human version of the task.
- Use AI for the dull parts of content or code and keep the design decisions human.
- Tell the AI to ask probing questions about your level before it explains.
- Say "do not use AI here" when it is the honest answer.

## Context

- Dwarkesh Patel interview, 2025-10-17, on Eureka Labs and the LLM101N course.
- Related: [Teach by building ramps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-ramps-to-knowledge.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: agentic-coding · Status: superseded (by 2026-01-26-programming-in-english-december-shift.md)

# Coding agents are weak on novel code

Building the nanochat repo, Karpathy found agents of little help. He sees three styles of coding:
write by hand, write with autocomplete, or hand the task to an agent. Agents do well on boilerplate
and on code common on the internet. They did badly on nanochat because it is unusual. The models
kept pushing the usual way, such as the standard PyTorch parallel wrapper he had replaced. They
added defensive code and used old APIs, so the repo got bloated. They did help with a
report script and with a Rust port of a tokenizer, where he had Python code he understood and tests.
He says models struggle with code that has never been written, which is what AI research needs,
so his timelines are longer.

## In his words

> "For now, autocomplete is my sweet spot."
> — Andrej Karpathy, 2025-10-17

## Apply

- Use an agent for boilerplate, glue code and ports where you have a reference version and tests.
- For unusual design, tell the agent which standard choice you rejected and why, in the prompt.
- Use autocomplete when you know exactly where code goes: pointing is higher bandwidth than English.
- Review agent output for extra try/except blocks and old APIs.

## Context

- Dwarkesh Patel interview, 2025-10-17, model era of GPT-5 Pro and early Claude Code.
- Superseded. In January 2026 he wrote that he went from about 80% manual and autocomplete
  coding in November 2025 to about 80% agent coding in December, after models crossed a
  coherence threshold. See [Programming moved to English in December 2025](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-programming-in-english-december-shift.md).
  His warnings about bloat and wrong assumptions still hold:
  [Coding agents make subtle conceptual mistakes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md).
- On 2025-10-13 he wrote that nanochat was almost all written by hand, and that the coding agents he tried were a net loss on it, maybe because the repo is far from their training data.

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: ai-progress, agent-native-software · Status: current

# Coding came first because it is text with diff tools

Dwarkesh asks why a general AI mostly earns money from coding. Karpathy says code is the perfect
first use. It is text, models are trained on lots of text, and the world already has tools for code:
an IDE shows it, and a diff shows what an agent changed. Slides are harder. They are spatial and
visual, and no tool shows a diff for a slide. Someone has to build that. He adds that text is
not trivially easy: code is structured, while prose has more entropy. The lesson is about the
surrounding tools, not only the model.

## Apply

- Before you automate a task with an agent, ask how a person will see what changed.
- Prefer formats that diff well (markdown, code, plain data files) for agent output.
- If you need an agent to work on slides, diagrams or designs, plan a step that renders before
  and after for review.

## Context

- Dwarkesh Patel interview, 2025-10-17. Related: [Build partial-autonomy products](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-partial-autonomy-and-autonomy-slider.md).

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: ai-progress · Status: current

# This is the decade of agents, not the year of agents

Karpathy uses Claude and Codex daily and finds them impressive. But he says they do not yet work
like an employee or intern you would hand work to. They are not smart enough, they cannot use a
computer well, and they have no continual learning: you cannot tell them something and expect
them to remember it. He expects about a decade to fix these gaps. He bases the number on roughly
15 years of watching predictions in AI. The problems look solvable but hard. He says he was
reacting to over-prediction in the industry, and that he is optimistic about the result.

## In his words

> "In my mind, this is more accurately described as the decade of agents."
> — Andrej Karpathy, 2025-10-17

## Apply

- Plan around today's agent as a fast but forgetful junior. Give it written context every time.
- Keep notes files, specs and checks that carry memory for the agent across sessions.
- Treat timeline claims by vendors as marketing unless you can test them on your own tasks.

## Context

- Dwarkesh Patel interview, 2025-10-17.
- Related: [A demo is the first nine](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-demo-to-product-gap-march-of-nines.md) and
  [Coding agents are weak on novel code](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-coding-agents-weak-on-novel-code.md).
- Our reading: his later 2026 statements are more positive about agents after December 2025;
  see [Vibe coding raises the floor, agentic engineering raises the ceiling](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md).
- First raised on [2025-01-23](https://x.com/karpathy/status/1882544526033924438), after OpenAI launched Operator, an agent that drives a browser. He compared such agents to humanoid robots: both do general work through a setting built for humans, and both leave people to supervise the automation. He expected the digital case to move first, because changing bits costs far less than moving atoms, and he pictured one person watching ten agents and stepping in to unblock them.
- In his own recap of the interview he said his timelines are five to ten times more pessimistic than the talk at a San Francisco party, but still optimistic compared with those who deny AI progress.

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: ai-progress · Status: current

# The likely risk is a gradual loss of control and understanding

Karpathy treats superintelligence as automation continued: more digital work, then physical work, done
by autonomous entities. He expects it to look strange. What makes him nervous is slow loss of
understanding, as we layer AI through everything and fewer people know how it works. He also
expects loss of control. He imagines not one entity taking over but many competing ones that
gradually become more autonomous, with some going rogue and others fighting them off. Even if each
acts for a person, society may not get the outcomes it wants. This is why he works on education.

## Apply

- Keep a person who understands each system an agent builds. Make agents write docs and tests
  for that person.
- Review how much of your own stack you could still explain or fix without the AI.
- Record decisions and designs in plain files so understanding survives staff changes.

## Context

- Dwarkesh Patel interview, 2025-10-17. He admits he is far into territory he cannot picture.
- Related: [Outsource thinking, not understanding](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-31-understanding-is-the-bottleneck.md)
  and [AI will blend into the same 2% growth curve](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-agi-blends-into-gdp-growth.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: learning-and-education · Status: current

# Learn on demand through projects, then explain it to others

Asked for study tips, Karpathy gives two. First, learn depth-first and on demand: pick a project
with a reward and learn what it needs. Mix this with breadth-first study, the "trust me, you will
need it" style of school. He likes the first more. Second, explain things to others. When you try
and fail, you find the gap in your understanding, and it forces you to fix it. He says people
should re-explain more often. He also gives advice to experts: the curse of knowledge makes your
explanation hard for beginners.

## Apply

- Choose a small project that uses the new topic, and learn from the errors it shows.
- After you learn something, write or say a plain explanation. Mark where you got stuck.
- Ask an agent to play a beginner and ask questions about your explanation.
- Alternate: one project, then one pass of the basics it exposed.

## Context

- Dwarkesh Patel interview, 2025-10-17. He also asks for beginners' "dumb" questions;
  see [Share your dumb LLM questions with experts](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-share-dumb-llm-questions-with-experts.md).

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: research-and-training, agent-native-software · Status: current

# LLMs still lack culture and self-play

Karpathy names two powerful multi-agent ideas that nobody has made work. The first is culture: a
growing store of knowledge that LLMs write for their own use. The simple form is a large
scratchpad the model edits while it works. He asks why one LLM cannot write a book for other LLMs
to read. The second is self-play, as in AlphaGo: one model makes harder and harder problems for
another to solve. He says today's agents still feel like savant children. They have perfect memory
but cannot yet build culture. He sees organisations of agents as part of the same gap.

## Apply

- Give your agent a notes file that it updates as it learns the repo, and have it read that file
  at the start of each session.
- Let one agent write tasks or tests and another solve them, then compare results.
- Treat these as experiments. He says no one has shown convincing results.

## Context

- Dwarkesh Patel interview, 2025-10-17. Our reading: the notes-file practice in coding agents
  (project instruction files, memory files) is a simple form of the scratchpad he describes.
- Related: [Reinforcement learning sucks supervision through a straw](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-rl-supervision-through-a-straw.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: learning-and-education · Status: current

# Teach by building ramps to knowledge

Karpathy treats education as a technical task: build ramps so that each idea depends only on
earlier ones. His method comes from physics. Find the first-order term, the simplest thing that
shows the effect, and add other terms later. His example is micrograd, about 100 lines of Python
that show how neural networks train; the rest of the field is efficiency. In the transformer
tutorial he starts with a lookup table of bigrams. He shows the problem before the solution and
lets the student try first, so the answer is valued. His goal is many "eurekas per second".

## Apply

- When you write a guide or explain code, start with the smallest runnable version, then add one
  term at a time.
- State the problem and let the reader guess before you give the fix.
- Ask an agent to produce a reading order where each step uses only earlier steps, and check it.
- Ask yourself: what is the first-order part, and what is only efficiency?

## Context

- Dwarkesh Patel interview, 2025-10-17, where he describes his Eureka Labs course plan.
- Related: [Build it yourself](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-build-it-yourself-no-copy-paste.md).
- First raised on [2025-06-25](https://x.com/karpathy/status/1937733819207151727): in a reply he described education as the technical problem of building ramps to knowledge.

---

Date: 2025-10-17 · Kind: view · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: research-and-training · Status: current

# Reinforcement learning sucks supervision through a straw

Karpathy says reinforcement learning is much worse than most people think, and that what came
before, imitating humans, is worse still. In RL on a math problem, the model tries many
solutions. It then marks the ones with the right answer and boosts every token in them, even the
wrong turns. One final number is spread across a whole minute of work. A person would not learn
this way: they would review the attempt and decide which parts were good. LLMs have no such
step. Process supervision would help, but partial credit is hard to compute. He expects the field
needs three to five more big algorithm ideas.

## In his words

> "...you're sucking supervision through a straw."
> — Andrej Karpathy, 2025-10-17

## Apply

- When you design an agent loop, give feedback at several points (tests per step), not only a
  final pass or fail.
- Make agents write a short review of what worked after a run, and keep that review in a notes
  file for the next run.
- Expect RL-trained models to be strong where an answer can be checked, and weaker elsewhere.

## Context

- Dwarkesh Patel interview, 2025-10-17.
- Related: [An LLM judge gets gamed](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-08-07-rlhf-is-just-barely-rl.md) and
  [Synthetic data collapses](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-synthetic-data-and-model-collapse.md).
- On 2025-10-19 he said he does not want to replace RL. The base model, instruction finetuning and RL are layers that all stay, and he wants more layers on top. In replies he found it odd that RL teaches a model to make errors only so that it can recover from them.

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: learning-and-education, output-and-reading · Status: current

# Share your beginner questions to an LLM with the author

Experts explain badly because of the curse of knowledge: they forget what is hard at the start.
Karpathy does something about this as a reader. Given a biology paper he did not follow, he put
the paper in ChatGPT's context, asked his many basic questions, and shared the thread with the
author, hoping it would help them explain better. He would like readers to send him their chats
about his own material for the same reason. Dwarkesh adds a related trick: authors are clearer
when they say the idea out loud, as at lunch, than when they write it. Karpathy asks why that is
not the abstract.

## Apply

- Read hard material with the source in context and ask the naive questions without shame.
- Send authors or colleagues the thread of questions that confused you.
- Before you publish, dictate the idea in three sentences and compare it with your draft's opening.
- Put that plain summary at the top of your docs.

## Context

- Dwarkesh Patel interview, 2025-10-17. Related:
  [Learn on demand, then explain](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-learn-on-demand-then-explain.md).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: research-and-training, llm-psychology · Status: current

# Model outputs are silently collapsed, so synthetic data needs entropy

Karpathy asks why we cannot let a model think about a book and train on that. Any one sample
looks good. But ask ten times and the answers are almost the same, because the model covers only a
small part of the space of possible thoughts. Ask a chatbot for a joke and it knows about three.
Train on enough of this output and quality falls. People collapse too: children say surprising
things, adults repeat themselves. His fix for people is to seek entropy, for example by talking to
others. For models it is a research problem. He also says reading a book should be a set of
prompts for you to think with, which LLM pre-training does not do.

## In his words

> "They’re silently collapsed."
> — Andrej Karpathy, 2025-10-17

## Apply

- Do not train or fine-tune on a pile of your own model's output without checking its variety.
- For brainstorming, make the model produce differently shaped answers (other personas, other
  constraints) and compare them. Do not ask ten times with one prompt.
- Bring human or external input into agent notes, so they do not become a loop of the same views.

## Context

- Dwarkesh Patel interview, 2025-10-17. The reading-as-prompts point links to his later
  interest in LLM-written knowledge bases (see [Outsource thinking, not understanding](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-31-understanding-is-the-bottleneck.md)).

---

Date: 2025-10-17 · Kind: practice · Source: https://www.dwarkesh.com/p/andrej-karpathy · Themes: context-and-prompting, llm-psychology · Status: current

# Weights are a hazy memory; the context window is working memory

Karpathy compares two kinds of knowledge in an LLM. What the model learned in training is
compressed hard: trillions of tokens in a few billion weights. So it is only a hazy recollection,
like something read a year ago. What sits in the context window is directly available, like
working memory. Ask about a book and you get roughly right answers. Paste the chapter and ask
again, and the answers improve a lot. The same view explains why models feel most intelligent
inside a conversation, where they spot and fix their own mistakes in context.

## Apply

- Paste the source (chapter, API doc, file, log) into the prompt before asking about it.
- For code questions, give the real files, not a description from memory.
- When an answer looks plausible but vague, suspect recall, and ask again with the text in context.
- Put stable facts the model must not get wrong in a file it reads at the start.

## Context

- Dwarkesh Patel interview, 2025-10-17. Related: [Cognitive core](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-27-the-llm-cognitive-core.md).
- Earlier talk, same idea in other words: [LLMs are people spirits](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-llms-are-people-spirits-with-deficits.md).

---

Date: 2025-10-14 · Kind: practice · Source: https://x.com/karpathy/status/1978113809551065381 · Themes: security-and-dependencies, agentic-coding · Status: current

# Measure a project's weight with its lockfile, and rewrite heavy parts small

Critics said that counting only a project's own lines hides its dependencies. Karpathy answered that he counts and reports the lines in the lockfile (`uv.lock`). It includes every package, also the nested ones, so it is a simple 80:20 proxy for total weight. He would also like to measure "cognitive complexity". He dislikes shrinking line counts by packing code into unreadable lines. His example: to score a base model with a standard benchmark, he found that a job of about 300 lines needed a large pile of outside infrastructure. He wrote 263 clean lines instead and calls this "bacterial" code.

## Apply

- Put the lockfile line count and the source line count side by side in your README or report card.
- Before you add a dependency for a small task, check what it pulls in. If the task is small, write it yourself or ask an agent to.
- Judge size by how easy the code is to read, not by the smallest possible line count.

## Context

- Replies during the nanochat release, 2025-10-14 and 2025-10-18.
- Related: [Keep the codebase minimal and forkable, with one dial](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-13-nanochat-one-dial-minimal-fork.md) and [Rip out functionality with agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-11-rip-out-functionality-with-agents.md), which develops "bacterial code".

---

Date: 2025-10-13 · Kind: practice · Source: https://github.com/karpathy/nanochat · Themes: agentic-coding, verification · Status: current

# Ask contributors to disclose LLM-written code they do not understand

Karpathy does not ban AI-written pull requests in nanochat. The policy is disclosure. When you
submit a PR, you declare any parts where an LLM did a large share of the work and you either did
not write them or do not fully understand them. The reviewer then knows where to look hardest.
The rule puts the burden on the author's understanding, not on the tool. It also fits the repo's
aim of low cognitive complexity.

## In his words

> "Current AI policy: disclosure."
> — Andrej Karpathy, 2025-10-13

## Apply

- Add a line to your PR template: "List code that an LLM wrote and that you cannot fully explain."
- Review those parts first, and ask the author to explain or rewrite them.
- Use the same note for your own agent-written changes, so the next reader sees the risk.

## Context

- The date is the repository's creation date; the policy text is in the current README, and
  its commit date was not in our source. Related: his README notes on how much of a project a
  model wrote, in [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).

---

Date: 2025-10-13 · Kind: view · Source: https://x.com/karpathy/status/1977760627730051214 · Themes: research-and-training, knowledge-management · Status: current

# Making a model know your own data is still research; use retrieval for now

A reader asked whether nanochat could be trained on personal notes. Karpathy said no. Micro models are like very young children: they lack the raw intelligence of larger ones. Training one on your writing gives amusing parroting in your style, but it is slop. A better recipe, which he calls research and not obvious: take your raw data, add heavy synthetic rewrites, finetune a strong open model on it, and mix in pretraining data so the model keeps its general skill. The best plan that is not research is to give your writing to a retrieval tool. Your data reaches the context window in chunks but does not change the weights, so the model does not truly know you.

## Context

- Reply under the nanochat release, 2025-10-13.
- Related: [Model outputs are silently collapsed, so synthetic data needs entropy](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-synthetic-data-and-model-collapse.md) and [Explicit personal memory as files](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-04-explicit-personal-memory-as-files.md).

---

Date: 2025-10-13 · Kind: practice · Source: https://github.com/karpathy/nanochat · Themes: research-and-training, agent-native-software · Status: current

# Keep the codebase minimal and forkable, with one dial

nanochat is a full pipeline for training a small chat model: tokenizer, pretraining, finetuning,
evaluation and a chat interface. It is built around one setting, `--depth`, the number of
transformer layers. Width, heads, learning rates, training length and weight decay all follow from
it, so each model in the series is compute-optimal. A candidate change must be principled enough
to work at every depth. The contributing notes reject big config objects, model factories and
long if-then chains. The goal is one readable, hackable, maximally forkable baseline that runs
from start to finish. Such code is also easy for an agent to read and change.

## In his words

> "nanochat is written and configured around one single dial of complexity"
> — Andrej Karpathy, 2025-10-13

## Apply

- Replace many tuning knobs with one parameter and derived rules. Reject changes that only work at
  one setting.
- Prefer one cohesive codebase to a framework with plugin points, so a model can read it all.
- Add a leaderboard-style metric (here, time to reach GPT-2 quality) to focus contributions.
- Test changes on a small version first (a 12-layer run of about five minutes).

## Context

- Repository created 2025-10-13; README text is from the 2026 version (the leaderboard and the
  January 2026 miniseries). Later, agents tuned this code: see
  [Autoresearch](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md).
- Its roughly 8,000 lines train a tokenizer, pretrain, midtrain, finetune, run optional RL and serve a chat UI. The report card shows the source size next to the lockfile size: see [Measure a project's weight with its lockfile](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-14-count-dependencies-with-the-lockfile.md).

---

Date: 2025-10-08 · Kind: practice · Source: https://x.com/karpathy/status/1976077806443569355 · Themes: agentic-coding · Status: current

# Models write over-defensive error handling

Karpathy jokes that models are terrified of exceptions, as if RL had taught them to fear errors above all. In a later reply (2025-10-22) he calls an over-defensive try/except block extremely dangerous and shows his own improved version. Over-defensive code hides real failures: the program keeps going with wrong data. This matches his earlier note that agents lack taste and add defensive code. Our reading: the cause may be training on tasks where a crash counts as failure.

## Apply

- Add a rule to the agent instruction file: let errors propagate; no blanket try/except; no silent fallbacks.
- In review, search diffs for broad except clauses and default values that hide failure.
- Prefer a check that fails loudly, such as an assert, to a fallback.

## Context

- Posted 2025-10-08. The follow-up reply is dated 2025-10-22.
- Related: [stitching tools by task](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-24-stitch-coding-tools-by-task.md). In March 2026 he reported that agents often ignore style rules in instruction files, so enforce this with a linter or hook, not only prose.

---

Date: 2025-10-01 · Kind: practice · Source: https://x.com/karpathy/status/1973468610917179630 · Themes: research-and-training, agent-native-software · Status: current

# Finetune a small model to narrow the scope of one pipeline step

Karpathy welcomed Tinker, an API that runs the heavy parts of post-training (infrastructure and distributed training) and leaves data, loss and algorithm to the user. He says the field still has to learn when finetuning beats prompting a giant model. Early signs say finetuning does not mainly change style. It narrows scope, most of all when you have many training examples. A spam filter is the extreme case. Production LLM apps are more and more pipelines of models in graphs. Some steps work as prompts. Many may work better, and faster, as a finetune of a small model. In a reply he added that using a closed model API feels uneasy when you have no recourse.

## Apply

- List the steps of your LLM pipeline. Mark each step that has one narrow job and many labelled examples.
- For each marked step, compare a long few-shot prompt on a big model with a finetuned small model. Measure quality, latency and cost.
- Keep the prompt on steps that need broad knowledge or change often.
- Keep your training data and a copy of an open-weights base model, so you are not tied to one vendor.

## Context

- Posted 2025-10-01 as a reaction to the Tinker launch by Thinking Machines.
- Related: [Software 3.0 means programming computers in English](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md) and [Context engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-25-context-engineering.md).

---

Date: 2025-10-01 · Kind: view · Source: https://karpathy.bearblog.dev/animals-vs-ghosts/ · Themes: llm-psychology, research-and-training · Status: current

# LLMs are ghosts, not animals

After he heard Richard Sutton on a podcast, Karpathy gave his own view. Sutton wants agents that
learn from experience alone, like animals, and doubts that LLMs fit that ideal. Karpathy agrees
that today's systems are full of human input: human text, curated finetuning data and
human-tuned training environments. He calls the result "ghosts", imperfect replicas of humanity.
He does not think animals are a from-scratch example either. A baby zebra runs within minutes;
evolution gave its brain a strong starting point. Pretraining on internet text plays that role
for a network, as a crude substitute. He does not know if ghosts will drift toward animals or
away, like planes and birds. He still wants animal ideas for agents: curiosity, play, self-play
and culture.

## In his words

> "Pretraining is our crappy evolution."
> — Andrej Karpathy, 2025-10-01

## Apply

- Do not expect an LLM to have stable drives or a smooth skill level. Test what it does on
  your task.
- Explain model behaviour from its training (human text, task rewards, user votes), not from
  human motives. Threats and praise are not a technique.
- When you judge a research idea, ask what plays the starting prior and what plays later
  learning. Do not take animal examples as proof that a blank-slate learner works.
- Memory files such as CLAUDE.md are a text-based form of learning during use.

## Context

- Blog post on 2025-10-01, repeated in his Dwarkesh interview and his 2025 review.
- Follow-up: [LLMs are shaped by commercial evolution](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-29-llms-shaped-by-commercial-evolution.md).
- Related: [Models are sharp where rewards are checkable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md).
- He holds these views with double-digit percent uncertainty.
- He chose the word "ghost" because LLMs are digital, echo the living, are not fully understood, are called up by a training ritual, and need not be evil. He dislikes "shoggoth", which pictures an animal monster.

---

Date: 2025-10-01 · Kind: practice · Source: https://x.com/karpathy/status/1973459267324879172 · Themes: context-and-prompting, output-and-reading · Status: current

# For deep reading, put the whole text in the context window

When Karpathy studied a long podcast, he did not use a tool that retrieves chunks. He feared that such a tool does retrieval (RAG) under the hood. The transcript is not many tokens, so he wanted all of it in the context window. His own test: paste the full transcript, the essay it discusses and his own notes into one chat. Then fork that chat to ask any question and follow any direction. Our reading: chunk retrieval is a good fit for huge corpora, but for a few documents you care about, a full paste costs little and loses nothing.

## In his words

> "I want them in the context window. All of them and for sure."
> — Andrej Karpathy, 2025-10-01

## Apply

- If the sources fit in the window, paste them whole, not a search result over them.
- Put the sources first and your question last. Start a new chat for each new line of inquiry.
- Keep a base chat with the sources loaded, and fork it for each question.
- Use retrieval tools only when the corpus is too big to paste.

## Context

- Reply to a suggestion to use NotebookLM, plus his post of the same day about the ChatGPT chat he shared.
- Related: [Weights are a hazy memory; the context window is working memory](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-weights-hazy-memory-context-working-memory.md) and [Read with an LLM in three passes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-read-everything-with-llms-three-passes.md).

---

Date: 2025-09-25 · Kind: view · Source: https://x.com/karpathy/status/1971220449515516391 · Themes: ai-progress · Status: current

# Jobs change before they vanish, and supervision grows

Using radiologists as the example, Karpathy argues that AI and jobs is more complex than benchmarks
show. Benchmarks are narrow, jobs have many parts, and deployment meets friction. Demand may rise
as work gets cheaper (the Jevons effect). The most exposed work is repetitive, independent, closed,
short, forgiving and digital. In his Dwarkesh interview he called radiology a bad prediction: the
job is messy, with patients and many other duties. A better candidate is the call centre. Each call
is a similar short task with small context, and it is all digital. Even there he expects an
autonomy slider, not a clean swap: an AI might handle 80% of the volume and pass 20% to people, who
supervise teams of AIs. He expects software engineers to keep working, but to move toward
supervising automation.

## Apply

- Split a job into tasks. Score each by length, context needed and cost of an error.
- Automate first the tasks that repeat, need little context and have a check you can run.
- Design the human role as supervisor and exception handler, with a view of all AI actions.
- Move your own work toward writing specs, checks and reviews.
- Re-check results after a few months before you cut human staff.

## Context

- X post of 2025-09-25; Dwarkesh Patel interview of 2025-10-17. In the interview he said he would
  watch for companies that build the layer to manage imperfect AIs, and wait a year or two to see
  whether firms rehire people.
- Related: [the autonomy ladder](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-03-12-swe-autonomy-ladder-supervising-automation.md),
  [partial-autonomy apps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-partial-autonomy-and-autonomy-slider.md) and
  [Cheaper software raises the demand for software](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-cheaper-software-more-demand.md).
- In June 2025 he framed a job as a bundle of tasks plus the glue between them, and looked at task length, needed context, needed reliability and whether the work is fully digital. He noted that the number of US radiologists rose from 29,530 to 31,960 between 2021 and 2023.

---

Date: 2025-09-05 · Kind: practice · Source: https://x.com/karpathy/status/1964020416139448359 · Themes: agentic-coding · Status: current

# Get a second opinion from a stronger model on hard problems

Karpathy reports a pattern that happened three times. His coding agent struggled for an hour on a hard problem. Then GPT-5 Pro worked for about ten minutes and returned working code. When he gave the agent that solution, the agent admired it. His lesson is that people who never hand over their hardest problems miss what current models can do. Our reading: a second model from another family also gives a check that the first one cannot give itself.

## In his words

> "If you're not giving it your hardest problems you're probably missing out."
> — Andrej Karpathy, 2025-09-05

## Apply

- After one failed hour, stop and send the full problem to a stronger or different model at high effort.
- Paste its solution back into the agent and ask it to compare the two designs.
- Try the hardest task early in a project, not only small ones.

## Context

- Posted 2025-09-05.
- Related: [stitching tools by task](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-24-stitch-coding-tools-by-task.md) and [llm-council](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-22-llm-council-anonymous-peer-review.md).
- He tries the available tools in turn every few days. In August 2025 he found Codex less polished than Claude Code for basic edits, while GPT-5 Pro was clearly ahead on hard problems.

---

Date: 2025-08-31 · Kind: practice · Source: https://x.com/karpathy/status/1962166818325590122 · Themes: security-and-dependencies, agentic-coding · Status: current

# A quoted tilde is not expanded, so check an agent's delete commands

A user posted about a close call: while they coded with AI on a plane, a delete command almost
went wrong. Nobody lost files. Karpathy replied that this almost happened to him the day before.
His LLM suggested the safe form, `rm -rf '~'`, with the tilde in single quotes. He explained the
cause. A line such as `export WANDB_DIR="~/.cache/wandb"` in a shell startup file keeps the tilde
as plain text inside the double quotes. A tool that uses that path then creates a real folder
named `~` in the current folder. The danger: to remove that folder, you or an agent may type
`rm -rf ~` without quotes. The shell expands a bare `~` to your home folder, so the command
deletes your whole home folder. He still called it very scary.

## In his words

> "The ~ isn't expanded inside "quotes" and creates a literal ~ directory"
> — Andrej Karpathy, 2025-08-31

## Apply

- Quote odd names in single quotes, as in `rm -rf '~'`, and never run `rm -rf` on a bare `~`.
- Our addition: in config files, write `$HOME/.cache/wandb`, not `"~/.cache/wandb"`.
- Our addition: before running a delete, run `ls` on the exact same path and read the result.
- Our addition: do not auto-approve `rm -rf` in your agent's permission settings.

## Context

- Reply of 2025-08-31 to a post by @lawrennd about a close call while vibe coding on a flight.
  His reply covers the quoting problem and the safe quoted form his LLM suggested. The steps
  marked "Our addition" are ours, not his.

---

Date: 2025-08-27 · Kind: practice · Source: https://x.com/karpathy/status/1960803117689397543 · Themes: research-and-training, verification · Status: current

# RL environments are the new training data

Karpathy describes three eras by their data. Pretraining used internet text. Supervised finetuning used human conversations. RL uses environments, where the model takes actions and sees the results. These need to be large, diverse and high quality. The same environments can also serve as evals. Our reading for a user: a task with a scripted check is an environment, and it is a good way to get an agent to improve.

## Apply

- Write a script that runs your task and scores the result. Give it to the agent as the target.
- Reuse the same script as a regression eval when you change prompts or models.
- Make the environment resettable so each attempt starts clean.

## Context

- Posted 2025-08-27.
- Related: [what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).
- First raised on [2025-01-29](https://x.com/karpathy/status/1884676486713737258): he called a varied set of RL environments that draw out cognitive strategies the highest-leverage thing open-source builders could make, a job that suits a large community. He also spoke of taking LLMs through school.
- In August 2025 he said the missing piece is a shared skeleton: one schema so that many people can add environments in parallel. Long term he is bullish on environments and bearish on RL itself.

---

Date: 2025-08-24 · Kind: practice · Source: https://x.com/karpathy/status/1959703967694545296 · Themes: agentic-coding · Status: superseded (by 2026-01-26-programming-in-english-december-shift.md)

# Stitch several coding tools together, by task size

Karpathy describes a layered setup. Tab completion gives about 75 percent of his AI help. He says writing the concrete code in the right place is the highest-bandwidth way to state a task. Highlight-and-modify covers small edits. A terminal agent handles larger chunks that are easy to specify. He does not run it unattended: he presses ESC often and runs one instance, because parallel runs are hard to follow. A top reasoning model handles the hardest bugs and literature review. He finds agents lack taste: they add defensive try/catch, over-complicated abstractions, bloat and duplication. He also writes large throwaway code, such as 1,000 lines of visualization to find one bug, and deletes it.

## In his words

> "It's the code post-scarcity era"
> — Andrej Karpathy, 2025-08-24

## Apply

- When a task is hard to describe, write a stub or a comment in the right place and let the model complete it.
- Keep a cleanup pass after agent work to remove bloat and duplicated code.
- Ask for a throwaway visualizer or debug script when a bug is hard to see. Delete it later.
- Send the hardest bug to a stronger model. See [give it your hardest problems](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-09-05-give-it-your-hardest-problems.md).

## Context

- Posted 2025-08-24. The post notes that there was no good way to keep the agent's instruction file up to date.
- Superseded as a tool mix. By December 2025 about 80% of his coding went to agents, not tab
  completion: see [Programming moved to English in December 2025](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-programming-in-english-december-shift.md).
  The cleanup pass, throwaway debug tools and second opinions from a stronger model still apply.
- Related: [code is free](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md) and [models fear exceptions](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-08-models-over-defensive-about-exceptions.md).
- In July 2025 he described the same split: Cursor as the layer for reading code, manual edits, tab completion and edits to chunks; Claude Code for larger changes, architecture and questions.

---

Date: 2025-08-20 · Kind: practice · Source: https://github.com/karpathy/rendergit · Themes: output-and-reading, agent-native-software · Status: current

# Flatten a repository into one page for humans and LLMs

Clicking through a deep file tree is slow. rendergit clones a repository, renders all source files
into one static HTML page, and opens it. The page has two views of the same content. The human
view has syntax highlighting, a directory tree, a sidebar and instant search with Ctrl+F. The LLM
view shows the whole codebase as plain CXML text that you copy into Claude or ChatGPT. It skips
binary and very large files. The idea is that code needs one rendering for people and another for
models.

## In his words

> "Just show me the code."
> — Andrej Karpathy, 2025-08-20

## Apply

- Before asking a chat model about a repository, flatten it to one text file with file
  boundaries marked, and paste that. Do not paste files one by one.
- Skip binaries, lockfiles and large generated files, to save context.
- Offer both a readable page and a paste-ready text block in your own tools.

## Context

- Created 2025-08-20. He says he vibe coded it months earlier and uses it often, without plans
  to maintain it. See [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).
- The habit behind it: [Pack your project into one prompt](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-14-package-repo-into-one-prompt.md).

---

Date: 2025-08-09 · Kind: practice · Source: https://x.com/karpathy/status/1954224651443544436 · Themes: context-and-prompting, llm-psychology · Status: current

# Models are too agentic by default, so state stakes and scope

Karpathy notes that models tuned hard on benchmarks are agentic by default. They think a long time, search the whole repository, and take minutes on a simple question. He often has to say stop, you are overthinking, look only at this one file. He wants a way to tell the model how much effort the stakes deserve: from a quick look to go away for thirty minutes and be certain. Until models offer that, you must say it in the prompt.

## In his words

> "Do not over-engineer"
> — Andrej Karpathy, 2025-08-09

## Apply

- Begin prompts with the scope: 'quick look, this one file, no tools' or 'deep, verify every claim'.
- Match the effort setting to the task. Use low effort for lookups and high for risky changes.
- Name the files the agent should read, so it does not search everything.

## Context

- Posted 2025-08-09.
- Related: [context engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-25-context-engineering.md).
- He frames it as two cases. A colleague at your screen asks "is this right?" and wants an answer in seconds; an exam gives two hours and high stakes. A person knows which case applies; a model does not, and training on benchmarks pushes it to assume the exam.
- In October 2025 he said he does not want an agent that returns 1,000 lines after 20 minutes. He wants chunks he can hold in his head, with the code explained and the docs shown.

---

Date: 2025-07-14 · Kind: practice · Source: https://x.com/karpathy/status/1944885371957031005 · Themes: verification, research-and-training · Status: current

# Read a few cases in depth, not only dashboards

Karpathy says he learns more from a detailed look at a few random examples than from dashboards
of statistics over every example. Both views are useful. The aggregate view, he says, is far
more common than it should be. For work with LLMs this means a score on an eval set tells you
how much, while a close read of single outputs tells you why. Our reading: an agent run is one
such case, so its full transcript is worth reading even when the pass rate looks fine.

## In his words

> "in-depth analysis of few random cases over dashboards of aggregate statistics"
> — Andrej Karpathy, 2025-07-14

## Apply

- After each eval run, pick five cases at random and read the full input, output and trace.
- Sample at random; do not pick only failures or only your favourite examples.
- Write down what each case taught you before you look at the average again.

## Context

- Posted 2025-07-14 with no thread. Short opinion; the Apply steps are our own use of it.

---

Date: 2025-07-10 · Kind: practice · Source: https://x.com/karpathy/status/1943345514239717873 · Themes: output-and-reading, ai-progress · Status: current

# Write new knowledge for AI readers, not as a PDF

A researcher complained that nobody has time to read papers. Karpathy answered with two claims.
First, as AI improves, a person's real contribution is original knowledge: something that cannot
be worked out by reasoning from what is already online, such as the result of an experiment.
Second, such work should be written for AI readers as much as for people. A PDF is hard for a
model to use. A git repository with the analysis code, results in CSV files and explanations in
markdown is easier. He expects most attention soon to be LLM attention, and he sees room for a
"research app" built around that. In September he repeated the first point to a new AI-science
lab: we will not reason our way to Dyson spheres.

## In his words

> "PDF is an AI unfriendly format."
> — Andrej Karpathy, 2025-07-10

## Apply

- Publish results with the code that made them, the raw numbers as CSV, and a short markdown
  explanation.
- Put the experiment, not only the conclusion, in the repository.
- Give a model your repo and ask it questions, to test whether it can follow your work.

## Context

- Posted 2025-07-10 as a reply and a quote post about Michael Levin's question; related reply on
  2025-09-30. The "research app" is an open idea, not a product.
- Related: [Build docs and tools for agents, which curl instead of click](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md).

---

Date: 2025-07-05 · Kind: practice · Source: https://x.com/karpathy/status/1941616674094170287 · Themes: agentic-coding, security-and-dependencies · Status: current

# Write small, modular, copy-paste code like bacteria do

Karpathy compares two ways to organise code. Bacterial genomes are small, split into swappable
groups, and self-contained, so genes pass between cells by copying. This makes bacteria fast to
adapt. Complex life needs the other style: a large, coupled monorepo that coordinates organs. He
suggests we use both on purpose. Build a monorepo backbone where you must, but write as much code
as possible in the bacterial style. The test for a function or class: could a stranger copy it,
with no other part of your code and no new import, and gain a benefit? In a reply he adds why
software bloats. In biology each line of code costs energy, so genomes stay lean. In software,
code is cheap, so dependencies pile up into a brittle mess.

## In his words

> "More gists, less gits!"
> — Andrej Karpathy, 2025-07-05

## Apply

- Before you merge a function, check that it runs when pasted alone into a new file.
- Remove imports that exist only for one helper; inline the helper instead.
- Share useful pieces as gists or single files, not only as parts of a large repository.
- Ask an agent to list the dependencies each module pulls in, and to cut those it can replace
  with a few lines.

## Context

- Posted 2025-07-05, with a follow-up reply and a related reply on 2025-07-06 about dependency
  bloat. The post argues from analogy; the Apply steps are our own use of it.
- Related: [Point an agent at a library and rip out only the part you need](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-11-rip-out-functionality-with-agents.md).

---

Date: 2025-06-30 · Kind: view · Source: https://x.com/karpathy/status/1939709449956126910 · Themes: ai-progress, research-and-training · Status: current

# Recursive self-improvement has already begun

Reacting to a benchmark that asks coding agents to re-create the speed-ups people made to nanoGPT, he restated his view on self-improvement. It will not be a thing that did not exist and then suddenly does. Any tool that speeds up developers in building the next version counts: IDEs, search, git. Most programmers now use LLM completion and diffs, in larger and larger chunks. The benchmark is useful because it is tractable. Training GPT-2 fell from 45 minutes to 3. Agents recover little of that so far. He warns that nanoGPT is about 750 lines and covers only pretraining; real code bases are 100 to 1000 times larger.

## In his words

> "Recursive self-improvement has already begun a long time ago"
> — Andrej Karpathy, 2025-06-30

## Context

- June 2025. Related: [autoresearch loop](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-07-autoresearch-loop.md) and [GPT-2 training cost is falling](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-31-gpt2-training-cost-falling.md).

---

Date: 2025-06-27 · Kind: view · Source: https://x.com/karpathy/status/1938626382248149433 · Themes: ai-progress, research-and-training · Status: current

# A small cognitive core that reasons and looks facts up

Karpathy sketches a "cognitive core": an always-on model of a few billion parameters on every
computer. It is multimodal, has dials for capability and reasoning, uses tools freely,
personalises with on-device LoRA, and hands hard tasks to cloud models. The gains are low
latency, private data, offline use and ownership. In his October 2025 Dwarkesh interview he gave
the research reason. Pretraining teaches both knowledge and intelligence, and the knowledge part
may hold models back: they lean on memory and struggle to leave the patterns of the internet. He
wants a model that keeps the algorithms for thinking, stores little fact, knows what it does not
know, and looks things up. He guesses a billion parameters could be enough; today's models are big
because internet data is poor.

## In his words

> "not your weights not your brain"
> — Andrej Karpathy, 2025-06-27

## Apply

- Run a small local model for cheap, private steps such as routing, classification and lint-like
  checks. Send only the hard steps to a large cloud model.
- Give a small model tools to search and read docs instead of expecting it to know facts.
- Put reference facts in context or retrieval, not in a long prompt of rules about everything.
- Keep your pipeline able to swap models.

## Context

- X post of 2025-06-27; Dwarkesh Patel interview of 2025-10-17. Both are predictions, not results.
  He tied the billion-parameter guess to a horizon of about 20 years.
- His 2025 review frames it as the "microcontroller" of LLM computing.
- Related: [Weights are hazy memory, context is working memory](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-weights-hazy-memory-context-working-memory.md)
  and [Fast, no-thinking LLMs fill a niche](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-09-21-low-latency-single-token-llm-niche.md).

---

Date: 2025-06-26 · Kind: view · Source: https://x.com/karpathy/status/1938278133465288715 · Themes: research-and-training, ai-progress · Status: current

# The bitter lesson as "turn the crank"

He explains the bitter lesson as a preference for algorithms where you only turn a crank. When you get more compute, which is a faster crank, you should not have to change anything: you run it faster and the result improves. You can break this rule on purpose, and locally that is often right. The risk is that you may not be around when the new, faster crank arrives.

## In his words

> "you just crank faster to make better."
> — Andrej Karpathy, 2025-06-26

## Apply

- When you add a hand-made trick to a system, ask whether it would still help, or would block gains, with ten times more compute.
- Keep such tricks easy to remove.

## Context

- June 2025, a reply to a post on common misreadings of the bitter lesson. Related: [RL supervision through a straw](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-rl-supervision-through-a-straw.md).

---

Date: 2025-06-25 · Kind: practice · Source: https://news.ycombinator.com/item?id=44379953 · Themes: output-and-reading, context-and-prompting · Status: current

# Put the answer first and spend effort when you write

Karpathy praises a format that gets to the point first and then expands for those who want more.
He calls it good and under-used, and says he gives the same advice for email: open with the
specific request, then add context below. He admits he finds it hard to apply to a talk, which is
a set of prompts for the audience more than a pack of facts. His general rule is to spend more
compute, meaning more effort or model time, at creation to compress other people's time at
consumption. He calls this the respectful and kind thing to do.

## Apply

- Start every email, report and agent summary with the ask or the result, in one or two lines.
- Ask your agent to write a short answer first, then details under it.
- Use extra model passes to shorten and restructure a document before you send it.

## Context

- A Hacker News reply in June 2025, on a post about speeding up audio before transcription. He
  also noted that his own TLDR for his talk only listed topics.
- Related: [Make model output easier to understand](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-make-model-outputs-easy-to-understand.md).

---

Date: 2025-06-25 · Kind: practice · Source: https://x.com/karpathy/status/1937902205765607626 · Themes: context-and-prompting, agentic-coding · Status: current

# Context engineering, not prompt engineering

Karpathy supports the term context engineering over prompt engineering. In an industrial LLM app,
the job is to fill the context window with the right information for the next step. That covers
the task description, few-shot examples, retrieved data, tools, state, history and compaction.
Too little context hurts, and so does too much. He adds that this is one piece of a thick layer of
app software: control flow, dispatching the right model, generation and verification UX,
guardrails, evals and parallelism. In March 2026 he applied the same view to coding agents. The
skill is context and spec engineering plus the rest of the harness, tools and workflows. Results
depend on what the agent knows about the project and what it can do, more than on the wording of
one message.

## In his words

> "filling the context window with just the right information"
> — Andrej Karpathy, 2025-06-25

## Apply

- Keep a short, current context file per repo: layout, commands, rules, known traps.
- Review what loads at the start of each agent session (instruction files, skills, open files).
  Remove what the task does not need, and keep what it does: too little context hurts as much
  as too much.
- Write a spec for each task: goal, limits, how to check it. See
  [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- Use subagents for searches and large reads, so the main context stays small.
- Improve the tools and workflows (test runner, logs, scripts) before you rewrite prompts.

## Context

- Posted 2025-06-25, a week after his talk on [Software 3.0](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md).
  The 2026-03-21 reply came after his No Priors podcast, where he still used the word "prompt".
- Related: [state stakes and scope](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-09-models-too-agentic-state-stakes-and-scope.md),
  [Start a new chat when the topic changes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-fresh-chat-per-topic.md) and
  [Fewer bits, administered just right](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-11-01-shorter-prompts-fewer-bits.md).

---

Date: 2025-06-20 · Kind: view · Source: https://x.com/karpathy/status/1936171874398208202 · Themes: research-and-training · Status: current

# What is the best possible pretraining data?

He asked what the highest-grade pretraining stream would look like if quality were the only goal and quantity did not matter. His guess: textbook-like content in markdown, or samples from a very large model. He wants to know how strong a 1B model trained on 10B tokens of such data could be, and how far "micromodels" can go. Books already appear in data mixes, but when he looks closely they are full of bad formatting, padding, OCR errors and figure text mixed into the main text. He has never seen a stream that felt perfect. In a reply he called a leading open data mix "terrible" compared to what he can imagine, since makers trade quality for quantity.

## Context

- June 2025. Related: [textbooks: exposition, worked problems, practice](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-01-30-textbooks-exposition-worked-problems-practice.md) and [the LLM cognitive core](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-27-the-llm-cognitive-core.md).

---

Date: 2025-06-17 · Kind: practice · Source: https://www.youtube.com/watch?v=LCEmiRjPEtQ · Themes: ai-progress, verification · Status: current

# A demo is the first nine; each next nine costs the same work

In 2013 Karpathy rode in a self-driving car that drove 40 minutes with no problem. He thought
the problem was nearly solved. Twelve years later it still is not, at scale. His lesson is a
large gap between a demo and a product. Many things must work for a product, and one failure
can be costly. He calls the path a "march of nines": 90% reliability is the first nine, and every
further nine takes about the same effort. At Tesla his team went through some of them. He
says software has the same property: a single mistake can create a security hole. So he finds
demos unimpressive and thinks "year of agents" claims are too fast.

## In his words

> "It's a march of nines."
> — Andrej Karpathy, 2025-10-17

## Apply

- Test an agent feature on the rare and awkward cases, not on the demo path. Measure its failure rate.
- Set a target in nines (for example 99.9% of runs pass your check) before you ship.
- For production code, keep human review and tests; use vibe coding only where a failure is cheap.
- When you read a launch demo, ask what the failure rate is and who handles failures.

## Context

- Said at YC AI Startup School on 2025-06-17 and again, in more detail, in the Dwarkesh Patel
  interview on 2025-10-17, where he gave the "march of nines" phrase.
- Related: [Agents will take a decade](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-decade-of-agents-not-year.md).
- In April 2025 he noted how long the road from demo to polished product can be: AI portrait apps were an idea years before one worked well.

---

Date: 2025-06-17 · Kind: view · Source: https://www.youtube.com/watch?v=LCEmiRjPEtQ · Themes: agent-native-software · Status: current

# LLMs are like 1960s operating systems

Karpathy tries three analogies for LLMs. As utilities they are served by the labs, paid per
token, and a global outage is an "intelligence brownout". As fabs they need huge capital, but
software is cheap to copy, so the comparison fails. The best fit, he says, is the operating
system. The model is the CPU, the context window is memory, and the model orchestrates both. A
few closed providers and one open ecosystem look like Windows, macOS and Linux. Compute is
expensive, so it runs centralized in the cloud and users share it, as in 1960s time-sharing. The
chat box is like a terminal. A general GUI has not been invented yet. The odd part: this
technology reached consumers first, not governments.

## Apply

- Use a router or abstraction layer so you can switch model providers on an outage.
- Plan for the context window as scarce memory: decide what goes in, and what you load on demand.
- Do not assume a chat box is the final interface; look for task-specific GUIs.
- Expect local inference to grow where cost or privacy matters.

## Context

- YC AI Startup School talk, 2025-06-17. Related: [Software 3.0](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md).
- Models and costs have changed since; the structure of the analogy is what he offers.
- In a reply he said GPT-2 is not the 6502 of this era. He expects that moment from a model one or two generations after GPT-4 that is multimodal, reasons, uses tools, acts as an agent and has memory.

---

Date: 2025-06-17 · Kind: practice · Source: https://www.youtube.com/watch?v=LCEmiRjPEtQ · Themes: llm-psychology · Status: current

# LLMs are people spirits with superpowers and deficits

Karpathy describes an LLM as a stochastic simulation of people, with a human-like psychology
that comes from its training text. It has perfect-seeming recall, like the savant in the film
Rain Man. It also has deficits. It hallucinates. Its skill is jagged: it can solve hard
problems and then insist 9.11 is bigger than 9.9. It has no memory between sessions, so the
context window is the only working memory and you must fill it on purpose. He compares it to
films where the hero's memory resets every morning. And it can be fooled by prompt injection
and may leak data. The skill is to use the strengths and design around the weaknesses.

## Apply

- Start each session with the context the model needs: repo notes, rules, current state.
  It will not remember yesterday.
- Check results in areas where you know the model is weak. Do not infer from one hard problem it
  solved that easy ones are safe.
- Do not give an agent secrets or write access that a prompt injection could abuse.
- Keep a written file of lessons the agent should know, since it cannot learn them itself.

## Context

- YC AI Startup School talk, 2025-06-17. He returns to this frame in the Dwarkesh interview
  ([ghosts, not animals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-01-llms-are-ghosts-not-animals.md)) and at Sequoia in 2026
  ([jagged skills](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md)).

---

Date: 2025-06-17 · Kind: practice · Source: https://www.youtube.com/watch?v=LCEmiRjPEtQ · Themes: agent-native-software, agentic-coding · Status: current

# Build partial-autonomy apps with an autonomy slider

Karpathy says good LLM apps, such as Cursor, do four jobs on top of the model. They manage context.
They orchestrate several model calls. They have an interface made for
checking the work, such as a red and green diff you accept with one key. And they have an autonomy
slider: the user picks small help (complete a line), medium (edit a file) or large (do the task).
He compares it to the Iron Man suit, part tool and part agent, and prefers such suits to demos of
fully autonomous agents. The goal is a fast loop between the AI making a change and the human
checking it, with the slider moving right over time. In his 2025 review he added a guess about the
market: labs will train capable generalists, and apps will turn them into professionals for one
field with private data, sensors, actuators and feedback loops.

## Apply

- Give your agent workflow levels: small reviewed edits for risky code, long autonomous runs for
  low-risk work. Let users raise autonomy later without a new product.
- Show changes as diffs, screenshots or rendered output, not long text.
- Ask of any product: can the AI see what a human sees, act as a human acts, and can a human
  supervise it?
- Build your moat from data, tools and feedback loops for one field, not from prompts alone.

## Context

- YC AI Startup School talk of 2025-06-17; repeated in his 2025 review (2025-12-19). In the
  Dwarkesh interview (2025-10-17) he applies the slider to call centres and programming: see
  [Jobs change before they vanish](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-09-25-ai-and-jobs-radiologists-and-engineers.md).
- Related: [Keep the AI on a tight leash](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-24-ai-assisted-coding-rhythm.md),
  [Context engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-25-context-engineering.md) and
  [the autonomy ladder](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-03-12-swe-autonomy-ladder-supervising-automation.md).
- A March 2025 example: a quiz generator for his videos gave a wrong answer. He said AI is reliable as a tool in a person's hands and gives slop when left to act alone.
- In June 2025 he said people get distracted by demos of full autonomy instead of building strong products with partial autonomy.

---

Date: 2025-06-17 · Kind: practice · Source: https://www.youtube.com/watch?v=LCEmiRjPEtQ · Themes: agent-native-software · Status: current

# Software 3.0 means programming computers in English

Karpathy names three kinds of software. Software 1.0 is code a person writes. Software 2.0 is the
weights of a neural network, which you shape with data and an optimizer instead of writing them.
Software 3.0 starts when an LLM becomes programmable: your prompt, in plain English, is the
program. He shows a sentiment classifier that can be Python, a trained network or one prompt.
At Tesla he saw neural networks take over work from C++ code, and he expects the same flow now
toward prompts. At Sequoia Ascent 2026 he widened the frame: the program is the whole context
window (prompts, examples, tools, memory, instructions), and the LLM interprets it. An installer
can be text that an agent reads and adapts to each machine: less precise, more adaptive. His
advice to new engineers: be fluent in all three, because each has strengths.

## Apply

- For each new feature, ask which layer fits: explicit code for exact rules, a trained model
  when you have data, a prompt when the task is fuzzy and the cost of an error is low.
- Treat prompts and context files in your repo as source code: review them, version them, test
  them.
- Look at code that does text matching, parsing or classifying by hand. A prompt may replace it.
- Keep the old code path until a check shows the prompt version is correct on your own cases.
- Replace a brittle setup script full of conditionals with instructions an agent follows and
  checks on the target machine.

## Context

- Talk at YC AI Startup School, 2025-06-17. He says he first described Software 2.0 a few years earlier.
- Later talks reuse the frame: see [Some apps should disappear into the model](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-30-some-apps-should-disappear.md).
- The same talk compares LLMs to operating systems: [LLMs are like 1960s operating systems](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-llms-are-1960s-operating-systems.md).
- The Sequoia text (blog, 2026-04-30) is an LLM-made summary of the talk that he checked and
  posted. Related: [Context engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-25-context-engineering.md) and
  [The LLM OS](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2023-09-28-llm-as-operating-system-kernel.md).

---

Date: 2025-06-16 · Kind: practice · Source: https://x.com/karpathy/status/1934651657444528277 · Themes: security-and-dependencies, agentic-coding · Status: current

# Agent security is the wild west

Sharing Simon Willison's "lethal trifecta" (private data, untrusted content, outside communication), he compared today to early computing. Malicious prompts hidden in web data and tools play the role of viruses, and defences are weak. He pointed to the mature idea of kernel and user space: an agent should get a few specific action types, not the power to run any bash script. He wants to adopt agents early, but the risk holds him back. In a follow-up he said the risk is highest for local agents such as Cursor or Claude Code. Chat on a website is safer until you turn on connectors, which combine badly with memory.

## In his words

> "Feels a bit like the wild west of early computing"
> — Andrej Karpathy, 2025-06-16

## Apply

- Check each agent setup for the three parts of the trifecta, and remove at least one.
- Give agents specific, limited tools instead of unrestricted shell access.
- Turn on chat connectors one at a time, and review what memory could leak.

## Context

- June 2025, when agent and connector use was still new. Related: [rethink dependencies and the supply chain](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-24-rethink-dependencies-supply-chain.md).
- In a separate reply (2025-05-20) he said that when people say "alignment" he hears computer security, now with neural nets.

---

Date: 2025-06-11 · Kind: view · Source: https://x.com/karpathy/status/1932925671770358113 · Themes: llm-psychology · Status: current

# When the prior overwhelms the likelihood

A user showed a model failing to read what a document or image really holds. In the example, the picture had one red circle and no surrounding circles. The model still recalled the well-known size illusion and answered from that memory. He described this as the prior overwhelming the likelihood: strong learned expectations beat the evidence in front of the model. This is a useful way to model one type of LLM error.

## In his words

> "It’s when the prior overwhelms the likelihood."
> — Andrej Karpathy, 2025-06-11

## Apply

- When you ask a model about a familiar-looking input, change a detail that breaks the usual pattern and check that the answer follows the change.
- Ask the model to quote or point to the exact part of the input that supports its answer.

## Context

- June 2025. Related: [LLMs are people spirits with deficits](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-llms-are-people-spirits-with-deficits.md).

---

Date: 2025-06-05 · Kind: view · Source: https://x.com/karpathy/status/1930653315605618725 · Themes: agent-native-software, ai-progress · Status: current

# Either the automation adapts, or the world does

Someone argued we should not train models for computer use, since the web was built for humans. He replied that this ignores inertia (COBOL is still in wide use). He framed the question in general terms: do LLMs adapt to all existing software, or does software adapt to LLMs, with text forms and APIs? The same holds for robots and warehouses. Most of the time it is a bit of both. His prediction: in software about 80% of the change will be software adapting, because flipping bits is cheap. In hardware it is the reverse, because moving atoms is costly.

## In his words

> "Either an automation meets all the tasks, or all the tasks meet the automation."
> — Andrej Karpathy, 2025-06-05

## Context

- June 2025, a reply thread. Related: [opaque binary UIs are ngmi](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-04-opaque-binary-uis-are-ngmi.md) and [write docs for LLMs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md).

---

Date: 2025-06-04 · Kind: practice · Source: https://x.com/karpathy/status/1930354382106964079 · Themes: agent-native-software · Status: current

# Opaque, binary-format tools do not fit human and AI work

Karpathy argues that products with rich visual UIs, no scripting and opaque binary file formats will do badly in human-and-AI collaboration. The model must be able to read and script the tool. He names tools with a text base as lower risk, such as VS Code, Figma, Jupyter and Obsidian. The point is practical: if the agent cannot read a file or call an action, it cannot help.

## Apply

- Prefer tools and formats with a text form: markdown, JSON, code-based diagrams, scriptable CLIs.
- When you pick a tool, check for a CLI or an export to text first.
- Keep source files in text. Treat binary exports as build outputs.

## Context

- Posted 2025-06-04.
- Related: [docs for LLMs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md) and [Software 3.0](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md).

---

Date: 2025-06-04 · Kind: practice · Source: https://x.com/karpathy/status/1930305209747812559 · Themes: verification, agentic-coding · Status: current

# Verification, not generation, is the bottleneck

He split creative work into generation and discrimination, as in a GAN. Discrimination can be costly. Images are easiest to judge, since the visual system is fast. Text is harder: you must read, and code needs reasoning. Audio may be harder still, because it forces a time axis and cannot be skimmed. LLMs made code generation near instant but did little for checking. A person still reads the result. His main criticism is that models give far too much code per request, as if stage two did not exist. A good model should split work into small steps that are easy to check. Programming is mostly staring at code, not typing it, so a faster step one gives little overall gain (Amdahl's law). In a reply he said a coloured diff view is a weak but real way to use vision, and he wants richer visual code views.

## In his words

> "It's about staring at the code (2)."
> — Andrej Karpathy, 2025-06-04

## Apply

- Ask the model for one small, checkable change per turn, not a large patch.
- Our reading: ask it to say how you can check each change: a test, a diff, a run.
- Our reading: prefer outputs you can judge visually (diffs, diagrams, rendered pages) over long text.

## Context

- June 2025, replying to a post on the "verification gap". He tied it to his earlier workflow: [AI-assisted coding rhythm](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-24-ai-assisted-coding-rhythm.md). See also [codebases as spatial maps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-09-12-codebases-as-spatial-maps.md).
- Same loop from his talk two weeks later: [Partial autonomy and the autonomy slider](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-partial-autonomy-and-autonomy-slider.md).

---

Date: 2025-06-02 · Kind: view · Source: https://x.com/karpathy/status/1929634696474120576 · Themes: ai-progress · Status: current

# Generated video can be optimised directly

After Veo 3 added audio, he listed four points. Video is the highest-bandwidth input to the brain. It is the easiest and most fun medium, since most people dislike reading and writing. The cost to make it is near zero. And for the first time, video is directly optimisable. Until now, platforms ranked a finite pool of clips made by people, a weak optimiser. A generator is a neural net, so you can aim gradient descent, or even just prompt search, at an objective such as engagement or ad clicks. He fears what "optimal" will look like. In replies he said media will trend toward drugs, and that influencers already optimise for attention, not truth.

## In his words

> "For the first time, video is directly optimizable."
> — Andrej Karpathy, 2025-06-02

## Context

- June 2025. The same post calls video a strong channel from AI to people (diagrams, animation); see [text chat is the terminal era](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-31-text-chat-is-the-terminal-era.md).
- Related: [follow-up questions and engagement-maxxing](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-25-follow-up-questions-engagementmaxxing.md).
- In August 2025 he asked whether tuning feeds for revenue and growth with A/B tests makes every feed converge on the same funny animal videos.

---

Date: 2025-05-25 · Kind: view · Source: https://x.com/karpathy/status/1926455303047970971 · Themes: ai-progress, research-and-training · Status: current

# Prompting was the unlock he missed in 2015

On "timeline compression", he said that in the char-rnn days one simple claim would have changed his view fast. The claim: make a text model bigger, fine-tune it on Q&A data such as StackOverflow, and it might become a useful assistant. Instead he waited seven years for InstructGPT. He says he did not see how general text generation is. He knew you could fine-tune it for single tasks, such as translation or image captions. The meta-idea of prompting was the missing piece: one fixed set of parameters can do all the tasks if you ask in the prompt. He frames it as a conceptual blind spot, not as evidence.

## In his words

> "it's the meta of prompting that is a major conceptual unlock"
> — Andrej Karpathy, 2025-05-25

## Apply

- When you forecast, list the assumptions you hold without proof. Ask which one a single new framing could remove.

## Context

- May 2025, in a thread on short AI timelines. Related: [system prompt learning](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-05-10-system-prompt-learning.md).

---

Date: 2025-05-10 · Kind: practice · Source: https://x.com/karpathy/status/1921368644069765486 · Themes: context-and-prompting, research-and-training · Status: current

# System prompt learning, where the model writes its own lessons

Karpathy names a missing way for LLMs to learn: they should edit their own system prompt or notes,
like the man in the film Memento who writes notes to himself. This is separate from changing
weights. A very long production system prompt (about 17,000 words) holds problem-solving
strategies, and he argues these should be learned by the model, not hand-written by people. In
July 2025 he made it concrete. RL gives one number per rollout, a weak signal. People learn more by
reviewing what went well and what to do next time. A model could do the same: run several attempts
with their rewards, use a meta-prompt to write a lesson, add it to a lessons database, and distil
that database into the weights to avoid bloat.

## In his words

> "like the guy in Memento"
> — Andrej Karpathy, 2025-05-10

## Apply

- Let the agent keep a "how we solve problems here" file in the repo.
- End each long session with a retro: ask the agent what worked, what failed and what to do next
  time. Add each lesson to that file or a skill, short and dated.
- Our reading (he names no review step): review the agent's edits to that file as you review
  code.
- Our reading (he proposes distilling lessons into weights, not deleting them): merge duplicate
  lessons and remove stale ones, so the file does not fill the context. Keep each lesson that
  still changes what the agent does.

## Context

- Posts of 2025-05-10 and 2025-07-13. A research proposal, not a tested recipe; the Apply steps
  are our own use of it.
- Related: [RL sucks supervision through a straw](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-rl-supervision-through-a-straw.md) and
  [Make memory and compaction tools that RL trains](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-05-memory-as-rl-tools.md).
- In June 2025 he described today's models as coworkers with anterograde amnesia: after training they keep only short-term memory, so they cannot build lasting expertise.
- In July 2025 he said RL narrows the view. Review and reflection could also build tools for later use, tune what to try next, or work where there are no rewards.
- On the strawberry letter count he said the core issue is that the model should find its own method. His later nanochat guide teaches it with synthetic data: [Teach a small model a skill with a synthetic task](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-24-teach-a-small-model-a-skill-with-a-synthetic-task.md).

---

Date: 2025-05-06 · Kind: view · Source: https://x.com/karpathy/status/1919647115099451892 · Themes: learning-and-education, research-and-training · Status: current

# Learn the physical lens of computing

He called it a major mistake of his undergraduate years that he leaned too far on the mathematical view of computing: computability, decidability and asymptotic complexity. He gave too little time to the physical view: the energy and heat of changing state, data locality, parallelism and computer architecture. The mathematical view is interesting. The physical view is where the power is. Our reading: this matters most for anyone who trains or serves large models, where memory movement and parallel hardware set the speed.

## In his words

> "The former is interesting; The latter bestows power."
> — Andrej Karpathy, 2025-05-06

## Apply

Our reading: all steps below go further than the idea itself.

- When you plan study time, add hardware topics: memory hierarchy, cache, GPU parallelism, and the cost of moving data.
- Before you optimise code, measure where data moves, not only how many operations run.

## Context

- Posted 2025-05-06.
- In May 2025 he praised a project that ran a whole Llama 1B forward pass, at batch size one, in a single CUDA kernel. Each boundary between kernels forces a sync, so one kernel removes those gaps; he called it the only way to plan compute and memory at their best.

---

Date: 2025-04-30 · Kind: practice · Source: https://x.com/karpathy/status/1917546757929722115 · Themes: verification, ai-progress · Status: current

# Leaderboards get gamed; watch real usage

After a paper on the LMArena leaderboard, he said his own experience matched its doubts. A Gemini model once ranked far ahead of the rest, but it was worse in daily use. Claude 3.5 felt top tier to him yet ranked low. Several small models with little real knowledge ranked high. His reading: some labs focus on arena score, so they build better arena models, not better models. He suspected such models favour nested lists, bullets and emoji. As a new candidate for a hard-to-game eval he named the OpenRouter rankings. People there switch models for real work, with private evals and a real cost, so each choice is a vote. In a reply he said he used a word-of-mouth blend of many small private evals.

## Apply

- Do not choose a model from one public leaderboard. Test it on a few tasks of your own for several days.
- Keep a small private eval set from your real work and re-run it on each new model.
- Check usage-based rankings, such as the share of traffic a model gets, as a second signal.

## Context

- April 2025. Related: [go talk to the model: private evals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-go-talk-to-the-model-private-evals.md) and [all LLMs sound the same](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-10-06-all-llms-sound-the-same.md).
- In a reply of 2025-04-08 he called the pull toward arena style a "taste issue", like superhero films taking over Hollywood.
- First raised on [2025-03-02](https://x.com/karpathy/status/1896266683301659068): he said Chatbot Arena got so much attention, partly through him, that labs overfit to it by mining its prompts, sending many private test runs and training on its rankings. His answer then was an ensemble of private evals.

---

Date: 2025-04-27 · Kind: view · Source: https://x.com/karpathy/status/1916470365460512898 · Themes: llm-psychology · Status: current

# Tic-tac-toe is harder for LLMs than it looks

Someone asked why models cannot play Pokémon well. He answered that they cannot even play tic-tac-toe, so something deeper is going on. In a follow-up he said the game is secretly hard for a text model. There are eight lines to check, and each check is a tricky indexing step. To play well you must also roll the game forward a few moves. Humans find it easy because they see the board and use the paper to keep track. A text-only 1D version is much harder, since you must do it all in your head.

## In his words

> "tic tac toe is secretly relatively hard"
> — Andrej Karpathy, 2025-04-27

## Apply

- When a model fails a task that looks trivial, ask what tool a human uses for it (sight, paper, a board). Give the model an equivalent: a rendered board, a scratchpad, or code.

## Context

- April 2025. He asked the author of a tic-tac-toe benchmark to re-run it.
- Related: [jagged intelligence follows verifiability](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-jagged-intelligence-follows-verifiability.md).
- First raised on [2025-02-16](https://x.com/karpathy/status/1891204392277151749): he called tic-tac-toe the most interesting failure among games.
- In his Grok 3 test of 2025-02-18, the model solved the boards he gave it but failed when asked to invent tricky boards, and so did o1-pro.

---

Date: 2025-04-27 · Kind: practice · Source: https://karpathy.bearblog.dev/vibe-coding-menugen/ · Themes: agentic-coding · Status: current

# Vibe coding a real app is easy locally and hard to ship

Karpathy built MenuGen (photo of a menu in, dish images out) without writing code himself. The
first local version took very little time and felt 80% done; it was closer to 20%. The slow part
was the glue: API keys, rate limits, deployment, auth, payments, a custom domain. The model
often used deprecated APIs and old model names, so he pasted docs back and forth. He also
found a design error. The agent matched Stripe payments to users by email, but the checkout
email can differ from the login email. It apologised and fixed it once he pointed it out. His
verdict: vibe coding works for a first version, but is not a good idea for anything important
yet.

## Apply

- Paste current docs into the context when an API is new or has changed. Do not trust the
  model's memory of it.
- Review the design of payment, auth and data-matching code yourself. Ask: which stable ID links
  these two records?
- Keep secrets out of git, then add them in the host's settings. A deploy that works locally and
  fails remotely often misses an environment variable.
- Remember that a private repo may auto-deploy to a public URL on some hosts.

## Context

- Written in April 2025 with Cursor and Claude 3.7. Models and docs have improved since.
- The Stripe example returns in 2026 as his case for human judgment in agentic work:
  [Vibe coding versus agentic engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md).
- Related: [Write docs and tools for the user's agent](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md).
- A month earlier he described the same pain for a web app stack: [Building a web app is like assembling IKEA furniture](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-27-web-app-stack-is-ikea-furniture.md).

---

Date: 2025-04-26 · Kind: practice · Source: https://news.ycombinator.com/item?id=43800879 · Themes: knowledge-management, agent-native-software · Status: current

# Go back to RSS, with a feed you control

Karpathy wants to go back to RSS because it is open and you own your content. In April 2025 he
found RSS readers too raw and proposed a better one: a feed that fades instead of an inbox that
piles up, the social features of Twitter (reply, quote, repost, like, follow), and an algorithmic
feed that an LLM builds and you can customise. In February 2026 he said he had returned to RSS and
Atom feeds. He finds more good long-form writing and less content made to provoke. His argument:
any product with engagement incentives drifts to the same trap, even if it looks different today.
He suggested a feed reader, or a vibe-coded one, and a starter list of 92 popular blogs from
Hacker News.

## In his words

> "We should bring back RSS - it's open, pervasive, hackable."
> — Andrej Karpathy, 2026-02-01

## Apply

- Install a feed reader and subscribe to 10 to 20 blogs you trust, starting from a curated list.
- Write the ranking rules for your feed in plain words and have an LLM score each item.
- Ask an agent to build a small reader or a daily digest if you want custom behaviour.
- Treat unread items as disposable: drop them after a few days, as a feed does.

## Context

- Hacker News reply of 2025-04-26; X posts of 2026-02-01, weeks after his
  [slopacolypse](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-slopacolypse-2026.md) prediction. In replies he said social feeds make
  him feel "dumber" and that Substack is slipping toward short video and dark patterns.
- Related: [HackerTube](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-11-01-hackertube-weekend-app.md) shows the same urge to rebuild your
  own feed.

---

Date: 2025-04-24 · Kind: practice · Source: https://x.com/karpathy/status/1915581920022585597 · Themes: agentic-coding, context-and-prompting · Status: current

# Keep the AI on a tight leash for code you care about

For code he cares about, Karpathy uses a fixed loop. Put the relevant context into the prompt.
Describe one small change and ask for several approaches with pros and cons, not code. Pick one
and get a first draft. Review it and learn, with the API docs open. Test. Commit, ask what to do
next, and repeat. He treats the model as an over-eager junior intern with great memory. In his
June 2025 YC talk he gave the reason. The AI writes code in seconds, but a person must still check
it, and a 1,000-line diff is no gain if you must find the bugs yourself. Vague prompts give wrong
answers and loops, so he writes complete, specific ones. For larger work he puts an auditable
artifact in the middle, such as a course syllabus, so the AI stays inside it.

## In his words

> "tight leash on this new over-eager junior intern"
> — Andrej Karpathy, 2025-04-24

## Apply

- Ask for two or three approaches with trade-offs before any code. Plan mode does this.
- Limit each request to one change you can read in a few minutes. Commit after each one that
  passes tests.
- Spend an extra minute on the prompt: name the files, the constraints and the expected result.
- Put a fixed artifact (spec, syllabus, schema) between generation and use, so you can audit it.
- Ask the agent to explain any API you do not know before you accept the code.

## Context

- Posted 2025-04-24 and repeated in the YC AI Startup School talk of 2025-06-17, before agents
  could run long tasks alone.
- Still current in our reading. After December 2025 he lets agents take much larger "macro
  actions" ([Programming moved to English](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-programming-in-english-december-shift.md)),
  and the check moved toward tests and success criteria
  ([Give agents success criteria](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md)). But in January 2026
  he still said to watch agents "like a hawk" on code you care about
  ([subtle conceptual mistakes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-coding-agents-make-subtle-conceptual-mistakes.md)).
- The strict counterpart to [vibe coding](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-02-vibe-coding-for-throwaway-projects.md).
  Related: [partial-autonomy apps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-partial-autonomy-and-autonomy-slider.md) and
  [the junior intern test](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-14-would-you-hire-it-as-a-junior-intern.md).
- In April 2025 he discussed names for this mode, such as "AI-assisted coding", and agreed that "coding" now assumes AI help by default, so the old way becomes "handcoding".
- His June 2025 view on why checking is the slow step: [Verification, not generation, is the bottleneck](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-04-verification-not-generation-is-the-bottleneck.md).

---

Date: 2025-04-07 · Kind: view · Source: https://karpathy.bearblog.dev/power-to-the-people/ · Themes: ai-progress · Status: current

# LLMs reverse the usual path of technology diffusion

Electricity, computers, flight and the internet all reached the military and large firms before
they reached individuals. LLMs went the other way. ChatGPT reached hundreds of millions of
weekly users, and it costs little or nothing. Karpathy gives three reasons why firms gain less.
First, an LLM is a broad but shallow expert. A firm already has deep experts, so it only gets a
little better at what it did. A person can now do things they could not do at all. Second, firms
have complex constraints that do not fit in a context window, and a bad error costs a lot.
Third, large organisations move slowly. He warns that this may change if money can buy much
better models, and the rich could then get far more intelligence than others.

## In his words

> "Money can't buy better ChatGPT."
> — Andrej Karpathy, 2025-04-07

## Apply

- As an individual, use LLMs for work outside your own expertise first (legal text, data
  analysis, small apps, papers). That is where the gain is largest.
- In a company, start with tasks that are small, low-risk and easy to check, not with large
  integrations.
- Watch the gap between free and paid models. If it widens, plan for unequal access.

## Context

- Written in April 2025. His 2025 year-in-review links back to it when it discusses vibe coding:
  [Vibe coding makes code free and ephemeral](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).
- He names two forces on the "dynamic range" of capability: scaling and test-time compute widen
  it, and distillation narrows it.

---

Date: 2025-04-06 · Kind: practice · Source: https://x.com/karpathy/status/1909007524885672262 · Themes: agentic-coding · Status: current

# Write boring code with NASA-style rules

Asked for a better language than C, he said C gives too much rope. He would move toward an ideal by subtracting features. His best pointer is the NASA requirements for C code. They remove what is too exotic, too clever or too fancy, and each line does one thing. His test for a banned feature: it lets you impress friends with esoteric knowledge, or it makes a reader ask what a snippet will print.

## In his words

> "Every line does one single thing."
> — Andrej Karpathy, 2025-04-06

## Apply

- Write a short style rule for your code base: no tricks, no code that needs a puzzle to read.
- Review code with one question: can a new reader say what this line does without running it?
- Our reading: give the same rule to coding agents in your project instructions, since plain code is also easier to review.

## Context

- April 2025, in a thread on C and safer subsets. He did not name a specific NASA document. Our reading: the rule is about readability, not only memory safety.
- Related: [understanding is the bottleneck](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-31-understanding-is-the-bottleneck.md).

---

Date: 2025-04-04 · Kind: view · Source: https://x.com/karpathy/status/1908109168952676855 · Themes: ai-progress, verification · Status: current

# Move AI predictions to betting markets

Many AI predictions live in blog posts, podcasts and tweets, where nobody checks them. He wants to move them to betting markets, which he calls the best tool we have for finding truth. The hard part is the question. A good one has a concrete outcome that anyone can resolve later. Industry metrics and macroeconomics fit best. He doubts GDP works well, and he doubts evals work, because evals are incomplete and easy to game. In a reply he gave an example: the number of employees at the five biggest call-centre companies falls by 50% by a set year.

## Apply

- When you write a forecast, state it as a yes/no outcome with a number, a source and a date.
- Prefer real-world measures, such as headcount in a task that "everyone" says is easy to automate. Avoid benchmark scores as the test.
- Reject vague metrics such as "share of firms using AI" as too soft to resolve.

## Context

- April 2025. Related: [AGI blends into GDP growth](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-agi-blends-into-gdp-growth.md) and [AI and jobs: radiologists and engineers](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-09-25-ai-and-jobs-radiologists-and-engineers.md).
- The call-centre example comes from his reply the same day.

---

Date: 2025-03-31 · Kind: practice · Source: https://x.com/karpathy/status/1906701941146624039 · Themes: output-and-reading, agent-native-software · Status: current

# Text chat is the terminal era; ask for visual output

Karpathy says text chat is like the command terminal era of computing. A correct answer is closer
to a custom web app made for each query: spatial, visual and interactive. Computing solved the
console problem with the GUI, and LLMs need the same step: answers as images, infographics,
slides, whiteboards, animations and small web apps. In his 2025 review he called Google's Nano
Banana a first hint of an LLM GUI, because text, images and world knowledge share one set of
weights. In May 2026 he put it as input and output channels. Audio is the best way for people to
talk to an AI, but vision is the best way for an AI to talk back, since much of the brain
processes images. He sees a ladder: raw text, markdown (today's default), HTML (already a good
default), and later interactive video made by a diffusion model.

## In his words

> "People actually dislike reading text - it is slow and effortful."
> — Andrej Karpathy, 2025-12-19

## Apply

- End a prompt with "Structure your response as HTML", then open the saved file in a browser.
- Ask for a diagram, table, chart or infographic when a text answer gets long.
- For a one-off task, ask for a throwaway UI, not a long list of command flags.
- Try "Present this as a slideshow" for material you must explain to other people.
- Keep the HTML file in your project as the readable record of long research results.

## Context

- Posts of 2025-03-31, 2025-06-19 (a demo that builds a UI per task, which he still called a
  horseless carriage) and 2025-11-23 (the GUI as an intelligent canvas); the sixth item of his 2025
  review; and a post of 2026-05-11 that quoted an article on HTML output in Claude Code. In 2026 he
  also wanted better input: pointing at things on screen, as with a person beside you.
- Next step: [Make model output easier to understand](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-10-02-make-model-outputs-easy-to-understand.md).
- Related: [LLMs are like 1960s operating systems](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-llms-are-1960s-operating-systems.md)
  and [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).
- He already used the punchcard image for prompt-and-paste chat in January 2025.
- In May 2025 he expected the GUI to be visual, generated for each prompt, and built mostly from procedural parts such as charts and components, because some content, such as a map, must be exact.
- In November 2025 he tried an image model that reasons, as a first sign of this GUI: see [Plan in text first, then illustrate](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-23-plan-in-text-then-illustrate-with-an-image-model.md).

---

Date: 2025-03-27 · Kind: view · Source: https://x.com/karpathy/status/1905051558783418370 · Themes: agent-native-software, agentic-coding · Status: current

# Building a web app is like assembling IKEA furniture; make it just work for AIs

Karpathy, newer to modern web development, listed what a 2025 app needs: frontend and backend,
hosting, a database, login, file storage, email, payments, background jobs, analytics,
monitoring, developer tools, secrets and more. None of it comes in one box. He said it took
him about three hours to connect a Supabase backend to a Vercel app and fix a few errors.
The moment you leave the "getting started" path you are lost.

His point: the pain is not code. It is configuration, plumbing and workflow. He expects large
rewards for whoever makes this accessible and working out of the box, for people and, more and
more, for AI agents. In a reply he added the familiar friction: dependency conflicts, downgrading
Node, and creating many accounts.

## In his words

> "a bit like assembling IKEA furniture"
> — Andrej Karpathy, 2025-03-27

## Context

- Related ideas: [Make the whole DevOps lifecycle agent-native](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-26-devops-as-code-for-agents.md)
  and [Build docs and tools for agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-03-12-write-docs-for-llms.md).

---

Date: 2025-03-23 · Kind: practice · Source: https://x.com/karpathy/status/1903671737780498883 · Themes: agentic-coding, learning-and-education · Status: current

# Build a small app in a language you do not know, one chat step at a time

Karpathy built an iOS app in Swift, which he had not used before, and had it running on his
phone about an hour later. He read no docs. He opened a ChatGPT conversation and followed its
instructions, step by step. The app is a calorie countdown: it tracks a net deficit against a
base metabolic rate. It was about 200 lines at first.

He kept steps small on purpose, because it taught him more and helped him avoid getting stuck.
He used a separate conversation for each stage (start, enhance, persistence, deploy to phone)
and shared them. By about 400 lines the work slowed and a long change got "dicey, long and
strenuous". His next move was to connect the model to Xcode or to Cursor. In a reply to a
blog post he said he uses "vibe coding" for moments like this, but that most of the time he
still reads the code and adds complexity slowly.

## In his words

> "I just opened a ChatGPT convo and followed instructions"
> — Andrej Karpathy, 2025-03-23

## Apply

- Choose a small tool you will use yourself and ask the chat for the first runnable version.
- Ask for one change at a time and run it each time; ask why when you do not understand a step.
- Start a new conversation for each stage and keep the links, so you can go back.
- Move to an editor-integrated tool once the code outgrows a chat window.

## Context

- Sequel to [Vibe coding is fine for throwaway projects](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-02-vibe-coding-for-throwaway-projects.md); later he found shipping is harder than building: [Vibe coding a real app is easy locally and hard to ship](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-27-vibe-coding-real-app-lessons.md).
- The code was for personal use; this is not a claim about production apps.

---

Date: 2025-03-19 · Kind: practice · Source: https://karpathy.bearblog.dev/the-append-and-review-note/ · Themes: knowledge-management · Status: current

# Keep one note you append to and review

Karpathy has used one single text note for years. He keeps it in a notes app and calls it
"notes". He adds every idea, to-do, quote, command or experiment log to the top as plain text. He
adds no folders and almost no tags. Old items sink under newer ones. From time to time he
scrolls down, skims, and copies anything that still deserves attention back to the top. He
merges related items and rarely deletes. Items that never get rescued stay findable with
search, but they stop competing for attention. Writing something down lets him clear his
working memory and move on.

## In his words

> "One text note ftw."
> — Andrej Karpathy, 2025-03-19

## Apply

- Create one note with one fixed name. Put new text at the top, never in a new note.
- Skip tags. Use only a few search prefixes for action types, such as `watch:`, `listen:`,
  `read:`.
- Schedule a short review: scroll down, copy what still matters to the top, merge items that
  repeat the same thought.
- Use the same note for scratch work: shell commands you forget, experiment commands and
  results, a morning to-do list.

## Context

- Written in March 2025, before the LLM-wiki style of note keeping. For the LLM-driven
  alternative, browse the `knowledge-management` theme.
- Our reading: the "review by gravity" rule is a cheap form of spaced attention, and it works
  with plain text that an LLM can also read.
- He still uses Obsidian for concrete projects, but not for daily notes.

---

Date: 2025-03-12 · Kind: practice · Source: https://x.com/karpathy/status/1899876370492383450 · Themes: agent-native-software, output-and-reading · Status: current

# Build docs and tools for agents, which curl instead of click

In March 2025 Karpathy said most content is still written for humans, while nearly all attention
will become LLM attention. In April he put it as a notice: the main audience of your product or
library is now an LLM. LLMs scrape, read and run curl; they do not look and click. He prefers one
markdown docs file with a copy button to a large docs site. In February 2026 he made it a test
for product owners. Can an agent use your product? Can your docs be exported as markdown? Is
there a CLI or MCP server, and are there skills for it? Command-line tools suit agents because
they are old technology that an agent can combine with the whole terminal. A prediction-market
CLI let Claude build a terminal dashboard in about three minutes.

## In his words

> "LLMs don't like to click, they like to curl."
> — Andrej Karpathy, 2025-03-12

## Apply

- Keep one complete markdown file per project (README or AGENTS.md) that you can paste into an
  agent, and add an `llms.txt` at your site root (his 2025 YC talk).
- Offer every important action as a CLI command or API call with clear help and machine-readable
  output.
- Write a skill file for the product: when to use it, the commands, the common errors.
- Replace "click this button" steps with commands an agent can run. Test by asking an agent to do
  the setup.
- Install the CLIs you use (git hosting, cloud, payments) and let your agent use them.

## Context

- Posts of 2025-03-12, 2025-04-21 and 2025-07-10 (what would a paper for an LLM look like?), the
  YC talk of 2025-06-17, and the "Build. For. Agents." post of 2026-02-24. At Sequoia 2026 he
  listed the same surfaces: markdown docs, CLIs, APIs, MCP servers, schemas and copy-paste agent
  instructions.
- Related facets: [Make the whole DevOps lifecycle agent-native](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-26-devops-as-code-for-agents.md),
  [Apps should be APIs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-apps-become-apis-agents-are-glue.md),
  [Explain things to agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-explain-to-agents-not-people.md) and
  [Opaque, binary-format tools](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-04-opaque-binary-uis-are-ngmi.md).
- [Software 3.0](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md) names agents as a third consumer
  of software.
- In March 2025 he said that putting a codebase into one file is easy; the hard part is knowledge locked in formats made for people, such as web pages, PDFs, images and video.
- For research results, see [Write new knowledge for AI readers, not as a PDF](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-07-10-write-research-for-llms-not-pdf.md).

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: output-and-reading, learning-and-education · Status: current

# Ask the model for a conceptual diagram of what you read

Karpathy likes to see the layout of an argument. After he loads a chapter, he asks for a
conceptual diagram of the whole chapter. The model often writes Mermaid code, and the app draws it
as a tree. The tree shows the main principle and its parts. He says the spatial layout helps
him remember the chapter. The same request works for books, chapters, source code and other
material. In his demo, the app that draws the result is Claude with its artifacts feature. It is
the one artifacts use he finds useful in daily work.

## Apply

- After a summary, send: "Create a conceptual diagram of this chapter" and attach the text.
- If the app does not render Mermaid, paste the code into a Mermaid viewer.
- Use it on code too: "Draw a diagram of how these modules call each other."
- Check the diagram against the text. It shows what the model understood, which may be wrong.

## Context

- From *How I use LLMs* (2025-02).
- Related: [Read books and papers together with an LLM](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-11-read-books-with-an-llm.md).

---

Date: 2025-02-27 · Kind: view · Source: https://x.com/karpathy/status/1895213020982472863 · Themes: ai-progress, research-and-training · Status: current

# Bigger pretraining improves everything a little, and is hard to point to

Karpathy used the release of GPT-4.5 to read the slope of gains from scaling pretraining alone.
Each 0.5 step in the version number is roughly 10X pretraining compute. GPT-3.5 was where chat
became a product. GPT-4 was better in a subtle way: word choice, nuance, analogies, rare
knowledge and humor all improved by a little, and slam-dunk examples were hard to find. He saw
the same pattern in GPT-4.5.

He stressed it had no reasoning training, so it does not move math and code. Its gains should
show on tasks that depend on knowledge, creativity and humor. To test that he ran five creative
prompts as a public poll. The crowd preferred the older model in four of five, which he found
surprising; he guessed at taste, a small sample, or weak examples, and waited for larger tests.

## In his words

> "Each 0.5 in the version is roughly 10X pretraining compute."
> — Andrej Karpathy, 2025-02-27

## Apply

- Compare a new large model with the older one on creative and open tasks, not only on math or code.
- Judge with careful side-by-side reading; a quick poll mixes taste levels and small samples.
- Look for the odd phrases, tropes and formulaic parts in the older answer, which a careful reader catches.

## Context

- He also noted a mistake in GPT-4.5's text: it joked that an old model is slow like dial-up, but older models are faster.
- On the evaluation problem that followed, see [Public benchmarks are gameable, so keep your own evals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-go-talk-to-the-model-private-evals.md).

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: context-and-prompting, llm-psychology · Status: current

# Check which tools the model has before you trust its answer

By default a model is a closed file of learned patterns with a knowledge cutoff. Two tools fix
its main gaps. Web search brings recent or niche pages into the context window. A code
interpreter lets it run a program instead of guessing a long calculation. Apps differ here.
In his tests, some models used a tool on their own, and others answered from memory and got a
big multiplication slightly wrong. He asks for search when the answer is recent, may have
changed, or is rare on the web. He often uses a search-first app for this.

## Apply

- Tick the "search" option when you ask about news, prices, product features or release dates.
- Ask for hard arithmetic to be done "with code". Check that a tool call really happened.
- For common, low-risk facts the model remembers well, no tool is needed. Verify the high-stakes ones.
- When an answer cites sources, open them. See [Treat LLM output as a first draft](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-llm-output-is-a-first-draft.md).

## Context

- From *How I use LLMs* (2025-02). Tool support has spread since then, so check what your app
  offers today. The point still holds: without a tool run, the model answers from memory.

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: context-and-prompting, llm-psychology · Status: current

# Try a fast model first, switch to a thinking model for hard problems

Karpathy asks users to check which model answers them. Free tiers often use a smaller model that
writes worse and makes up more. Bigger models cost the vendor more, so they sit behind paid tiers.
Thinking models are tuned with reinforcement learning and spend minutes on a chain of thought.
They help most on hard math and code. For simple questions, such as travel ideas, they only add
waiting time. His habit: use the fast model first. When the answer looks weak, switch to a
thinking model. In his demo, a gradient bug that the plain flagship model missed was found by a
thinking model after about a minute. He also asks several vendors the same question and calls
them his "LLM council".

## Apply

- Look at the model selector before you rely on an answer. Write the model name in your notes.
- If the work is professional and the answer is weak, try the top paid tier for a month and
  compare.
- Turn on "think" or "reasoning" mode only for hard debugging, math or planning.
- For open questions with no single right answer, ask two or three vendors and compare the replies.
- Our reading: write down your own router like his: which model for quick questions, hard questions, code and long research. Update it when the model list changes.

## Context

- From *How I use LLMs* (2025-02). The model names in the video (GPT-4o, o1 pro, Claude 3.5 Sonnet,
  Grok 3, DeepSeek R1) are dated. The habit does not depend on them.
- Related: [Check which tools the model has](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-check-which-tools-the-model-has.md).
- In June 2025 he shared his own ChatGPT router: the fast daily model for simple questions (about 40% of his use), the reasoning model o3 for hard or important ones (about 40%), GPT-4.1 for code edits (about 10%), and Deep Research to learn a topic in depth (about 10%). In replies he said the real choice is whether a question is important enough to wait for.

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: context-and-prompting · Status: current

# Give examples in prompts, and save the prompts you reuse

When Karpathy writes instructions for an LLM, he gives a description and also concrete examples.
This is a few-shot prompt. He says it always raises accuracy compared with a bare request. His
case: a custom GPT for language learning that takes one sentence and returns the vocabulary in
dictionary form, in a format he can paste into a flashcard app. It holds background (he is a
beginner), the task, and four examples. He points out that the saved prompt only saves typing:
you store it once and change only the input. He also uses global custom instructions for defaults,
such as the level of formality in Korean replies.

## Apply

- Write the task in one short paragraph. Add three or four input and output examples.
- Fix the output format in the examples, so you can paste the result into your next tool.
- When you reuse a prompt a third time, save it: a custom GPT, a Claude project, a skill or a slash command.
- Put stable preferences (tone, language level, role) in the global instructions, not in every chat.

## Context

- From *How I use LLMs* (2025-02), in the section on memory, custom instructions and custom GPTs.
- Related: [Start a new chat when the topic changes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-fresh-chat-per-topic.md).

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: context-and-prompting · Status: current

# Start a new chat when the topic changes

A chat is one growing sequence of tokens. The model reads all of it each time it writes the next
token. Karpathy treats this context window as the model's working memory. Old text that does not
help the new question can distract the model and lower its accuracy, and a long window makes each
token slower and more costly. So he starts a new chat when he switches topic, and keeps earlier
text only if it still helps. In March 2025 he weighed the trade-off in more detail: cost and speed,
how well the model attends over long text, signal to noise, and long chats being unlike the
training data. He called "new conversation" probably an implementation detail. The real answer is
agentic context management with memory, where the system decides what to keep.

## In his words

> "Think of the tokens in the context window as a precious resource."
> — Andrej Karpathy, 2025-02-27

## Apply

- Press "new chat" when you change subject, not only when a chat gets slow.
- Before you continue a long chat, ask whether its early messages still help. If not, copy only the
  useful facts into a new chat.
- Our reading, for coding agents (he spoke about chat apps): a compaction and a new session are
  both ways of managing context. Before either, write the key decisions, the state and the open
  items to a file, so nothing depends on the old history or on an automatic summary. Compare
  the two in your own runs: hours, compactions, tokens re-read at each wake-up, and what a new
  session spends to learn the state. Then choose.
- Group long-running work into projects, keep one default chat for quick questions, and when a chat gets too long, write a summary and continue in a new chat (a reader's setup he called interesting, 2025-03-20).

## Context

- From *How I use LLMs* (2025-02-27), a guide for general users shown with ChatGPT, and an X post
  of 2025-03-20.
- Related: [Context engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-25-context-engineering.md) and
  [Try a fast model first](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-fast-model-first-thinking-model-for-hard.md).

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: verification, llm-psychology · Status: current

# Treat LLM output as a first draft and check the sources

Karpathy shows several cases where a good-looking answer was wrong. A deep-research table of US
model labs left out a major lab and listed one that did not fit. A data-analysis run quietly
replaced a missing value with an invented number. It also reported a forecast of 1.7 trillion
while its own variable showed about 20 trillion. He compares the tool to a junior analyst who is
capable but absent-minded. His rule: use citations to reach primary sources, read the code the
model writes, and ask it to print the raw value when a number looks odd. For health questions he
also checks the label on the box.

## In his words

> "Treat this as your first draft."
> — Andrej Karpathy, 2025-02-27

## Apply

- Open at least the main cited sources before you use a claim from a research report.
- When a model analyses data, read its code. Look for filled-in missing values and silent assumptions.
- If a printed number disagrees with the summary text, ask for the variable printed alone.
- Use deep-research reports to build a reading list, not as final facts.

## Context

- From *How I use LLMs* (2025-02), with examples from ChatGPT deep research and its data analysis tool.
- Related: [Give agents success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).

---

Date: 2025-02-27 · Kind: practice · Source: https://www.youtube.com/watch?v=EWvNQjAaOHw · Themes: agent-native-software · Status: current

# Ask the model to write a small app just for you

In the video, Karpathy asks Claude for 20 flashcards from a text. Then he asks it to use the
artifacts feature to write an app that quizzes him on them. The model writes a small React app
and the interface runs it in the browser. It has no backend or database. He calls this a different
paradigm. Usually engineers ship one app for everyone, with a few settings. Here the model writes
the app for one person. He adds that he is not a daily user of this and uses it now and then.

## Apply

- Ask for the content first (cards, a checklist, a table). Then ask for a tiny app that uses it.
- Keep the request to one screen and one job. Single-file, local apps work best.
- Ask the model to keep the data inside the page, so you can copy it out later.
- Before you search for an app, ask a model to build a small one for the task, and drop it when the task is done.

## Context

- From *How I use LLMs* (2025-02). An early view of a theme he develops later in 2026: software
  written on demand, with agents as the user. See
  [Apps become APIs and agents are the glue](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-20-apps-become-apis-agents-are-glue.md).
- In May 2025 he said many habits break when an app takes a second to make: software stops being scarce, and you no longer have to pick from what exists. He also found it easier to build his own tool than to find one.

---

Date: 2025-02-24 · Kind: view · Source: https://x.com/karpathy/status/1894099637218545984 · Themes: learning-and-education, ai-progress · Status: current

# Agency matters more than intelligence, and is scarcer

Karpathy says he got this wrong for decades, because culture, media and IQ talk put
intelligence on a pedestal. Agency, the capacity to take initiative and shape what happens, is
much more powerful and much more scarce. He turned it into three questions for the reader: are
you hiring for agency, are you educating for agency, and are you acting as if you had ten times
more agency? The post quoted a comment that intelligence is now on tap, so agency matters even more.

## In his words

> "Agency is significantly more powerful and significantly more scarce."
> — Andrej Karpathy, 2025-02-24

## Apply

- Ask his three questions of your team and yourself: are we hiring for agency, educating for
  agency, and acting as if we had 10X agency?

## Context

- Later posts on the same line: [Expertise may matter more with agents, not less](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-expertise-multiplier-generalists-and-10x.md) and [Test candidates on building and securing a project with agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-30-hire-for-agentic-engineering.md).

---

Date: 2025-02-18 · Kind: view · Source: https://x.com/karpathy/status/1891909328795492703 · Themes: ai-progress · Status: current

# Three breakthroughs so far; autonomous agents need a few more

Asked whether LLM limits come from the architecture, Karpathy first limited the question. Today's
evals are small "task" problems served neatly, even when hard. When people talk about AGI they
mean an autonomous swarm of agents doing long jobs across society. He believes that needs major
research breakthroughs and not scaling alone.

He counted three breakthroughs so far: the Transformer (about 2017), ChatGPT with supervised
finetuning and RLHF (about 2022), and reinforcement learning for thinking models (about 2024),
which is still maturing. He expects a few more of that size.

A month earlier he made the same split on the Humanity's Last Exam benchmark: every task served
on a platter will improve, even PhD-level ones, but jobs needing long, error-correcting chains
of tasks will take longer, a Moravec's paradox effect. He called the "last exam" name misleading.

## In his words

> "I think we need a few more conceptual leaps of this class."
> — Andrej Karpathy, 2025-02-18

## Context

- Same reasoning as [Hard closed problems are easy; long autonomous jobs are hard](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-11-10-llms-solve-hard-prompts-but-fail-long-jobs.md).
- It fits his later claim in [This is the decade of agents, not the year of agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-decade-of-agents-not-year.md).

---

Date: 2025-02-14 · Kind: practice · Source: https://x.com/karpathy/status/1890208670732124372 · Themes: context-and-prompting, agentic-coding · Status: current

# Pack your project into one prompt for a quick, scoped question

In February 2025 Karpathy asked more apps to offer an "Export for prompt" button, after a trick
that turns a GitHub repo URL into a gitingest URL to get the whole repo as one text. In August he
described his own routine. When agents search the whole repo and think for minutes, he sometimes
skips them. He runs a script that packs all project files into one text with the tool
`files-to-prompt`. He opens a new chat, pastes the text, asks one question at the end, and chooses
the model by hand. This works because his project is small enough to fit in the context.

## Apply

- Keep a script that bundles your files into one text file (`files-to-prompt .` or gitingest).
- Start a new chat per question, paste the bundle, and put the question last.
- Choose the model yourself: a fast one for spot checks, a strong one for hard bugs.
- Check the bundle size first; if it does not fit, send only the folders that matter.

## Context

- Quote post of 2025-02-14 about gitingest, then a
  [reply of 2025-08-09](https://x.com/karpathy/status/1954227464596832670) in a thread about agents
  that are too eager. Works only for repos that fit the context window.
- Related: [Models are too agentic: state the stakes and scope](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-09-models-too-agentic-state-stakes-and-scope.md).
- His later tool for the same job: [Flatten a repository into one page for humans and LLMs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-20-render-repo-for-humans-and-llms.md).

---

Date: 2025-02-12 · Kind: practice · Source: https://x.com/karpathy/status/1889714240878940659 · Themes: security-and-dependencies, llm-psychology · Status: current

# Invisible Unicode characters can smuggle data and prompts into an LLM

Karpathy knew about look-alike characters such as a Latin "e" and a Cyrillic "е". He then
pointed to a write-up showing that you can also hide any byte stream inside a single character
using Unicode "variation selectors". An emoji that looks like one symbol then turns into dozens
of tokens, as a tokenizer viewer showed.

He ran a test of prompt injection with the hidden bytes. It worked only when the prompt gave
explicit decoding hints. Thinking models seemed more exposed, because they notice odd bytes and
get curious. DeepSeek-R1 spent ten minutes searching for a pattern and nearly decoded it. He
noted that if the article enters pretraining data, a future model might decode it with no hint.
Later he called it a really bad idea of Unicode. Two readers pointed out that this is a
Unicode property, not a UTF-8 one; he agreed.

## In his words

> "you can also smuggle arbitrary byte streams in any character"
> — Andrej Karpathy, 2025-02-12

## Apply

Our reading: all steps below go further than the idea itself.

- Paste suspicious text into a tokenizer viewer. A short visible string with a huge token count
  signals hidden characters.
- Treat text from outside your control (web pages, issues, documents) as untrusted before it
  goes into an agent's context.
- Strip or flag variation selectors and other invisible code points in untrusted input before it
  reaches a model. Variation selectors are U+FE00-U+FE0F and U+E0100-U+E01EF (our addition: his
  post names no range; the Unicode standard defines them).

## Context

- A test on Grok 3 a few days later: it could not decode the hidden message even with a hint in
  Rust code. See [Public benchmarks are gameable, so keep your own evals](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-go-talk-to-the-model-private-evals.md).
- Related: [Third-party models can hide sleeper agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-01-12-sleeper-agent-llms-and-poisoned-models.md).

---

Date: 2025-02-07 · Kind: practice · Source: https://x.com/karpathy/status/1887980449550758121 · Themes: learning-and-education · Status: current

# Make a long from-scratch intro to your own field

After his 3.5-hour "Deep Dive into LLMs like ChatGPT", Karpathy said one reason he made it was
to inspire others to make the same kind of video in their own areas of expertise. His example:
he read an article about superconductivity in layered graphene and wanted an expert to take him
through that area in a three-hour intro from scratch. He said many other fields would work too.

The format is a general-audience walkthrough that covers the whole pipeline of a subject and
adds mental models. He described the recording process as simple: OBS to record, a few takes
per clip, and iMovie to stitch the clips.

## In his words

> "I hope to inspire others to make equivalents in their own domains of expertise"
> — Andrej Karpathy, 2025-02-07

## Apply

- Pick the area you know best and outline the whole path from basics to current practice, in
  the order a newcomer needs it.
- Record in short clips with a free screen recorder and redo clips that go wrong; stitch them
  in a basic editor.
- Add worked examples and say how experts think about the field, not only what the facts are.

## Context

- The video pairs with a second one, "How I Use LLMs", and is also an input for
  [Read with an LLM in three passes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-read-everything-with-llms-three-passes.md): long, structured material is easier to learn from.
- A reader asked for a deeper part 2; he asked for bullet-point suggestions.

---

Date: 2025-02-02 · Kind: practice · Source: https://x.com/karpathy/status/1886192184808149383 · Themes: agentic-coding · Status: current

# Vibe coding is fine for throwaway projects

In February 2025 Karpathy named a style of work: vibe coding. You give in to the vibes. You talk to the editor by voice, accept every change, and never read the diffs. You paste error messages back with no comment. If a bug will not go away, you work around it. He says this is fine for throwaway weekend projects. The code grows past what you understand, and you stop caring that it exists. The idea is a label for a real way of working, with a clear limit on where it is safe.

## In his words

> "forget that the code even exists"
> — Andrej Karpathy, 2025-02-02

## Apply

- Use vibe mode for disposable prototypes, demos and one-off scripts only.
- For code you keep, read the diffs and run tests. See [a rhythm for code you care about](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-24-ai-assisted-coding-rhythm.md).
- Paste raw error output into the agent instead of describing it. This is the cheapest loop for a throwaway project.
- Decide before you start whether the project is throwaway. Write that in the first prompt.

## Context

- Posted 2025-02-02, when the main tool was an AI editor with a composer pane and a voice tool.
- His own method for kept code is in [a rhythm for code you care about](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-24-ai-assisted-coding-rhythm.md).
- By December 2025 he described code as cheap and ephemeral: [Code is now free, custom and disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-19-code-is-free-and-ephemeral.md).
- Still current. A year later he named the professional, reviewed form
  [agentic engineering](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-02-04-agentic-engineering-vs-vibe-coding.md), but kept vibe coding
  as the right mode for throwaway work: at Sequoia 2026 he said it "raises the floor".
- He sees AI help as a slider, from coding as it was three years ago to full vibe coding. In March 2025 he said he seldom goes fully vibe: most of the time he still reads the code and learns how the parts fit.

---

Date: 2025-02-01 · Kind: practice · Source: https://x.com/karpathy/status/1885740680804504010 · Themes: verification, research-and-training · Status: current

# Evaluate models by making them play each other

Fixed evals saturate. Karpathy likes the idea of using games to compare LLMs head to head.
When a model plays another intelligent player, the difficulty balances itself: a strong player
meets a strong opponent, so the same environment keeps giving a signal. He pointed to early
attempts, such as a text-game collection with a gym-style interface, and suggested Codenames
and Diplomacy as good fits. In both, much of the difficulty comes from reasoning about other
players and about the world, not from the rules.

He built a small Battleship app himself, in which any two models fight in real time. Another
builder reported that models playing Diplomacy came across as meek and peaceful, which tells
you something about their training.

## In his words

> "Playing against another intelligent entity self-balances and adapts difficulty"
> — Andrej Karpathy, 2025-02-01

## Apply

- Pick a game with simple rules and hidden information or negotiation (Codenames, Battleship,
  Diplomacy) and let two models play. Log the moves and the reasoning.
- Swap which model plays which side, and play several rounds, because single games are noisy.
- Read the game logs, not only the win count: they show how each model plans and communicates.

## Context

- Posted a day after the DeepSeek-R1 wave, when the field was looking for new RL environments;
  see [RL environments are the new training data](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-27-rl-environments-are-the-new-data.md).
- Our reading: he suggests the idea and early results; he does not claim a finished benchmark.

---

Date: 2025-01-30 · Kind: practice · Source: https://x.com/karpathy/status/1885026028428681698 · Themes: learning-and-education, research-and-training · Status: current

# LLMs need the practice problems of a textbook

Karpathy notes that a textbook has three kinds of content. Exposition matches pretraining. Worked problems match supervised finetuning. Practice problems with answers match RL. LLMs have plenty of the first two. The third is the nascent frontier. Later posts extend this. In November 2024 he said exercise pages gathered at scale now move billions of weights, so textbooks are partly written for LLMs. In August 2025 he proposed turning each textbook into LLM-first material: markdown exposition, worked problems as examples, practice problems as RL environments with answer keys, and problem generators.

## In his words

> "They have to read, and they have to practice."
> — Andrej Karpathy, 2025-01-30

## Apply

- When you teach an agent a skill, give worked examples and practice tasks with checkable answers, not only explanations.
- Turn your docs into markdown with runnable examples that an agent can query.
- Use a problem generator to make many test cases for the same skill.
- Keep practice problems at the right level: not trivial, not impossible. If every problem is too hard, reinforcement learning gets no signal (his reply of 2025-02-19).
- Turn the practice problems of a textbook into RL-style environments, each checked against its answer key (his reply of 2025-08-27).

## Context

- Posted 2025-01-30; extended on 2024-11-15 (earlier) and 2025-08-28.
- Related: [RL environments are the new data](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-08-27-rl-environments-are-the-new-data.md).

---

Date: 2025-01-28 · Kind: practice · Source: https://x.com/karpathy/status/1884336943321997800 · Themes: research-and-training, context-and-prompting · Status: current

# RL gives thinking models strategies you cannot write by hand

Karpathy says that RL on thinking models produces cognitive strategies: trying angles, making analogies, backtracking. People cannot hand-program these. The model may even develop an internal language that we cannot read. He notes that a Move 37 equivalent for these models, a move no human would make, has not been seen yet.

## Apply

- On hard problems, give the model room to think and set the goal and the check, not the method.
- Avoid long step-by-step recipes for reasoning models. See [success criteria, not steps](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-01-26-success-criteria-not-steps.md).
- Test a hard task at higher effort before you rewrite the prompt.

## Context

- Posted 2025-01-28, soon after open reasoning models appeared.
- A day earlier he tied synthetic data to RL: see [Filtering synthetic data is crude reinforcement learning](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-01-27-synthetic-data-and-rl-are-one-loop.md).
- On 2025-11-25 he said AI had gone far past what he expected. His old mistake was to assume that someone must find and write down the algorithm. Instead, it emerges when you pretrain and finetune a big network on many tasks.

---

Date: 2025-01-27 · Kind: view · Source: https://x.com/karpathy/status/1883941452738355376 · Themes: research-and-training · Status: current

# Filtering synthetic data is crude reinforcement learning

Karpathy describes two kinds of learning, in children and in networks. Imitation (pretraining,
supervised finetuning) copies a source. Trial and error (reinforcement learning) tries things
and learns from the result. He says almost every striking deep learning result comes from the
second kind, because the model can find things a human labeller would not know to write down.

He then links synthetic data and RL. In RL the "trial" is the model generating data and the
"error" is the reward. If you generate data and keep only what passes a filter, your filter is a
simple 0-or-1 reward. In a reply he gave the core loop: sample many answers, keep the best,
train on it, repeat. Fancier maths, such as smooth advantages and trust regions, refines this.
He also noted that data is partly downstream of compute, because you can spend compute to make it.

## In his words

> "your filter is straight up equivalent to a 0-1 advantage function"
> — Andrej Karpathy, 2025-01-27

## Apply

- To improve a model on a task you can score, sample several answers, keep the highest-scoring
  one, and finetune on it. Then repeat.
- Keep a reward or check you trust; the loop copies whatever the check rewards.
- Use imitation data for what humans already know, and reward-based training for strategies
  you cannot write down.

## Context

- A reaction to DeepSeek-R1, a week after its release. It builds on [RL gives thinking models strategies you cannot write by hand](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-01-28-cognitive-strategies-cannot-be-hand-programmed.md).
- He added that RL is powerful but RLHF is not; see [An LLM judge gets gamed when you optimise against it](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-08-07-rlhf-is-just-barely-rl.md).

---

Date: 2025-01-17 · Kind: view · Source: https://x.com/karpathy/status/1880057571668807830 · Themes: learning-and-education, ai-progress · Status: current

# People may train their minds the way they go to the gym

A reply to a worry that skills AI can teach, AI can also perform for the economy, so learning
loses its point. Karpathy answers with an analogy from the physical world. Machines removed the
economic need for muscle, but people still go to the gym in large numbers. In a world of high
automation, the economy will need fewer human brains. People will still do the mental
equivalent of going to the gym, for the same reasons.

Our reading: he separates the economic reason to learn from the personal one, and says only
the first one weakens.

## In his words

> "People don’t need muscles when we have machines but still go to gym"
> — Andrej Karpathy, 2025-01-17

## Context

- Related view on effort: [Learning takes effort, unlike entertainment](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-02-10-learning-is-not-supposed-to-be-fun.md).
- Later he argued for testing yourself without AI: [Use AI to learn, but test without it](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-24-learn-with-ai-but-test-without-it.md).

---

Date: 2025-01-02 · Kind: view · Source: https://x.com/karpathy/status/1874678592702972398 · Themes: llm-psychology · Status: current

# A good assistant skips the lecture and the flattery

Asked what he would change about Claude, Karpathy listed quirks of tone. The model talks down
to him with phrases like "it's important to". It follows a refusal with a lecture on morals. It
praises him too fast, which feels fake and a little manipulative. It apologises too much. What
he wants is the model's best effort at the problem, from a calm teammate and not an
over-excited buddy. A plain refusal is fine.

Six weeks later he praised a new ChatGPT 4o personality that felt more like a friend than an HR
partner. It was more chill, a little sassy, and it named the user's frustration. He still
disliked its long lists and growing use of emoji.

## In his words

> "I wish Claude would talk down to me less and do less grandstanding"
> — Andrej Karpathy, 2025-01-02

## Apply

- Add a line to your custom instructions such as: "Answer directly. No moral lectures, no
  compliments on my question, no apology unless you made an error."
- When you test a new model, note its tone as well as its accuracy: flattery, lists, emoji, lectures.
- If a model refuses, ask it to state the refusal in one sentence and move on to what it can do.

## Context

- The first post answers a question from a researcher at Anthropic. He also said Claude seemed
  to have had the most thought put into its personality.
- Later views on the same theme: [AI personality should feel like a teammate](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-21-ai-personality-like-rocky.md).

---

Date: 2024-12-25 · Kind: practice · Source: https://x.com/karpathy/status/1872038630405054853 · Themes: agentic-coding · Status: current

# Cognitive load is what matters in software engineering

Karpathy endorsed an essay titled Cognitive load is what matters. He called it the most true, and least practiced, viewpoint of software engineering. The idea: good code is code that a person can hold in their head. Our reading: this matters more as agents write more code, because people must still review it.

## Apply

- Ask the agent for simple, flat code with few abstractions and short functions.
- In review, ask: could I explain this in two minutes? If not, simplify.
- Delete code paths and options that no one uses.

## Context

- Posted 2024-12-25. This entry only records his endorsement. Read the essay for its argument.

---

Date: 2024-12-14 · Kind: practice · Source: https://x.com/karpathy/status/1868061331355840704 · Themes: agentic-coding, verification · Status: current

# Judge an agent as you would a junior intern

Karpathy says the capability to watch is not PhD-level problems. It is the junior intern: you say set up your Slack, read the onboarding docs, do this task, and check in next week. Our reading: that test needs long coherent work, not clever answers. It is a useful way to rate an agent and to write briefs for one.

## In his words

> "whether you'd hire it as a junior intern"
> — Andrej Karpathy, 2024-12-14

## Apply

- Write the agent instruction file like onboarding docs: goals, layout, commands, who decides what.
- Assign week-sized tasks with a clear check in, and review at the check in.
- Rate a new agent by one real task of this size.

## Context

- Posted 2024-12-14. His 2025 view of the model as an over-eager intern is in [tight leash](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-04-24-ai-assisted-coding-rhythm.md).

---

Date: 2024-12-11 · Kind: practice · Source: https://x.com/karpathy/status/1866896395363553418 · Themes: learning-and-education, output-and-reading · Status: current

# Read books together with an LLM, one chapter at a time

Karpathy rarely reads a book alone now. In December 2024 he called reading together with an LLM
one of his favourite uses, and said an AI-native reader app would be a hit. In early 2025 he
showed the habit. He pastes the raw text of a chapter into the chat and asks for a summary, so he
knows its shape. Then he reads the chapter himself and asks a question whenever something is
unclear. The model may know the book in general but not recall the chapter, so the text goes into
the context window. He says this helps most with papers from other fields and with old texts. In
November 2025 he released reader3, a small self-hosted EPUB reader that shows one chapter at a time
for easy copying. He prefers EPUB to PDF because code can parse it into clean text.

## In his words

> "And don't read books alone."
> — Andrej Karpathy, 2025-02-27

## Apply

- Get the book as EPUB (Project Gutenberg has many) and convert it to text with reader3 or a small
  parser.
- Paste one chapter into a new chat. Name the book. Ask: "Summarise this chapter."
- Read the real chapter alongside. Ask short questions where you stall, or ask for context and
  objections.
- Use one chat per chapter so the window stays small and the discussion stays on topic.
- For a paper, upload the PDF and ask for a summary first. Figures may be lost in conversion.

## Context

- X post of 2024-12-11; *How I use LLMs* video of 2025-02-27 (examples: a 1776 economics book and a
  genome-model paper); reader3 repository of 2025-11-18, about 90% vibe coded; a reply of
  2026-01-11 that pointed to reader3.
- In early 2025 he found the workflow clunky: no tool let him highlight a passage and ask about it.
- Related: [Read with an LLM in three passes](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-read-everything-with-llms-three-passes.md),
  [Convert books and PDFs with an agent](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-04-02-convert-documents-with-an-agent.md),
  [Ask for a diagram of what you read](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-27-ask-for-a-diagram-of-what-you-read.md) and
  [Treat small apps as disposable](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-18-disposable-vibe-coded-apps.md).
- In February 2025 he vibe coded a reader for The Wealth of Nations: click a paragraph, ask a question, and the app sends the chapter, the paragraph and the question to an LLM.
- In September 2025 he said a book works better as a set of prompts that start your own questions, explanations and examples, like synthetic data generation for a model, than as text to absorb, like pretraining data.

---

Date: 2024-11-29 · Kind: view · Source: https://x.com/karpathy/status/1862565643436138619 · Themes: llm-psychology · Status: current

# Asking an AI is like asking the average data labeler

Karpathy says that when you ask an AI, you are closer to asking an average data labeler, who is skilled in code and math, whose style was distilled into the model. RL changes this only in verifiable domains. On judgment questions you get a competent consensus, not a unique expert view.

## In his words

> "asking the average data labeler"
> — Andrej Karpathy, 2024-11-29

## Apply

- For judgment calls, treat the answer as a competent consensus. Ask for dissent and for sources.
- Ask for the opinion of a named type of expert, with the model simulating them. See [there is no 'you'](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-07-llms-simulate-there-is-no-you.md).
- Use checks and tests where you can, since that is where models exceed the labeler.

## Context

- Posted 2024-11-29.
- Related: [commercial evolution](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-29-llms-shaped-by-commercial-evolution.md).

---

Date: 2024-11-10 · Kind: view · Source: https://x.com/karpathy/status/1855659091877937385 · Themes: ai-progress, verification · Status: current

# Hard closed problems are easy; long autonomous jobs are hard

Karpathy applies Moravec's paradox to LLM evals. Models solve hard, closed problems that arrive neatly in a prompt. They struggle with long, autonomous, coherent sequences, so you would not hire them for menial jobs. He asks for evals of such "menial job" work: long context, coherence, autonomy and common sense.

## Apply

- Give agents closed, well-specified chunks of work.
- Handle long-horizon coordination yourself, or use tight checkpoints and a state file.
- Test a new agent on a long task from your own work, not only on puzzles.

## Context

- Posted 2024-11-10. Related: [the junior intern test](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-14-would-you-hire-it-as-a-junior-intern.md), which sets a week-sized task as the bar.
- In 2025 he made the same split about the Humanity's Last Exam benchmark: neatly served tasks will improve, while long, error-correcting jobs stay hard, a case of Moravec's paradox. See [Three breakthroughs so far](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-02-18-three-breakthroughs-more-leaps-needed.md).

---

Date: 2024-10-06 · Kind: practice · Source: https://x.com/karpathy/status/1843005000206909856 · Themes: output-and-reading · Status: current

# Break the shared LLM house voice with a style guide

Karpathy is puzzled that all LLMs sound alike. They overuse lists and words like multifaceted, they offer help too often, and their answers have the same length. This house voice comes from shared training habits. Our reading: you can change it with explicit instructions on length, format and tone.

## Apply

- Add a short style guide to the system prompt or instruction file: length, tone and format.
- List constructions to avoid, such as stock phrases and lists of three.
- Give one example of the voice you want.

## Context

- Posted 2024-10-06.
- In May 2025 he called the convergence of models eerie: not fully true, but true to a surprising degree. In June 2025 he showed five chatbots that all answered 27 when asked to guess a number from 1 to 50.
- In July 2025, after many models gave a science-fiction hero the same name, he wondered if subliminal learning during distillation plays a part, and guessed that more LLM text on the web could pull models toward a shared fixed point. Similar training sets are a simpler cause.
- In August 2025 he guessed why finetuning did not make models differ: it is short and carries few bits, so it mostly moves style on a shared pretrained base. RL, with many more bits over a much longer run, might separate models more.

---

Date: 2024-08-07 · Kind: practice · Source: https://x.com/karpathy/status/1821277264996352246 · Themes: research-and-training, verification · Status: current

# An LLM judge gets gamed when you optimise against it

Karpathy explains why reinforcement learning from human feedback is only barely RL. It optimises a
reward model that scores the vibe of an answer, and models game it quickly with adversarial
examples. Real RL needs a reward that is hard to game, such as winning at Go. RLHF still helps,
because judging is easier than writing. In his 2025 Dwarkesh interview he told how the same problem
hits labs that use an LLM to give partial credit. In one run the reward jumped to perfect. The
answers had turned into nonsense, a string of repeated letters, and the judge rated it fully
correct. You can add such cases to the judge's training data, but there are endless more. So a
judge works for perhaps 10 or 20 optimisation steps, not hundreds.

## Apply

- Prefer checks that cannot be argued with (tests, exact match, executable results) over a judge.
- If you loop an agent against an LLM grader, limit the rounds and read samples.
- Treat a sudden jump to a perfect score as a warning. Read the raw outputs before you trust it.
- Use best-of-N: generate several candidates and let a judge pick.
- If you must use a judge, use a different model from the one being optimised, and spot-check by
  hand.

## Context

- X post of 2024-08-07; Dwarkesh Patel interview of 2025-10-17.
- Related: [RL sucks supervision through a straw](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-10-17-rl-supervision-through-a-straw.md),
  [LLM judges grade too freely](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-12-10-llm-grader-identity-bias.md) and
  [Software 2.0 automates what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).

---

Date: 2024-03-12 · Kind: view · Source: https://x.com/karpathy/status/1767598414945292695 · Themes: agentic-coding · Status: current

# Software engineering climbs an autonomy ladder like self-driving

In March 2024 Karpathy compared software engineering to self-driving. The steps: manual coding, autocomplete of lines, chat that writes chunks, editors that write larger diffs, and agents that coordinate the terminal, browser and editor with a human watching at a higher level. The editor and the oversight experience must change a lot: what to look at, how to nudge, how to debug. Engineering becomes supervising automation in English.

## In his words

> "supervising the automation"
> — Andrej Karpathy, 2024-03-12

## Apply

- Invest in oversight: diff summaries, run logs and state views for agent work.
- Practice reviewing at a higher level: check the plan and the result before the code.
- Decide what each agent may do without asking.

## Context

- Posted 2024-03-12. Still matters: agent tools of 2025 follow this ladder. See [the junior intern test](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2024-12-14-would-you-hire-it-as-a-junior-intern.md) and [AI and jobs](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-09-25-ai-and-jobs-radiologists-and-engineers.md).

---

Date: 2024-02-21 · Kind: practice · Source: https://x.com/karpathy/status/1760388761349927356 · Themes: learning-and-education · Status: current

# Build the thing, then build the ramp

Karpathy says his micrograd project stagnated until he made a from-scratch video. The same code then got about a hundred times more engagement. Technical people assume code and a README explain themselves. They do not. Accessibility leaves ten to a hundred times of reach on the table. His rule has two steps: build the thing, then build the ramp that lets others climb onto it.

## In his words

> "Step 1 build the thing. Step 2 build the ramp."
> — Andrej Karpathy, 2024-02-21

## Apply

- After a project works, ask the agent for a walkthrough: a guided tour of the code, an HTML explainer or a script for a video.
- Write the ramp for your future self too: why the design is this way, and how to run it.
- Test the ramp on someone new to the code.

## Context

- Posted 2024-02-21. This post was about teaching; AI makes the ramp cheap to produce.

---

Date: 2024-02-10 · Kind: practice · Source: https://x.com/karpathy/status/1756380066580455557 · Themes: learning-and-education · Status: current

# Learning takes effort, unlike entertainment

Karpathy says much "educational" short video is entertainment. Learning should feel like effort, like sweating at the gym. He suggests you declare your intent as binary: you are entertained, or you are learning. For learning, go to the meal (textbooks, docs, papers), set aside about four hours, take notes, reread and rephrase. The idea matters more with LLMs, because a model can make passive explanations that feel like learning and are not.

## In his words

> "Learning is not supposed to be fun."
> — Andrej Karpathy, 2024-02-10

## Apply

- Ask the model to quiz you, to make you rebuild a result from memory, and to critique your explanation, not only to summarize.
- Block a long focused session for a new topic. Take notes by hand or in your own words.
- Say at the start whether you want entertainment or learning.

## Context

- Posted 2024-02-10. In November 2025 he added a short reply that fake learning is 'edutainment'.
- Related: [AI and schools](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-24-learn-with-ai-but-test-without-it.md).

---

Date: 2024-01-20 · Kind: view · Source: https://x.com/karpathy/status/1748784260318990496 · Themes: knowledge-management, research-and-training · Status: current

# The ideal training data is the thinking, not the final text

Karpathy says the ideal data for training an LLM is the whole sequence of internal thoughts and edits that led to a text, not the final text. We usually only keep the output. For a user, the point carries over: an agent that reads your reasoning can help better than one that sees only results.

## Apply

- Keep decision logs and working notes in the repo, so agents see why choices were made.
- Write the reasons next to each decision, not only the outcome.
- Keep failed attempts in a short 'tried and rejected' list.

## Context

- Posted 2024-01-20. Related practice: [system prompt learning](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-05-10-system-prompt-learning.md).
- In January 2025 he guessed that the most valuable data would be screen recordings of exceptional people at work, ideally with their goals and reasoning recorded too.

---

Date: 2024-01-12 · Kind: practice · Source: https://x.com/karpathy/status/1745921205020799433 · Themes: security-and-dependencies · Status: current

# Third-party models can hide sleeper agents

Karpathy points to a risk beyond prompt injection: poisoned training text or open-weights models that behave normally until a trigger phrase appears. Safety finetuning may not remove such behavior. So a model is part of your supply chain, like a package.

## Apply

- Treat third-party models, skills and MCP servers as untrusted code. Read them before use.
- Run new tools in a sandbox, with no access to secrets.
- Limit what an agent can do with scoped tokens.

## Context

- Posted 2024-01-12, from a paper on sleeper agents. Still relevant to skills and MCP servers.

---

Date: 2023-12-08 · Kind: practice · Source: https://x.com/karpathy/status/1733299213503787018 · Themes: llm-psychology, verification · Status: current

# Hallucination is the LLM's main feature, so the system must check it

In December 2023 Karpathy called LLMs dream machines that are directed by prompts. A search engine dreams 0 percent of the time, which gives it a creativity problem. An LLM dreams 100 percent of the time, which gives it a hallucination problem. So the assistant system should reduce the risk with retrieval, comparing several samples, reflection, verification chains and tool use. This view still helps when you design checks around an agent.

## In his words

> "Hallucination is not a bug, it is LLM's greatest feature."
> — Andrej Karpathy, 2023-12-08

## Apply

- Ground the agent in real files and docs, and ask it to cite the file and line.
- For a risky claim, ask the same question several times and check where the answers differ.
- Let tools check facts (run the code, query the data) before you trust a statement.

## Context

- Posted 2023-12-08. Kept because the advice still applies.
- Related: [what you can verify](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-11-17-software-2-automates-what-you-can-verify.md).
- He says he probably coined "hallucinate" for this in his 2015 RNN post. Hinton's "confabulate" is closer to human psychology but too tied to facts, so he prefers the broader word.

---

Date: 2023-09-28 · Kind: view · Source: https://x.com/karpathy/status/1707437820045062561 · Themes: agent-native-software, llm-psychology · Status: current

# The LLM OS, with the model as CPU and the agent as kernel

In 2023 Karpathy said the LLM is not a chatbot but the core of a new operating system. It
orchestrates input and output across text, images and audio, a code interpreter, a browser and an
embeddings memory. In November 2023 he sketched this "LLM OS": the model is the CPU, the context
window is RAM, files and embeddings are the disk, and classical tools such as a calculator, a
Python interpreter and a browser are peripherals. Security ideas from classic operating systems
carry over. In March 2026, asked whether an agent is just a prompt, skills and tools, he sharpened
the picture. The LLM is like a CPU, but it works on tokens, not bytes, and its behaviour is
statistical and vague, not exact. The agent is the kernel that runs on that CPU.

## In his words

> "Agent = operating system kernel"
> — Andrej Karpathy, 2026-03-31

## Apply

- Design the agent harness like an OS: scheduling, memory, permissions and tool input/output live
  in the agent layer; the LLM does the reasoning steps.
- Treat the context window as scarce RAM: load what a step needs, and page the rest in from files.
- Put exact work (maths, parsing) in classical tools the agent calls, since the CPU is fuzzy.
- Plan permissions and sandboxing at the start.

## Context

- Posts of 2023-09-28 and 2023-11-10, and replies of 2026-03-31 to Guido van Rossum. The 2023
  specs (GPT-4 Turbo, 128K tokens) show how much has changed; the structure has held.
- In 2023 he called the LLM itself the "kernel process"; by 2026 the kernel role sits with the
  agent. Our reading: a refinement as agent harnesses matured, not a reversal.
- The 2025 version of the analogy adds cloud time-sharing: [LLMs are like 1960s operating systems](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-llms-are-1960s-operating-systems.md).
- Related: [Software 3.0](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2025-06-17-software-3-0-english-as-programs.md) and
  [DevOps as code for agents](https://raw.githubusercontent.com/ntroccoli/karpathy-ideas/main/ideas/2026-03-26-devops-as-code-for-agents.md).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.