The night shift

In this chapter, we will learn what karpathy/autoresearch is, which three files it is made of, and why the whole idea fits in one sentence: the human writes the instructions, the agent runs the experiments, and a single number decides. We will count how many experiments fit in a night. You need no machine learning background. If you have run a program and read its output, you can follow this book.

What autoresearch is

Autoresearch is a small, real model-training setup plus one page of instructions that lets an AI coding agent run experiments on its own, all night. Andrej Karpathy published it on GitHub in March 2026 under the MIT licence. By 10 October 2026 it had 97,600 stars and 13,600 forks. His own summary of the idea:

"Give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model."

The README opens with a joke written from the future, in which research "used to be done by meat computers in between eating, sleeping, having other fun", and ends: "This repo is the story of how it all began." The joke carries a serious claim. Experiments that a person runs at the speed of thought, with meals and sleep in between, can be run by an agent at the speed of the hardware, with nothing in between.

The training code is a cut-down, single-GPU version of nanochat, Karpathy's small language-model project. The model is tiny and the data set is small on purpose. Nothing about autoresearch depends on the model being good. It depends on the model being measurable in five minutes.

Three files

The repository is deliberately small, and only three files matter. The README says so in those words. Here they are, with who is allowed to touch each one.

prepare.py   data download, tokenizer, the evaluation,     READ-ONLY. The agent must not touch it.
             the fixed constants (time budget, sequence
             length)

train.py     the whole model, the optimizer, the           THE ONLY FILE THE AGENT EDITS.
             training loop. "Everything is fair game:       Architecture, hyperparameters, batch size,
             architecture, hyperparameters, optimizer,      model size: all of it.
             batch size, etc."

program.md   the standing orders: setup, the loop,         THE ONLY FILE THE HUMAN EDITS.
             what may change, what may not, how to
             log, when to keep, never to stop

Read the right-hand column again. The division of labour is the whole design. In the README's words: "you're not touching any of the Python files like you normally would as a researcher. Instead, you are programming the program.md Markdown files that provide context to the AI agents and set up your autonomous research org." The human programs the organisation. The agent programs the model.

The README calls program.md "essentially a super lightweight skill", and says the shipped version "is intentionally kept as a bare bones baseline, though it's obvious how one would iterate on it over time to find the 'research org code' that achieves the fastest research progress". Chapter 5 is about that iteration.

One number, one budget

Every run trains for exactly five minutes of wall-clock time and reports one number, val_bpb. The number is validation bits per byte: how many bits the model needs, on average, to encode one byte of text it has never seen. Lower is better. The README picks it because it is "vocab-size-independent so architectural changes are fairly compared". A model with a bigger vocabulary cannot win by changing the units.

The five minutes exclude start-up and compilation. They are the same on every machine, which has a consequence the README states both ways: experiments are "directly comparable regardless of what the agent changes", and your results "become not comparable to other people running on other compute platforms". The loop finds the best model for your machine in five minutes. That is the honest version of the question.

How many experiments fit in a night

program.md states the pace as arithmetic. Here it is, from the file:

one experiment     about 5 min of training, plus a few seconds of start-up and eval
per hour           about 12
one night          "a total of about 100 over the duration of the average human sleep"

Before you move anything: if the training budget were two minutes instead of five, how many runs would a night hold, and what would be lost?

A shorter run gives more experiments and a worse measurement. A two-minute model has not learned as much as a five-minute one, so a change that helps late in training may look useless. The budget is a dial between how many ideas you can test and how much each test tells you. Karpathy picked five minutes; the Mac Mini and Windows ports in the README's "notable forks" (miolini, trevin-creator, jsegov and andyluo7 are the authors named) shrink the model instead of the minutes.

Who it is for

Anyone with a number a program can compute. The repository is about training language models because that is Karpathy's problem. Nothing in the loop knows what a language model is. The loop knows a file to edit, a command to run, a line to grep, and a rule for keeping. Chapter 6 shows the loop pointed at a templating engine and chapter 7 shows it pointed at a database, a memory layer, a voice cloner, a voice twin and a website.

Here is the shape of the rest of the book.

Part I    1. The night shift          what it is                      (you are here)
          2. The loop, step by step    the seven steps and the ledger

Part II   3. One number nobody edits   the metric and the locked harness
          4. The ratchet               keep, discard, and git as memory
          5. The program, not the code what program.md holds and how to improve it

Part III  6. What people got           the reported results, with sources and caveats
          7. Five loops of our own     real ledgers from our projects

Part IV   8. Write your own loop       a toy you can run in a minute, and the checklist

Try it yourself

Open the repository and read program.md top to bottom. It is short. Count the sentences that tell the agent what it may not do. Then count the sentences that tell it what it may do. The second number is smaller. Keep that ratio in mind for chapter 5.

Common questions

Do I need a GPU to understand this book?

No. You need a GPU to run the original repository (it is tested on an H100 and needs one NVIDIA GPU), and the community ports run on a Mac or a Windows machine with a consumer card. Chapter 8 gives you a loop that runs on any laptop with Python, no GPU and no packages.

Is val_bpb the same as loss?

It is a loss, expressed per byte of text instead of per token, so that models with different tokenizers can be compared. Lower is better in both. The README's reason for choosing bits per byte is exactly that comparability.

Which agent does it use?

Any. The README says to "spin up your Claude/Codex or whatever you want in this repo (and disable all permissions)" and then point it at program.md. The agent is replaceable; the program and the harness are the design.

Why five minutes and not until the model converges?

Because the budget is a fixed window, every run is comparable to every other run on the same machine, and a bad idea costs five minutes. A run that trains "until done" has no fixed cost and no fixed meaning.

Carry this

• Three files: the harness nobody edits, the one file the agent edits, the one file the human edits.
• One number (val_bpb, lower is better) and one budget (five minutes) make every run comparable.
• About 12 experiments an hour and about 100 a night, by program.md's own arithmetic.
• The human programs the organisation; the agent programs the model.

Check yourself

1. Which of the three files does the agent edit, and which does the human edit?

Answer

The agent edits train.py and nothing else. The human edits program.md. prepare.py is read-only for both during a run.

2. Why does the README choose bits per byte rather than loss per token?

Answer

Bits per byte does not depend on the vocabulary size, so a change to the tokenizer or the architecture can be compared fairly.

3. What does a fixed five-minute budget give up?

Answer

Comparability with other machines. The README says your results become not comparable to people running on other compute platforms; in exchange, every run on your machine is comparable with every other.