CS2680 Modern AI Systems: Agents and System Optimizations
Lecture 23 — LLM for Systems Research I

For eleven weeks the arrow has pointed one way: agents were the workload and systems were the thing we built for them. Today it reverses, and the honest framing is that this is the least settled material in the course — so the emphasis is on how to evaluate a claim rather than on results that will age badly. The thesis is narrow and defensible: an agent's usefulness on a task is governed by the cost of checking its work, and systems tasks are unusually blessed with cheap, legible, non-gameable checkers that were built for other reasons and were never designed to be fooled. Compilers, test suites, and benchmarks are exactly the verifiers Lecture 5 §5.2 priced at turning a 36% task into an 82% one for 25% more wall-clock. That is why the first credible results in this area are in software engineering and GPU kernels rather than anywhere else, and it is why the interface — not the model — has been the dominant variable. By the end you should be able to say what makes a systems task tractable, explain why an agent-computer interface moved SWE-bench scores more than model choice did, read a two-part benchmark metric like fast_p correctly, and apply a measurement checklist that most papers in this literature currently fail.

Optional content — not lectured, not examinable. This ran as the opening of Part III on an earlier version of the schedule and came off it when the semester ran out of meetings; the notes are written and stay up, and nothing later assumes them. It is the most directly useful note for Assignment 5 and for a final-project tool, since both turn on building a verifier you can trust. Its companion — LLM for Systems Research II — is also written. The readings are on the readings page.

Required SWE-agent (Yang et al., 2024) — read it as a design paper, not a results paper. The contribution is the agent-computer interface: a file viewer with a bounded window, an editor that lints on every write and rejects broken edits, and a search that returns a summary rather than a dump. Read the ACI section and the ablations closely — the ablations are the paper — and as you go, match each design decision to a principle from Lecture 4 §4.2 on tool schemas. Hold one question: which of their interface decisions had you already made by accident in Assignment 2, and which would you now change?

Required KernelBench (Ouyang et al., 2025) — the benchmark that makes this course's own subject the task. A model is asked to write a GPU kernel; correctness is checked against a reference implementation and speed is measured. Read the task construction, the level structure, and above all the fast_p metric — fraction of problems that are both correct and at least p× faster than baseline — until you can say why a single-number metric would have been dishonest here. Read the failure analysis. Hold one question: every kernel in Lecture 9 is a candidate task. Which of that lecture's four ideas would you expect a model to find, and which to miss?

Optional LLMs for compiler optimization — the same bet one layer down, where the verifier is a compiler and the reward is a measured instruction count. OpenHands — the interface question asked as a platform rather than an experiment; useful for Assignment 5 if you are building tooling. SWE-bench Multimodal — what happens when the verifier can no longer be a test suite, which is the sharpest way to see how much the verifier was doing. BountyBench — tasks with real economic stakes and adversarial structure; read for how they handle the fact that a successful attack is its own verifier. Always measure one level deeper (Ousterhout) — not about LLMs at all, and the most useful thing on the list. §23.7 is this paper applied to this literature.

Where this sits

Part I built the vocabulary: an agent is a control loop around a stateless model (Lecture 3), the loop and its tool interfaces are yours to design (Lecture 4), and a change is only a change if a frozen task set says so (Lecture 5). Part II spent eleven meetings making the machine underneath it efficient, and ended by making the program legible to the scheduler (Nov 18, Nov 23).

Part III asks the reverse question, and the connective tissue is not thematic but technical: every tool Part I gave you for judging an agent is exactly the tool needed to judge these claims. Lecture 5 §5.2's verifier arithmetic explains which tasks are tractable; §5.3's frozen set explains why most reported numbers are softer than they look; §5.5's diagnose-in-order rule explains why interface work beat model work. Its companion note, LLM for Systems Research II, takes the next step up — agents attempting research rather than tasks — and today is the groundwork for it.

What today is not. It is not a survey of model capabilities, and nothing here should be read as a claim about what models can do in general. Capability results in this area have a half-life measured in months, so the durable content is the evaluation apparatus: what a task needs in order to be attemptable, what a benchmark measures and misses, and which numbers a paper owes you. Those outlast any particular model, and they are what the self-check tests.

Instructor notes — Timing plan

75-minute class (Mon/Wed 11:15am–12:30pm, SEC LL2.221). Instructor-led, two required papers, opening Part III. The temptation is to spend the hour on capability anecdotes; resist it. The two protected segments are both about evaluation.

TimeSegmentNotes
0–7§23.1 The arrow reversesSay plainly that this is the least settled material, and why that changes how we read it.
7–20§23.2 Verifiers decideProtected — never cut. Re-derive 0.36 → 0.82 at +25%. This is the lecture's thesis.
20–34§23.3 SWE-agent's interfaceThe ablation is the paper. Map each decision to a Lecture 4 §4.2 principle.
34–46§23.4 KernelBench and fast_pWhy two-part metrics. Then the pass@k inflation arithmetic.
46–55§23.5 Why kernels are the ideal testbedReuse Lecture 9's numbers as the task. The 768-config search space.
55–65§23.6 What benchmarks missContamination, pass ≠ correct, one-shape kernels.
65–72§23.7 The checklistProtected. Five questions to ask every paper in Part III.
72–75§23.8 WrapPoint at the companion note. End on the verifier line.

Reading-only, not scheduled: §23.8's agenda in full, and the BountyBench/multimodal material.

If running long: compress §23.5 to the search-space number and drop §23.6's third paragraph. Never cut §23.2 or §23.7 — the verifier arithmetic is why this material belongs in this course rather than in a general AI course, and the checklist is what students will actually reuse on the companion note.

Learning objectives

By the end of this class you should be able to:

  1. State what makes a task tractable for an agent, in terms of the cost and properties of its verifier rather than the difficulty of the task.
  2. Re-derive Lecture 5's compounding arithmetic and use it to explain why systems tasks are unusually favourable.
  3. Name the three properties a verifier needs, and give a systems example that has all three and one that lacks a specific one.
  4. Describe SWE-agent's interface decisions and connect each to a tool-design principle from Lecture 4.
  5. Explain why fast_p is a two-part metric and what a single-number version would hide.
  6. Compute how much pass@k inflates an apparent success rate, and convert it into cost per completed task.
  7. Explain why GPU kernel generation is an unusually good testbed, and estimate the size of a kernel configuration search space.
  8. Apply a five-question measurement checklist to a paper in this area and say what it fails to report.

23.1 The arrow reverses, and what that costs us

Part II's question was "how do we serve this workload efficiently?", and it had the property that made it teachable: the answers are arithmetic. A ridge point is 295 FLOP/byte whether or not you like it, and a 25.6× cache reduction is a division. Today's question — "can a language model do systems work?" — has no such property. It is empirical, the evidence is a few years old, and the underlying capability is moving faster than the literature evaluating it.

Two consequences shape the rest of Part III.

Results will age; methods will not. Any number in the assigned papers about what fraction of tasks a model solves is a statement about a particular model on a particular set on a particular date, and will be wrong within a year in the optimistic direction. What will still be true is the structure of the question: which tasks admit cheap checking, what an interface is worth, what a benchmark's metric conceals. So the durable content is evaluation, and that is what today is about.

This course is unusually well equipped for it. Not because we know about models — we have deliberately never asked whether a model is good (Lecture 1 §1.1) — but because Part I built the measurement apparatus. Lecture 5 spent a full meeting on frozen task sets, cost per completed task, sample sizes, and the discipline of changing one thing at a time. That apparatus is exactly what this literature needs and frequently lacks, and applying it is a contribution a student in this room can make.

Instructor notes

Minutes: 7. Board: "Part II: arithmetic. Part III: evidence." Then "results age, methods do not." Ask the room: "Which lecture in Part I is the most useful thing you own for reading these papers?" Lecture 5. Make them say it. Expect confusion: Students expect a capabilities lecture. Set the frame explicitly in the first two minutes or you will fight it for the rest of the hour.

23.2 Verifiers decide what is attemptable

This is the thesis, and it is Lecture 5 §5.2 restated as a claim about task selection rather than about agent design.

An agent working a multi-step task compounds its per-step failure rate. Lecture 5's arithmetic: at 95% per step, a 20-step task succeeds 0.95²⁰ = 36% of the time. Insert a verifier inside the loop that catches a fraction c of errors, and the effective per-step reliability becomes q' = 1 − (1 − q)(1 − c); at c = 0.8 that is 0.99 per step, so 0.99²⁰ = 82%, bought for about 25% more wall-clock. A single component more than doubles the completion rate.

`q'` = 1 − (1 − `q`)(1 − `c`) success = `q'`^`n` over `n` steps

Now read that as a statement about which problems to point an agent at. The multiplier depends on having a verifier that is cheap (you can afford to run it every step), legible (its output tells the model what to fix, or the retry fails identically), and non-gameable (passing it means the work is right, not that the check was defeated). Lecture 5 named those three properties for an agent you build; the same three decide whether a domain is attemptable at all.

And here is why systems is the domain where this landed first. Its verifiers already exist, and they were not built to be gamed. A compiler rejects your program with a line number. A test suite fails with an assertion and a stack trace. A benchmark returns a number in microseconds. Each of these is cheap, extremely legible, and — crucially — was designed decades ago by people who had no idea it would one day serve as a reward signal, which is the best possible guarantee that it is not measuring something optimized-for.

The three properties, checked against four systems verifiers

VerifierCheap?Legible?Non-gameable?
Type checker / compiler~1 sexcellent — line, column, expected typeyes, within its scope
Unit test suitesecondsgood — assertion and traceno — tests can be edited, or special-cased
Kernel correctness check vs a reference~1 smoderate — a numerical mismatch, not a causeyes — the reference is not yours to change
Wall-clock benchmarksecondspoor — "slower", with no reasonyes, if the harness is trusted

Note the pattern: the two with the best legibility are the two with a gameability or scope problem, and the two that are hardest to fool tell the model least about why it failed. No single verifier has all three properties, which is why the working recipe is a ladder — type check, then tests, then build, then benchmark — exactly as Lecture 5 §5.2 proposed.

The corollary is the useful part for research taste. Where the verifier is weak, results should be distrusted in proportion. Ask of any claim in this area: what checked the work, what would it accept that a human would reject, and could the agent have optimized against it? That question is the reason §23.6 exists and the reason SWE-bench Multimodal is on the reading list — it is what happens when the test suite is taken away.

Instructor notes

Minutes: 13. Protected. Board: 0.95²⁰ = 0.36, then 0.99²⁰ = 0.82, then "+25%". Then the three properties. Then the four-verifier table, filling the last column last. Ask the room: "Which of these four could an agent defeat?" The test suite — by editing tests. Let them find it; it is the whole of §23.6's contamination worry in miniature. Expect confusion: Students hear "verifier" and think "evaluation". It is inside the loop, not after it. Lecture 5 §5.2's placement argument is worth thirty seconds.

23.3 SWE-agent: the interface was the variable

SWE-agent's result is the one that should most interest a systems audience, and it is not a score. It is that holding the model fixed and changing the interface moved performance more than changing the model did. That is a systems finding about a machine-learning system, and it is the same claim Lecture 4 §4.2 made from first principles — the model re-reads your tool documentation on every call, with no compiler to catch a misunderstanding, so the schema is the program — now with a benchmark behind it.

Their agent-computer interface makes four decisions worth naming, and each maps onto a Lecture 4 principle.

A file viewer with a bounded window rather than "here is the file". Lecture 4 §4.2's output cap, enforced in the tool rather than in the prompt — and Lecture 4 priced the alternative: one uncapped 20,000-token dump at step 4 of a 20-step task rides in 16 subsequent prompts for 320,000 tokens, more than three times the whole task's budget.

An editor that lints on every write and rejects a syntactically broken edit rather than accepting it and failing later. This is Lecture 4 §4.5's errors as data, plus Lecture 5 §5.2's legibility: the model is told immediately, in a form it can act on, at the moment the context still contains what it was trying to do. A broken edit accepted now becomes an inscrutable test failure ten steps later.

A search that returns a summary with locations rather than every match. Context is the scarce resource (Lecture 3 §3.5), and a search that floods it has spent the task's budget to answer one question.

A small, closed command set rather than a general shell. Fewer ways to be wrong, and every command's output shaped deliberately — which is Lecture 4's argument that a small agent you fully understand beats a capable one you assembled.

The reason to spend class time on this is not the individual choices; it is what the ablation implies about where effort should go. The interface is the part you control, it is cheap to iterate on, and it was worth more than waiting for a better model. For a systems audience that is a familiar shape — it is the same reason Lecture 2 §2.9 said PyTorch beat TensorFlow by optimizing the human rather than the throughput — and it is directly actionable in Assignment 5.

Instructor notes

Minutes: 14. Board: Four ACI decisions in a column; beside each, the Lecture 4 or 5 section it instantiates. Ask the room: "Which of these did you already build into your Assignment 2 agent?" Then: "which one would you add now?" The second question is the useful one. Expect confusion: Students read the ACI as ergonomics. It is a token-budget and legibility intervention, and both are quantitative. Use the 320,000-token figure.

23.4 KernelBench, and why one number would have lied

KernelBench asks a model to write a GPU kernel for a specified operation, then checks it two ways: correctness against a reference implementation on test inputs, and speed against a baseline. Its headline metric is deliberately two-part.

`fast_p` = fraction of problems where the generated kernel is CORRECT and at least `p`× faster than the baseline

That shape is the right one, and it is worth being explicit about what each single-number alternative would have concealed. Correctness alone rewards emitting a wrapper around the reference implementation — correct, zero speedup, and arguably the highest-scoring degenerate strategy available. Speed alone rewards a kernel that returns garbage very quickly, which is the classic reward-hacking failure. Mean speedup over correct kernels hides how few were correct, and is dominated by outliers. Only the conjunction, swept over p, describes the frontier — and sweeping p is what turns a metric into a curve, so fast_1 (any correct speedup) and fast_2 (a real one) can be reported separately and tell different stories.

There is a second measurement issue in this literature that KernelBench's structure makes visible, and it is Lecture 5 §5.3's arithmetic in a new costume: pass@k inflation.

What pass@k does to an apparent success rate

Let p be the per-attempt success probability. Independent attempts give pass@k = 1 − (1 − p)^k.

At p = 0.20: pass@1 = 20%, pass@5 = 67%, pass@10 = 89%.

So the same agent can be described as solving one task in five or nine in ten, depending on a reporting choice. Now price it in Lecture 5 §5.3's currency — cost per completed task: at p = 0.2 you spend 1/0.2 = 5 attempts per success, and reporting pass@10 = 89% obscures that you paid for 10 and completed 0.89, i.e. 11.2 attempts per completed task.

Compare Lecture 5's own numbers, which are the same phenomenon: 22.3 calls per completed task against 15.6 attempted and 9 that a demo shows — a factor of 2.5× between the honest figure and the demonstrated one.

The general point, which applies to every paper in Part III: pass@k for k > 1 is a statement about a sampling budget, not about capability, and it is only interpretable alongside the per-attempt rate and the cost. It is the benchmark-scale version of the "large language monkeys" observation, and the reason discussion seed 3 asks you to convert every reported number into cost per completed task before comparing anything.

Instructor notes

Minutes: 12. Board: fast_p's definition, then three crossed-out alternatives with their degenerate strategies. Then 20% / 67% / 89% and "11.2 attempts per completed task". Ask the room: "What is the highest-scoring cheat against correctness-only?" Wrap the reference. Getting there in ten seconds is a good sign about the room. Expect confusion: pass@k is read as capability. It is a budget. Say it twice and put the cost figure beside it.

23.5 Why kernels are the right testbed for this course

Of all the tasks one could pose, GPU kernel generation is unusually well suited, and the reasons are worth enumerating because they double as a recipe for constructing a good task in any domain.

Both halves of the objective are machine-checkable, and neither is a proxy. Correctness is a numerical comparison against a reference; speed is a measurement in microseconds. There is no rubric, no judge model, and no human in the loop — which removes the largest source of noise in agent evaluation and the largest opportunity for gaming.

The search space is large but structured. This is the part that makes it a search problem rather than a knowledge problem, and it is worth a number.

The size of a kernel's tuning space

A Triton matmul kernel's autotuning space, on the conventions of Sep 28: BLOCK_M, BLOCK_N, BLOCK_K each from {32, 64, 128, 256}; num_warps from {2, 4, 8}; num_stages from {2, 3, 4, 5}.

4 × 4 × 4 × 3 × 4 = 768 configurations — for one kernel, at one shape, before any change to the algorithm.

And the algorithmic space above it is not enumerable at all: Lecture 9's four ideas — tiling, online softmax, recomputation in the backward pass, split-K — are structural rewrites, not parameter choices.

The reward is dense and immediate. A wrong kernel fails in a second; a slow one reports a number. Contrast a task whose signal arrives only after a full training run, or after a human reviews a design document — Lecture 5 §5.2's compounding arithmetic simply does not apply when the verifier is expensive.

And, for this room, every task is one you have done. Lecture 9 is a catalogue of candidate problems. An agent asked to write a fast attention kernel is searching for exactly the transition that lecture derived by hand: unfused attention sits at 64 FLOP/byte and 22% of an H100's peak, and keeping the score matrix off HBM moves intensity to S/2, crossing the ridge at 590 tokens — a 4.6× on the kernel. That is a well-posed, machine-checkable, non-obvious optimization with a known answer, which makes it an unusually informative test: you can ask not merely whether the agent got a speedup but whether it found that speedup, and if not, which of the four ideas it missed.

The honest counterpoint, and it is the reason §23.6 follows immediately: a kernel that is correct and fast on the benchmark's shapes may be neither on yours. Lecture 9 §9.5 made exactly this point about FlashAttention's own gains — 4.6× on the kernel, 1.18× on a 4K prefill, 1.73× at 32K — so a kernel-level result is not a deployment result, and a benchmark that fixes the shapes cannot tell you which you are getting.

Instructor notes

Minutes: 9. Board: 768, then "and the algorithmic space is not enumerable". Then Lecture 9's 64 → S/2 → 590. Ask the room: "Which of Lecture 9's four ideas would a model find?" Tiling and split-K are in the literature it trained on; online softmax as a derivation is the interesting case. There is no settled answer — that is the point of the question.

23.6 What these benchmarks do not measure

Four gaps, in rough order of how much they should change your reading.

Contamination, and the frozen-set problem. SWE-bench tasks are drawn from public repositories whose issues, discussions and fixes are in training corpora. A model may have seen the patch. This is not a hypothetical objection — it is the reason held-out and freshly-collected variants keep being constructed — and it is Lecture 5 §5.3's frozen-set discipline colliding with the fact that you cannot freeze a set against a model whose training data you do not control. The best available responses are tasks created after a model's cutoff, tasks whose solutions were never public, and reporting on both. None is fully satisfying.

Passing the tests is not being correct. A patch that satisfies a repository's test suite may be wrong in ways the suite never checked — that is what it means for a suite to be incomplete, and every suite is. Worse, the agent can sometimes reach the tests themselves, at which point the verifier's non-gameability (§23.2) fails outright. The standard mitigation is to hide the tests and forbid editing them, which is exactly an admission that the verifier only works when the agent cannot reach it.

No signal on the qualities engineers actually argue about. Nothing in a pass/fail harness measures whether a patch is maintainable, whether it fits the codebase's conventions, whether its abstraction is right, or whether a reviewer would accept it. Those are the properties that dominate real engineering time, and they are absent by construction because they are not cheaply checkable — which is §23.2's thesis biting back: the tasks that got attempted first are the tasks whose checkable part is the whole task, and that is a selection effect, not a discovery about difficulty.

Single-shape, single-machine kernel results. As §23.5 closed: correct and fast at the benchmark's shapes and on the benchmark's GPU. A kernel tuned for one shape can be slower than the baseline at another, and Lecture 9's Amdahl arithmetic means even a genuine 4.6× kernel win is a 1.18× end-to-end win at 4K. A benchmark reporting kernel-level speedups is reporting the larger of the two numbers.

Instructor notes

Minutes: 10. Board: Four bullets. Under the third, write "selection effect, not a discovery". Ask the room: "If you had to fix one of these, which?" Contamination is the one with no clean answer; the maintainability gap is the one that matters most in practice. Expect confusion: Students treat contamination as a solved problem because "they held out a test set". Holding out from your training run does not hold out from the model's.

23.7 The checklist: five questions for every paper in Part III

This is the part to keep. Ousterhout's Always measure one level deeper argues that the most common measurement failure is stopping at the top-level number and never asking what produced it, and this literature currently fails that test routinely. Five questions, in the order that catches the most.

1. What was the verifier, and could the agent reach it? §23.2's three properties. If the check was a test suite the agent could read or edit, the result is about the harness.

2. What is the per-attempt rate, and what is the cost per completed task? §23.4. A pass@k with no k, no per-attempt rate, and no token or GPU cost is uninterpretable — and Lecture 5's 22.3-against-9 gap says the honest number is typically 2–3× the demonstrated one.

3. What is the interval, and how many tasks? Lecture 5 §5.3: 20 tasks gives ±20 points at p ≈ 0.7, and halving that interval costs the tasks. Most reported differences in this area are smaller than their own error bars, and the paper usually does not say. A 2-point improvement on a 50-task suite is not a result.

4. What was the baseline, and was it configured competently? This course's own running example: Lecture 12 Exercise 5 showed a 4.5× disaggregation claim can be manufactured entirely by leaving chunked prefill off in the baseline. The agent equivalent is comparing against a bare model with no interface, when §23.3 says the interface was the variable — so "our agent beats the raw model" measures the ACI, not the agent.

5. Where did the time and money go? Lecture 5 §5.4's invariant, model + tool + gap = wall-clock, and the token ledger that must sum to the input. If a paper reports a success rate and no cost, it has reported the numerator of a fraction.

The checklist applied to a plausible claim

"Our agent resolves 43% of SWE-bench Verified, up from 33%, with GPT-class model X."

(1) Verifier: repository test suites, hidden from the agent — good, though contamination remains. (2) Per-attempt vs pass@k: if 43% is pass@1, fine; if pass@5, the per-attempt rate is nearer 11% and the cost is ~9 attempts per completed task. The paper must say which. (3) SWE-bench Verified is 500 tasks, so a 43% estimate carries about ±4.3 points at 95% — meaning a 10-point gain is real and a 3-point gain would not be. Sample size rescues this one. (4) Baseline "33%": same interface? Same scaffold? If the 33% is a different scaffold, the comparison is of scaffolds, not of the change claimed. (5) Cost: absent from most such claims. At 20 steps and Lecture 3's ledger, a single attempt is order $0.18–$0.55, so 500 tasks × 5 attempts is a real number the reader should be told.

Instructor notes

Minutes: 7. Protected. Board: The five questions, numbered. They are the rubric for the companion note as well. Ask the room: "Which of the five does the required reading answer?" SWE-agent answers 1 and 4 well (the ablations are the baseline discipline) and 5 poorly.

23.8 The agenda, in this course's vocabulary

Reading-only; not scheduled in class.

Stated as open questions rather than predictions, and deliberately in the terms this course has built.

Which systems tasks have cheap verifiers that nobody has wired up yet? §23.2 says this is the question that determines what becomes attemptable. Candidates from Part II: an eviction policy scored on a replayed trace, which the parked competition page still specifies and which is therefore a task an agent could attempt; a scheduling policy scored on goodput in a simulator; a kernel scored by fast_p. Each has a machine-checkable objective and a large structured search space.

Can an agent do the analysis rather than the search? Lecture 9's win came from writing FLOPs in one column and bytes in the other and noticing a matrix nobody wanted. That is a different act from sampling 768 configurations, and it is what the course has been training humans to do. Whether it is reachable by a model is genuinely open and is the most interesting question on this list.

How do you evaluate a change when the verifier is the expensive part? All of today's tractable tasks have sub-second checks. The tasks that matter in systems research often have checks measured in GPU-hours — does this optimization help on real traffic? — and Lecture 5 §5.2's compounding arithmetic collapses when the verifier cannot run every step.

What is the cost per accepted contribution, and is it falling? Question 2 of the checklist, asked of the field rather than of a paper. It is the only number that would settle whether this is a research direction or a product.

Discussion seeds

  1. A verifier that already exists. The parked competition specifies an eviction policy scored on a replayed trace — cheap, non-gameable, and with a numeric objective. Specify it as an agent task: the interface, the step budget, the failure modes. Would you expect an agent to beat your own submission?
  2. Which of Lecture 9's four ideas? Tiling, online softmax, backward recomputation, split-K. Predict which a model finds and which it misses, then say what your prediction assumes about the difference between recall and derivation.
  3. Convert the numbers. Take any three reported results from today's readings and convert each into per-attempt rate, cost per completed task, and a confidence interval. Which of the three survives?
  4. The gap that matters. §23.6 says nothing measures maintainability because it is not cheaply checkable. Propose the cheapest verifier you can that captures some of it, and say what it would accept that it should not.
  5. When the tests are gone. SWE-bench Multimodal removes the test-suite verifier. Predict what happens to scores, and say what that predicts about how much of current performance is attributable to the verifier rather than to the model.
  6. The uncomfortable question. If an agent can write a better eviction policy than you can, what changes about this course — and what does not?

Key takeaways

  • Part III's results will age and its methods will not, so the durable content is the evaluation apparatus — which Part I already built. Lecture 5 is the most useful thing you own for reading these papers.
  • Verifiers decide what is attemptable. Lecture 5's arithmetic: 0.95²⁰ = 36% becomes 0.99²⁰ = 82% for about 25% more wall-clock when a verifier catching 80% of errors sits inside the loop. A domain is tractable when it has verifiers that are cheap, legible, and non-gameable.
  • Systems is where this landed first because its verifiers already existed and were not built to be gamed — compilers, test suites, benchmarks. No single one has all three properties (the most legible are the most gameable), so the working recipe is a ladder.
  • SWE-agent's finding is a systems finding: holding the model fixed, the interface moved performance more than the model did. Its four decisions are Lecture 4 §4.2 and §4.5 instantiated — bounded file windows (the output cap, worth 320,000 tokens on a 20-step task), lint-on-write (errors as data, legible at the moment of the mistake), summarizing search (context is scarce), and a small closed command set.
  • fast_p is two-part on purpose. Correctness alone rewards wrapping the reference; speed alone rewards fast garbage; mean speedup over correct kernels hides the correctness rate. Only the conjunction, swept over p, describes a frontier.
  • pass@k is a budget, not a capability. At a 20% per-attempt rate, pass@1 = 20% and pass@10 = 89% — the same agent — and the honest figure is 11.2 attempts per completed task. Lecture 5's own 22.3-against-9 gap is the same phenomenon at 2.5×.
  • Kernels are the ideal testbed: both halves of the objective are machine-checkable with no rubric and no judge, the reward is immediate, and the space is large but structured — 768 configurations for one Triton matmul at one shape, with a non-enumerable algorithmic space above it. And every task is one Lecture 9 already did: unfused attention at 64 FLOP/byte, 22% of peak, becoming compute- bound past 590 tokens, a 4.6× on the kernel.
  • What they miss: contamination (you cannot freeze a set against a model whose training data you do not control), passing ≠ correct, no signal on maintainability — which is a selection effect rather than a discovery — and single-shape kernel results, where Lecture 9's 4.6× is 1.18× end-to-end at 4K.
  • The five questions: what verified it and could the agent reach it; per-attempt rate and cost per completed task; interval and sample size (20 tasks = ±20 points); was the baseline configured competently; and where did the time and money go.

Numbers worth memorizing

QuantityValueSource
Compounding without a verifier0.95²⁰ = 36%Lecture 5 §5.2
With a verifier catching 80%0.99²⁰ = 82%, at +25% wall-clockq' = 1−(1−q)(1−c)
Uncapped tool output, step 4 of 20320,000 prompt tokensLecture 4 §4.2
pass@1 vs pass@10 at p = 0.220% vs 89% — same agent1 − (1−p)^k
Cost per completed task at p = 0.2, k = 1011.2 attempts10 ÷ 0.89
Lecture 5's demo gap22.3 vs 15.6 vs 9 calls (2.5×)Lecture 5 §5.3
Interval on 20 tasks±20 points; 4× tasks to halve itLecture 5 §5.3
Triton matmul tuning space768 configs, one kernel, one shape§23.5
The kernel task, from Lecture 964 FLOP/byte → S/2; ridge at 590 tokens; 4.6×Lecture 9 §9.2
Kernel win vs end-to-end win4.6× vs 1.18× at 4K, 1.73× at 32KLecture 9 §9.5

Self-check

  1. What makes a task tractable for an agent, and why is that a claim about verifiers rather than about difficulty?Because multi-step reliability compounds: at 95% per step a 20-step task succeeds 36% of the time, and a verifier catching 80% of errors inside the loop lifts that to 82% for 25% more wall-clock. So the binding constraint is whether you can check the work cheaply enough to check it every step. Difficulty matters, but a hard task with a cheap check is more attemptable than an easy task with an expensive one — which is why the field started with code rather than with design.
  2. Name the three properties a verifier needs and give a systems verifier that fails one.Cheap, legible, non-gameable. A unit test suite is cheap and legible and gameable — the agent can edit the tests or special-case the inputs, which is why harnesses hide them and forbid edits. A wall-clock benchmark is cheap and hard to fool but barely legible: it says "slower" without saying why, so a failure carries almost no information for the retry. No single verifier has all three, hence the ladder.
  3. SWE-agent held the model fixed and changed the interface. Why should that surprise nobody in this course?Because Lecture 4 §4.2 derived it: the model re-reads your tool documentation on every call with no compiler to catch a misunderstanding, so the schema is the program, and the failure modes are token budget and legibility rather than intelligence. Lecture 4 even priced one of them — an uncapped 20,000-token output at step 4 of 20 rides in 16 later prompts for 320,000 tokens. SWE-agent supplies the benchmark evidence for a claim this course made from arithmetic.
  4. Why is fast_p two-part, and what does each single-number alternative reward?Because correctness and speed can each be satisfied degenerately. Correctness alone rewards wrapping the reference implementation — correct, zero speedup. Speed alone rewards returning garbage quickly. Mean speedup over the correct subset hides how small that subset is and is outlier-dominated. Only "correct and at least p× faster", swept over p, describes a frontier rather than a point — and reporting fast_1 and fast_2 separately tells you whether the speedups are real.
  5. An agent is reported at 89% pass@10. What have you been told?That the sampling budget was 10 and the union of 10 attempts succeeded 89% of the time. If attempts are independent that implies a per-attempt rate of about 20%, and a cost of 10 attempts per task for 0.89 completions — 11.2 attempts per completed task. pass@k is a statement about budget, not capability, and it is uninterpretable without the per-attempt rate and the cost. Lecture 5's 22.3-vs-9 gap is the same inflation seen from inside a single agent.
  6. Why are GPU kernels an unusually good testbed, and what is the catch?Both halves of the objective are machine-checkable with no rubric and no judge model, the reward is immediate (a second), and the space is large but structured — 768 autotuning configurations for one Triton matmul at one shape, over a non-enumerable algorithmic space. The catch is generality: correct and fast at the benchmark's shapes on the benchmark's GPU. Lecture 9 §9.5 already showed the gap between a kernel result and a deployment result — 4.6× on the kernel, 1.18× on a 4K prefill.
  7. Why is "nothing measures maintainability" a selection effect rather than a gap in the benchmarks?Because §23.2 says tasks get attempted when their verifier is cheap, and maintainability has no cheap verifier. So the tasks that entered the literature first are precisely those whose checkable part is the whole task. That is not evidence that the unchecked properties are unimportant — they dominate real engineering time — it is evidence about what was measurable. Reading a benchmark's coverage as a map of difficulty is the error.

Exercises

  1. Specify a scored policy task for an agent. The parked competition scores an eviction policy on a replayed trace. Write it as a task specification an agent could attempt: the interface (what it can read and call), the step budget, the verifier, and the three failure modes you would expect. Then say which of §23.2's three properties your verifier has. Solution sketch: Interface: a policy file exposing admit(entry) and evict() -> victim, a replay(trace, capacity) -> score command returning recovered-prefill under the competition metric, and a read-only trace summary (length distribution, reuse-distance histogram) — deliberately not the raw trace, since dumping it floods context (Lecture 3 §3.5). Budget: 30 steps, one replay per step at a few seconds each. Verifier: replay on a held-out trace slice — cheap (seconds), poorly legible (a scalar, no reason), non-gameable if the held-out slice is hidden. Expected failures: (a) overfitting to the visible slice — the classic gameable-verifier outcome, caught by the held-out split; (b) proposing LRU with cosmetic changes, since it is the strongest thing in the training data (Lecture 13 §13.8 says LRU is a defensible baseline, so this scores respectably and teaches nothing); (c) a policy that ignores the tree constraint and evicts interior nodes, which is a correctness bug the replay harness must reject explicitly. The interesting design choice is adding a legible second verifier — a diagnostic that reports hit rate by prefix length — converting a scalar into a signal the agent can act on, which by §23.2's arithmetic is worth more than any prompt change.
  2. Convert three claims. Given: agent A reports 43% pass@1 on 500 tasks; agent B reports 71% pass@5 on 100 tasks; agent C reports 58% pass@1 on 50 tasks. Compute each one's interval, B's implied per-attempt rate, and rank them by evidence quality. Solution sketch: A: 1.96√(0.43·0.57/500) = ±4.3 points. B: pass@5 = 0.71 → per-attempt p = 1 − (1 − 0.71)^(1/5) = 1 − 0.29^0.2 = 1 − 0.7788 = 22.1%; interval on the pass@5 estimate is 1.96√(0.71·0.29/100) = ±8.9 points, and the cost is 5 attempts per task for 0.71 completions = 7.0 attempts per completed task. C: ±1.96√(0.58·0.42/50) = ±13.7 points. Ranking: A (tight interval, honest pass@1), then B (usable once converted, but its headline is 3.2× its per-attempt rate), then C — whose ±13.7 points means it cannot distinguish itself from anything within 27 points. A and C's headline numbers differ by 15 points and C's interval swallows it, which is exactly Lecture 5 §5.3's "+2 of 20 is a coin flip" at benchmark scale.
  3. Price the experiment you would demand. You want to know whether interface change X improves a 43% baseline by 5 points, at 95% confidence. Compute the tasks needed unpaired and paired, then the dollar cost at Lecture 3's per-session figures. Solution sketch: Unpaired, detecting a 5-point difference near p = 0.43 needs the interval on the difference below 5 points: 1.96√(2·0.43·0.57/n) ≤ 0.05 → √(0.49/n) ≤ 0.0255 → n754 per arm, 1,508 runs. Paired (same tasks both arms) removes task difficulty as a variance source; with a per-task McNemar-style comparison and a plausible discordance rate of 20%, n300 tasks run twice, 600 runs. Cost at Lecture 3's ≈$0.18–$0.55 per 10–20-step session: 600 runs × $0.35 ≈ $210, plus harness time. The experiment costs a couple of hundred dollars and is almost never run — which is the exercise's point, and the reason §23.7's question 3 catches so much. Pairing is worth 2.5× here and is free.
  4. The degenerate strategies. For each of four metrics — correctness only, speedup only, mean speedup over correct kernels, and fast_2 — write the highest-scoring useless strategy, and say which metric survives. Solution sketch: Correctness only: emit return torch.matmul(a, b) — a wrapper around the reference. Scores 100%, speedup 1.0×. Speedup only: return an uninitialized tensor of the right shape; arbitrarily fast, always wrong. Mean speedup over correct kernels: solve one trivial problem with a 50× win and fail everything else; mean speedup 50×, coverage 1/n. fast_2: requires correct and ≥2×, so wrappers score 0 (1.0× < 2), garbage scores 0 (incorrect), and the one-lucky-kernel strategy scores 1/n — honestly. fast_2 survives all three attacks. The residual attack on fast_2 is shape-specific tuning: hard-code the benchmark's dimensions, which is correct and fast on the suite and useless in deployment — defeated only by evaluating on unseen shapes, which is §23.6's fourth gap and the one no current benchmark fully closes.
  5. Apply the checklist to the required readings. For SWE-agent and KernelBench, answer each of §23.7's five questions, and name the single strongest and weakest point of each paper's evidence. Solution sketch: SWE-agent. (1) Verifier: repository test suites, hidden — strong, though contamination is unaddressed by construction. (2) pass@1 reported — good. (3) SWE-bench's several hundred tasks give intervals of a few points, adequate for the effect sizes claimed. (4) Strongest point: the ablations are baseline discipline — they vary the interface with the model fixed, which is precisely Lecture 5 §5.5's change-one-thing rule, and it is why the interface conclusion is credible. (5) Weakest: cost per completed task is under-reported, so the reader cannot compare scaffolds on an efficiency basis. KernelBench. (1) Verifier: reference comparison plus timing — cheap and hard to game, poorly legible. (2) Reports pass-style rates and fast_p at several p — good. (3) Task counts per level are modest, so per-level differences carry wide intervals. (4) Baseline is PyTorch eager/compiled — reasonable and clearly stated. (5) Strongest point: fast_p itself, which forecloses the three degenerate strategies of Exercise 4. Weakest: single-shape, single-GPU evaluation, so a benchmark result does not transfer to a deployment — the gap Lecture 9 §9.5 quantified as 4.6× against 1.18×.

Reading guide

Required — SWE-agent. Read the ACI section first and the ablations second, and treat the ablations as the contribution: they are the part that establishes interface rather than model as the variable, and they are a clean instance of Lecture 5 §5.5's change-one-thing discipline. As you read each interface decision, write down which Lecture 4 principle it instantiates — bounded windows are the output cap, lint-on-write is errors-as-data plus legibility, summarizing search is context-as-scarce-resource, and the closed command set is the small-agent argument. Skim the benchmark-scores discussion; treat absolute numbers as dated. Hold this question: which of their decisions did you make by accident in Assignment 2, which did you get wrong, and what would you change in Assignment 5 tomorrow?

Required — KernelBench. Read the task construction and level structure, then spend real time on the metric: be able to say what fast_p forecloses that each single-number alternative permits (Exercise 4 is that list). Read the failure analysis carefully, since it is the most information-dense part for a systems reader — the kinds of mistakes tell you what the search is and is not exploring. Then read it against Lecture 9: pick two problems and ask which of tiling, online softmax, recomputation, or split-K a solution would need. Hold this question: the benchmark fixes shapes and hardware — write down what you would need to add to make a result predictive of a deployment, and estimate what that would cost to run.

Optional — LLMs for compiler optimization. The same bet with a compiler as the verifier and instruction count as the objective. Question: the verifier here is more legible than a wall-clock benchmark and less gameable than a test suite — does that predict better results, and does the paper bear it out?

Optional — OpenHands. The interface question as a platform. Read if you are building tooling for Assignment 5. Question: which of SWE-agent's four decisions does it generalize, and which does it abandon?

Optional — SWE-bench Multimodal. The sharpest instrument on this list for §23.2's thesis, because it changes the verifier and holds much else fixed. Question: how much of current performance is attributable to the verifier rather than the model — and what does your answer imply about which domains will see results next?

Optional — BountyBench. Tasks with real stakes and adversarial structure, where a successful exploit verifies itself. Question: self-verifying tasks have perfect non-gameability — why has that not made them the easiest domain?

Optional — Always measure one level deeper. The most useful reading on the list and the shortest. §23.7 is this paper applied to this literature. Question: Ousterhout's central complaint is stopping at the top-level number — find the top-level number in each required reading and say what is one level beneath it.

Looking ahead

LLM for Systems Research II — the companion note, also optional — moves up a level: from agents doing tasks with cheap verifiers to agents attempting research — MLGym and AI Scientist-v2 — where the verifier is expensive, contested, or human. §23.7's five questions are that meeting's rubric, and §23.8's third open question is its central difficulty: Lecture 5's compounding arithmetic simply does not apply when the check cannot be run every step. Come having decided what you think a credible claim would even look like there.

Assignment 5 is due Dec 2, and the final project — announced Oct 26, proposal due Oct 28, report due Dec 8 — is where today's material is most directly useful: a project that builds a tool has to build the verifier that says whether the tool earned its time, which is §23.2's whole argument turned into a deliverable.

One thing to carry out of the room. The reason this material belongs in a systems course is not that agents write code. It is that the binding constraint on what an agent can do is the cost and quality of the check, and building cheap, legible, non-gameable checks is a systems-design problem — the same problem, in a different costume, as the one Lecture 5 set you in September. Everything in Part III is downstream of it.