CS2680 Modern AI Systems: Agents and System Optimizations
Fall 2026

Announcement

  • Sept 4, 2026We have increased capacity to 80 students.
  • Aug 28, 2026Audit policy: everyone is free to audit as long as it is permitted by school policy and the classroom has space. However, we do not have computing resources for auditing students.
  • Aug 17, 2026Welcome to CS2680 Modern AI Systems: Agents and System Optimizations! Class meets Monday and Wednesday in SEC LL2.229.

Overview

This course follows one agentic system from the agent loop down through the serving stack. You first use a modern coding agent (Claude Code), then design your own agent using provided API access, measure it, and optimize it. Then you serve an open-weight model yourself and take responsibility for the layers underneath: batching and scheduling, KV-cache and prefix reuse, routing, quantization, and speculative decoding.

Class time and location
Monday and Wednesday 11:15am – 12:30pm SEC LL2.229
Prerequisites
CS61, and at least one graduate-level course in computer systems. Comfort with Python and PyTorch is assumed.
Assessment
Five assignments and a final project. Class participation throughout the semester. There is no exam.
Textbook. No textbook is required. Background reading:
Format. The course runs in two parts.
  • Part I — Introduction to LLMs and agents. You will learn about how to use coding agents (Claude Code) and how to design an agent. You will also explore various agent architectures, use cases and optimizations.
  • Part II — Systems for LLMs and agents. We will talk about LLM serving (batching, scheduling, routing, caching, quantization, and speculative decoding) and agentic systems.
The assignments walk down that stack in five steps. First, Mads-Lens1 lets you watch the agent work (Assignment 1). Second, you build the loop in Mads-Loop (Assignment 2). Third, you make every token count in Mads-Opt (Assignment 3). Fourth, you own the model in Mads-Serve by serving it on the provided GPUs (Assignment 4). Fifth, you optimize the full stack in Mads-Stack (Assignment 5).

1 “Mads” abbreviates Modern Agent Design and Systems, the arc of this course, and nods to MaDSys, Harvard’s Measurement and Design of Systems research group, which teaches and supports it.

What this course expects. This course will not teach you every piece of knowledge you need for the assignments. It focuses instead on the skills to learn with, and points you in directions worth going. Expect to spend a significant amount of time learning and experimenting on your own after class. If you want an A, plan on it.
Grading.
  • Assignment 1 Mads-Lens: watch the agent work10%
  • Assignment 2 Mads-Loop: build the loop10%
  • Assignment 3 Mads-Opt: make every token count20%
  • Assignment 4 Mads-Serve: own the model10%
  • Assignment 5 Mads-Stack: optimize the full stack20%
  • Project20%
  • Class participation10%
  • Bonus optional presentations up to 4% each; course feedback up to 6%; problems current AI cannot solve, 2% eachup to 10%

See policy page for details.

Course website. https://cs2680.com

Staff

Juncheng Yang
Juncheng Yang
Instructor
Yiyu Liu
Yiyu Liu
TF
Advaith Ravishankar
Advaith Ravishankar
TF
Charles Wang
Charles Wang
TF
Yunjia Zheng
Yunjia Zheng
TF
Contact: cs2680-staff@g.harvard.edu Please use this address for all course-related questions.
Office hours:
  • Instructor: Appointment only
  • TFs: See Ed + by appointment

Schedule

Tentative and subject to change. Rows with Students as the speaker are student-led: either a paper discussion (see the paper discussion page) or an assignment sharing session. Paper presentations are optional and can earn up to four bonus percentage points each.

Week Date Speaker Topic Readings Notes
Part I — Introduction to LLMs and agents
1 2026-09-02 Juncheng Yang Course overview and logistics [slide] Assignment 1 out (Mads-Lens: Watch the agent work); Sign up for AWS
2 2026-09-07 No class — Labor Day
2 2026-09-09 Juncheng Yang Agents from a user’s perspective I: How Claude Code Works [slide] How I use LLMs; Best practices for Claude Code; Agentic Engineering Patterns; more reading →
3 2026-09-14 Juncheng Yang Agents from a user’s perspective II Research acceleration: The view inside OpenAI; The Shift to Agentic AI; more reading →
3 2026-09-16 Juncheng Yang Agents from a designer’s perspective I: context management and tool design Assignment 1 due Sun Sep 20, 11:59pm
4 2026-09-21 Students Student sharing I What you learned from using an agent (Assignment 1): surprises, failures, lessons, insights, new tools/skills/plugins Assignment 2 out (Mads-Loop: Build the loop)
4 2026-09-23 Juncheng Yang Agents from a designer’s perspective II: failure modes, safety, and multi-agent designs
5 2026-09-28 Juncheng Yang Agents from a designer’s perspective III: evaluation, agentOps, self-evolving harness
5 2026-09-30 Juncheng Yang Large language models Assignment 2 due Sun Oct 4, 11:59pm
6 2026-10-05 Harsh Nilesh Pathak (Snowflake) Guest lecture: Developing and Evaluating Agents in Enterprise Setting Assignment 3 out (Mads-Opt: Make every token count)
Part II — Systems for LLMs and agents
6 2026-10-07 Juncheng Yang Efficient LLM serving: paging, batching and scheduling
7 2026-10-12 No class — Indigenous Peoples’ Day
7 2026-10-14 Juncheng Yang Efficient LLM serving: KV-cache optimization I
8 2026-10-19 Students Efficient LLM serving: KV-cache optimization II Assignment 3 due Tue Oct 20, 11:59pm
8 2026-10-21 Juncheng Yang Efficient LLM serving: prefix cache Assignment 4 out (Mads-Serve: Own the model)
9 2026-10-26 Juncheng Yang and Students Assignment Feedback and Student sharing II TF: Common pitfalls in your submission; Students: What you optimized and what it bought (Assignment 3)
9 2026-10-28 Juncheng Yang Efficient LLM serving: routing and load balancing Project proposal due
10 2026-11-02 Juncheng Yang Efficient LLM serving: pruning and quantization I
10 2026-11-04 Students Efficient LLM serving: pruning and quantization II Assignment 4 due Sun Nov 8, 11:59pm
11 2026-11-09 Xianzheng Kou (Meta) Guest lecture: Why Infrastructure sets the pace of AI Assignment 5 out (Mads-Stack: Optimize the full stack)
11 2026-11-11 Juncheng Yang Efficient LLM serving: speculative decoding
12 2026-11-16 Students Efficient agent serving systems I
12 2026-11-18 Juncheng Yang Efficient LLM serving: others
13 2026-11-23 Students Efficient agent serving systems II Assignment 5 due Sat Nov 28, 11:59pm
13 2026-11-25 No class — Thanksgiving recess
14 2026-11-30 Juncheng Yang and Students Assignment Feedback and Student sharing III TF: Common pitfalls in your submission; Students: Showcase your full stack optimizations (Assignment 5)
14 2026-12-02 Students Student sharing IV Showcase your full stack optimizations (Assignment 5)
14 (TBD) Students Final project poster and demo Present your final project as a poster and a live demo. Final project due Tue Dec 8, 11:59pm

Compute resources

Setup instructions for every environment the course uses are on the computing setup page. Compute for the assignments and the final project comes from:

Sponsors

We are grateful for the support of our sponsors.