CS2680 Modern AI Systems: Agents and System Optimizations
Blogs to Watch

The systems this course studies change faster than the literature that describes them. A scheduling policy, a cache layout, or a disaggregation strategy usually appears in a serving engine's release notes, a vendor's engineering post, or an independent teardown well before anyone writes it up for a conference. This page lists the sources worth tracking in that gap, grouped by subject.

Nothing here is required reading. Use it when you are looking for a final project topic, checking whether an idea has already shipped in production, or trying to find a current number for something a paper measured two years ago.

Must watch is grouped by subject, not ranked. Those sources publish something relevant often enough to justify following the group you care about, and the most prolific and most technical of each group is listed first. Others collects systems blogs that seldom write about AI directly and reward a weekly scan for the storage, scheduling, and performance engineering instead. Every link was checked on 26 August 2026.

Must watch

Grouped by subject. Within each group the sources that publish most often, and most technically, come first.

Agent frameworks and runtimes

  • Anthropic Engineering. Agent harnesses, tools, MCP, context, long-running agents, and security. Probably the highest-signal agent-engineering blog at the moment, covering managed-agent architecture, containment, evaluation, tool design, and MCP code execution.
  • OpenAI. Codex, agent runtimes, orchestration, environments, and evaluation. Most useful for agent infrastructure: harness engineering, computer environments, data agents, and agent security.
  • LangChain. LangGraph, harnesses, memory, orchestration, observability, and evaluation. Worth reading less for the framework tutorials than for the production-agent architecture patterns that show up in them.
  • Microsoft Foundry. Microsoft Agent Framework, runtime, state, observability, multi-agent orchestration, and enterprise agent infrastructure.
  • LlamaIndex. Retrieval-heavy agents, workflows, memory, and data agents.
  • OpenHands. Open coding-agent architecture, benchmarks, and agent deployment.
  • Google Developers Blog. The Agent Development Kit (ADK), Gemini agents, tool integration, and agent protocols.
  • GitHub Blog. Copilot agents, coding workflows, and GitHub-native agent infrastructure.

Coding-agent builders

  • Cursor Research. Coding agents, harnesses, cloud agents, swarms, and model routing. One of the better sources on practical agent systems, spanning cloud-agent infrastructure, agent swarms, automated review, and GPU kernels. The product blog and changelog are separate, and implementation changes there are often meaningful for harness design.
  • Replit. Coding agents, evaluation, reliability, sandboxing, and environments. Publishes concrete detail on long agent trajectories, decision-time guidance, evaluation loops, and runtime infrastructure. The AI, Engineering, and Infrastructure categories are the relevant ones.
  • Cognition. Devin architecture, coding-agent capabilities, and software-engineering automation.
  • Vercel. The AI SDK, sandboxes, agent deployment, and frontend-agent infrastructure.

Serving engines and runtimes

  • vLLM. Serving, scheduling, caching, speculative decoding, and disaggregation. High publication rate and unusually technical, with recent coverage of context parallelism, MoE serving, attention and FFN disaggregation, and RL infrastructure.
  • SGLang and LMSYS. RadixAttention, prefix caching, serving, kernels, MoE, and distributed RL. The counterpart to vLLM, and particularly relevant to agent workloads because of prefix reuse and low-latency serving.
  • Anyscale. Ray Serve, distributed inference, autoscaling, and LLM serving.
  • llm-d. Kubernetes-native distributed inference. Follow the project releases as well as the engineering posts.

Inference companies

  • Baseten. Production inference, kernels, model optimization, and serving infrastructure. One of the strongest vendor blogs on inference engineering and production serving specifically.
  • Modal. Serverless GPU systems, sandboxes, inference, and agent runtimes. Interesting because agent execution environments and inference infrastructure meet inside one platform.
  • Hugging Face. Open models, serving, agents, inference providers, and kernels. Broad, and the most reliable way to track what is happening in the open ecosystem.
  • Fireworks AI. Inference engine design, serving optimization, and model performance.
  • Together AI. Inference kernels, quantization, serving, and open-model infrastructure.
  • Cerebras. Useful precisely because the hardware and system architecture are so different from the GPU baseline the rest of the course assumes.

Hardware and accelerator stack

  • NVIDIA Technical Blog. Agentic inference, Dynamo, TensorRT-LLM, GPU kernels, and networking. Essential for the systems side of agents, because NVIDIA is explicitly optimizing the whole inference stack around long-context, multi-step agent workloads.
  • Google Cloud AI Infrastructure. TPUs, GKE, inference routing, storage, and networking. Strong on end-to-end infrastructure: KV caches, accelerator orchestration, inference gateways, network fabrics, and agent sandboxes.
  • AMD ROCm. Kernels, inference frameworks, and MI-series performance.
  • Intel Developer. Lower frequency, but worth monitoring for Gaudi, CPU inference, and systems work.

Large-scale production systems

These are not AI blogs. They are here because agent infrastructure keeps rediscovering classic distributed-systems problems, and these groups solved the earlier versions at scale.

  • Meta Engineering. Distributed systems, AI infrastructure, networking, storage, and accelerators.
  • Google Cloud Infrastructure. Compute, storage, networking, Kubernetes, and hardware/software co-design.
  • AWS Machine Learning. Large-scale deployment, Trainium and Inferentia, and managed serving.
  • Cloudflare. Workers, serverless execution, networking, inference, and distributed agent infrastructure.
  • Netflix TechBlog. Distributed systems, storage, observability, experimentation, and ML infrastructure.
  • Uber Engineering. Distributed systems, ML platforms, storage, and compute scheduling.
  • Dropbox Tech. Storage, databases, networking, and large-scale infrastructure.
  • LinkedIn Engineering. Distributed data systems, serving, and ML infrastructure.
  • Databricks Engineering. Data systems, Spark, inference, training, and model serving.

Others

Scan these weekly. Most posts will have nothing to do with AI, and the ones that matter are about storage, scheduling, and performance engineering done carefully enough to be worth imitating.

  • TigerBeetle. Databases, storage, replication, correctness, and performance engineering.
  • Oxide Computer. Hardware/software co-design, operating systems, networking, and cloud infrastructure.
  • Tailscale. Networking, distributed systems, identity, and production engineering.
  • ClickHouse. Databases, storage, query execution, and performance.
  • ScyllaDB. Storage systems, CPU and memory architecture, distributed databases, and performance.
  • Redpanda. Streaming, storage engines, networking, and performance.
  • Fly.io. VMs, networking, distributed applications, and infrastructure engineering.
  • Cockroach Labs. Distributed databases, transactions, storage, and resilience.

Independent sources

These synthesize across companies instead of presenting one vendor's view, which makes them the fastest way to find out whether a claim in a vendor post holds up.

  • SemiAnalysis. The most important independent source for AI infrastructure, GPUs, inference economics, clusters, networking, and increasingly agentic inference.
  • Chips and Cheese. Microarchitecture, GPU, CPU, memory, and accelerator analysis, usually with its own measurements.
  • Simon Willison. Fast coverage of agents, coding agents, MCP, tool use, and new model capabilities, usually with hands-on experiments.
  • Latent Space. Agents, AI engineering, infrastructure, and interviews with the people building them.
  • Interconnects. Models, post-training, RL, and the open-model ecosystem. Useful context for the workloads systems researchers will soon have to serve.
  • Artificial Analysis. Not a blog, but a standing cross-provider comparison of model quality against latency, throughput, and price. Reach for it whenever you need a current number rather than a vendor's own.

A compressed watchlist

If you want the shortest list that still covers the course, monitor these twenty.

Filtering topics for an automated monitor

Summarizing every new post from twenty sources produces more text than anyone reads. If you build a monitor, filter on the terms below and summarize only what matches. Building one is a reasonable warm-up for a final project, since the hard part is not the fetching but deciding what deserves your attention.

The course runs one, at news.cs2680.com. It reads every source on this page once an hour and files each post under the topic it scores highest against, using the terms below. Its index is also served as JSON at /posts.json, so you can build on it rather than re-fetching forty-five blogs yourself.

Agents. agent, agentic, coding agent, harness, orchestration, multi-agent, subagent, tool use, MCP, A2A, skills, memory, context engineering, computer use, sandbox, isolation, agent evaluation, long-horizon, reliability, fault tolerance, prompt injection, agent security.

AI systems. inference, serving, KV cache, prefix caching, prefill, decode, disaggregated serving, speculative decoding, scheduling, batching, MoE, attention, linear attention, kernels, quantization, GPU, TPU, NPU, RDMA, networking, storage, checkpointing, model loading, autoscaling, distributed inference, RL infrastructure, energy, sustainability, cost, latency, throughput.

General systems. storage systems, caching, distributed systems, databases, networking, scheduling, serverless, virtualization, containers, hardware/software co-design, observability, failure recovery.