CS 6465 - Advanced OS Implementation

cs6465 teaches advanced topics in operating systems through a combination of hands-on engineering approach and studying implementation of modern full-featured kernels like Linux, virtualization stacks like Xen and KVM, various operating system organizations like unikernels and microkernels, etc.

This class is for students who plan to stay in the systems field, not just building new OS kernels and hypervisors, but working on parts of modern cloud and datacenter infrastructure, be an expert in low-level systems security, be able to work on cycle-level code optimizations for warehouse scale AI and big data systems, etc. I.e., you will have a rare skill to work with the teams developing macOS and iOS kernels, XNU, Windows and Linux kernels, novel hypervisors like Amazon Nitro and Firecracker, etc. This is a rare and potentially well-compensated skill.

The idea is that you can learn enough about modern commodity operating systems like Linux, hypervisors like KVM, Xen and VMware, container technologies and a range of cutting edge research topics that you can either get a job in this area or start working on competitive research.

You will study, in detail, organization of modern commodity kernel like Linux, commodity hypervisors like KVM and Xen, understand performance and security problems of modern operating systems. You will further understand major research directions in the areas of security, performance and reliability of operating systems. We will cover ideas of microkernels, exokernels, unikernels and library kernels, ideas of retrofitting isolation into commodity kernels, modern hardware isolation mechanisms, software fault isolation and WASM, basics of software verification and its potential for development of formally correct operating systems.

To master the concepts, cs6465 is organized as a series of in-class presentations that most often take a deep dive into organization of one of the modern mucrokernels, hypervisors, or kernels and often will require reading of a research paper. The homeworks will follow the lectures, but will require heavy kernel engineering effort, e.g., build a memory allocator for xv6 that follows design of the Linux memory allocator, extend xv6 with support for virtualization through a layer of paravirtualization like in Xen or hardware-assisted virtualization like in KVM, add etc.

Note: The 2026 version of this class will be different from 2025 (aka build your own microkernel in Rust) and 2024 (mostly lectures + a project). In the future the idea is to have two versions offered despite sharing the same name.

Aug 24

Aug 26

Aug 31

Sept 2

Sept 7

Sept 9

Sept 14

Sept 16

Sept 21

Sept 23

Sept 28

Sept 30

Approximate schedule for Fall 2026

Virtualization

Modern deployments

Hypervisors vs microkernels

Security

Scalability and performance

Modern kernels