12.1-12.6 Virtualization, Containers, and CNI

1. Overview

Linux virtualization splits the VM into a userspace device model and a kernel execution engine. QEMU owns VM construction, emulated devices, and migration streams, while KVM enters guest mode, handles VM exits, and maintains the nested MMU state that maps guest memory onto host pages.

Containers use a different boundary. Instead of emulating a machine, Linux gives ordinary processes separate kernel views through namespaces, then limits shared host resources through cgroup v2 controllers.

2. Key Data Structures

A KVM virtual machine starts at /dev/kvm. QEMU creates a struct kvm, registers guest RAM as memslots, creates one struct kvm_vcpu per virtual CPU, and shares a struct kvm_run page with the kernel for exits that need userspace emulation.

VMCS fields are grouped by responsibility: some fields are restored into the guest on entry, some restore the host on exit, and the control fields decide which guest operations trap back to KVM.

EPT is a second page table owned by the hypervisor. Guest page tables translate guest virtual addresses to guest physical addresses; EPT then translates those guest physical addresses to host physical pages.

Ballooning and KSM are memory overcommit tools. A balloon driver voluntarily pins guest pages so the host can reclaim their backing pages, while KSM scans for identical anonymous pages and maps them copy-on-write.

A network namespace owns a separate struct net. That object points at namespace-local routing tables, socket tables, protocol state, and network devices, so a container can have an eth0 that is unrelated to the host physical eth0.

Cgroup v2 is a single hierarchy. Controllers such as CPU, memory, and I/O attach to the same tree, so orchestration systems can place a pod or container in one cgroup and apply all resource limits there.

Virtio block storage uses the same split as virtio networking: the guest driver fills virtqueue descriptors, notifies the backend, and receives completions through the used ring.

3. Core Mechanism

The hardest point is the VM run loop: guest execution is not interpretation. KVM asks the CPU to enter guest mode; hardware runs guest instructions directly until a configured event forces a VM exit.

Background

Suppose the guest writes to a virtio device register. A normal RAM write should stay in guest mode, but a device register write must be emulated by KVM or QEMU. VMX controls make that register range exit without trapping every instruction.

Plan

1. QEMU calls KVM_RUN on the vCPU file descriptor.
2. KVM loads pending interrupt, register, and MMU state into VMCS fields.
3. The CPU executes the guest until a VMEXIT condition occurs.
4. KVM handles exits that are pure kernel work, such as EPT faults or idle HLT.
5. If a device model is needed, KVM fills struct kvm_run and returns to QEMU.

Walkthrough

For an EPT violation, the CPU has already translated through the guest page table. The missing part is the second translation from GPA to HPA, so the VM exits to KVM, KVM installs an EPT entry, and the same guest instruction is retried.

Live migration uses the same separation of responsibilities: KVM tracks which guest pages changed, QEMU copies memory and device state, and the final stop-and-copy window transfers the last dirty pages plus vCPU state.

The downtime window is intentionally small and late. Most RAM moves while the source VM is still running; only the final dirty set and CPU/device state require pausing the source.

In a network-heavy migration, memory is not the whole state. If the VM datapath can be XDP, OVS, or a mixed transition, the migration controller must preserve map entries, flow tables, tunnel state, and packet steering so the destination host sees the same traffic contract.

Container Isolation Walkthrough

The hardest container concept is that isolation is assembled from multiple kernel views. A process can be root inside a user namespace, PID 1 inside a PID namespace, and attached to a veth device inside a network namespace while still sharing the same host kernel.

Background

Suppose Kubernetes starts a pod. The container runtime creates namespaces for process and filesystem isolation, then the CNI plugin creates a veth pair so the pod has a normal-looking eth0.

Plan

1. Create the container process with clone() flags such as CLONE_NEWPID, CLONE_NEWNET, and CLONE_NEWNS.
2. Attach the process to a cgroup v2 node and write controller files such as memory.max, cpu.weight, and io.max.
3. Create a veth pair, move one endpoint into the pod network namespace, and attach the host endpoint to a bridge or eBPF datapath.
4. Program routing, policy, service load balancing, and overlays through the CNI plugin.

Example

For two pods on the same node, the packet never needs the physical NIC. It exits pod A through the veth pair, the Linux bridge learns the destination MAC, and the frame enters pod B through the second veth.

Across nodes, the CNI plugin wraps the pod packet in an underlay packet, commonly VXLAN or IPIP. The remote node decapsulates it and delivers the original pod packet to the destination veth.

Cilium replaces much of the iptables and kube-proxy path with eBPF programs and maps. Service load balancing, policy, and connection tracking are map lookups in the datapath rather than long rule-chain traversal.

Storage virtualization has a similar choice: emulate a block device through QEMU for compatibility, move the backend to userspace polling with vhost-user-blk and SPDK for throughput, or pass a physical NVMe device directly to the VM with VFIO.

NVMe pass-through removes most host storage virtualization overhead, but it also gives the guest ownership of the device function. The IOMMU is the safety boundary that restricts DMA to the guest memory assigned by VFIO.

4. Minimal C Demo

This demo models the KVM ioctl lifecycle and the shared kvm_run exit buffer. It is not using real kernel headers; it keeps only the control flow that matters.

This demo models dirty page logging during pre-copy migration. Each round copies the current dirty bitmap, clears it, and lets the next round capture newly written pages.

This demo models the cgroup v2 file interface. Real runtimes write controller files under /sys/fs/cgroup; the principle is that limits are normal kernel control files, not a separate daemon API.

This demo models how a veth pair gives a pod a private eth0 while keeping the peer endpoint in the host network namespace for bridge, routing, or eBPF handling.

5. Kernel Source Pointers

6. Interview Prep