Outlines

• Hypervisor designs and UEFI

  • Lectures: various use of hypervisors, UEFI module-based hypervisors, comparison with kernel module-based hypervisors, and UEFI/EDK2

• VT-x Basics

  • Lectures: processor modes, VMCS, host vs.guest, VM-exit, VM-entry, high-level design options, tools and techniques to diagnose bugs, and the trick of navigating specifications
  • Lab: source-level debugging with VMware
  • Lab: configuring and starting host and guest, monitoring CPUID execution
  • Lab: troubleshooting VMX instruction errors with Bochs

• OS Boot

  • Lectures: system boot phases, boot time vs runtime, physical vs virtual mode, and UEFI runtime drivers
  • Lab: controlling VM-exits with MSR bitmaps
  • Lab: booting Windows and separating resources between guest and host
  • Lab: tracing guest page faults with exception interception and event injection
  • Advanced lectures and demos: analysis of Hyper-V configurations and common vulnerabilities in pass-through hypervisors

• Extended Page Tables (EPT)

  • Lectures: traditional x64 address translation vs EPT-enabled translation, EPT setup and activation, EPT-induced VM-exits
  • Lab: building and enabling pass-through EPT
  • Lab: tracing guest execution with EPT
  • Advanced lectures and demos: memory types emulation, caches invalidation, VPID, stealth hooking with EPT, MBEC, VT-rp (HLAT), device virtualization and VT-d (IOMMU/DMA remapping)

• Multi-processors Support

  • Lectures: multi-processor protocol, processor activity state, application processors startup, unrestricted guest, Hypervisor Top Level Functional Specification (TLFS), and enlightenment
  • Lab: virtualizing all processors
  • Lab: booting multi-processor Windows by emulating INIT-SIPI-SIPI

• Control Register Shadowing

  • Lectures: control register guest/host mask, read shadow VMCS, and complexities with emulation of control register access
  • Lab: booting Ubuntu by properly emulating MOV-to-CRx

• Additional Demos and Resources

  • Snapshot-based fuzzing hypervisors, hardware debuggers (DCI), nested virtualization techniques (software-based, VMCS shadowing, enlightened VMCS, EPT virtualization strategies), Intel TXT and PPAM, and helpful open source projects

Contents may change in a way that does not impact the learning objectives.

Description

Virtualization technologies are critical components in software security and analysis. How can hypervisors be used to secure system software? How can custom hypervisors be written to perform reverse engineering and fuzzing more efficiently?

This class will teach you the foundation to answer those questions by developing simple hypervisors together! The class is designed so everything is built from scratch and optimized for learning. This allows you to understand the building blocks of real-world applications of virtualization technologies and expand the knowledge for your interests afterward.

This class is hands-on-oriented. We believe that we can learn and retain knowledge best by tackling concrete challenges rather than being taught. With this philosophy, the class is designed with lab activities as the primary learning opportunities and lectures to explain the theories behind them. We will spend 30-40% of the time on hands-on exercises.

At the beginning of the class, you will receive a skeleton implementation of a hypervisor and incrementally update it through a series of exercises. We will also discuss other design options to understand their pros and cons.

As we learn foundations, we will analyze various applications and their implementations. This includes snapshot-based system-level fuzzing, performant system hardening with MBEC and HLAT (VT-rp), HyperGuard, HVCI, and KDP on Windows, dynamic analysis with stealth hooking, and SMM security reporting with Intel TXT (PPAM).

You will also receive two additional hypervisor implementations for reference:

• The minimalistic one in Rust🦀 It supports Intel and AMD processors and compiles into a UEFI module and Windows driver. This is an excellent reference for those when you review the “must do” parts and rebuild your hypervisor from scratch for AMD or as a Windows driver or simply prefer the language.
• The full version of our hypervisor. This includes the implementation of advanced concepts, such as stealth-hooking hypercall, use of VT-d (DMA protection), guest hardening, host hardening with CET, SMAP, and UMIP, and handling of uncommon events like microcode update, NMI, and MTRR updates. This version can complement your understanding of advanced topics and be a reference to explore more as you wish.

Who should attend?

Software developers, security researchers, and anyone interested in expanding their knowledge of virtualization technologies, the x86_64 system architecture, and UEFI should attend the class. Many past students enjoyed discovering details of new system architecture aside from learning Intel VT-x!

Class requirements

Prerequisites:

• Fluency in C (or C++) programming.
• Familiarity with the x86_64 architecture, such as privilege levels, interrupts, page tables, and system registers at the concept level.
• System programming experience, such as kernel-module development, is a plus but not a requirement.

You will receive links to recommended pre-class learning materials 2-3 weeks before the class.

Hardware and Software Requirements:

You need to have the following hardware and software:

• The host machines with the Intel processors, SSD, 8GB+ RAM, and 50GB+ of free storage space
• For Windows users
  • Windows 10 build 22631 (a.k.a. 22H2)+ without Virtualization-base Security (VBS) enabled
  • Ubuntu 22.04+ on WSL version 1
  • VMware Workstation Pro (Recommended) or VMware Workstation Player 17
• For Linux users
  • Ubuntu 22.04+
  • VMware Workstation Pro 17
• For macOS users
  • macOS 11+
  • VMWare Fusion Pro 12 or VMWare Fusion Player 12
  • Homebrew and git

Again, an Intel processor-based machine is required.

Newer operating systems and software are supported. Older software and another Linux distro may be workable but not tested. Other hypervisors, such as KVM, Hyper-V, or VirtualBox, cannot be used. If the host machine cannot be arranged locally, it can also be a cloud-provided machine. Contact the trainer for suggestions if you need a cloud-provided machine.

You will receive setup instructions 2-3 weeks before the class and must complete them before the class.

What is included?

• Materials
  • Training materials (slides and sample code) will be shared 2-3 weeks before the class.
  • Recording will be available shortly after the end of each day.
• Support
  • 3 weeks of asynchronous consultation on Slack and email from the last day of the class.
  • The indefinite right to request the latest slides and code for no additional cost.