# User Mode I: The Wall, the Trampoline, and the Trapframe ## CS 326 Operating Systems L22 · November 10, 2026 · exercise `48k_user_mode` (Nov 13) --- ## Learning Objectives - Explain why privilege levels alone do not isolate a process - Enumerate what user mode forbids, and the trap each violation raises - Describe an rv6 user address space and the role of `PTE_U` - State the trampoline problem: the instruction *after* `csrw satp` - Trace `uservec`, naming `a0` and `sscratch` at every step - Derive why the kernel cannot push user registers onto a stack --- ## The Promise > Run this program. If it misbehaves, only the program dies. Not achievable in software: - A program that can execute any instruction the kernel can → rewrites the kernel - A program that can name any address the kernel can → reads its secrets, forges its structures The hardware has to refuse **on the kernel's behalf, every cycle, at zero cost**. --- ## Two Mechanisms — You Built Both | Mechanism | Restricts | Where you met it | |---|---|---| | Privilege levels | Which **instructions** are legal | `43k_traps` | | MMU + page tables | Which **addresses** exist | `33k_paging`, `39k_virtual_memory` | <div class="highlight-box"> Neither suffices alone. Privilege without paging → an ordinary <code>ld</code> reads the kernel. Paging without privilege → <code>csrw satp</code> installs any table you like. Isolation is the <em>conjunction</em>. </div> --- ## Rings: a Short History - **Multics (1965)** — eight nested protection rings - **Intel 80286** — four rings; every real OS used exactly two - **ARM** — exception levels EL0–EL3 - **RISC-V** — three: M, S, U (+ optional hypervisor) Rings model a *linear* order of trust. Real systems need a *lattice* — mutually untrusting peers — which comes from separate address spaces, not rings. --- ## The Third Privilege Level <div class="mermaid"> stateDiagram-v2 [*] --> M: reset M --> S: mret, MPP=S S --> U: sret, SPP=U U --> S: ecall U --> S: interrupt U --> S: fault S --> S: kernel trap </div> Dropping S → U is exercise 43k's `mret` one level down: clear `sstatus.SPP`, set `sepc`, `sret` (`usermode.rs:455`, `:459`). --- ## What U-Mode Forbids | U-mode may not | Result | |---|---| | `csrr`/`csrw` any supervisor CSR | illegal instruction, `scause = 2` | | `sret`, `sfence.vma`, `wfi` | illegal instruction, `scause = 2` | | Touch a page without `PTE_U` | page fault, `scause` 12 / 13 / 15 | | Touch an unmapped address | page fault | Arithmetic, loads, stores, branches: all legal. User mode is not a sandbox that *inspects* — it is a set of refusals that cost nothing. --- ## Two Doors Back The CSR ban includes **reads**: a program cannot read `sstatus`, `satp`, or `stvec`. The wall is opaque from the far side — which is why `getpid()` must be a system call. - **`ecall`** — the program asks; `scause = 8` - **Interrupt or fault** — the hardware forces it <div class="info-box"> <code>ecall</code> is not a jump and not a call. It is a synchronous exception — same vector, same <code>sepc</code>, same bookkeeping as a page fault. <strong>A system call is a fault raised on purpose.</strong> </div> --- ## A Private Address Space ```text kernel page table a user page table 0x3F_FFFF_F000 TRAMPOLINE R X 0x3F_FFFF_F000 TRAMPOLINE R X <- same page 0x3F_FFFF_E000 (unmapped) 0x3F_FFFF_E000 TRAPFRAME R W <- this proc ... ... 0x8000_0000 KERNBASE R W X (nothing at all up here) 0x0C00_0000 PLIC R W 0x0001_1000 <- initial sp 0x1000_0000 UART0 R W 0x0001_0000 stack page R W U ... (guard gap) 0x0000_0000 (unmapped) 0x0000_0000 program image R X U ``` One page table per process since `34k_processes` — today `satp` finally points at it. --- ## `PTE_U` Is the Wall `PTE_U = 1 << 4` (`vm.rs:23`) — one bit per leaf PTE. - The kernel's UART, PLIC, and 128 MiB of RAM are simply **absent** from the user table - The two entries that *are* present — trampoline, trapframe — have `PTE_U` **clear** (`proc.rs:164-165`) **It cuts both ways.** With `sstatus.SUM = 0` the *kernel* may not load or store through a `PTE_U` page either — a deliberate guard against dereferencing a user pointer by accident. Hence `walkaddr` (`vm.rs:252-261`), which translates by hand. --- ## `MAXVA`, Address 0, the Gap - **`MAXVA = 1 << 38`** (`memlayout.rs:49`) — Sv39 gives 39 bits, but bits 63:39 must sign-extend bit 38. Stop one bit short and never think about it again. - **Address 0 is ordinary.** Nothing sacred about zero in a private space; it is where xv6 loads programs. Null-pointer traps are a luxury of `49k_exec`. - **The guard gap** between the image and the stack at `0x1_0000` turns a runaway into a clean page fault. --- ## Linux Did the Opposite The kernel lives in the **high half of every address space**, supervisor-only. Entering the kernel needed no page-table switch at all — just a privilege change. That held until **January 2018**: Meltdown showed speculation could leak supervisor-only pages that were merely *mapped*. <div class="highlight-box"> KPTI unmaps the kernel from user tables and switches <code>CR3</code> on entry — so the switching code must be mapped in both. Linux calls that page the <strong>entry trampoline</strong>. Same design, arrived at from the opposite direction, twenty years later. </div> --- ## The Trampoline Problem Entering the kernel means changing `satp`. But `satp` is not data — it is the map for *every* address, including the next instruction fetch. ```text va 0x8000_5000: ld t1, 0(a0) # t1 = kernel satp va 0x8000_5004: csrw satp, t1 # <- the world changes HERE va 0x8000_5008: jr t0 # <- fetched through the NEW table ``` The `csrw` retires; PC becomes `...5008`; the fetch uses the new table. Different page → execute garbage. Unmapped → instruction page fault, delivered through an `stvec` that may itself be unmapped → dead hart. <div class="info-box"> Not a TLB problem. <code>sfence.vma</code> fixes <em>stale</em> translations; here the translation is fresh, correct, and points somewhere else. </div> --- ## The Resolution: One Page, Every Table The instructions that write `satp` must live at a virtual address that means the same thing in the old table **and** the new one. <div class="mermaid"> graph TB KR["kernel page table root"] --> KE["leaf PTE\n255 / 511 / 511\nR X V, no U"] P1R["process 1 root"] --> P1E["leaf PTE\n255 / 511 / 511\nR X V, no U"] P2R["process 2 root"] --> P2E["leaf PTE\n255 / 511 / 511\nR X V, no U"] KE --> PHYS["ONE physical page\nuservec + userret"] P1E --> PHYS P2E --> PHYS PHYS -.-> VA["mapped at VA 0x3F_FFFF_F000\nin every table"] </div> `TRAMPOLINE` is a constant of the design, not a per-process value. --- ## It Gets Its Own Physical Page The assembly is linked into the kernel image, sharing a page with unrelated code. Mapping *that* page would expose the neighbors. ```rust let tramp = kalloc::kalloc(); // vm.rs:158 ptr::copy_nonoverlapping(src as *const u8, tramp, len); // vm.rs:167 asm!("fence.i"); // vm.rs:168 mappages(root, TRAMPOLINE, PGSIZE, tramp as usize, PTE_R | PTE_X)?; // vm.rs:169 ``` `fence.i`: we just wrote **instructions** through the data path. RISC-V does not promise the fetch path sees them without it. --- ## Mapped Without `PTE_U` `vm.rs:169` and `proc.rs:164` — `PTE_R | PTE_X`, no `PTE_U`, in the *user's* table. - The trampoline is kernel code that happens to be addressable in the user's space - If user code could execute it: jump past the `csrw satp` in `userret`, reload registers of your choosing, read `kernel_satp` out of the trapframe - Without `PTE_U`, the only way to land there is a **trap** — which already raised privilege before the first byte was fetched **In the address space and unreachable from it.** --- ## `sfence.vma` Brackets Every `satp` Write ```asm sfence.vma zero, zero csrw satp, t1 sfence.vma zero, zero ``` - Before: flush stale entries for the table you are leaving - After: guarantee the MMU consults the new one QEMU often forgives their absence. Hardware with a real TLB does not — and the failure is *intermittent*, which is worse. --- ## The Trapframe: Why Not a Stack? The `ecall` retires. Privilege is S — and every register still holds the **user program's** value, all 31 of which must come back bit-exact. `kernelvec` (`trap.rs:90-107`) solves this by pushing: ```asm addi sp, sp, -128 sd ra, 0(sp) ``` That works because a kernel trap interrupts **kernel** code. From user mode: 1. `sp` holds a *user* value — the program chose it; it may point anywhere 2. Even a good `sp` points into the *user's* space, and `satp` still holds the user's table **No valid kernel stack pointer exists in any register when `uservec` begins.** --- ## Trapframe Layout ```text offset field who writes it 0 kernel_satp usertrapret (usermode.rs:449) 8 kernel_sp usertrapret (usermode.rs:450) 16 kernel_trap usertrapret: address of usertrap (:451) 24 epc usertrap on entry; +4 for ecall (:397, :401) 32 kernel_hartid unused; keeps the xv6 layout 40 ra 48 sp 56 gp 64 tp 72 t0 ... 112 a0 ... 168 a7 ... 280 t6 ``` `#[repr(C)]` (`usermode.rs:33-71`) is what makes the Rust field offsets and the assembly's byte offsets the same numbers. **Do not reorder.** --- ## Trapframe ≠ Context | | `Context` | `Trapframe` | |---|---|---| | Size | 14 registers | 31 registers + 4 notes | | Why | `swtch` is an ordinary call; the ABI spilled the rest | a trap strikes between *any* two instructions | | Lives | on the kernel stack | on its own page, fixed VA | The ABI has promised nothing about the moment a trap arrives. --- ## `sscratch`: the Chicken and the Egg `uservec` needs a register holding the trapframe address. To load an address it must destroy a register. Every register holds unsaved user state. ```asm csrrw a0, sscratch, a0 # usermode.rs:94 ``` One instruction **swaps** register and CSR: - `a0` ← `TRAPFRAME` - `sscratch` ← the user's `a0` (parked where user mode cannot read it) Zero memory accesses. No stack. Nothing lost. --- ## Who Arms `sscratch`? `userret` does, on the way **out** (`usermode.rs:180`): ```asm li a0, {trapframe} # :144 a0 = TRAPFRAME ... csrrw a0, sscratch, a0 # :180 a0 = user a0, sscratch = TRAPFRAME sret # :181 ``` <div class="highlight-box"> The exit path arms the entry path. A process can only reach user mode through <code>userret</code>, so "<code>sscratch</code> holds <code>TRAPFRAME</code> whenever U-mode runs" holds from the program's very first instruction. No separate init exists. </div> --- ## `uservec`: Four Phases <div class="mermaid"> graph LR A["swap a0 / sscratch"] --> B["park 31 registers\ninto TRAPFRAME"] B --> C["load kernel_sp\nkernel_trap\nkernel_satp"] C --> D["sfence + csrw satp\n+ sfence"] D --> E["jr t0\ninto usertrap"] </div> `stvec` points here while user code runs (`usermode.rs:443-445`). --- ## Phases 1 and 2 ```asm csrrw a0, sscratch, a0 # :94 a0 = TRAPFRAME, sscratch = user a0 sd ra, 40(a0) # :96 ... 30 stores, a0 skipped sd sp, 48(a0) ... sd t6, 280(a0) csrr t0, sscratch # :126 t0 = the user's a0 sd t0, 112(a0) # :127 park it ``` `satp` is still the **user's** table — the trapframe is reachable because it is mapped there. `t0` was saved at offset 72 already, so it is free to clobber. --- ## Phases 3 and 4 ```asm ld sp, 8(a0) # :129 kernel stack top ld t0, 16(a0) # :130 address of usertrap() ld t1, 0(a0) # :131 kernel satp sfence.vma zero, zero # :133 csrw satp, t1 # :134 <- the instruction this page exists for sfence.vma zero, zero # :135 jr t0 # :137 fetched through the NEW table ``` <div class="info-box"> The kernel cannot <em>look anything up</em> on entry — so it leaves itself a note on exit. The trapframe is the only memory reachable at this instant. </div> Note the ordering: `sp` is a kernel address, still unmapped. Load it; do not touch it. --- ## The Road Back `usertrapret` (`usermode.rs:440-466`), in Rust, with full addressability: 1. `stvec` ← `TRAMPOLINE + (uservec - trampoline)` (`:443-445`) 2. Write `kernel_satp`, `kernel_sp`, `kernel_trap` into the trapframe (`:449-451`) 3. `sstatus.SPP = 0`, `SPIE = 1` (`:455-456`) 4. `sepc` ← saved `epc` (`:459`) 5. Call `userret` at its trampoline address with the user `satp` in `a0` (`:461-466`) Symmetrically, `usertrap`'s first act is `stvec` ← `kernelvec` (`:387`). --- ## One Way Out Every return to user mode — first entry, a syscall return, a timer-interrupt return, a forked child's first breath — goes through `usertrapret` → `userret`. That single path is why: - the `sscratch` invariant holds - `51k_fork_wait` gets a working child by **copying the parent's trapframe and zeroing one field** --- ## Key Concepts | Concept | Role | |---|---| | **User mode (U)** | No CSRs, no privileged instructions, no non-`PTE_U` pages | | **`PTE_U`** | The wall — and with `SUM=0` it bars the kernel too | | **`TRAMPOLINE`** | `0x3F_FFFF_F000`, top page of *every* address space | | **Trampoline page** | One physical page, same VA in all tables, `R X`, no `U` | | **`TRAPFRAME`** | `0x3F_FFFF_E000`, this process's register parking lot | | **`sscratch`** | Holds `TRAPFRAME` while U-mode runs; armed by `userret` | | **`kstack`** | Per-process kernel stack; never the user's `sp` | | **`sfence.vma`** | Brackets every `satp` write | --- ## Summary 1. **Isolation is a conjunction** — privilege levels restrict instructions, page tables restrict addresses; either alone leaves the kernel open 2. **`PTE_U` is the wall, and it cuts both ways** — the kernel translates user addresses by hand through `walkaddr` 3. **The trampoline exists because of the instruction *after* `csrw satp`** — that fetch uses the new table 4. **One physical page, one VA, every table**, mapped without `PTE_U` so only a trap can land there 5. **The trapframe exists because there is no usable stack on entry** — `sp` is a user value in a user address space 6. **`sscratch` breaks the chicken-and-egg, and `userret` arms it** — one `csrrw`, nothing lost, valid from the program's first instruction --- ## Next - **Friday's exercise (November 13):** `48k_user_mode` — `map_user_pages`, `usertrap`'s ecall branch, `dispatch`, `copyin` - **Next Tuesday (L23):** User Mode II — the system-call ABI, `epc += 4`, and reading a buffer that exists only in someone else's address space - **Read:** the lecture page, and `usermode.rs` top to bottom `oslings run 48k_user_mode`