Boot: From Reset to kmain

# Boot: From Reset to `kmain`

## CS 326 Operating Systems

L10 · September 24, 2026 · exercises `30k_kernel_basics` (Oct 2), `31k_boot` (Oct 8)

---

## Learning Objectives

- Describe a RISC-V hart's state at reset, and what is *not* initialized
- Explain what firmware does, and what `-bios none` removes
- Decode QEMU's six-instruction `virt` boot ROM
- Justify every line of `kernel.ld`: `ENTRY`, `.`, `*(.entry)`, `etext`, `end`
- Trace one byte from `write_volatile` to a character on the terminal
- Predict the failure of a kernel with no `sp`, and of a non-`volatile` MMIO write

---

## The Machine at Reset

Stop QEMU before it executes anything (`-S`) and dump the registers:

```text
 pc       0000000000001000
 mhartid  0000000000000000
 mstatus  0000000a00000000        <- MPP = 0: MACHINE mode
 mtvec    0000000000000000        <- no trap handler
 satp     0000000000000000        <- paging OFF, addresses are physical
 x2/sp    0000000000000000        <- NOT a stack
```

That is the entire state that matters.

---

## Four Facts That Govern Everything

1. **`pc = 0x1000`**, not your code — the board's reset vector
2. **Machine mode** — the only privilege mode that exists at reset
3. **Paging off** (`satp = 0`) — virtual addresses do not exist yet
4. **`sp` is meaningless** — QEMU zeroes it; real silicon leaves garbage

<div class="highlight-box">
The RISC-V spec leaves general registers <strong>unspecified</strong> at reset.
Never write boot code that assumes a register starts at zero.
</div>

No allocator. No interrupt handler. No "process". On real hardware, not even
working RAM until a DRAM controller is programmed.

---

## Normally, a Lot Runs Before You

<div class="mermaid">
flowchart TD
    A1["Power on: hart 0 leaves reset"] --> A2["Mask ROM / ZSBL"]
    A2 --> A3["OpenSBI, M-mode\nx86: BIOS / UEFI"]
    A3 --> A4["U-Boot or GRUB"]
    A4 --> A5["Linux, S-mode\ncalls SBI for console and timers"]
</div>

**OpenSBI** stays resident in M-mode and offers the Supervisor Binary
Interface — `ecall` services for console, timers, reset — so an S-mode kernel
need not know which UART the board has.

---

## What `-bios none` Deletes

QEMU loads the `-kernel` ELF straight into RAM and points the reset vector at
RAM base. **Your kernel is the firmware.**

- No SBI → `putc` cannot be an `ecall`; it must be a store to a device register
- No bootloader → nothing relocates you; the linker script must place you exactly
- You start in **machine mode**, because no M-mode resident dropped you down

<div class="info-box">
<strong>"No firmware" is not "no boot ROM."</strong> Six ROM instructions always run.
What disappears is OpenSBI and the bootloader — tens of thousands of instructions.
</div>

---

## The Entire Boot Process of This Course

```asm
0x1000:  auipc  t0, 0            # t0 = 0x1000
0x1004:  addi   a2, t0, 40       # a2 = 0x1028, fw_dynamic info struct
0x1008:  csrr   a0, mhartid      # a0 = 0
0x100c:  ld     a1, 32(t0)       # a1 = [0x1020] = 0x87e0_0000, device tree
0x1010:  ld     t0, 24(t0)       # t0 = [0x1018] = 0x8000_0000
0x1014:  jr     t0               # go
```

Both loaded constants are data QEMU patched into the ROM at startup.

---

## What the ROM Hands You

| Register | Value | Meaning |
|---|---|---|
| `a0` | `0` | hart ID (`-smp 1`) |
| `a1` | `0x87e0_0000` | device tree blob, 2 MiB below `PHYSTOP` |
| `a2` | `0x1028` | `fw_dynamic` struct, magic `"OSBI"` |
| `t0` | `0x8000_0000` | jump target = RAM base |

Linux reads `a1` and discovers its memory map from it.
**rv6 ignores `a1`** and hardcodes `memlayout.rs` — which is exactly why rv6
would not boot on a different RISC-V board.

---

## The `virt` Physical Map: RAM and Devices Together

<div class="highlight-box">
<strong>Memory-mapped I/O:</strong> a load or store in a device's range is not a
memory access at all — it <em>operates the device</em>. Print the map yourself
with <code>info mtree -f</code> in the QEMU monitor.
</div>

---

## Five Regions, Five Exercises

| Region | Base | What it is | rv6 |
|---|---|---|---|
| Test finisher | `0x0010_0000` | write a magic word, QEMU exits | ex 01 |
| CLINT | `0x0200_0000` | core-local timer: `mtime`, `mtimecmp` | ex 14 |
| PLIC | `0x0c00_0000` | routes device IRQs to a hart | ex 15 |
| UART0 | `0x1000_0000` | NS16550A serial port | ex 01 |
| RAM | `0x8000_0000` | 128 MiB, `KERNBASE`..`PHYSTOP` | everywhere |

**CLINT is inside the CPU's world** — it interrupts *this* hart, M-mode only.
**PLIC is outside** — it collects peripheral lines (UART = source 10).

<div class="info-box">
<strong>Why <code>0x8000_0000</code>?</strong> Everything below it is device
space or ROM; RAM begins there and nowhere else, so that is the ROM's jump
target, so that is where the kernel's first instruction must be.
</div>

---

## `kernel.ld`

```text
OUTPUT_ARCH( "riscv" )
ENTRY( _entry )              /* kernel.ld:12 */

SECTIONS {
  . = 0x80000000;            /* kernel.ld:16 */

.text : {
    *(.entry)                /* kernel.ld:19  <- the trick */
    *(.text .text.*)
    . = ALIGN(0x1000);
    PROVIDE(etext = .);      /* kernel.ld:22 */
  }
  .rodata : { . = ALIGN(16); *(.srodata .srodata.*) *(.rodata .rodata.*) }
  .data   : { . = ALIGN(16); *(.sdata   .sdata.*)   *(.data   .data.*)   }
  .bss    : { . = ALIGN(16); *(.sbss    .sbss.*)    *(.bss    .bss.*)    }

PROVIDE(end = .);          /* kernel.ld:43 */
}
```

---

## `ENTRY` and the Location Counter

**`ENTRY(_entry)`** writes the address into the ELF header's entry field.

<div class="highlight-box">
QEMU's boot ROM <strong>never reads that field</strong>. It jumps to
<code>0x8000_0000</code> unconditionally. <code>ENTRY</code> is how a debugger
knows where to break.
</div>

**`. = 0x80000000`** sets the *location counter*, the linker's placement cursor.
Change it and your kernel is unbootable — the ROM's jump target does not move
with it.

---

## `*(.entry)` First — The Whole Trick

- `.entry` is a section name nothing in Rust's output claims
- `entry.rs:11` puts exactly one function in it with `#[link_section = ".entry"]`
- The script lists `*(.entry)` **before** `*(.text .text.*)`
- So `_entry` lands at offset 0 of `.text` = `0x8000_0000`

<div class="info-box">
Delete that line and the linker orders functions as it pleases.
<code>0x8000_0000</code> then holds an arbitrary Rust function, entered with a
garbage stack pointer — and the kernel dies <strong>silently</strong>.
</div>

---

## `etext`, `end`, and the Page Allocator

`PROVIDE(etext = .)` is a page-aligned boundary for mapping code R-X; rv6 does
not use it yet, so `nm` will not show it. `PROVIDE(end = .)` sits past every
byte of the image — the linker's runtime answer to *"where does my kernel stop?"*

```rust
extern "C" {
    static end: u8;                          // kalloc.rs:14
}

pub unsafe fn init() {
    let start = &end as *const u8 as usize;  // kalloc.rs:22
    free_range(start, PHYSTOP);              // kalloc.rs:23
}
```

`static end: u8` declares a *byte* whose value is meaningless — the allocator
wants its **address**. That is how you reach a linker symbol from Rust.

---

## What a Real Build Produces

```text
Section     Address       Size      Note
.text       0x8000_0000   0x1000    _entry at offset 0
.rodata     0x8000_1000   0x022e    string literals
.eh_frame   0x8000_1230   0x0058    the script never named this
.data       0x8000_1288   0x0008
.bss        0x8000_1290   0x4000    all of it is STACK0
                                    -> end = 0x8000_5290
```

<div class="highlight-box">
<code>.eh_frame</code> is there although the script never mentioned it. A linker
script does not restrict output to the sections it names — which is why you read
<code>end</code> at runtime, not as ".bss start plus size".
</div>

---

## The Chicken and the Egg

Every function Rust compiles begins with a prologue:

```asm
kmain:
    add  sp, sp, -32        # carve 32 bytes off the stack
    sd   ra, 24(sp)         # save the return address there
```

Both instructions dereference `sp`. You cannot fix that from inside Rust,
because the fix would itself have a prologue.

<div class="info-box">
<strong>The stack must be established by code that does not use a stack.</strong>
</div>

---

## `entry.rs` — Twenty Bytes of Assembly

```rust
const STACK_SIZE: usize = 4096 * 4;                      // entry.rs:5

#[no_mangle]
static mut STACK0: [u8; STACK_SIZE] = [0; STACK_SIZE];   // entry.rs:8

#[no_mangle]
#[link_section = ".entry"]                               // entry.rs:11
pub unsafe extern "C" fn _entry() -> ! {
    asm!(
        "la sp, {stack}",     // sp = bottom of our stack
        "li t0, {size}",      // t0 = stack size
        "add sp, sp, t0",     // sp = top (it grows downward)
        "call kmain",         // enter Rust; never returns
        stack = sym STACK0,
        size = const STACK_SIZE,
        options(noreturn),
    );
}
```

---

## Stacks Grow Down

```text
   high addresses
   0x8000_5290  +---------------------+  <- sp starts HERE (STACK0 + 0x4000)
                |                     |
                |   16 KiB of         |     sp moves DOWN as calls nest
                |   kernel stack      |
                |                     |
   0x8000_1290  +---------------------+  <- STACK0, symbol address
   low addresses
```

`la sp, STACK0` gives the array's *lowest* address. Starting there means the
first push writes **below** the array. The `li`/`add` pair moves `sp` one byte
past the top.

---

## Four Pseudo-Instructions, Six Real Ones

```asm
0000000080000000 <_entry>:
    80000000:  00001117    auipc  sp, 0x1
    80000004:  29010113    add    sp, sp, 656      # sp = 0x80001290 <STACK0>
    80000008:  6291        lui    t0, 0x4          # t0 = 0x4000 = 16384
    8000000a:  9116        add    sp, sp, t0       # sp = 0x80005290
    8000000c:  00000097    auipc  ra, 0x0
    80000010:  20e080e7    jalr   526(ra)          # -> 0x8000021a <kmain>
```

- `la` = `auipc` + `add` — RISC-V has no 64-bit immediate
- `li t0, 16384` collapsed to one compressed `lui` (16384 is `4 << 12`)
- `call` = `auipc` + `jalr`, leaving a return address `kmain` never uses

---

## Skip It: The Silent Trap Loop

Delete the three stack instructions. It builds cleanly. QEMU prints **nothing**.

```text
 pc       0000000000000000
 mcause   0000000000000001        <- instruction access fault
 mtvec    0000000000000000
 x1/ra    0000000080000008        <- we did reach kmain
 x2/sp    fffffffffffffff0        <- 0 + (-16)
```

`sp - 16` wrapped → store fault → jump to `mtvec` = 0 → fetch
fault at 0 → forever.

<div class="highlight-box">
A silent hang plus an OSlings timeout <strong>is</strong> the signature of a
broken stack pointer.
</div>

---

## Printing Is One Store

No operating system, no C library, no file descriptor, no system call:

```rust
const UART0: *mut u8 = 0x1000_0000 as *mut u8;   // uart.rs:15

pub fn putc(c: u8) {
    unsafe { write_volatile(UART0, c); }         // uart.rs:24
}
```

The device is an **NS16550A**, descended from the 8250 on the 1981 IBM PC.
The 16550's FIFO was famously broken; the **16550A** fixed it, and that part
number became the universal serial interface — still, forty years later.

---

## Eight Registers, `0x1000_0000`–`0x1000_0007`

| Offset | On write | On read | rv6 |
|---|---|---|---|
| +0 | THR — transmit holding | RBR — receive buffer | `uart.rs:6-7` |
| +1 | IER — interrupt enable | IER | `uart.rs:8` |
| +2 | FCR — FIFO control | IIR — interrupt ident | `uart.rs:9` |
| +3 | LCR — line control | LCR | `uart.rs:10` |
| +4 | MCR — modem control | MCR | `uart.rs:11` |
| +5 | — | LSR — line status | `uart.rs:12` |

`LSR_DR` (bit 0) = a byte waits in RBR. `LSR_THRE` (bit 5) = safe to transmit.
**Same offset, two registers, depending on direction.**

---

## Following the Byte

<div class="mermaid">
flowchart TD
    A["uart::putc(0x48) — uart.rs:24\nwrite_volatile: emit exactly once"] --> C["sb a1, 0(a0)\na0 = 0x1000_0000, a1 = 0x48"]
    C --> D["satp = 0: no translation\n0x1000_0000 goes on the bus"]
    D --> E["decode: 'serial' MemoryRegion, not RAM"]
    E --> F["QEMU serial write, offset 0 = THR"]
    F --> G["chardev (-serial mon:stdio) → your terminal"]
</div>

After exercise 33k turns on Sv39, the UART page must be **explicitly mapped** or
this very same store faults.

---

## Why `volatile` Is Not Optional

Replace `write_volatile(UART0, c)` with `*UART0 = c`, build with `--release`:

```asm
0000000080000016 <kmain>:
    80000016:  lui   a0, 0x10000       # a0 = 0x1000_0000
    8000001a:  li    a1, 10            # a1 = '\n'   <- the LAST byte only
    80000026:  sb    a1, 0(a0)         # one store. thirty were deleted.
    8000002a:  sw    a2, 0(a3)         # test finisher: 0x5555
```

Thirty-one stores to one address, no intervening read → thirty are dead.
The terminal receives **one newline**.

<div class="highlight-box">
A missing <code>volatile</code> can pass in debug and fail in release.
<code>unsafe</code> turns off Rust's <em>safety</em> checks, not the optimizer.
</div>

`volatile` constrains the **compiler**, not the hardware: it guarantees the
instruction is emitted, and says nothing about caches or what another hart
observes. Cross-hart ordering needs fences — exercise 37k.

---

## Stopping the Machine

```rust
const TEST_FINISHER: *mut u32 = 0x10_0000 as *mut u32;   // testdev.rs:11
const FINISHER_PASS: u32 = 0x5555;                       // testdev.rs:13
const FINISHER_FAIL: u32 = 0x3333;                       // testdev.rs:14

pub fn exit_failure(code: u16) -> ! {
    unsafe { write_volatile(TEST_FINISHER, FINISHER_FAIL | ((code as u32) << 16)); }
    loop { core::hint::spin_loop(); }                    // testdev.rs:30-33
}
```

A kernel has nowhere to return to — hence `-> !`. x86 powers off through ACPI;
real RISC-V calls the SBI reset extension. The finisher is what lets
`oslings run 31k_boot` get a real exit status back.

---

## The Whole Session

<div class="mermaid">
sequenceDiagram
    autonumber
    participant HW as Hardware / QEMU
    participant ROM as Boot ROM @ 0x1000
    participant E as _entry @ 0x8000_0000
    participant K as kmain (Rust)
    participant U as UART @ 0x1000_0000
    HW->>ROM: reset: pc=0x1000, M-mode, satp=0, sp=garbage
    ROM->>ROM: a0 = mhartid, a1 = device tree, t0 = 0x8000_0000
    ROM->>E: jr t0
    E->>E: la sp, STACK0 / li t0, SIZE / add sp, sp, t0
    E->>K: call kmain
    K->>U: write_volatile(0x1000_0000, b'r')
    K->>HW: write_volatile(0x10_0000, 0x5555)
    HW-->>HW: QEMU exits, status 0
</div>

---

## Compared With xv6 and Linux

- **xv6-riscv** boots under OpenSBI, starts every hart, gives each a stack slice.
  rv6 uses `-smp 1` and one `STACK0` — removing a class of concurrency bugs
- **Linux `head.S`** also sets `sp` to a static `init_thread_union` before
  calling C — then relocates itself, parses the DTB, enables paging with a
  temporary map, and calls `start_kernel`

<div class="info-box">
Your twenty bytes at <code>0x8000_0000</code> do the job of the first page of
<code>head.S</code>.
</div>

---

## Where This Goes Next

| Exercise | What it adds |
|---|---|
| 02 | reads `end`, turns RAM above it into a free list |
| 03 | Sv39 page tables; the UART store works only if you map it |
| 13 | `start.rs` between `_entry` and `kmain`; `mret` into S-mode |
| 14 | CLINT timer — a heartbeat |
| 15 | PLIC — an interrupt-driven console |

Everything after today elaborates one sequence diagram.

---

## Summary

1. **A hart at reset gives you almost nothing** — M-mode, `satp = 0`, `mtvec = 0`, `sp` meaningless

2. **`-bios none` removes firmware, not the boot ROM** — your kernel *is* the firmware

3. **`0x8000_0000` is not a choice** — everything below it is device space

4. **`*(.entry)` first is the whole mechanism** that puts `_entry` at RAM base

5. **`end` is the linker's runtime answer** to "where does the kernel stop"

6. **Set `sp` before any Rust runs** — or get a silent trap loop at `pc = 0`

7. **Printing is one `volatile` store** to `0x1000_0000`; drop `volatile` and the optimizer eats it

---

## The Exercises: `30k_kernel_basics` (Oct 2), `31k_boot` (Oct 8)

```bash
oslings run 30k_kernel_basics
oslings watch
```

- `30k` passes when the crate **compiles** for `riscv64gc-unknown-none-elf`
- `31k` passes when QEMU prints `OSLINGS:PASS` on the serial console
- A timeout with no output means `sp` never got set up

Read the exercise `README.md` first — it teaches the *how*.
This deck and the lecture page carry the *why*.