QBao: bao as a Resource Partitioner on RVA23

There is an excellent RISC-V board on my desk: a SpacemiT PicoITX/K3. And an excellent emulator installed on my workstation: QEMU with RVA23 support. So I thought: since this is the most advanced virtualization technology available, which hypervisor matching my idea of a "Resource Partitioner" already exists?

Bao, of course. There was a very interesting recording about it recently, from this year's RISC-V summit in Bologna.

Bao is a static partitioning hypervisor. No scheduler, no dynamic memory, no shared drivers: each guest gets its own cores, its own memory and its own devices, all decided at build time and never renegotiated. A few thousand lines of code you can actually read.

Given that I already have HFI BIOS and mr-bml, I thought: why not apply the same approach to improving bao that I already use in HFI BIOS? Trimming sources, patching, etc....

The name suggested itself: QBao 😃

Step 0: bao with no guest OSes at all

The first attempt was to run bao with no guest OSes whatsoever. It worked.

Trimming turned out to be simpler than with U-Boot: bao's build leaves a .d file per object, listing exactly what the compiler actually opened. So the list of needed files is not guessed but read back out of the build itself: 581 files at the top, 121 at the bottom. What survives is src/arch/riscv with the AIA interrupt controller, src/core and its MMU half, src/lib, the qemu-riscv64-virt platform and the sbi_uart driver.

Three things worth noting straight away.

A commit, not a tag. I pinned to a commit on main rather than to the v2.0.0 tag: the tag predates the switch of the qemu-riscv64-virt platform from PLIC to AIA, and the vector, Zicbom/Zicboz and Sstateen work. All of that is RVA23 territory.

Debian ships only lp64d. Bao builds with -mabi=lp64 (soft float, the right call for a hypervisor), so the very first <limits.h> reaches through features.h → gnu/stubs.h for gnu/stubs-lp64.h, which does not exist. A freestanding build does not want the libc headers anyway, so the build drops them (-nostdinc), hands back GCC's own, and adds an empty limits.h for the #include_next that GCC's limits.h ends on. Not a bao defect; a multilib gap.

There is no such thing as an empty configuration. Bao's configuration IS its guest list. An empty one yields CONFIG_VM_NUM 0 and therefore a zero-length array in vmm.c, which the build rejects. Upstream has configs/null with one dummy VM, purely so the tree compiles; it does not survive a run. And it does not survive it quietly: an ERROR() inside mem_init() reaches console_write() before console_init() has run, and spins there in while (!console_ready). The machine simply says nothing, not even a banner.

Hence my own configuration: one VM with zero cpus. Nothing is ever scheduled, and the hypervisor says so:

Bao Hypervisor v2.0.0-50-g53f2b4c-dirty (Sep  1 2026 - 15:45:29)
BAO ERROR: cannot start guest OS: configuration declares no guest cpus

Step 1: the same thing, off a disk and through real firmware

Next, the long way round, the way it will be on the board. Nothing is injected with -kernel:

QEMU reset -> HFI BIOS SPL (M-mode, -bios)
           -> u-boot.itb off this disk's ESP, found by GPT type
              -> OpenSBI (M) -> HFI/U-Boot (HS) -> the BIOS front end
                 -> mr-bml (ESP: EFI/boot/bootriscv64.efi)
                    -> QBao, in HS-mode

This is where it turned out that bao.elf cannot be loaded by any loader. Bao sets its location counter to BAO_VAS_BASE and never gives the loadable sections a load address, so every PT_LOAD carries p_paddr == p_vaddr == 0xffffffc0.... QEMU's -kernel does not care: it drops the flat bao.bin at 0x80200000 and enters there. A real loader, however, copies each PT_LOAD to its p_paddr, and no such physical address exists.

The fix is one objcopy --change-section-lma shifting every LMA down by BAO_VAS_BASE - 0x80200000. The entry point stays high, so the loader's own translation (e_entry - p_vaddr + p_paddr) lands exactly where OpenSBI enters bao today. bao.bin comes out byte-identical, which is the check that nothing else moved.

A second detail: the machine needs one hart more than bao uses. platform.cpu_num is 4, and bao's master starts harts 0..3 itself over SBI HSM, so all four must be free. The VideoBIOS, meanwhile, keeps a hart of its own for its character generator and never gives it back. You can see it in the log: VideoBIOS controller owns hart mask 0x10 — hart 4, just past bao.

The main thing: guests are linked into the hypervisor binary

And here the reason for all of this became visible. Bao's guest list is a C file, and VM_IMAGE(name, path) is an .incbin. The guest images live inside bao.elf, so changing the partitioning means rebuilding the hypervisor.

I want the opposite: the guest list should arrive with the boot.

Why Multiboot3 rather than kernel + initrd

mr-bml has a kernel command which would load the hypervisor perfectly well. But it can hand over exactly one blob in the role of an initrd, because a device tree can carry linux,initrd-start only once. That is not a convention but a structural limit: it is a property of the /chosen node, and property names are unique within a node. Trivially checked:

$ dtc -I dts -O dtb dup.dts
dup.dts:6.9-47: ERROR (duplicate_property_names):
        /chosen:linux,initrd-start: Duplicate property name

module3, on the other hand, appends. Every call adds a module with its own command line, so the guest list becomes a list:

menuentry 'QBao hypervisor + QSOE/N + Linux' {
    multiboot3 /boot/qbao/bao-load.elf
    module3    /boot/qbao/qbao.cfg       qbao.cfg
    module3    /boot/guests/skimmer.bin  qsoe-n
    module3    /boot/guests/modpkg.cpio  qsoe-modpkg
}

The first word of a module's command line is its name, and that is what joins this menu to the configuration. The Multiboot3 path in mr-bml had to be finished first: the hand-off function for the case without EFI boot services was a stub calling mrbml_fatal(), and by then the EFI console was already gone. A silent death with no diagnostic. That went into mr-bml 0.9.

modrunner

QBao's own part is called modrunner/. It attaches without any patch to the build system: bao already compiles an out-of-tree CONFIG_REPO/CONFIG directory into the hypervisor and puts its inc/ on the include path — exactly the hook a front end needs. The same relationship BIOS/ has with U-Boot in HFI BIOS.

The Multiboot3 loader enters it with a0 holding the magic, a1 the address of the boot information, and no stack at all. modrunner then walks the tags, reads the partition map out of qbao.cfg, stages the guest images into memory bao owns, fills in the same struct config bao was going to read, and enters bao.

The configuration is mandatory — inventing one would be a worse answer than stopping:

Guest QSOEN {
    module = qsoe-n
    initrd = qsoe-modpkg
    cpus   = 2
    base   = 0x80200000
    size   = 0x10000000
    entry  = 0x80200000
}

Five traps, all of them silent

Each of these produces a machine that says nothing at all, for the reason described above: an ERROR() raised in early memory has nowhere to go. Recorded here because each one cost hours.

Nothing that must survive may live in .bss. Bao clears .bss after modrunner has handed over. Anything the hypervisor is meant to read later goes in .datanocopy, which is loaded from the image and never cleared. A stack in .bss is perfectly fine: it is dead by then.

No pointer may be baked into an initializer. modrunner runs with the MMU off, so &x under -mcmodel=medany is PC-relative and comes out physical, while anything the linker resolved is virtual. Writing through such a pointer is a fault at an address starting 0xffffffc0. This needed an explicit conversion, anchored on a word in assembly holding the link-time address of _image_start — in assembly precisely so the compiler cannot fold it back into a PC-relative computation of the same symbol.

The entry stub must let secondary harts through. Bao brings its other harts up with sbi_hart_start(hartid, load_addr, 0), and load_addr is the start of the image — which is now my stub. They arrive with a0 = hartid rather than the magic, and must fall through into bao. Turn them away and the master waits at its boot barrier forever. The magic is exactly what distinguishes the two entries.

image.separately_loaded must be true. Otherwise config_init() rewrites load_addr = load_addr - BAO_VAS_BASE + img_addr. Correct for an .incbin'd image carrying a link-time address, nonsense for one the loader placed.

Guests are staged downward from the top of bao's region. They have to be staged at all because the loader takes memory from the bottom (around 0x80003000), while bao owns only what its platform description claims, from 0x80200000 up. And they go at the top because immediately above the hypervisor image sits the page pool's bitmap (root_pool_set_up_bitmap(), 128 KiB for a 4 GiB pool).

The console

The guest then ran, but had nowhere to put a word:

BAO WARNING: guest issued unsupport sbi extension call (1145193294)

1145193294 is 0x4442434E, "DBCN": the SBI debug console. Bao does not forward it. I started with console_write_byte (FID 2), where the byte arrives in a register and there is nothing to translate, and added a warning for the remaining functions. The very first run said function 0: QSOE/N writes buffers, not bytes.

And console_write hands over a buffer by guest-physical address, which the hypervisor's address space does not map. So each page of the buffer is walked through the guest's second stage and mapped in turn. A page at a time deliberately: guest-physical contiguity says nothing about host contiguity when the memory came from a page pool.

console_read I do not forward. One console and several guests is an input policy: only one of them can be listening, and deciding which belongs to the configuration, not to this file. For the same reason DBCN is deliberately not added to the extension table: a guest that asks whether the extension is available is told no, which is true of the extension as a whole.

Since the wire is shared for now, every guest line is labelled with the name its configuration gave it:

QSOEN-> ================================================
QSOEN->   QSOE/N microkernel ("Skimmer") v0.25
QSOEN-> ================================================
QSOEN-> *** PANIC at kernel/arch/riscv/fdt.c:1099:
        fdt_init: NULL FDT pointer (bootloader broke contract) ***

A device tree for each partition

The guest is right: bao enters it with a1 = 0, and vcpu_arch_reset() even carries a comment saying that ought to be the DTB address.

The host's tree will not do. It describes the whole machine: every hart, all of memory, every device. A partition owns a slice, and handing it the whole picture would be handing it a map of memory it must not touch and cpus that are not its own. So the tree is built from the partition's own description — the same qbao.cfg.

Two details I would not have guessed, and learned by asking the guest.

First, QSOE/N does not merely prefer an initrd: without /chosen/linux,initrd-* its fdt_init panics. Which closes the circle rather neatly, since that same property — the one a tree can carry only once — is precisely why the guest list needed module3.

Second, the APLIC node needs a phandle of its own. The guest collects APLIC candidates and then picks the one whose msi-parent points at the S-mode IMSIC, and a node without its own phandle never makes it into the candidate list at all. So: "no interrupt controller", with the controller described right there. One property.

The AIA addresses also have to go not only into the tree but into vm_config.platform.arch.irqc: bao emulates the APLIC at irqc.aia.aplic.base and places the guest's IMSIC files at irqc.aia.imsic.base + PAGE_SIZE * vcpu_id. A tree that disagrees points the guest at memory nothing answers.

The result: the guest reads the tree, believes it, and fences off exactly the regions modrunner placed:

QSOEN-> fdt: no PCI host bridge -- PCI surface disabled
QSOEN-> timer: timebase 10000000 Hz, tick 10000 cycles, scan hart 0
QSOEN-> FDT: aplic@0xd000000 srcs=96 imsic@0x28000000 ids=255 gidxbits=0 stride=0x1000
QSOEN->      ram@0x80200000 size=0x10000000
QSOEN->      initrd@0x900cf000 size=0x120e00
QSOEN-> physmem: 1 bank(s), 3 exclusion(s)
QSOEN->   ram  [0x80200000 .. 0x90200000) fdt memory@ node
QSOEN->   excl [0x901f0000 .. 0x901f04ad) fdt blob
QSOEN->   excl [0x900cf000 .. 0x901efe00) initrd (modpkg cpio)
QSOEN->   excl [0x80200000 .. 0x81181000) firmware + kernel
QSOEN-> intc: selected aia backend
BAO ERROR: unknown synchronous exception (22)

Every number there came out of qbao.cfg. Nothing was recompiled to put a guest there.

Where it stands now

scause 22 is virtual instruction: an instruction the hardware refused to execute in VS-mode and handed to the hypervisor, which has no handler for it, so it dies as "unknown". QSOE/N drives siselect/sireg/stopei directly, and that is the likeliest culprit. Naming the instruction precisely would take htinst, which that handler already has to hand and simply discards.

Console input is open too: output works (that whole log is DBCN), but nothing can be typed at a guest, because no UART is assigned to one. Which partition owns which console is a configuration question.

Conclusion

The evening came to four patches against bao. Two of them are the price of grafting one's own front end onto any hypervisor: a diagnostic, and some room in the linker script. The other two are of a different kind: bao gives a guest neither a console nor a device tree.

And those are not oversights. They follow from where bao stands: the integrator bakes everything at build time, and the guest is a bare-metal RTOS compiled for one board, which needs no tree, gets its UART passed through, and asks almost nothing of SBI. QSOE/N and Linux are the opposite kind of guest: general operating systems that expect the SBI/Linux boot contract — hart id, device tree, console, HSM, timer. Every gap I hit today is that one mismatch, showing up in a different place.

Bao is well made. It is simply made for a different guest.

What does come out of this evening for certain: Multiboot3 works as a hand-off mechanism. It carried a hypervisor, a configuration file and two guest images off a GPT disk through real firmware, and the contract held. Along the way it found two things in mr-bml that would otherwise still be sitting there: an unimplemented hand-off function, and a trap in the width of the type field in tags (most tags begin with a 32-bit type, but MODULE and MEMMAP with a 64-bit one, so size sits at offset 8 rather than 4; a walker assuming 4 reads the zero upper half of the type as the size and loops on the spot forever).

gk208-videobios, part 8/8 — The Datasheet: Handing a Live Controller to an Operating System

Last in a series on building a firmware-free VideoBIOS for RISC-V. In Part 7 the card drew text I controlled — a grid of cells turned into pixels by a compute kernel on the GPU's own engines, with no firmware anywhere. That made it a video controller. This post is about the part that makes a controller worth building: letting something else drive it.

---

The story so far. A cold GK208, brought up from nothing by RISC-V code, now shows text I choose, generated on the card's own engines. But it was still my program doing the drawing. A controller that only its author can drive is not a controller.

The thing that draws is not the thing that stays

A video facility at power-on cannot be a program that has to keep running. The moment I began this series for — text on the screen before an operating system exists — works because the firmware draws it and then hands the machine on. So my program's job is narrow, and it ends: cold-init the card, draw the first thing on the screen, and then it is done. What it leaves behind has to be enough for the next stage to take over, because my code will not be there to help.

A block of memory that says a controller is up

Nothing in the GK208's silicon records that a text controller exists. The cell array is memory I chose; the repaint is a kernel I wrote; the hardware knows about neither. So the only authoritative sign that a controller is up is one I publish myself: a block of memory, marked with a private GUID and a magic word, holding everything a consumer needs — where the cells live, how a cell is encoded, where the framebuffer is, how to repaint. I called it the Controller Handoff Block. A consumer does not probe the PCI bus or read a single card register to find it, since a cold card would tell it nothing useful anyway. It looks for the block. Presence is the whole signal.

Exit, and persist

Handing off across an exit has one hard part. The block cannot live inside my program, because the program unloads when it exits — the description would leave with the describer. So the .efi copies the block into memory the firmware keeps after an application is gone, and registers it there as a configuration table. Then it exits. The controller stays up on its own — scanout is autonomous, the picture holds — and the instructions for driving it survive in a pool that outlives the program that wrote them.

The doorbell

Scanout holding by itself is not the same as new text appearing. When a consumer writes new cells, something has to turn them into pixels — wake the compute kernel — and that wake is a small fixed sequence: flush the framebuffer, write one entry into the channel's ring, preempt, then bump the put-pointer. You write all the cells first and ring once, never per character. The last step is where the hardware taught me something I had assumed wrong, and I left the lesson in a comment:

/* Writing GP_PUT here pokes the PBDMA's doorbell — a PRAMIN/VRAM write does not. */

The doorbell is a specific address that means "go," and writing the same value anywhere else in memory does nothing. Ring it and the kernel repaints the grid; write the cells without ringing and the screen never changes.

The loop closes

On the 1st of July 2026 the whole thing ran end to end. A cold GK208: my .efi inits it, draws a logo and a cursor, publishes the handoff block, and exits. Then the next boot stage — mr-bml, the loader that will become part of QSOE — finds the block by its GUID, writes its own menu into the cells, rings the doorbell, and its menu appears on the monitor. My program was already gone. The screen was being driven by software that had never touched the card, only the block of memory I left behind. That is the line between a demo and a controller, and it had finally been crossed.

Writing the datasheet

What is left is the work of making it a thing other software wants to use, and lately that has meant small, deliberate additions to the handoff block. A consumer-writable palette, so the operating system sets its own colours instead of mine. And a transparent cell: one bit that tells the generator to leave a cell's pixels untouched, so a BIOS can cut a rectangle out of the text grid and draw a logo straight into the framebuffer beneath it, with the text intact around the hole. The bit had to come from somewhere, and the cell had one to spare — glyph codepoints never reach 0x2600, so the top bit of every cell was free to take:

[31] transparent   [30:16] symbol (UCS-2)   [15] blink   [14:8] bg   [7:0] fg

In the kernel it costs three instructions: mask the bit, test it, and return without writing if it is set. The GPU documents none of this — not the cells, not the doorbell, not the transparent bit. The handoff block is the datasheet for a controller that exists only because I wrote it, and writing that datasheet down plainly, so someone else can build against it, is the actual product.

Saying hello

Now the RISC-V Personal Computer I've built 5 years ago finally has proper means of announcing information on the screen when it's powered on. It's very satisfying, and it paves the way towards the next challenge: real BIOS, with real POST screen and real Set-Up program.

gk208-videobios, part 7/8 — First Pixel: Content I Control, and the Character Generator

Seventh in a series on building a firmware-free VideoBIOS for RISC-V. In Part 6 a cold GK208, driven only by my own modeset, put out a live 720×400 analog signal the monitor recognized — but the picture it scanned held nothing I had put there. This post is about crossing that last gap: making the card show pixels I chose, and then making text with no text mode to lean on.

---

The story so far. With the x86 firmware gone, my own EVO modeset locked the pixel clock and the monitor read the exact mode back to me. The display pipe was live. What it presented was not — the head was scanning a surface I did not control.

An empty raster

A locked clock gives you timing, not a picture. The head was scanning, the sync was clean, and what reached the glass was whatever happened to sit in the surface the head pointed at — not a framebuffer I had filled. To present pixels I choose, the base channel has to show a real surface out of VRAM with an output LUT in the path, and that is where First Light stopped.

My first instinct was to add it on top of what already worked: keep the modeset that locked the clock, and poke in the base surface and the LUT after the fact.

INVALID_STATE

The display engine refused. The atomic update that added the surface and the LUT came back with an error I would stare at for days — 0x6101f0 reading 0x5080, a type-5 INVALID_STATE — and it refused every variation of the same idea. The reason turned out to be a property of the hardware, not a wrong value of mine: you cannot extend a head that has no coherent surface state by poking new pieces into it. The engine takes exactly one coherent initialization, and half a modeset plus later additions is not a state it will accept.

That is the same shape of wall as Part 4, and the same lesson in a new place: the engine advances when the state is real and whole, and rejects it when it is assembled in pieces.

One commit

So I threw out the bolt-on and built the modeset the way nouveau builds it — as a single atomic commit. Everything the head needs goes in together: mode timing, the core backing surface, the output LUT, the usage bounds, the DAC, procamp and dither and the viewport, all pushed into the 907D core channel; the real framebuffer, at VRAM 0x02000000, pushed into the 907C base channel; and the whole assembly made to take at once by one interlocked UPDATE — the core committing interlocked with the base, the base interlocked with the core, so neither lands without the other. Pushed as one consistent state, the engine accepted it. The wall that had stood for a very long time was gone in the time it takes to issue two methods.

First pixel

On the 25th of June 2026 I painted the letters GK208 into the framebuffer and they stood on the monitor — white glyphs on black, cold card, no nouveau anywhere, no firmware underneath. After First Light, where the monitor knew the mode but showed nothing of mine, this was the first time the screen carried something I had put there on purpose — five characters, and I sat looking at them.

The one register with colour in it

For a while everything I drew was some shade of gray, and I did not question it, because everything I drew was white on black anyway. When I finally pushed a colour through the pipe, the head handed me gray. The saturation had been zero the whole time — Kepler's procamp block wants a specific neutral value, and zero is not it:

push_mthd(0x0498, 1);
push_data(0x00040000);   /* SAT_COS = 0x400 (unity). Writing 0 here
                            zeroes saturation and the head outputs GRAYSCALE. */

One method, one value — 0x400, unity saturation — and the colour I was already sending arrived as colour. It was the one bit of Part 4 again, this time a single register: nothing downstream was wrong, one field upstream was neutralizing everything.

A character generator, not a mode

Here is what a cold Kepler will not give you: a text mode. The legacy VGA text path lights nothing on this card, which Part 6 settled. So text is not a mode you select — it is glyphs you draw, one bitmap at a time, into the framebuffer you now control. The first GK208 was exactly that, done on the CPU: walk the string, look up each glyph in the font, set the pixels.

That prints a logo. It is not a controller. A controller cannot spend a CPU drawing every character of every repaint, so the drawing has to move onto the card's own engines — and on these Kepler parts those engines run on nouveau's open microcode, with no signed firmware blob to load, so I could bring them up firmware-free. To put five letters on a screen I had ended up bringing a 2D engine and then a compute pipeline to life on a cold GPU, a stretch of yak-shaving I can only defend by where it led. It led to a character generator: a compute kernel of my own, hand-written and compiled to the card's native instruction set, that reads a grid of cells and a font and writes the glyph pixels itself. Each cell is one word:

[31:16] symbol (UCS-2)   [15] blink   [14:8] bg   [7:0] fg

The screen becomes a grid of those cells, and turning the grid into pixels is the kernel's work, not the CPU's.

It was never a demo

Once text is a cell array that the card's own engine turns into pixels, the program I had been writing quietly changed into something else. It was not a thing that draws a logo and holds it. It was a video controller — a text surface an operating system could write into and repaint, the way a PC's software has always driven the text console it was handed. I had set out to give my computer a voice at power-on and built, without quite naming it, the controller that voice would speak through. Handing that controller to an operating system — the ABI, the doorbell, the exit — is the last post.

Next: The Datasheet — Handing a Live Controller to an Operating System.

gk208-videobios, part 6/8 — First Light: the Modeset That Finally Locked the Clock

Sixth in a series on building a firmware-free VideoBIOS for RISC-V. In Part 5 a cold GK208 woke with no x86 anywhere in the machine — live chip ID, memory controller, privilege ring — and the monitor still showed black. This post is the modeset I wrote to light it, and the ordering that finally locked the pixel clock.

---

The story so far. I abandoned the card's x86 firmware and replayed a captured cold-init sequence from C on the RISC-V CPU. The silicon came up firmware-free. The screen did not, because waking the chip and lighting the display are different jobs, and the second one had killed the old road too.

One bit, again

The black screen from Part 4 had a single cause, and it had not changed: the pixel clock never locked. VPLL0's control register at 0x614140 read 0x02010002 on my card and 0x02030002 on a working one — bit 17, the lock bit, clear. On the x86 road I could blame the firmware for it, since it sat on a timed gate waiting for a warm machine mine was not. Now there was no firmware. The sequence going into the display engine was mine, and so was the unlit bit — days into the June 2026 rewrite, with the cold card freshly awake, nothing stood between it and a lit screen but code I had written.

VGA registers light nothing

I started where the capture pointed: program the legacy CRTC and attribute registers for mode 3, set the raster geometry, upload the 8×16 font. On these cards that scans out nothing. The VGA path Kepler carries is vestigial — present for compatibility, wired to nothing that reaches the DAC on its own. The display is driven by EVO, the class-based display channel: you allocate a core channel and push methods into it, and no amount of poking CRTC registers coaxes a raster to life. I had been aiming at the wrong altitude.

A modeset in the open

So I read nouveau again, the same way as before — as a specification, not code to ship. An EVO modeset is a sequence of methods on the core channel: bind a context DMA for the framebuffer, configure the head with the raster timings and the 28.322 MHz pixel clock, attach an output resource, then issue one UPDATE that makes the whole assembled state take at once. I wrote that in my own C, against the golden register values I had read off the working card. The live output on this board is DAC-1, not DAC-0 — a fact that had already cost me an evening in Part 4 — and this time I attached the right one from the start.

The cargo cult I left behind

In Part 4 I had tried to move the display supervisor by faking the signals it waits on, driving the advance myself so the firmware would go and program its clock. That is cargo-cult modesetting: reproducing the motions the state machine expects without establishing the state those motions are supposed to leave behind. It went through some of the phases and never locked. The coherent EVO sequence is the opposite. It puts the engine into the exact state the supervisor is checking for, so the supervisor advances because the modeset is real, not because I mimed the parts it could see.

The thing that locked the clock

Even with the method sequence right, the VPLL stayed unlocked until one more thing happened: the output was attached to the head. The lock is not something I could set by writing the PLL register directly — it follows from binding the DAC to the head that drives it. My lock check after programming the coefficients is deliberately short, because I learned it will never pass this early, and the comment says why:

/* on this board the VPLL does NOT lock until the DAC is attached
 * later in the supervisor handshake, so a long wait here always caps. */
while (!(rd(0x614140) & 0x00020000u) && ++spins < 400)
    usleep(5);

That 0x00020000 is bit 17, the same lock bit that read clear in Part 4. Attach the DAC, and it sets. The ordering has the shape of a rain dance, an arcane sequence performed in a fixed order to placate the hardware, and for a while I treated it as one. It is not voodoo. The clock locks because the output path the PLL feeds is finally whole, and calling that magic would only mean I had not yet understood it.

First light

I ran the cold init from Part 5, then the modeset, on a GK208 that had never been POSTed in its life. 0x614140 read 0x02030002 — the lock bit set, the same value the working card shows at a login screen. The monitor recognized the signal and, in its own on-screen menu, read the mode back to me: 720×400 at 70 Hz, analog — the exact timing I had programmed. There was no x86 running anywhere, no emulator in the binary, no firmware underneath: a RISC-V CPU had taken a stone-cold card to a live raster on its own.

A signal, not a picture

What lit was the pipe, not a picture I had chosen — the display engine scanning out a framebuffer whose contents I did not yet control. That is the honest size of it: the gate that had stopped both roads was open, and the clock the firmware could never lock on my machine locked on the first coherent modeset of my own. Putting content I choose into that raster — text, from a character generator of my own, with no VGA text mode to lean on — is a different problem, and it is the next post.

Next: First Pixel — Content I Control, and the Character Generator.

gk208-videobios, part 5/8 — Burning the Boats: Abandoning x86 Entirely

Fifth in a series on building a firmware-free VideoBIOS for RISC-V. In Part 4 the card's own x86 firmware ran to a clean far return and the monitor still showed black, because that firmware was written for a warm PC my board is not. This post is about giving up on it — and the better approach that giving up made obvious.

---

The story so far.I got a RISC-V CPU to run the GK208's x86 video BIOS: full init, sign-on banner, clean far return — onto a card that stayed dark. The firmware was waiting for a machine mine isn't, and more fidelity would not change that.

The last thing I did on the old road

The last commit on the x86 project is not a fix. It is a small tool that only watches: while a known-good driver brings the card up, it logs every register write — address and value — to a text file. I wrote it as one more debugging instrument, but it was the end of the x86 road. With that log, the question answers itself: if I can see every write a working card receives, why run the x86 firmware at all?

What the capture meant

A GPU init is a sequence of register writes. The x86 firmware is one elaborate way to produce that sequence — self-decompressing, ROM-reading, assuming a PC around it — but the card does not care which CPU issues the writes; the same values in the same order bring it up the same way. Everything I had built on the x86 road — the emulator, the disassembler fixes, the decompression hole, the warm-card handshake I never satisfied — existed only to generate that sequence, and I already had it captured.

Reading a driver instead of running a firmware

The known-good driver is nouveau, the open Linux driver for these cards. It mattered as a reference to read, not as code to run — I am not going to ship a Linux kernel to light the screen at power-on. Unlike the legacy VBIOS, nouveau is a cold-start driver: it assumes nothing warm underneath and brings the card up from nothing, in readable C. Where the firmware was a black box, nouveau was a specification. None of it ships; what ships is my own sequence, in my own C, checked against a driver that does the same job in the open.

Burning the boats

The hard part was throwing away work I liked. The emulator I had compiled for RISC-V, the disassembler I had repaired, the patches that went upstream — all of it served running the firmware, and I was deciding that running the firmware was the wrong goal. Not a failed attempt at a good plan, but a good attempt at a plan that could not work, because the firmware wants a PC and my machine is not one. Cortés burned his ships so no one could turn back; a new, empty repository with no x86 in it was the same move. Mostly it was a relief.

A cold init is a sequence of writes

The new program is short to describe: replay the captured sequence from C, on the RISC-V CPU, into the register window. It is not a flat list, because a cold init has to wait for the hardware in places, so the recipe becomes a small op-table of three step kinds — write a value, poll a register until it reads what I expect, wait a fixed time. Each poll is bounded by the GPU's own timer, so a step that never completes fails in milliseconds instead of hanging the machine. Clocks, memory controller, privilege ring: a few hundred lines, no interpreter, no x86 ROM.

Replay is not parroting

A blind replay does not work. Some steps are only values in order, but others depend on the state the card is in when you reach them, and the privilege ring is the clear case: it has to be brought up and re-enumerated before whole regions of registers will answer, and writing to them early returns a fault pattern instead of data. You have to know what each step is for, so that when a captured value does not produce the captured result you can tell whether you are early, wrong, or reading a region the ring has not connected yet.

The card wakes, firmware-free

I powered a cold GK208 that had never been POSTed, ran the op-table, and it came up: PMC_BOOT_0 read a live chip ID, the memory controller started, the privilege ring stood, the register map answered. No x86 ran anywhere, and no emulator was even in the binary.

And the monitor was still black. Waking the silicon and lighting the display are different jobs, and the x86 road had died at the second one — the clock that would not lock, the supervisor stuck at phase one. Burning the boats did not remove that problem; it only meant I would solve it in my own modeset, with no firmware underneath me. That is the next post.

Next: First Light — the Modeset That Finally Locked the Clock.

gk208-videobios, part 4/8 — The Wall: Why a POSTed Card Still Showed Black

Fourth in a series on building a firmware-free VideoBIOS for RISC-V. In Part 3 the card's own x86 firmware ran to a clean far return on my RISC-V machine, sign-on banner and all. This post is about what happened next: by every register I could read the card was up, and the monitor still showed nothing.

---

The story so far. I taught a RISC-V CPU to emulate x86 well enough to run the GK208's video BIOS from the first byte to its sign-on banner to a clean far return. Counted in instructions, POST finished. The screen stayed black.

Did the display code even run?

First I had to rule out the boring explanation: that the firmware's display code never actually executed and I'd fooled myself with a nice-looking trace.

The device-init handlers in this firmware are dispatch records with a code pointer a few bytes in. Once I found that structure I put a log on the pointer and watched the display handlers get called and run. So they ran. The problem was past "did it run."

Walking back from the glass

When the monitor is black, you start at the connector and walk backwards, asking at each stage whether the signal is there.

The DAC I could power on. Cost me an evening to learn the live output on this board is DAC-1, not DAC-0. The 8×16 font uploaded into the right plane. The mode-3 raster geometry programmed fine. Everything near the monitor looked plausible.

The signal died at the clock. The pixel clock on this head comes from a PLL the hardware calls VPLL0, and VGA text mode wants it locked at 28.322 MHz. No locked clock, no timing; no timing, no sync; no sync, black — no matter what's downstream. My VPLL0 wasn't locking.

One bit

To see what a locked VPLL looks like, I did what I'd end up doing over and over: booted the card-equipped machine into Linux with nouveau, let it bring up a login screen, and read the display registers off a card that was demonstrably working. That became my golden reference.

The difference was one bit. The VPLL control register at 0x614140 read 0x02030002 on the nouveau card and 0x02010002 on mine — bit 17, the lock bit, set on the one showing a login screen and clear on the one showing black. Everything else in my modeset matched the golden values closely enough not to matter. So it wasn't a wrong color or a font in the wrong plane. The display engine just never crossed from "configured" to "on," and it said so with one unlit bit.

The supervisor stuck at phase 1

That bit stays clear because of the display supervisor — a small state machine in Kepler's display controller that runs a modeset as a sequence of phases and enables the output at the end. Mine stalled at phase 1 and never advanced. No advance, no output enable, no PLL lock, black screen.

I spent days pushing on it. Instrumented the output teardown to find where it wedged. Wrote code to fake the supervisor-advance signals so the firmware would go program its own VPLL — and it did, I got the VBIOS programming the pixel clock itself. Still no lock.

So I took apart the display script the firmware decompresses at runtime (the self-inflating hole from Part 3), decoded its opcode table, and found the instruction that wouldn't let go: a timed condition-wait, one opcode among many, waiting for a hardware condition that on my machine never came true. That was the gate. Nothing was broken and my emulation wasn't wrong — the firmware was correctly waiting for something my cold card never gave it.

Two firmwares in one ROM

What it was waiting for got clearer when I looked harder at the ROM. It isn't one program. The image I'd been running for months is the legacy x86 video BIOS, the option-ROM that answers INT 10h. Sitting next to it in the same ROM is a second, separate image: a 64-bit x86 UEFI GOP driver — the module a modern PC's UEFI loads to drive this exact card.

That re-framed the first image. A GK208 in a real PC isn't brought up by the INT 10h path I'd been so carefully emulating; it's brought up by UEFI running the GOP driver, inside a pile of services and prior init the legacy path has quietly come to depend on. The old text-mode firmware is still there for compatibility, but it no longer carries the whole recipe for taking a stone-cold card to a lit display, because on the machines it actually runs on it never has to.

The warm-card model

This is what I started calling the warm-card model. The firmware doesn't expect the card I was handing it. It expects one that's already partway warm — touched by a real PC's UEFI, by the GOP driver, by the handshake a motherboard does around the video BIOS — and its display path is written for that. On a genuinely cold card with nothing under it but a RISC-V CPU running x86, the supervisor hits its timed gate, waits for a condition only a warm machine would have set up, and never advances.

The thing I was proud of in Part 3 — running the real firmware faithfully, with nothing faked — is exactly why it couldn't finish. A real firmware run faithfully will faithfully wait for the world it was written for.

The floor

This is the wall, and it's worth being clear about which kind it is: not the kind more of the same effort climbs. I could make the x86 emulation flawless, serve every byte of the PROM, match the golden registers digit for digit — and the card would still sit at that gate. The missing piece was never fidelity. It was a whole warm-machine context, a UEFI and a GOP and a PC, that my board isn't and that I had no intention of building. I'd set out to give my computer a small voice at power-on and found that this road ended at rebuilding most of a PC just to say hello.

The last commit on this road

At the very tip of that project's history there's a quiet commit, and I like it in hindsight, because it's where the whole approach turns. After all the fighting to make someone else's firmware finish, the last thing I did was write code to just watch: log the full sequence of register writes a real, working init performs, and save it. I didn't quite see it that night, but that little capture tool was me giving up on running the x86 firmware and starting to ask a better question — if I can see exactly what a working card wants written to it, why do I need the x86 firmware at all?

The rest of the series is the answer. It starts with throwing away everything above.

Next: Burning the Boats — Abandoning x86 Entirely.

gk208-videobios, part 3/8: Teaching a RISC-V Chip to Dream in x86

Third in a series on building a firmware-free VideoBIOS for RISC-V. In Part 2 I got a decades-old x86 emulator to compile and link for RISC-V for the first time. This post is about what happened when I finally let it run — when a RISC-V processor started interpreting, one instruction at a time, the x86 firmware that my graphics card had carried in ROM since the day it was manufactured.

---

The story so far. My RISC-V machine cannot execute the x86 program that knows how to wake up the graphics card, so I resurrected U-Boot's forgotten bios_emulator and made it build for RISC-V. Making it compile, though, was only ever the easy half.

From "it builds" to "it runs"

There is a particular kind of quiet that settles over you the first time you point an emulator at a piece of firmware that has never in its life been executed by anything but a real Intel-compatible CPU. The GK208's video BIOS had spent years being run only by the machines it was designed for, and now a RISC-V core was going to walk through it byte by byte, pretending — convincingly, I hoped — to be the x86 processor it expected to find.

The mechanics are less mysterious than they sound. The emulator maps the card's ROM image into the classic option-ROM location in emulated memory — segment C000 — sets up the x86 registers the way a real PC firmware would before calling a video BIOS, and starts interpreting at the entry point. The card's firmware announces itself the old-fashioned way, with the option-ROM signature 0x55AA at its very first bytes, followed by the PCIR and NPDE structures that identify it as the BIOS for this particular device. All of that parsed correctly, which was already a small thrill: the emulator and the firmware agreed on what they were looking at.

Then the firmware started executing in earnest, and it did what complex, real firmware always does when it meets a world that is subtly not the one it expects. It crashed.

Fixing the disassembler just to read the map

When an emulated program goes off the rails, the first thing you want is a readable trace — a running account of which instruction executed where, so you can find the exact point at which the firmware's expectations and my emulated machine parted ways. The emulator can produce exactly this, an instruction-by-instruction disassembly of everything it runs, and it became my single most important tool for the entire x86 adventure.

The trouble was that the disassembler itself was subtly broken, in ways that had apparently gone unnoticed for years because nobody had leaned on it this hard. Certain instructions printed garbled — the bit-scan BSF, the whole family of conditional SET instructions, and the near CALL and JMP forms that take an immediate address all came out either mangled or with the wrong target printed next to them. A disassembler that lies to you about jump and call targets is worse than no disassembler at all, because you spend hours chasing control flow that never happened, so before I could trust a single trace I had to stop and repair the emulator's own disassembly of those instructions. It is a strange feeling to fix the map before you can begin the journey, but that is genuinely where a lot of the early time went, and those fixes are among the patches that later made their way upstream.

With traces I could finally believe, two real mysteries came into focus.

The zero hole: code that unpacks itself

The first mystery was a hole. Reading through the ROM image, there is a stretch — roughly from offset 0xb000 to 0xf200 — that is simply full of zeros, and the firmware, at a certain point in its run, would happily jump straight into that empty region and execute nothing, which is to say it would run off into the weeds.

The explanation, once I saw it, was elegant and slightly infuriating. That region is not meant to be empty at runtime, because it holds compressed code, and early in its execution the firmware is supposed to run a decompressor that inflates the packed payload into the hole before anything tries to call into it. The packing turned out to be ordinary DEFLATE, the same algorithm that lives inside every zip file, which meant the card's firmware carries its own little inflate routine and unpacks part of itself into place at boot. On my emulated machine that decompressor was never firing, so the hole stayed zero and the firmware leapt into the void.

Finding where the decompressor lived was not something a static read of the ROM could tell me, because its destination is computed rather than obvious, so I did the thing you do in this situation and set a write-watchpoint across the whole 0xb000–0xf200 region, asking the emulator to stop and tell me the instant any instruction wrote into the hole. The moment the unpacker fired, the watchpoint caught it red-handed with its exact address, and from there the region filled in correctly and the firmware could call into code that actually existed.

The card reads its own ROM — and I had given it only half

The second mystery was stranger, and it turned out to be the one that had been quietly poisoning everything.

Buried in the firmware is a routine — I came to know it intimately as the code at 0x1a40 — that is, in effect, an option-ROM reader. It reaches back into the card's own PROM through a hardware window, parses the familiar 0x55AA / PCIR / NPDE structures, and pulls out data the rest of the init sequence depends on. In other words, the firmware does not treat its ROM as a passive image that someone else loads for it; partway through waking the card up, it turns around and reads itself.

Here is where my own shortcut came back to bite me. The ROM image I had been feeding the emulator was a truncated dump — enough, I had assumed, to hold the code that mattered. But when the firmware's own ROM-reader reached past the end of my truncated dump, looking for content that exists on the real card but not in my partial copy, the emulator did the only honest thing it could and read off the end of the buffer, and the whole thing came down with a segmentation fault:

exit 139 (segfault) — reading the real PROM beyond our dump runs past the buffer

The fix was almost embarrassingly simple once I understood the cause, which was to stop being clever and serve the firmware the entire 256 KB PROM exactly as it exists on the card, so that when it read itself it found everything it expected. The truncated dump had been the root of a whole family of failures I had previously blamed on far more exotic causes, and replacing it with the full image changed everything at once.

A clean return

With the full PROM in place and the hole filling itself in, the firmware ran, and ran, and then it did something I had half stopped believing I would see: it finished.

An x86 video BIOS, when it is done, returns to whoever called it with a far return, a RETF, back to the caller's address. Mine did exactly that, jumping from c000:2dcb — deep inside the card's firmware — back out to the return address a real PC firmware would have set, and the emulator exited cleanly with status zero and not a single skipped handler or faked-up redirect along the way. Everything the firmware asked for, it got, and everything it did, it did for real.

The part that made me trust the result was that it was boringly reproducible, run after run landing in the same place with the same amount of work behind it:

run 1: EXIT=0  RETF_hits=2  mem.wr=20041  mmio=4240
run 2: EXIT=0  RETF_hits=2  mem.wr=20041  mmio=4278

Twenty thousand memory writes, a few thousand hardware register accesses, two passes through the return path, and a clean exit both times. In emulation terms, the card's power-on self-test had completed — the firmware had walked its full initialization from entry to exit on a RISC-V processor, which as far as I know had never happened before.

It even tried to say its name

There was one more detail that, more than the clean return itself, made me feel I was genuinely close.

Near the end of its run, just before that final RETF, the firmware calls into a sign-on routine — its banner printer — sitting at 0x2c9e, invoked from 0x2d4c. That routine walks a small table of strings, the ones every one of us has seen a thousand times without thinking, the card's name and the BIOS version and the copyright line, and it prints each of them character by character through a helper at 0x2c6e. And the way that helper puts a character on screen is the most gloriously period-authentic thing in the whole story, because it issues the old INT 10h teletype call, AH=0Eh, through the firmware's own interrupt-10 handler at 0x1afb, pulling the text attribute from a byte the firmware had stashed away earlier — the exact same mechanism by which every PC has printed its boot text since before I ever sat down in front of that PS/2.

So the firmware did not merely initialize the card and return, it ran the complete software path to announce itself, character by character, through the classic video teletype service, precisely as it would have on a real motherboard. On the level of executed instructions, my RISC-V machine had run a real x86 video BIOS from its first byte to its sign-on banner to its clean far return.

The victory that wasn't quite

I want to be honest about how this felt, because it felt like winning. The card's own firmware, running to completion on a processor it was never written for, doing real work against real hardware, walking its banner path on the way out — measured in instructions executed, this was everything I had set out to do in the autumn of 2024, and there were a few evenings there where I let myself believe the hard part was behind me.

It was not, and the reason is a gap that sits at the very center of this whole project, which is the difference between the firmware ran its display code to a clean return and the monitor showed me something. Everything I have described so far happened inside the emulator and inside the card's registers, in a place I could see only through traces and memory dumps, and the one thing I could not produce — the one thing this entire quest was about — was a single readable character glowing on the actual screen in front of me.

Understanding why a fully, correctly, reproducibly initialized card still stared back at me with a black screen is where the x86 road stopped being an engineering problem and started being an education. That is the next post.

Next: The Wall: Why a POSTed Card Still Showed Black.