Pi-BoltReference
Benchmarks
Pi-Bolt against Pi as released, on stock Bun and on Node, on three machines. Medians of fresh processes, interleaved across the builds. Lower is better in every row.
How it is measured#
- The same work for every build. Each build runs the same prompts against a local server that replays the same model responses, so there is no network or provider variance.
- Pi as released. "Pi on Bun" and "Pi on Node" are Pi 1.1.0 built from its release tag, not Pi-Bolt's tree.
- Interleaved. Builds take turns, so a change in the machine's state affects all of them alike.
- CPU time is user plus system time of the process. Memory is the process's own memory: the pages it wrote, not file pages it shares with the system.
The benchmark scripts, the method in full and the raw data are in the repository: BENCHMARKS.md.
Linux#
AMD EPYC 7B13, pinned to four cores. Pi 1.1.0 as released.
| Benchmark | Pi-Bolt 0.8.5 | Pi on Bun 1.4.2 | Pi on Node 24 |
|---|---|---|---|
| Writing a 200 KB file through a tool call | 0.6 s | 24.0 s | 32.5 s |
| CPU, streaming a 60,000-character answer | 2.6 s | 40.2 s | 38.7 s |
| Memory after a 4.2M-token session | 134 MB | 253 MB | 536 MB |
| Memory of a session in tmux | 22 MB | 84 MB | 185 MB |
| CPU, interactive session (5 prompts) | 193 ms | 746 ms | 1,093 ms |
CPU, one prompt (-p) | 61 ms | 283 ms | 550 ms |
| Ready to type | 38 ms | 116 ms | 296 ms |
macOS#
Apple M5 (MacBook Air). Pi 1.1.0 as released.
| Benchmark | Pi-Bolt 0.8.5 | Pi on Bun 1.4.2 | Pi on Node 26 |
|---|---|---|---|
| Writing a 200 KB file through a tool call | 0.4 s | 22.6 s | 28.1 s |
| CPU, streaming a 60,000-character answer | 6.5 s | 37.9 s | 37.6 s |
| Memory after a 4.2M-token session | 38 MB | 92 MB | 1,581 MB |
| Memory of a session in tmux | 24 MB | 65 MB | 162 MB |
| CPU, interactive session (5 prompts) | 139 ms | 561 ms | 937 ms |
CPU, one prompt (-p) | 49 ms | 221 ms | 512 ms |
| Ready to type | 41 ms | 100 ms | 348 ms |
Windows#
Intel Core i5-1335U laptop. Pi 1.1.0 as released. Memory on Windows is the private working set.
| Benchmark | Pi-Bolt 0.8.5 | Pi on Bun 1.4.2 | Pi on Node 24 |
|---|---|---|---|
| Writing a 200 KB file through a tool call | 0.5 s | 38.1 s | 26.6 s |
| CPU, streaming a 60,000-character answer | 6.9 s | 41.1 s | 32.6 s |
| Memory after a 4.2M-token session | 122 MB | 248 MB | 413 MB |
| Memory of a session in a terminal (ConPTY) | 32 MB | 97 MB | 60 MB |
| CPU, interactive session (5 prompts) | 278 ms | 805 ms | 1,125 ms |
CPU, one prompt (-p) | 92 ms | 329 ms | 544 ms |
| Ready to type | 105 ms | 177 ms | 311 ms |
Against PiG, oh-my-pi and OpenCode#
Pi-Bolt against two other fast Pi builds: PiG 0.4.1, a port of Pi to Go
(based on Pi 1.0.3), and oh-my-pi 18.8.7, a fork of Pi with many more
built-in features, on Bun; and OpenCode 2.0, in a run of its own next to
Pi-Bolt on the same machines, with its own server in the same process tree (--standalone).
| Benchmark | Pi-Bolt 0.8.5 | PiG 0.4.1 | oh-my-pi 18.8.7 | OpenCode 2.0 |
|---|---|---|---|---|
| Writing a 200 KB file through a tool call | 0.6 s | 77.5 s | 3.3 s | 2.4 s |
| CPU, streaming a 60,000-character answer | 2.6 s | 16.7 s | 2.2 s | 8.6 s |
| Memory after a 4.2M-token session | 134 MB | 349 MB | – | 273 MB¹ |
| Memory of a session in tmux | 22 MB | 48 MB | 334 MB | –² |
| CPU, interactive session (5 prompts) | 193 ms | 2,458 ms | 2,800 ms | 8,593 ms |
CPU, one prompt (-p) | 61 ms | 227 ms | 1,524 ms | 2,220 ms |
Peak memory, one prompt (-p) | 123 MB | 55 MB | 505 MB | 346 MB |
| Ready to type | 38 ms | 31 ms | 890 ms | 1,687 ms |
| Benchmark | Pi-Bolt 0.8.5 | PiG 0.4.1 | oh-my-pi 18.8.7 | OpenCode 2.0 |
|---|---|---|---|---|
| Writing a 200 KB file through a tool call | 0.4 s | 56.8 s | 2.5 s | 1.7 s |
| CPU, streaming a 60,000-character answer | 6.5 s | 16.5 s | 5.1 s | 13.6 s |
| Memory after a 4.2M-token session | 38 MB | 330 MB | – | 260 MB¹ |
| Memory of a session in tmux | 24 MB | 55 MB | 330 MB | –² |
| CPU, interactive session (5 prompts) | 139 ms | 2,367 ms | 2,374 ms | 6,493 ms |
CPU, one prompt (-p) | 49 ms | 230 ms | 1,030 ms | 1,548 ms |
Peak memory, one prompt (-p) | 70 MB | 50 MB | 419 MB | 379 MB |
| Ready to type | 41 ms | 39 ms | 612 ms | 1,489 ms |
PiG starts a little sooner and its single-prompt process is smaller; oh-my-pi streams a long answer with a little less CPU. Pi-Bolt uses the least CPU in sessions and per prompt, the least memory over a session, and writes large files fastest. oh-my-pi compacts a long conversation on its own, so the long session does not compare it (–). ¹ OpenCode's process without its server process, so its memory is higher. ² OpenCode did not finish its answers in tmux.
Why a large file write takes so long on stock Pi#
Those 20 to 40 seconds are not the disk. A model writes a file by streaming the write tool call's arguments, the whole
file, a few characters at a time, so a 200 KB file arrives in thousands of chunks. Stock Pi parses and redraws the whole tool
call again for every chunk: each chunk costs more than the last, and the total grows with the square of the file's size.
Pi-Bolt processes only the new part of each chunk.
In the benchmark the model streams as fast as it can, so the cost shows as wall time. With a real model, which streams more slowly, it shows as CPU: about 40 seconds of it for a 200 KB file on stock Pi.