The Book
by Rory Roberts, FRM
What it actually costs to run AI on hardware you own — from cache lines to euros per million tokens.
EON Risk Services
From One GPU to a County AI Factory
Rory Roberts, FRM
Most writing about running AI locally is assembled from vendor specifications. This is assembled from one workstation — an RTX 5090 serving a 27-billion parameter model — with every load-bearing figure measured, dated, and recorded alongside its source.
Where a chapter could not measure something, it says so in its opening rather than leaving you to guess.
316 pages · 61 chapters · 1.4 MB
What happens when a program runs, number formats, CPUs, memory, storage and networking — because every later constraint is one of these wearing a larger hat.
Why generation is a memory problem rather than an arithmetic one, what PCIe and NVLink actually cost you, and the single measurement that explains serving economics.
Tokenisation, embeddings, attention and the key-value cache — traced from the characters you type to the data structure that decides how many people you can serve.
vLLM, SGLang, TensorRT-LLM, scheduling, speculative decoding and prefix caching — and how to benchmark them without fooling yourself.
Multi-GPU parallelism, server hardware, power and cooling, and the Irish grid conditions that decide whether you can connect at all.
Security, privacy, provenance, reliability and capacity planning, ending with one specification worked end to end into a cost per million tokens.
The book ends by designing an AI platform for a county of 150,000 people and costing it properly: twenty accelerators, five servers, one rack, and roughly €233 per million tokens — about twenty-five times what the same tokens cost commercially.
That is not a rounding error, and the book does not hide it. The cause is the shape of the demand rather than the hardware: capacity is sized for a thousand concurrent users while the cost is divided by an average of twenty-five.
The case for building it rests on where the prompts go, not on price. People tell an assistant things they would never put in a form, and a council holding that in confidence is answering a different question from a commercial buyer. The book makes that argument only after refusing to stretch the cost one.
A finished draft was audited adversarially by a different model. It found errors the author's own checking had missed — including a throughput calculation that ignored cache reads, and a design whose tensor-parallel groups could not actually be built.
Those corrections were then audited too, and that pass found five more defects the fixing had introduced. All of it is printed in the back of the book rather than quietly applied, because a book arguing for measurement should show that measuring is hard.