Show HN: I Turned DOOM into a Transformer

Doomformer compiles integer neural networks into ZDoom maps and runs them in ViZDoom. Voodoo dolls (the copies of the player that DOOM spawns for extra player-1 starts) do the arithmetic, and sector light levels and plane heights hold the sums. Python reads every exposed value back from the running game and compares it with a NumPy integer reference, tolerance 0. One value off by one and the run fails. It is slow: about 23,000 times slower than NumPy on a small MLP (engine time; 65,000 times with a timer around the whole call), and a handwritten digit takes 163 seconds of real-time DOOM in one map at the default schedule and 8 seconds at --opt 3.

Install · Proof · How it works · Films · Docs

Install and run

pip install git+https://github.com/mottopanikeiku/doomformer # Python 3.11+, Linux or macOS doomformer hello # 2 s: a small MLP, every value checked doomformer draw --digit 7 # 15 s: the marine walks a 7 and the level reads it

doomformer doctor checks ViZDoom, Freedoom and the display, builds ACC if it is absent, and compiles an ACS probe before reporting success. The first build needs git, cmake, a C compiler and network access to github.com. Linux uses dynamic runtimes: glibc-static is not required. On Fedora: sudo dnf install git cmake gcc gcc-c++ make. The Colab notebook needs no setup, but you draw on a browser canvas and doomformer mnist --image feeds it to DOOM, with no marine.

Proof: walk a digit, DOOM reads it

Holding FIRE, the marine inks a 3×3 brush of floor cells under him. READ makes the level write the 784 pixels into the network, and 1,355 voodoo dolls in the same map compute it. With --digit, Python stands in for you: it presses the marine's keys and USE once, then only advances the clock and reads.

▄▄▄▄▄▄▄▄▄▄▄▄▄▄ ▄████████████████ ████▀▀ ▀████▀ ▀▀▀ ▄███▀ ████ ███ ███ ███ ████ ███▀ ████ ▀███

DOOM reads: 7 (right) -- the winner's doll lit it on the 7-segment display EXACT: all 248 values DOOM computed [...] equal the NumPy integer reference (tolerance 0) READ pass: 273 game tics = 8 s of real-time DOOM at 35 tics/s, 3 s wall [...]

Abridged at [...]; docs/commands.md has the whole output. draw builds its map with --opt 3, so a READ takes 273 game tics; at --opt 0, the original schedule, it takes 5,705 (163 s of real-time DOOM). Without --digit the command opens a window and you walk and draw yourself; DOOM then runs in real time.

The dolls compute 784 pixels → 64 hidden units → 45 pairwise score comparisons → 10 win counts. Only the digit that wins all nine of its comparisons gets a doll through, and the line that doll crosses starts the level script that lights the digit on the display. Python reads back the 784 pixels, the hidden layer, every comparison and the answer and checks all 248 values (119 values, each read with its negated twin, plus 10 outputs).

EXACT means DOOM equals the integer network. It says nothing about whether the digit is right. On the first 100 MNIST test digits DOOM reads 99 correctly, and 966 of 1,000 sampled from all 10,000; every result is exact. The integer network scores 97.16 % on all 10,000 (the float model scores 97.58 %). The 30 walked digits (0 to 9, three scripted stroke variants each) all read correctly and exact at --opt 3, and the network is also trained on that stroke distribution: the first version of this demo, a 784-32-10 network trained on MNIST alone, reads only 41.08 % of such strokes. Through real X key events a drawn 7 read exact at the default --opt 3; a 7 and a 1 were tried at --opt 0 with an earlier network. Details · film

The museum map: light a digit, rewire one connection

doomformer museum build writes museum.wad, a map you can play without Python. You step on seven floor pads to light a digit and press READ. 136 voodoo dolls compute a small hand-written network: each digit's score counts the segments that agree with it, and the winner's doll lights the display. MUTATE rewires one connection of that network inside the running level: segment a to digit 7 goes from +1 to -1, by changing the specials of its two weight lines. The same 7 then reads as 1, and RESTORE puts it back. The HUD and a scoreboard show the ten scores after each READ, and a camera station shows digit 7's sixth score doll: it passes its gate before MUTATE and stops in front of it after.

$ doomformer museum verify 7 pads abc w_7a +1 DOOM reads 7 scores [4, 6, 3, 5, 4, 3, 2, 7, 3, 4] EXACT (148 values; 744 tics; [...]) 7 after MUTATE pads abc w_7a -1 DOOM reads 1 scores [4, 6, 3, 5, 4, 3, 2, 5, 3, 4] EXACT (148 values; 744 tics; [...]) 7 after RESTORE pads abc w_7a +1 DOOM reads 7 scores [4, 6, 3, 5, 4, 3, 2, 7, 3, 4] EXACT (148 values; 744 tics; [...])

Here Python plays: it presses the marine's keys and USE, and during each READ it only reads, with guarded readouts. It compares all 148 values with the NumPy reference of the network as it is wired at that moment. Without running DOOM, the static extractor also shows that the rewired map's netlist is the network with exactly that one weight changed. I have run the whole map only in ViZDoom. Its self-test map reads the digits 0 to 9, then the 7 after MUTATE and after RESTORE, without a player. On a GitHub runner it logs the exact values in GZDoom 4.14.2, LZDoom 3.88b, UZDoom 5.0.3 and ZDoom 2.8.1, and in Zandronum 3.2.1 only when every sector tag is below 32,768. Details

What is checked

Python reads sector light levels and floor and ceiling heights from the running game and compares each with an independent NumPy integer implementation of the same network. The check is integer equality over the runs that were made, and says nothing about inputs that were not run. The tests attack it: raising one weight line in a small compiled map from 5 to 6 must fail by exactly +1 on exactly the inputs whose dolls cross that line, and a readout that echoes a value Python wrote must fail (pytest tests/test_tamper.py, three tests, about 15 s).

Readback is guarded by default: before each page, Python poisons and checks its USER slots, then requires the page to replace them. Dropping a page cannot silently reuse a repeated input's answer or the previous page's values. guard=False is an explicit one-map historical opt-out. Guarded builds have new hashes (ACS and metadata, not geometry or arithmetic); the hash record explains the change.

doomformer fuzz --nets 40 compares every value of 40 random nets run in DOOM (306 passes, 0 mismatches, about 10 s with --workers 2; a planted compiler bug is found). fuzz --edge puts weights, biases and inputs at the range limits: 15,000 nets over ten fresh seeds, at --opt 0, at --opt 3 and as tile maps, made 113,605 passes and compared 1,987,632 values with no mismatch (the range check refused 47,686 of the inputs it drew, about 30 %), 3,000 more with the generator widened to plane starts near the bounds below (22,682 passes) and 2,000 more that also draw starts from 15,000 to 16,000 (14,782 passes), all with no mismatch. Earlier campaigns ran 2,000 random MLPs (15,374 passes, 276,594 values), 300 spiking nets and 100 random transformers over all 8 attention, norm and MLP variants (99 ran; one all-zero matrix is unsupported), after the scheduling fixes; 200 spiking --edge nets made 12,794 passes with no mismatch. fuzz --edge does not draw transformers. CI fuzzes 60 nets with a fresh, printed seed on every push and a longer campaign every Monday. The fuzzer and independent runs found four faults in this repo's own maps: two scheduling bugs and two plane limits. A plane move that starts more than 14,018 units from zero keeps its mover busy for three tics and drops the next request on that plane, and a plane that starts above 16,000 is exact on the first pass and wrong on later ones, so DOOM disagreed with the reference on nets that net.check had accepted. check and compile now refuse them: planes start inside [-15,000, 16,000] and move within ±14,018 (doomformer verify --plane-sweep, docs/verification.md ).

doomformer verify runs every input class of a small MLP (the classes agree with every input on 20 of 20 nets, under an assumption stated in the same page). df.coverage counts what is compared: on MNIST all 248 exposed values, which are 129 values the reference defines and 119 twins, and the count falls when a readout name is dropped. The map does not expose the 903 staircase counts and 1,351 gate comparisons of the same net; they are checked through the sums they feed. doomformer verify --extract net.wad --net net.npz reads the net back out of a compiled map without starting DOOM and compares every weight, bias and threshold with the source net: the shipped MNIST net (784-64-45-10-10) comes back equal from its map alone at --opt 0 to 3, and so do 484 of 490 random nets at --opt 0 (the other 6 are equal but break the plane contract) and 199 of 199 at each of --opt 1 to 3. EQUAL covers the arithmetic netlist, the plane contract, the trigger and repeat flags, the mover speeds, the reset script and the readout pages; timing and geometry are printed as NOT CHECKED. A changed weight line shows as exactly that weight. Of 23,622 single edits to a small map at --opt 0 the extractor detects 17,906, and the 98 undetected edits that were run in DOOM on 12 inputs were all exact. It shows what the map says; that DOOM runs the map as read is what the exactness checks test. A mutation harness changes one line of a compiled map: of the 1,004 sites run (of 5,578) 297 are killed, all weight, sign and target classes among them. Of the 707 survivors 590 match a rule (a redundant scroller, a freeze line a forward pass never needs) that aggregate controls support, and 117 are input gaps (walking floors that only some inputs expose); none is proven equivalent. doomformer lint --fuzz compared the static schedule checker with DOOM on 5,397 maps: 0 false negatives. The method is translation validation (Pnueli, Siegel and Singerman, 1998): a test of each compile, not a proof that the compiler is right.

Who does what:

- The dolls do every multiply-add, activation and comparison. ACS level scripts address cells, move input gates, rewind the dolls, sense the world and copy results out. None of the network's arithmetic runs in ACS. The toolkit's df.Agent is the one exception: it takes its argmax in ACS ( docs/agents.md ).