← heypcb.ai
heypcb · Benchmarksealed2026q4 v1 · 49 runs · 6 arms

Ten new boards.
One model built them all.

How far can an AI agent take a PCB from a written brief to a board that passes automated electrical, design-rule, sourcing and orderability checks? Six arms, ten new boards, one run per board, every run published; two arms ran only part of the set. With HeyPCB's design tools, Gemini 4 Argon in a plain loop passed all ten, at $1.32 a board. Inky 1.0 and 1.1 passed five and five.

Key takeaways

  • 10 of 10Gemini 4 Argon on its own finished every board. It ran alone in a plain loop with HeyPCB's design tools, at a median of $1.32 a board. With these tools a frontier model on its own can take a brief to a finished board.
  • 5 and 5 of 10Inky 1.0 and Inky 1.1 each passed five. GPT-6 Astra on its own passed none of its ten. Board for board, Inky 1.1 won five and lost none, a gap that holds on these ten boards.
  • 7 of 10Same model, with and without Inky. GPT-6 Astra on its own placed no parts on seven of its ten boards. Inside Inky 1.1's harness the same model built boards with a median of 30 parts and 288 tracks, observed on four boards.
  • 6 of 10Inky lost boards by stopping. Six of Inky 1.0 and Inky 1.1's ten failed runs ended after the two allowed Continue presses, 6.6 min to 27 min in, well inside the 45-minute cap.

Which arms finished boards?

Each row is one board, each column one arm. A pass means the board passed all six checks inside the caps. A cap is a run stopped at 45 minutes or billed past $30; it counts as a fail even when its board passed. "Not run" is a board the arm never reached.
Each board’s outcome for each arm, with the run’s cost
BoardInky 1.1Inky 1.0Inky 1.1 harness · GPT-6 AstraGPT-6 Astra loopOpus 5.5 loopGemini 4 Argon loop
s01Cap$13.9Cap$8.00Cap$37.5Fail$1.70Capboard passed$89.3Pass$1.60
s02Pass$4.88Pass$3.83Fail$16.4Fail$1.09Pass$19.8Pass$1.05
s03Pass$9.56Pass$3.33Pass$23.9Fail$1.08Pass$22.9Pass$1.00
s04Cap$8.41Fail$5.20Capboard passed$41.9Fail$1.37Pass$25.1Pass$1.84
s05Fail$6.19Fail$5.79Not runFail$1.26Cap$115Pass$1.53
s06Pass$1.25Fail$4.98Not runFail$1.03Not runPass$1.12
s07Fail$9.74Pass$7.17Not runFail$0.96Not runPass$1.98
s08Pass$3.92Pass$3.16Not runFail$1.35Not runPass$1.11
s09Pass$2.55Pass$1.85Not runFail$0.72Not runPass$1.28
s10Fail$1.46Fail$2.12Not runFail$1.65Not runPass$1.36
Fig. 01 · Outcome and cost, board by board

Opus 5.5 loop and Inky 1.1 harness · GPT-6 Astra stopped at the spending stop and were not finished. Their results cover only the boards they ran, and the page never scales them up. The method says why.

How close did the failures get?

Every run is scored on the same six checks. A run that passes five of them came close; a run that passes one built almost nothing. Read each strip left to right: the taller the bars on the right, the closer that arm got.
  • Inky 1.1
    4.9 of 6 per run
    Failed runs: 3.8 of 6, most often missing orderable and routed.
  • Inky 1.0
    5.1 of 6 per run
    Failed runs: 4.2 of 6, most often missing orderable and routed.
  • Inky 1.1 harness · GPT-6 Astra
    5.3 of 6 per run
    Failed runs: 5.0 of 6, most often missing wired and routed.
  • GPT-6 Astra loop
    1.5 of 6 per run
    Failed runs: 1.5 of 6, most often missing routed and orderable.
  • Opus 5.5 loop
    5.6 of 6 per run
    Failed runs: 5.0 of 6, most often missing checks clean and orderable.
  • Gemini 4 Argon loop
    6.0 of 6 per run
    No failed runs.
Fig. 02 · Runs by how many of the six checks they passed

The same runs, one check at a time. Each bar is the share of an arm’s runs that passed that check. A board can be routed and still fail sourcing, so the checks are not steps in order.

Inky 1.1

All six: 5 of 10 runs · 50%

Wired
100%
Routed
60%
Checks clean
100%
Orderable
50%
Sourced
100%
Matches the brief
80%

Inky 1.0

All six: 5 of 10 runs · 50%

Wired
100%
Routed
80%
Checks clean
80%
Orderable
60%
Sourced
100%
Matches the brief
90%

Inky 1.1 harness · GPT-6 Astra

All six: 1 of 4 runs · 25%

Wired
75%
Routed
75%
Checks clean
100%
Orderable
75%
Sourced
100%
Matches the brief
100%

GPT-6 Astra loop

All six: 0 of 10 runs · 0%

Wired
30%
Routed
0%
Checks clean
90%
Orderable
0%
Sourced
20%
Matches the brief
10%

Opus 5.5 loop

All six: 3 of 5 runs · 60%

Wired
100%
Routed
100%
Checks clean
80%
Orderable
80%
Sourced
100%
Matches the brief
100%

Gemini 4 Argon loop

All six: 10 of 10 runs · 100%

Wired
100%
Routed
100%
Checks clean
100%
Orderable
100%
Sourced
100%
Matches the brief
100%
Fig. 03 · Share of runs that passed each check

What each arm built on the hardest board.

Board s01 was the one the fewest arms passed. This is the brief every arm got, word for word, and what each one left on the board when its run ended. A dash is a count that could not be read.
A minimal RP2040 board: the RP2040 with a W25Q16 QSPI flash, a 12 MHz crystal, USB-C for power and programming, an AP2112K 3.3V regulator, a BOOTSEL button and a reset button, and 0.1 inch headers for all the GPIO.
What each arm built on board s01
ArmOutcomeParts placedTracksLeft unroutedMinutesCost
Inky 1.1Cap333375548 min$13.9
Inky 1.0Cap353265045 min$8.00
Inky 1.1 harness · GPT-6 AstraCap373275338 min$37.5
GPT-6 Astra loopFail3054.2 min$1.70
Opus 5.5 loopCapboard passed34577029 min$89.3
Gemini 4 Argon loopPass39592018 min$1.60
Fig. 04 · Board s01, every arm

What did it cost?

What one board cost each arm and how long it took, as the median of its runs: half cost less, half more.
InkyModel alone, plain loopRan fewer than 10 boardsBest value for money
Fig. 05 · Boards passed against median cost per board

Cost is price × tokens. A model that stops early is cheap and builds nothing; providers that cache repeated context discount most of a long run.

Higher passed more boards; further left cost less. The dashed line joins the arms that no other arm beats on both at once (a Pareto frontier). Arms that ran different boards do not compare like for like.

Fig. 06 · Boards passed against median minutes per board

Every arm

Every arm’s pass rate, median cost and median time per board
ArmPassMedian $Median time
1Inky 1.15/10$5.5313 min
2Inky 1.05/10$4.407.4 min
3Inky 1.1 harness · GPT-6 Astra1/4$30.726 min
4GPT-6 Astra loop0/10$1.173.2 min
5Opus 5.5 loop3/5$25.18.4 min
6Gemini 4 Argon loop10/10$1.3214 min

Pass is boards passed of boards run. On a phone, each point carries its number from this table.

The medians behind both charts. Billed is what the provider charged. Estimated is tokens at list price, or Inky's own metered cost.

Median cost and time per board, by arm
ArmRunsMedian per boardCost isMedian timeCap hits
Inky 1.110$5.53estimated13 min2
Inky 1.010$4.40estimated7.4 min1
Inky 1.1 harness · GPT-6 Astra4$30.7estimated26 min2
GPT-6 Astra loop10$1.17estimated3.2 min0
Opus 5.5 loop5$25.1billed8.4 min2
Gemini 4 Argon loop10$1.32billed14 min0
Fig. 07 · Median cost and time per board

Inky 1.1 against each arm, on the same boards.

Each row compares Inky 1.1 with one arm on only the boards both ran. The dot is how many more boards in a hundred Inky 1.1 passed (left of centre: the other arm passed more). The bar is the range the true gap could sit in, given this few boards (a 95% bootstrap interval).

Right of centre: Inky 1.1 passed more

  • Inky 1.0
    ±0 pts (−30 to +30)
    Won 1, lost 1, tied 8 of 10. Observed only: with this few boards, the range includes no difference at all.
  • Inky 1.1 harness · GPT-6 Astra
    +25 pts (±0 to +75)
    Won 1, lost 0, tied 3 of 4. Observed only: with this few boards, the range includes no difference at all.
  • GPT-6 Astra loop
    +50 pts (+20 to +80)
    Won 5, lost 0, tied 5 of 10. The range stays clear of zero, so the gap holds on these ten boards.
  • Opus 5.5 loop
    −20 pts (−60 to ±0)
    Won 0, lost 1, tied 4 of 5. Observed only: with this few boards, the range includes no difference at all.
  • Gemini 4 Argon loop
    −50 pts (−80 to −20)
    Won 0, lost 5, tied 5 of 10. The range stays clear of zero, so the gap holds on these ten boards.
Fig. 08 · Inky 1.1 against each arm, in boards per hundred

How it was run.

The protocol was written before the first scored run. Every change after that is dated in it and applied to every arm alike.
What passes
A run passes only when all six checks pass. The scorer reads them from the board’s own files when the run ends. Nobody looks at a board and decides.
The boards
10 briefs written for this benchmark: seven short hobby briefs, typed the way a person types them, and three detailed ones with exact sizes, colours and lettering. Each was used for the first time in the scored run. Only their hashes were in the repository until the run ended; here are the words. Every arm got each brief byte for byte.
  1. s01A minimal RP2040 board: the RP2040 with a W25Q16 QSPI flash, a 12 MHz crystal, USB-C for power and programming, an AP2112K 3.3V regulator, a BOOTSEL button and a reset button, and 0.1 inch headers for all the GPIO.
  2. s02A USB to serial adapter with a CH340C, USB-C on one end and a 6 pin header on the other (GND, CTS, VCC, TX, RX, DTR), plus a jumper to choose 3.3V or 5V on VCC and TX/RX activity LEDs.
  3. s03An adjustable bench supply board around an LM317: 12 to 24V in on a screw terminal, a 5k pot to set the output from 1.25V up, input and output capacitors, the protection diodes, and an output screw terminal with a red LED showing it is on.
  4. s04A battery temperature logger: an STM32L031K6 microcontroller, an SHT40 temperature and humidity sensor on I2C, a microSD card slot on SPI for the log, a single-cell LiPo with an MCP73831 charger from USB-C, and a 3.3V LDO from the battery. Keep the USB 5V, the battery and the 3.3V rail as separate nets.
  5. s05A stepper driver board for one NEMA 17 motor: a DRV8825 with its current-sense resistors, a 12V barrel jack with a big bulk capacitor, a 4 pin motor header, a STEP/DIR/EN/GND header for a microcontroller, and an AMS1117 for 3.3V logic.
  6. s06A 4-channel relay board: four 5V relays driven by a ULN2003A from a 6 pin input header (5V, GND, IN1 to IN4), a red LED on each channel, and a 3 pin screw terminal per relay for COM, NO and NC.
  7. s07A two-player reaction timer game: an ATtiny85, one push button and one LED for each player, a piezo buzzer that sounds the start, and a CR2032 coin cell holder. 50 x 20 mm board.
  8. s08A breadboard power supply: a 5.5 mm barrel jack input, an on/off slide switch, an AMS1117-5.0 and an AMS1117-3.3 with their capacitors, a green power LED, and two 2-pin headers that plug into a breadboard's power rails, carrying 5V and 3.3V. 53 x 33 mm board. Keep 5V and 3.3V separate.
  9. s09A spinner badge: a round board 50 mm across, 2-layer, blue solder mask and white silkscreen, with in the middle the lettering: SPIN. Eight red 0805 LEDs arranged in a circle, each with its own 330 ohm 0805 resistor, driven by an ATtiny84 so the light chases around the ring. A tactile button changes the speed, and a CR2032 holder on the back powers it. Set the board outline first, then add every part and wire every net. Just build it with sensible defaults, no questions needed. Place and route the PCB, then finish the complete board with zero DRC and ERC errors, every part sourced, and fab-ready.
  10. s10A USB power meter: a 40 x 20 mm 2-layer board, black solder mask and white silkscreen, with along the bottom edge the lettering: AMPS. A USB-C receptacle in and a USB-A receptacle out, with VBUS passing through a 0.1 ohm 1% shunt resistor measured by an INA219 current monitor. A 4 pin header carries VCC, GND, SDA and SCL to a microcontroller, with 4.7k pull-ups on SDA and SCL, and the INA219 gets its own 100nF decoupling capacitor. Set the board outline first, then add every part and wire every net. Just build it with sensible defaults, no questions needed. Place and route the PCB, then finish the complete board with zero DRC and ERC errors, every part sourced, and fab-ready.
The arms
Inky 1.0 and Inky 1.1 as shipped on commit 327e60560. Each may be continued twice, as a person pressing Continue would, and no more. The plain loop gives a model one system sentence, a short orientation, the brief and HeyPCB's design tools, and passes every result back. It has no plan step, no completion ladder, no nudges and no retries. The tools that hand work to Inky are refused, and a board Inky touched would not count. One more arm runs Inky 1.1's harness pinned to GPT-6 Astra, beside the GPT-6 Astra loop, so the two differ in the harness alone.
One run each
One run per arm and board, to keep the run inside its budget. The intervals are wide because of it, and the page makes no claim about how consistent any arm is.
Caps
45 minutes, enforced live, and $30, judged on the final bill, the same for every arm. A run past either cap counts as a fail, even when its board passed.
One machine
Four runs at once on one Mac, on two local stacks, every arm under the same load. Minutes are as measured under that load.
No web
No arm had web search. Every arm had the same frozen parts catalogue and the same datasheet tools. The plain loop passes text only: an image a tool returns is named, not sent.
Cost
OpenRouter returns each call's cost, so those arms are billed. OpenAI calls return tokens, priced here at list price, and Inky's cost is the product's own metered cost: both estimated. The plain loop sent no prompt-cache markers. OpenAI's and Google's models cache a repeated prefix on their own; Anthropic's cache only what a request marks, so every step of the Opus 5.5 loop paid full price for its whole history. That inflates its cost, $272 over five runs, never its pass or fail.
The ledger stop
The run had a $620 spending stop, and reached it partway through. The four cheaper arms then finished every board under the same rules. The Opus 5.5 loop stays at the five boards it ran; the Inky 1.1 harness · GPT-6 Astra stays at the four boards it ran, and both are shown as partial: a hollow point, and "not run" in the grid.
Failures
Every attempt is kept and published. A run broken by the harness, the stack or a provider outage may be rerun once; the rerun is what counts, and both are in the data.
Not EEBench
The official EEBench leaderboard is a different benchmark, and its numbers do not compare with these.

Where this favours Inky.

We wrote the benchmark and we build one of the arms. This is where that tilts the result.
  1. 01HeyPCB wrote the briefs.
  2. 02HeyPCB's tools are native to Inky and new to the plain loop, which gets only the tools' own descriptions and a short orientation.
  3. 03The plain loop is bare on purpose, to measure the model on its own. A team could build a stronger agent around the same model.
  4. 04The analog practice set, not on this page, is one Inky was developed on.
  5. 05No arm has web search, though Inky as shipped can read datasheets from the web.
  6. 06The plain loop passes text only, while Inky can look at its renders.
  7. 07Costs are list prices on the day of the run.
  8. 08Everything ran on one Mac.

Where this is hard on Inky.

The same setup also tilts the other way.
  1. 01Six of Inky 1.0 and 1.1's ten failed runs ended unfinished after their two allowed Continue presses, 6.6 min to 27 min in, well inside the 45-minute cap. The plain loop never stops to ask; it runs until it is done or out of time.
  2. 02Nobody pressed Continue a third time, as a person watching the board might have.

Check the numbers.

Every attempt, every check and every cost on this page, in one file.
Download the data (JSON)
Protocol sha256
49c0e64588bc07103f6da5b15dc6097a2d1605e1764f93c3bde1b93064cc3b2d
Commit
327e605604af64bd71adf09045a561ab0f732669
Board set
sealed2026q4 v1
Set fingerprint
21b1adcd931685cf41dddb53d41854317384a19dcf5ef8785755fefcc4f04016
Environment
e1fe3ac8ef33
Schema
heypcb.benchmark.boards.v1

To try Inky on your own brief, start building.