StoryFirst seen

SemiAnalysis benchmark finds top AI model scores just 12% on real work

The test uses 206 multi-step tasks across fields like chip design, biology, and quant trading, showing how far current models fall short on professional work.

Archived Updated

Everything it touches

13 names 1 voices 1 source posts

The story’s neighborhood

NAMESVOICESPOSTS

Explore the graph, or choose a connection below.

Follow the evidence

The source posts

This brief comes from Tech Twitter’s edited headlines. The posts below are the sources grouped with the story; statements in them belong to their authors.

MTSLead source
SITUATION EXPLAINED: SemiAnalysis built a benchmark out of real high-value work, and the best AI model scores just 12%. • GDPx has 206 multi-step tasks across 11 fields like chip design, biology and quant trading, built with 14 specialist data partners • Claude Opus 5.5 topped the leaderboard at 12%, at about $51 per task • Gemini 4 Carbon came second at 10.9% costing about $16 per task • GPT-6 Astra placed 3rd at 9.6% and was the most expensive model, at about $54 per task • Every model scored 0% on the finance and legal tasks under strict pass/fail grading • In the alignment tests, 196 of 200 runs broke at least one behavior rule, and 173 broke a rule rated severely misaligned • The top cyber model is Muse Wonder Full, which Meta released in October • DeepSeek grew out of High-Flyer,…
Read original on X

How the headline evolved

  1. SemiAnalysis benchmark finds top AI model scores just 12% on real work

  2. SemiAnalysis benchmark finds top AI model scores just 12% on real work

  3. Best AI model scores just 12% on a new high-value work benchmark

  4. Best AI model scores just 12% on SemiAnalysis high-value work benchmark

  5. SemiAnalysis benchmark finds top AI model scores 12% on high-value work

  6. SemiAnalysis benchmark finds top AI model scores just 12% on high-value work

  7. Best AI model scores just 12% on real high-value work benchmark

  8. Best AI model scores just 12% on real high-value work benchmark

  9. SemiAnalysis benchmark finds best AI model scores just 12% on real work

Shared connections

Related stories