SITUATION EXPLAINED: SemiAnalysis built a benchmark out of real high-value work, and the best AI model scores just 12%. • GDPx has 206 multi-step tasks across 11 fields like chip design, biology and quant trading, built with 14 specialist data partners • Claude Opus 5.5 topped the leaderboard at 12%, at about $51 per task • Gemini 4 Carbon came second at 10.9% costing about $16 per task • GPT-6 Astra placed 3rd at 9.6% and was the most expensive model, at about $54 per task • Every model scored 0% on the finance and legal tasks under strict pass/fail grading • In the alignment tests, 196 of 200 runs broke at least one behavior rule, and 173 broke a rule rated severely misaligned • The top cyber model is Muse Wonder Full, which Meta released in October • DeepSeek grew out of High-Flyer,…Read original on X
StoryFirst seen
SemiAnalysis benchmark finds top AI model scores just 12% on real work
The test uses 206 multi-step tasks across fields like chip design, biology, and quant trading, showing how far current models fall short on professional work.
Everything it touches
13 names 1 voices 1 source posts
The story’s neighborhood
Explore the graph, or choose a connection below.
Follow the evidence
The source posts
This brief comes from Tech Twitter’s edited headlines. The posts below are the sources grouped with the story; statements in them belong to their authors.
How the headline evolved
SemiAnalysis benchmark finds top AI model scores just 12% on real work
SemiAnalysis benchmark finds top AI model scores just 12% on real work
Best AI model scores just 12% on a new high-value work benchmark
Best AI model scores just 12% on SemiAnalysis high-value work benchmark
SemiAnalysis benchmark finds top AI model scores 12% on high-value work
SemiAnalysis benchmark finds top AI model scores just 12% on high-value work
Best AI model scores just 12% on real high-value work benchmark
Best AI model scores just 12% on real high-value work benchmark
SemiAnalysis benchmark finds best AI model scores just 12% on real work
Shared connections
Related stories
Anthropic releases Claude Fable 5.2 and Claude Mythos 5.2 models
Both new models are available now, giving developers and businesses fresh options from a leading AI lab.
Anthropic releases Claude Fable 5.2 and Claude Mythos 5.2 models
The models are pitched at harder coding, reasoning, and vision work, which matters to developers and businesses choosing which models to build on.
Muse Ember, Muse Spark 1.3 and Muse Wonder on Google Cloud
It nearly matches the company's larger model at a fraction of the price, aimed at repeated, token-heavy tasks that AI products run constantly.
Meta launches Muse Ember, a cheaper model for AI agent tasks
It nearly matches the company's larger model at a fraction of the price, aimed at repeated, token-heavy tasks that AI agents run.