Until recently AI benchmarks were exams: maths problems, trivia, coding puzzles. The models launched this autumn are being judged on whether they can do a job. Their scores tell any business owner where to automate and where to stay careful.
The new tests measure work
- AutomationBench (built by Zapier) asks a model to complete realistic business automations across connected apps.
- Remote Labor Index uses real freelance projects of the kind people are paid to do remotely.
- OSWorld checks whether a model can operate a computer: open apps, click, type and finish a task.
The scores
| Benchmark | Claude Fable 5.2 | Claude Opus 5.5 | Gemini 4 Argon |
|---|---|---|---|
| AutomationBench | 51.4% | 42.5% | 51.3% |
| Remote Labor Index | 41.3% | 21.3% | Not reported |
| OSWorld 2.1 (offline set, partial) | 89.3% | 87.2% | Not reported |
Figures as published by Anthropic (9 Nov 2026) and Google (30 Sep 2026). OpenAI says GPT-6 Astra is state of the art on AutomationBench but its announcement text does not give a number.
How to read them
- Operating a computer is nearly solved for well-defined tasks. Close to 90% on OSWorld means clicking through software is no longer the hard part.
- Finishing whole jobs is not. About half of realistic automations and about four in ten freelance projects are completed. The rest fail somewhere along the way.
- Progress is fast. Fable 5.2 roughly doubled Opus 5.5's Remote Labor Index score within weeks of Opus 5.5's release.
What to automate now
| Automate end-to-end | Automate with a human approval step | Keep human-led for now |
|---|---|---|
| Data entry from forms, invoices and emails | Customer replies and quotes | Pricing and credit decisions |
| Lead capture and first response | Collections reminders and follow-ups | Hiring and performance calls |
| Meeting notes and CRM updates | Report drafts and monthly reviews | Legal sign-off and filings |
| Appointment booking and reminders | Social posts and campaign copy | Anything a regulator could ask you to explain |
Design rules that work
- Give the agent one clear job, with a defined start and end, rather than a vague role.
- Let it act inside narrow permissions: read access everywhere, write access only where needed.
- Log every action so a person can check what happened and why.
- Add an approval step wherever a mistake costs money or a customer.
- Measure success on your own cases for two weeks before removing any human check.
For Indian examples and cost data, see The State of AI Automation for Indian SMEs (2026). Before hiring help, read nine questions to ask an AI automation vendor.
Sources
- Anthropic: Introducing Claude Fable 5.2 and Claude Mythos 5.2 (9 Nov 2026)
- Google: Gemini 4 Argon, our next era of frontier intelligence (30 Sep 2026)
- OpenAI Developer Community: Introducing GPT-6 Astra (3 Sep 2026)
Checked on 10 November 2026. Model names, prices and availability change quickly; confirm on the provider's page before you budget.
AI agents now finish about half of real office automations. What that means for your team