What happened
Sber released Cowork Bench, an open designed to test AI agents' capacity to perform complex office tasks. The suite includes 475 tasks spanning 12 business functions and 26 corporate systems, requiring agents to interact with spreadsheets, documents, email, and calendars. Initial testing of nine agent shells on a single open model showed completion rates ranging from 62% to 80%, with Sber's GigaCowork platform achieving a 75.2% success rate.
Sber introduced Cowork Bench, an open Russian-language specifically designed to assess the practical capabilities of AI agents in office environments. Unlike traditional benchmarks that evaluate the quality of a model's text generation, Cowork Bench measures whether an agent can successfully complete a defined task without human intervention. The benchmark comprises 475 distinct tasks organized into 12 business functions and integrated with 26 different corporate systems.
The tasks require agents to navigate a complex digital workspace, including working with spreadsheets, managing documents, handling email, scheduling via calendars, accessing data storage, and querying knowledge bases. Specific examples cited include budget analysis, where an agent must compare plan versus actual figures for four departments, calculate deviations, explain reasons, prepare reports and presentations, revise forecasts, and send emails. Another example involves organizing a business trip, requiring the agent to select employees, find tickets, calculate budgets, and add meetings to a calendar.
To establish a baseline, the developers of the GigaCowork platform tested nine different software shells for agents, including Goose, Claude Code, Codex CLI, OpenHands, and OpenCode. All tests were conducted using one open model with identical tasks and verification methods to ensure comparability. The results showed that the share of completed tasks varied significantly, ranging from 62% to 80% depending on the specific agent shell used.
Sber's own GigaCowork platform achieved a completion rate of 75.2% in these tests. This release follows the introduction of GigaPlatform AI Studio, which allows users to assemble AI agents from ready-made blocks without programming, and GigaAgent, which breaks down instructions into stages and works with calendars and documents. The is intended to validate the scenarios that these platforms are designed to handle.
Why it matters
This shifts the evaluation focus from static model responses to dynamic, end-to-end task completion in realistic business environments. By providing a standardized, open framework for measuring agent autonomy in office workflows, it offers a practical metric for enterprises assessing the reliability of AI agents for operational tasks. The results highlight current limitations in agent reliability, with even top performers failing nearly a quarter of complex tasks, which is critical for organizations planning to deploy autonomous agents in production settings.
The release of Cowork Bench is significant because it addresses a critical gap in AI evaluation: the ability of agents to execute multi-step, real-world business processes. While large language models are often tested on static questions, enterprise adoption depends on whether agents can reliably manage workflows involving multiple tools and systems. By focusing on task completion rather than answer quality, the provides a more relevant metric for operational AI deployment.
The open nature of the allows for independent verification and broader industry adoption. The fact that it is Russian-language may limit its immediate global applicability, but the underlying methodology for testing agent autonomy in office settings is universally relevant. The results, showing completion rates between 62% and 80%, indicate that while AI agents are becoming more capable, they are not yet fully reliable for unsupervised office work, which has direct implications for risk management and requirements in enterprise AI strategies.
This development also highlights the competitive landscape in agent frameworks. By comparing nine different shells on the same model, Sber provides a comparative analysis that helps developers and enterprises understand the impact of the agent framework itself on performance. This is distinct from model performance, suggesting that the choice of agent architecture is a critical factor in achieving high task completion rates.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
crm_get_transaction(id='4092').An agent must create a draft calendar event for Tuesday at 2 p.m. Which evidence would establish the requested result?
What to watch next
Monitor whether other major AI developers adopt or replicate the Cowork Bench methodology to validate their own agent capabilities. Watch for updates on the specific performance gaps identified in the 26 corporate systems, as these may drive targeted improvements in agent frameworks. Additionally, observe if Sber expands the to include multilingual tasks or integrates it into broader industry standards for enterprise AI evaluation.
Future updates to Cowork Bench may include additional business functions or systems, potentially expanding the scope beyond the initial 12 functions and 26 systems. Monitoring these expansions will provide a clearer picture of the evolving capabilities of AI agents in complex enterprise environments.
The adoption of this by other AI companies or research institutions will be a key indicator of its influence. If other major players begin using Cowork Bench or similar task-completion metrics, it could become a de facto standard for evaluating enterprise AI agents.
Specific improvements in the agent shells that performed lower in the initial tests may be driven by the insights gained from this . Watching for updates from developers of Goose, Claude Code, Codex CLI, OpenHands, and OpenCode to see if they address the specific failure modes identified in the 62-80% completion range will be important for understanding the trajectory of agent reliability.