MKA tests whether AI agents can turn dense visual files into usable professional work

Google DeepMind and Artificial Analysis launched the Multimodal Knowledge Agent benchmark on November 10, testing work that requires more than extracting text from a document. MKA asks agents to interpret visually complex inputs and produce finished artifacts such as reports, spreadsheets and presentations. Google announcement
The evaluation contains 200 tasks spanning 83 occupations and ten sectors. Domain experts wrote the tasks and their grading rubrics. Google says the work would typically take an experienced professional three to eight hours, with rubrics weighted 75% toward factual accuracy and 25% toward visual design. Task design
The headline score is not a completed-job percentage
Artificial Analysis reports a leading Climb Weighted Satisfaction Rate of 57% for Claude Fable 5.2 at maximum effort, followed by Gemini 4 Carbon at high effort on 54% and Claude Opus 5.5 at maximum effort on 53%. These are the published results at launch. Artificial Analysis launch results
Climb criteria are the harder rubric requirements. Regression criteria check baseline competence. The headline metric averages the weighted share of Climb requirements satisfied across tasks; it does not mean an agent independently completed 57% of entire professional assignments. Scoring methodology
That distinction is consequential for buyers. A document can satisfy many individual requirements and still contain a single error that makes it unusable. Teams evaluating high-stakes work need to identify those critical failures separately from an aggregate score.
A common working environment improves comparability
Artificial Analysis runs the agents in its Stirrup harness with code execution, web tools and image viewing, subject to a 250-turn limit. An agentic judge inspects submitted deliverables against the rubric. The main 200-task set stays private, while Google has released ten separate public examples. Evaluation environment, Public and private sets
A common setup reduces one source of variation, but results still describe that setup. An organization with different document converters, permissions or review tools may see different outcomes. The public examples are useful for understanding the task format without treating them as a substitute for the private evaluation.
Cost and specialty can change the choice
Artificial Analysis reports average costs of $31.97 per task for Fable 5.2 and $6.29 for Carbon. It also finds that no model leads all ten sectors. The difference between a top aggregate score and the best fit for a particular department is therefore material. Cost and sector results
A practical internal trial should include the documents people actually struggle with: dense drawings, scanned pages, complicated tables and mismatched references. Grade the resulting files for correctness, completeness and usability, then add review time and repeat attempts to the cost.
MKA offers a more relevant comparison for visually demanding knowledge work. Its strongest use is to help narrow a shortlist and design a better local test, while keeping the final acceptance decision tied to the finished work rather than a single leaderboard rank.
Sources and citations
Revision history
Published .