CONNECT WITH US
AI & Deeptech

AI & Deeptech

Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?

Hugging Face logo

Published on

Add as a preferred source on Google
Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?

There are nearly 200 agent harnesses to choose from. We gave three of them the same open-weight model and 90 AppWorld tasks, and changing the harness moved task completion from 30% to 81%. For enterprise use, CUGA adds what this benchmark does not test: rules that the harness enforces and deployment inside your own boundary.

Community lists such as awesome-agent-harness and best-of-Agent-Harnesses track nearly 200 of them, and they make one point clearly: the harness matters, not only the model. In one analysis that best-of-Agent-Harnesses cites, the same GLM-5.2 model went from 23% to 52% on SWE-bench Pro when only the harness changed.

So the right question is not "which harness is best?" but "which harness fits my workload, with my model?" Hermes from Nous Research is built for always-on personal assistants, and Deep Agents from LangChain for long, multi-step tasks such as research. Both are excellent at those jobs.

Enterprise work is a different job. Agents act on business systems, such as CRM and payments, and they must follow rules that hold whatever the model decides. CUGA (ConfigUrable Generalist Agent) is IBM's open-source harness built for that job, with open-weight or hosted models, inside your own environment, including sovereign clouds and air-gapped deployments. To measure what the harness adds, we kept the model the same and changed only the harness.

We ran CUGA, Hermes, and Deep Agents on AppWorld , a benchmark of multi-step tasks across 9 apps and 457 APIs, such as email, payments, and file storage. The apps are consumer apps, so AppWorld is a proxy: it tests a subset of the capabilities that enterprise automation also needs, such as logging in, reading API docs, paging through results, and changing state across several apps.


Source link
Related Topics:AI & Deeptech

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.