Automation

AI

Test Automation

Mixture of Agents, Explained Simply: How Hermes Uses Multiple Models

5 min read

Most AI tools ask one model to answer your question. That is fast and cheap, but it also means one model's blind spot can become your blind spot. Mixture of Agents (MoA) changes the pattern: ask several models for independent advice, then let one model combine the strongest parts into a final answer.

The simple way to think about it: a single model is like asking one senior engineer. MoA is like asking a small review panel, then asking the best writer in the room to turn the discussion into one clear response.

🎯

In Plain English

MoA is not a new LLM. It is a way to coordinate several LLMs during one answer.
Reference models go first. They produce separate answers or critiques.
The aggregator goes last. It reads the references and writes the final response.
Hermes exposes MoA as a virtual provider. You can select a MoA preset much like selecting a model.

What the Research Says

The research foundation is the 2024 paper Mixture-of-Agents Enhances Large Language Model Capabilities by Wang et al., later listed on OpenReview as an ICLR 2025 Spotlight. The paper describes a layered setup where multiple LLM agents write responses, later agents read those responses as extra context, and a final aggregator combines them.

One headline result from the paper: an MoA system using only open-source models scored 65.1% on AlpacaEval 2.0, compared with 57.5% for GPT-4 Omni in the paper's reported comparison. That does not mean MoA wins every task. It means model collaboration can beat a single model on some instruction-following benchmarks.

Important Distinction

The academic MoA architecture and Hermes Agent's MoA implementation are related, but not identical. The paper studies a layered benchmark system. Hermes turns the pattern into a practical agent feature: reference models run in parallel, then an aggregator model writes the actual response.

How Hermes Uses MoA

In the Hermes Agent repository, MoA is implemented as a virtual provider. That means a named MoA preset can appear in the model picker like a model, even though it is really a small workflow behind the scenes. The aggregator is the acting model: it writes the assistant message and can emit tool calls.

Reference models are used as advisers. Hermes fans them out in parallel, collects their responses, and hands them to the aggregator. If the aggregator decides a tool is needed, Hermes continues through its normal agent loop. That is the important part: MoA changes who thinks before the answer, but it still lives inside Hermes.

Flow diagram showing a user prompt sent to multiple reference models, then to an aggregator model, then back into the Hermes agent loop.

Hermes MoA in one picture: reference models advise, the aggregator writes the final answer, and Hermes keeps the normal agent loop around it.

The Pieces

Piece	Plain-English Meaning
Reference model	A model that gives an independent answer, critique, or angle.
Aggregator model	The model that reads the reference answers and writes the final response.
Virtual provider	A provider entry that behaves like a model selection, even though it runs a multi-model process.
Preset	A saved recipe: which reference models to ask and which aggregator should combine them.
Tool call	An action the agent can request, such as searching, editing files, or running a command, depending on its enabled tools.

A Simple Example

Imagine you ask Hermes to review a risky database migration. With a single model, one model reads the task and answers. With MoA, Hermes can ask several models for separate reviews first. One model may focus on schema risk, another on rollback planning, another on test coverage. The aggregator then turns those notes into one practical plan.

Code

Single model:
  User prompt -> One model -> Final answer

Mixture of Agents:
  User prompt -> Reference model A
              -> Reference model B
              -> Reference model C
              -> Aggregator reads A+B+C
              -> Final answer or tool call

Why This Helps

Different models often make different mistakes. One may be better at code, another at long-form reasoning, another at spotting missing assumptions. MoA is useful when those differences matter. It is not magic, but it gives the final answer more raw material than one model would have produced alone.

For Hermes specifically, the interesting bit is that MoA is not isolated from the rest of the agent. It can sit beside the things Hermes already does: provider switching, tools, memory, skills, desktop/dashboard management, and gateway-style usage.

When MoA Is Worth It

Use a Single Model When	Use MoA When
The task is simple or repetitive.	The task is high-impact or ambiguous.
You need the fastest possible answer.	Correctness matters more than latency.
You are formatting, summarizing, or doing quick Q&A.	You are doing architecture review, security review, migration planning, or hard debugging.
The cost of a wrong answer is low.	A second or third opinion could prevent expensive rework.

The trade-off is simple: MoA costs more because it calls more models. If you use three reference models plus one aggregator, you are paying for several model calls instead of one. For small tasks, that is waste. For important decisions, it can be cheap insurance.

A Good Way to Explain It

MoA is not about replacing every single-model workflow. It is about using multiple models when the question deserves more than one point of view. Think of it like code review: you do not need a review panel to rename a file, but you probably want one before changing a production database, designing a new architecture, or trusting an agent with a risky task.

That is why Hermes makes MoA interesting. It brings the review-panel idea into an agent that can still use tools, remember context, switch providers, and continue its normal loop after the aggregator produces the final response.

Bottom Line

Hermes Agent's MoA mode is best understood as a built-in review panel for an AI agent. It asks multiple models for perspective, then lets an aggregator produce one answer inside the normal Hermes loop. That makes it useful for serious reasoning work, but unnecessary for every prompt.

The Practical Takeaway

Use one model when speed and cost matter most. Use MoA when you want a second opinion before the agent acts, especially for code review, production design, debugging, and security-sensitive work.

Sources

Hermes Agent GitHub repository - primary source for the current MoA virtual-provider implementation notes.
Hermes Agent provider documentation - official source for provider and model switching behavior.
Mixture-of-Agents paper on arXiv - primary paper record and benchmark summary.
ICLR 2025 OpenReview page - confirms the paper's ICLR 2025 Spotlight status and abstract.

About the Author

Dhiraj Das | Automation Consultant | 10+ years building automation systems that expose failures, reduce flakiness, and make complex workflows repeatable. He now applies that discipline independently to AI-agent validation, run replay, LLM testing, and postmortems.

Creator of many open-source tools solving what traditional automation can't: waitless (flaky tests), sb-stealth-wrapper (bot detection), selenium-teleport (state persistence), selenium-chatbot-test (AI chatbot testing), lumos-shadowdom (Shadow DOM), and visual-guard (visual regression).

Share this article: