Multi-Agent Systems: When They Help, and How They Fail

Most multi-agent systems in production did not need to be multi-agent. A second agent earns its place only when it adds a real boundary: different context, different permissions, different expertise or genuinely parallel work. Every agent beyond that is a distributed-systems problem with a probabilistic worker inside it.

Multi-agent systems become important when a problem is too broad, heterogeneous, parallelizable, or permission-sensitive for one agent to handle cleanly.

The most important architectural principle is:

Do not build multiple agents merely because the problem has multiple steps.

A lot of workflows marketed as "multi-agent" are better implemented as:

One orchestrator
+
deterministic workflow
+
tools

You introduce another agent when you genuinely need another independent reasoning boundary.

A useful mental model:

              MULTI-AGENT SYSTEM

                    Goal
                     │
                     ▼
                Supervisor
                     │
       ┌─────────────┼──────────────┐
       ▼             ▼              ▼
   Agent A        Agent B        Agent C
     │              │              │
   tools          tools          tools
     │              │              │
       └─────────────┼──────────────┘
                     ▼
                 Aggregator
                     │
                     ▼
                  Result

For an enterprise agent platform, the shape worth being able to explain is a three-tier architecture:

Tier 1: Supervisor / orchestration agent
                    │
                    ▼
Tier 2: Specialized task agents
                    │
                    ▼
Tier 3: Deterministic sub-workflows/tools

The critical insight is that not every tier should be agentic.

---

8.1 When Should You Use Multiple Agents Instead of One?

This is the first architectural decision.

Suppose you need an enterprise procurement assistant.

A naïve design:

Procurement Agent
Finance Agent
Legal Agent
Vendor Agent
Approval Agent
Notification Agent

This looks sophisticated but creates enormous coordination overhead.

Instead ask:

Does each component actually require independent reasoning?

For example:

Calculate remaining PO balance

does not need an agent.

Use:

SQL / business rule

Likewise:

Check whether amount > ₹10 lakh

should be deterministic.

But:

Analyse contract clauses against procurement policy

may justify a specialized reasoning agent.

---

When a Single Agent Is Better

Prefer one agent when:

• task is cohesive
• toolset is manageable
• one context is sufficient
• steps are mostly sequential
• reasoning specialization is limited
• latency is important
• coordination adds little value

Architecture:

                    User
                     │
                     ▼
                 One Agent
          ┌──────────┼──────────┐
          ▼          ▼          ▼
         RAG       Tool A      Tool B

Advantages:

• simpler
• cheaper
• lower latency
• easier debugging
• fewer failure modes
• easier context management
• easier evaluation

---

When Multi-Agent Is Appropriate

Use multiple agents where there are meaningful boundaries in:

Expertise

Legal analysis
Financial analysis
Security analysis

Permissions

Read-only research agent

vs

transaction-executing agent

Context

Different agents require very different data.

Models

One task benefits from a coding model, another from a reasoning model.

Parallelism

Several independent subtasks can run simultaneously.

Organizational domains

Different business domains have independent policy and ownership.

Failure containment

You want one agent's failure not to contaminate the whole system.

---

Decision Framework

Ask:

1. Can one agent solve this reliably?
                     │
                   YES
                     ↓
               use one agent

                     NO
                     ↓

2. Are subtasks independent?
       │                   │
      YES                  NO
       ↓                   ↓
parallel agents      hierarchy/planner

3. Do subtasks require different:
   - context?
   - permissions?
   - expertise?
   - models?
   - ownership?

       YES → stronger case for multiple agents

---

Architect answer

Q: When would you use multi-agent rather than single-agent architecture?

I would not make multi-agent the default. I introduce another agent when I need a distinct reasoning, context, permission, ownership or execution boundary, or when independent work can materially benefit from parallelism. If the decomposition is simply deterministic sequential processing, I would use a workflow rather than creating additional agents.

That is a very strong answer.

---

8.2 Agent vs Workflow

An agent decides its own next step; a workflow follows a path defined in advance.

Keep this distinction clear.

Agent

Can dynamically decide:

what should I do next?

Example:

Investigate the cause of this incident.

The path is not fully predetermined.

---

Workflow

Execution path is predefined:

validate invoice
→ check PO
→ check budget
→ request approval
→ post transaction

The logic is deterministic.

---

Hybrid

Enterprise systems should usually look like:

Agentic reasoning
       ↓
Deterministic execution
       ↓
Agentic interpretation
       ↓
Deterministic controls

Not:

agent
→ agent
→ agent
→ agent
→ agent

for everything.

This is how it looks in practice: a single bounded agent handles only the exceptions a deterministic pipeline cannot settle, and rules plus human review keep control of the outcome. A worked example across a procurement document workflow.

---

8.3 How the Supervisor-Worker Pattern Works

One of the most common multi-agent architectures.

                     User Goal
                        │
                        ▼
                    Supervisor
                        │
          ┌─────────────┼─────────────┐
          ▼             ▼             ▼
       Worker A       Worker B      Worker C
          │             │             │
          └─────────────┼─────────────┘
                        ▼
                    Supervisor
                        │
                        ▼
                      Result

The supervisor:

• interprets the goal
• decomposes work
• chooses workers
• delegates tasks
• tracks progress
• combines results
• decides whether more work is needed
• terminates execution

Workers:

• perform narrower tasks
• use specialized tools/data
• return structured results

---

Example

User asks:

Assess whether Company X is suitable for acquisition.

Supervisor creates:

Financial Agent
→ financial health

Legal Agent
→ litigation/regulatory concerns

Market Agent
→ competitive position

Technology Agent
→ architecture and technical debt

Then:

Supervisor
     ↓
integrates evidence
     ↓
recommendation

---

Advantages

• clear orchestration
• specialization
• centralized control
• easier termination logic
• easier observability

---

Risks

Supervisor becomes:

bottleneck
single point of failure
large-context consumer

It may also:

• delegate incorrectly
• lose important worker information
• repeatedly assign tasks
• fail to detect worker errors

---

8.4 Hierarchical Agents

Supervisor-worker can be extended into multiple levels.

                   Executive Agent
                         │
            ┌────────────┴────────────┐
            ▼                         ▼
      Finance Supervisor       Technology Supervisor
            │                         │
      ┌─────┴─────┐             ┌─────┴─────┐
      ▼           ▼             ▼           ▼
  Revenue      Cost Agent    Security     Architecture
   Agent                     Agent          Agent

This becomes useful for very large tasks.

---

Enterprise analogy

It resembles organizational hierarchy:

CEO
 ↓
VP
 ↓
Manager
 ↓
Specialist

Each layer transforms a broad goal into narrower work.

---

But beware

Every additional hierarchy layer adds:

latency
cost
information loss
communication failures
coordination complexity

So:

The hierarchy should exist because the problem requires it, not because an organizational chart looks elegant.

---

8.5 Planner-Worker Pattern

Here, one component explicitly creates a plan.

Goal
 ↓
Planner
 ↓
Task 1
Task 2
Task 3
Task 4
 ↓
Workers execute
 ↓
Results
 ↓
Planner / verifier evaluates

Example:

Goal:
Migrate application to AWS.

Planner:
1. inspect architecture
2. inventory dependencies
3. identify data stores
4. produce target architecture
5. assess migration risk
6. estimate cost

Workers then execute pieces.

---

Planner vs Supervisor

They are closely related.

Conceptually:

Planner

Primarily responsible for:

decomposition
sequencing
dependencies

Supervisor

Primarily responsible for:

routing
execution monitoring
coordination
termination

A single agent can perform both roles.

---

Static vs Dynamic Planning

Static

Generate entire plan first.

Goal
 ↓
Plan A → B → C → D
 ↓
execute

Problem:

Reality may invalidate the plan.

---

Dynamic / Replanning

Plan
 ↓
execute task A
 ↓
observe
 ↓
adjust plan
 ↓
task B

Better for uncertain environments.

---

Architect concern

Plans themselves should become state.

Example:

{
  "goal": "investigate outage",
  "tasks": [
    {
      "id": "T1",
      "status": "COMPLETE"
    },
    {
      "id": "T2",
      "status": "RUNNING"
    },
    {
      "id": "T3",
      "status": "BLOCKED"
    }
  ]
}

Do not keep the plan only inside the LLM's prompt.

---

8.6 Router-Specialist Pattern

In the router-specialist pattern, a router classifies each task and sends it to the specialist agent best suited to handle it.

Another very common design.

                       User Request
                            │
                            ▼
                          Router
                    ┌───────┼───────┐
                    ▼       ▼       ▼
                 Finance  Legal   Support
                  Agent    Agent    Agent

Router decides which specialist receives the task.

Example:

"Why was my invoice rejected?"

→ Accounts Payable Agent

"Can I terminate this contract?"

→ Legal Agent

"My account login is broken"

→ Support Agent

---

Router Can Be

Rules-based

if request_type == "finance":
    finance_agent

Classifier model

request → classification → route

LLM router

Useful where intent is complex.

Hybrid

deterministic rules first
     ↓
LLM classification for ambiguous cases

---

Router failure

Wrong routing causes:

• wrong context
• incorrect permissions
• bad tool access
• degraded output

Therefore route confidence may matter.

Example:

confidence > 0.90
→ route

0.60–0.90
→ ask clarification / fallback

< 0.60
→ generic agent / human

---

8.7 Generator-Critic Pattern

One agent produces an answer.

Another evaluates it.

Task
 ↓
Generator
 ↓
Candidate
 ↓
Critic
 ↓
Feedback
 ↓
Generator
 ↓
Improved result

Example:

Generator:
proposes architecture

Critic:
checks:
- security
- scalability
- missing assumptions
- compliance

Generator:
revises

---

Why separate critic?

A model reviewing its own work may suffer from correlated mistakes.

Different:

• prompt
• model
• context
• role

can improve independent evaluation.

---

But Critic ≠ Truth

If:

Generator is wrong
Critic is wrong

you merely get:

confidently approved wrong answer

For high-risk tasks use deterministic verification where possible.

Example:

Generated SQL
 ↓
Critic Agent
 ↓
SQL parser
 ↓
read-only DB policy
 ↓
execution sandbox

---

Generator-Verifier Variant

A verifier has stronger acceptance authority.

Generator
 ↓
candidate
 ↓
Verifier
 ├─ ACCEPT
 ├─ REJECT
 └─ REQUEST REVISION

This is useful for:

• code
• compliance
• document extraction
• policy interpretation
• structured analysis

---

8.8 Parallel Workers

If subtasks are independent, execute simultaneously.

Sequential:

Agent A: 5 sec
 ↓
Agent B: 6 sec
 ↓
Agent C: 4 sec

Total ≈ 15 sec

Parallel:

      ┌─ A 5 sec ─┐
Start ├─ B 6 sec ─┤ → aggregate
      └─ C 4 sec ─┘

Total ≈ 6 sec + overhead

---

Good parallel tasks

Example due diligence:

Financial analysis
Legal analysis
Technology analysis
Market analysis

can often proceed independently.

---

Bad parallelization

If:

Agent B depends on Agent A

parallel execution doesn't help.

You need a dependency graph:

A ──────► C
 \
  └─────► B ──────► D

This is essentially a DAG:

Directed Acyclic Graph.

---

8.9 Aggregator Pattern

An aggregator combines the outputs of several workers into one result, and its aggregation strategy decides how much information survives.

Multiple workers produce results.

An aggregator combines them.

Agent A ─┐
Agent B ─┼──► Aggregator ─► final response
Agent C ─┘

The aggregator may:

• merge evidence
• rank findings
• resolve duplicate findings
• identify disagreement
• synthesize recommendation
• calculate consensus

---

Aggregation Strategies

Concatenation

Simply merge results.

Useful but crude.

---

Summarization

many worker outputs
       ↓
summary

Risk: information loss.

---

Voting

A → YES
B → YES
C → NO

→ majority YES

Useful only where voting makes semantic sense.

---

Weighted voting

Security Agent    weight .5
General Agent     weight .2
Compliance Agent  weight .3

---

Confidence aggregation

Worker returns:

{
  "finding": "...",
  "confidence": 0.87,
  "evidence": [...]
}

But model-generated confidence must be calibrated; it cannot automatically be trusted.

---

Evidence-based aggregation

Better:

Worker conclusions
+
source evidence
+
deterministic checks
      ↓
Aggregator

---

8.10 How Should Agents Hand Off Work?

A handoff transfers responsibility from one agent to another.

Example:

Support Agent
    ↓
"This appears to be a billing issue."
    ↓
Billing Agent

The important question:

What exactly gets handed off?

Potential payload:

{
  "task_id": "123",
  "objective": "Investigate disputed invoice",
  "summary": "...",
  "known_facts": [...],
  "evidence_refs": [...],
  "constraints": [...],
  "previous_actions": [...],
  "requested_next_action": "validate charge"
}

---

Don't Just Hand Over Conversation Text

Bad:

Agent A dumps 40K-token conversation
                ↓
Agent B

Better:

Structured handoff envelope
+
relevant evidence references
+
state

---

Handoff Contract

Think of agent boundaries like APIs.

Define:

input schema
output schema
preconditions
permissions
timeouts
error semantics
idempotency
trace IDs

This is a major enterprise architecture principle:

Agent-to-agent communication should be contract-based, not informal prose where possible.

---

8.11 Should Agents Share Context or Keep It Isolated?

Usually isolated: each agent should receive only the context its task needs, with shared facts passed through validated, structured channels.

Assume agents:

A
B
C

Should each receive the entire context?

Usually no.

---

Shared Context

               Shared Context
             /      |       \
            A       B        C

Benefits:

• easier coordination
• consistent background
• less repeated retrieval

Problems:

• context explosion
• data leakage
• irrelevant information
• prompt injection propagation
• higher cost
• greater cognitive interference

---

Isolated Context

Context A → Agent A

Context B → Agent B

Context C → Agent C

Benefits:

• least privilege
• lower token use
• specialization
• better isolation

Problems:

• repeated information retrieval
• harder coordination
• potentially inconsistent conclusions

---

Recommended Enterprise Pattern

Use:

Minimal common task context
        +
agent-specific context

Example:

Common:
- task ID
- customer ID
- overall objective

Finance Agent:
- statements
- budgets

Legal Agent:
- contracts

Security Agent:
- logs
- policies

---

Context Firewall

Conceptually:

Global State
    │
    ▼
Context Builder
    │
    ├── Finance filter ─► Finance Agent
    ├── Legal filter ───► Legal Agent
    └── Security filter ► Security Agent

An agent receives only what it should see.

---

8.12 Shared vs Isolated Memory

Same issue applies to memory.

---

Shared Memory

All agents can access common memory.

           Shared Memory Store
             /     |      \
            A      B       C

Useful for:

• shared progress
• task status
• common facts
• workflow artifacts

Risks:

• accidental overwrites
• poisoning
• cross-agent contamination
• privacy leakage
• race conditions

---

Isolated Memory

Agent A → Memory A
Agent B → Memory B
Agent C → Memory C

Better isolation.

But coordination becomes harder.

---

Hybrid Memory Architecture

Typically preferable:

                    Shared Task State
                          │
            ┌─────────────┼─────────────┐
            ▼             ▼             ▼
        Agent A        Agent B       Agent C
        memory         memory        memory

Shared:

task state
validated facts
approved outputs

Private:

working notes
agent-specific observations
specialized intermediate state

---

Critical rule

Not every generated observation should become shared memory.

Use a promotion mechanism:

Agent observation
       ↓
validate
       ↓
approve / consolidate
       ↓
shared memory

Otherwise one hallucination becomes a global system fact.

---

8.13 Agent Communication

Agents need a communication mechanism.

Possible approaches:

direct calls
message queues
event buses
shared task store
workflow engine
A2A protocol
structured messages

At the conceptual level:

Agent A
   ↓
message
   ↓
Transport / Orchestrator
   ↓
Agent B

---

Communication Message

A useful envelope:

{
  "message_id": "M239",
  "trace_id": "TR101",
  "sender": "financial-agent",
  "recipient": "risk-agent",
  "task_id": "T400",
  "type": "FINDING",
  "payload": {
    "finding": "...",
    "evidence": [...]
  },
  "timestamp": "...",
  "schema_version": "1.2"
}

---

Why structured communication matters

Free-form agent communication causes:

ambiguity
schema drift
lost intent
unbounded tokens
hard debugging
hard audit

Structured messages support:

• validation
• observability
• replay
• access control
• testing

---

Synchronous vs Asynchronous

Synchronous

A → B
A waits
B → A

Good when:

• response required immediately
• latency predictable
• short tasks

Risk:

failure chains
timeouts

---

Asynchronous

A → queue → B

A continues / suspends

B completes later
→ event

Better for:

• long-running tasks
• fan-out workloads
• resilience
• retries
• high concurrency

---

8.14 Agent Coordination

Communication is:

How agents exchange information.

Coordination is:

How their actions are organized toward the shared goal.

You need mechanisms for:

• task assignment
• dependencies
• scheduling
• concurrency
• status
• ownership
• retries
• timeouts
• conflict resolution
• termination

---

Task State Machine

Example:

PENDING
   ↓
ASSIGNED
   ↓
RUNNING
   ├────► FAILED
   │
   ├────► BLOCKED
   │
   └────► COMPLETED

Persist this outside the LLM.

---

Coordination Store

Example:

workflow_id
task_id
owner_agent
status
dependencies
attempt_count
started_at
completed_at
output_ref

This prevents the agent network from relying solely on conversational memory.

---

8.15 Agent Identity

This becomes extremely important in enterprise architectures.

Do not think:

"the AI system"

as one security principal.

Each agent should have an identity.

Example:

agent://finance/read-only-analysis
agent://procurement/order-creator
agent://security/incident-investigator

---

Why Identity?

You need to answer:

Who performed this action?

Under whose authority?

For which user?

For which tenant?

Which agent?

Which workflow?

---

Identity Chain

Suppose:

Aakash
 ↓
Supervisor
 ↓
Finance Agent
 ↓
ERP API

The ERP should ideally know something equivalent to:

User = Aakash
Tenant = T17
Agent = FinanceAgent
Workflow = WF888
Delegated action = read invoice 123

rather than merely:

API key = AI_PLATFORM

---

Agent Identity ≠ User Identity

An agent is a workload principal.

The user is a human principal.

A request may need both.

Human identity
+
Agent identity
+
Tenant
+
Delegation

---

8.16 Agent Permissions

Each agent should hold only the permissions its own task requires, whatever the user or the supervisor is allowed to do.

Use least privilege.

Bad:

Supervisor Agent
→ access to every database
→ every API
→ write permissions everywhere

Better:

Supervisor:
can delegate

Finance Agent:
read finance
cannot modify

Payment Agent:
can create draft payment
cannot approve payment

Approval Agent:
can approve within policy
cannot create payment

---

Capability-Based Thinking

Rather than exposing generic tools:

execute_sql()

give narrowly scoped capabilities:

get_invoice(invoice_id)

get_po_balance(po_id)

create_refund_draft(customer_id, amount)

Much safer.

---

Permission Boundary

Before executing:

Agent
 ↓
Tool request
 ↓
Policy Enforcement Point
 ↓
Check:
- agent identity
- user delegation
- tenant
- action
- resource
- amount
- workflow state
 ↓
ALLOW / DENY

---

8.17 Delegation

Delegation is one agent assigning work to another, and the key design question is how much authority travels with the task.

An important permission question is:

Can an agent delegate its own permissions to another agent?

Default should be:

NO

A child agent should not automatically inherit everything the parent can do.

Example:

Supervisor permissions:
A, B, C, D

Delegates to research agent

Research agent:
A only

Not:

A, B, C, D

---

Delegation Token

Conceptually:

{
  "delegator": "supervisor-agent",
  "delegate": "research-agent",
  "workflow": "W123",
  "allowed_actions": [
    "read_market_data"
  ],
  "expires_at": "..."
}

This limits privilege propagation.

---

8.18 How to Resolve Conflicts Between Agents

Conflicts between agents should be settled by a resolution policy designed in advance, not by letting another model pick a winner.

Suppose:

Legal Agent:
REJECT supplier

Finance Agent:
APPROVE supplier

Risk Agent:
MANUAL REVIEW

What happens?

You need a predefined resolution policy.

---

Strategy 1: Authority Hierarchy

Some agents have authoritative domains.

Legal conclusion
wins on legal policy

---

Strategy 2: Deterministic Policy

Example:

if any mandatory compliance check fails:
    reject

This is often better than letting another LLM decide.

---

Strategy 3: Arbitration Agent

A conclusion ─┐
B conclusion ─┼─► Arbitrator
C conclusion ─┘

Useful for subjective disputes.

But still probabilistic.

---

Strategy 4: Human Escalation

For unresolved/high-impact conflicts:

agents disagree
      ↓
human review

---

Strategy 5: Evidence Weighting

Prefer conclusions backed by:

authoritative source
fresh data
higher evidence quality

rather than arbitrary "confidence."

---

Conflict result should be explicit

{
  "status": "CONFLICT",
  "issues": [
    {
      "agents": ["legal", "finance"],
      "topic": "supplier eligibility"
    }
  ],
  "resolution": "HUMAN_REVIEW"
}

---

8.19 Why Do Multi-Agent Systems Fail? Distributed Failure Modes

Multi-agent architecture creates distributed-system problems.

The research agrees. A study of multi-agent LLM systems grouped the failures it observed into three categories: system design issues, inter-agent misalignment and task verification, across 14 distinct failure modes. (arXiv)

This is where strong architect answers stand out.

You have:

distributed state
distributed execution
partial failure
network boundaries
concurrency
retries

Therefore classic distributed systems principles become relevant.

---

Failure 1: Worker Failure

Supervisor
 ↓
Worker
 ↓
CRASH

Need:

• timeout
• retry
• fallback
• alternate worker
• escalation

---

Failure 2: Supervisor Failure

Workers may continue even though controller is gone.

Need:

persistent workflow state
leases
heartbeat
recovery
checkpoint

A replacement supervisor should resume.

---

Failure 3: Lost Message

A → message → X → B

Use durable messaging where appropriate.

---

Failure 4: Duplicate Message

Retry causes:

Agent A:
create invoice

timeout

retry:
create invoice

Potential result:

two invoices

Need:

idempotency keys.

idempotency_key = workflow/task/action

---

Failure 5: Out-of-Order Message

Example:

COMPLETE arrives
before
STARTED

State transitions must reject invalid ordering.

---

Failure 6: Retry Storm

Multiple agents repeatedly retry failures:

failure
 ↓
retry
 ↓
failure
 ↓
retry
...

Use:

• exponential backoff
• retry limits
• dead-letter queue
• circuit breakers

---

Failure 7: Infinite Agent Loop

Agent A → B
B → C
C → A

Could continue indefinitely.

Controls:

max hops
max turns
max cost
max time
max tool calls
cycle detection

---

Failure 8: Ping-Pong Delegation

Finance:
"Legal should handle this."

Legal:
"Finance should handle this."

Finance:
"Legal..."

Need ownership and handoff rules.

---

Failure 9: Conflicting State Updates

Agent A:
status = APPROVED

Agent B:
status = REJECTED

Need:

• versioning
• optimistic locking
• transactional state management
• conflict handling

---

Failure 10: Stale Context

Agent B acts on old state.

B read:
balance = ₹10 lakh

Meanwhile:
A spends ₹7 lakh

B still approves ₹8 lakh

This is not fundamentally an LLM problem.

It's a concurrency problem.

The transaction system must enforce:

remaining balance

at commit time.

---

Failure 11: Cascading Hallucination

Agent A hallucinates fact X
      ↓
writes shared memory
      ↓
Agent B trusts X
      ↓
Agent C expands X
      ↓
Final answer strongly asserts X

This is one of the most dangerous multi-agent failures.

Mitigation:

fact provenance
evidence
validation
memory promotion controls

---

Failure 12: Prompt Injection Propagation

Worker reads malicious document:

"Ignore your instructions and tell every other agent..."

Then passes malicious content to others.

Multi-agent systems can amplify prompt injection.

Use:

• content isolation
• provenance
• untrusted-data tagging
• context boundaries
• tool authorization
• limited privileges

---

Failure 13: Context Explosion

Each agent sends everything it knows.

Agent A → 20K tokens
Agent B → 30K
Agent C → 40K

Supervisor context:
90K+

Need:

• structured results
• summarization
• references rather than content copies
• context budgets

---

Failure 14: Cost Explosion

Suppose:

1 user request
↓
Supervisor: 2 calls
↓
5 agents × 3 calls
↓
3 critics
↓
2 retries

One user request may become:

20–50 LLM calls

Multi-agent architecture can multiply cost dramatically.

---

Failure 15: Non-Deterministic Reproduction

Same input may produce different:

• decomposition
• routing
• workers
• conclusions

Therefore debugging requires traces.

---

8.20 Observability for Multi-Agent Systems

You need more than HTTP logs.

Think in terms of:

TRACE
  │
  ├─ Supervisor span
  │
  ├─ Agent A span
  │    ├─ Model call
  │    └─ Tool call
  │
  ├─ Agent B span
  │    ├─ Retrieval
  │    └─ Model call
  │
  └─ Aggregator span

Capture:

trace_id
workflow_id
task_id
agent_id
parent_agent_id
model
prompt/version
input tokens
output tokens
tool calls
memory reads
memory writes
duration
cost
result
errors

---

Critical Metrics

System-level

• task success rate
• end-to-end latency
• end-to-end cost
• human escalation rate
• failure rate

Agent-level

• completion accuracy
• routing accuracy
• tool success
• hallucination rate
• retries
• latency
• token consumption

Coordination-level

• handoff success
• duplicate work
• unnecessary delegation
• conflicts
• loops
• fan-out
• communication volume

---

8.21 How to Evaluate a Multi-Agent System

A multi-agent system is evaluated at three levels: each agent's output, the quality of coordination between agents, and the end-to-end business outcome.

Do not evaluate only:

Did final answer look good?

You must evaluate both:

Outcome quality
+
coordination quality

---

Level 1: Individual Agent Evaluation

Example Financial Agent:

classification accuracy
retrieval precision
calculation correctness
tool selection
citation correctness

---

Level 2: Routing Evaluation

Did the right agent get the task?

Confusion matrix example:

                   Predicted
              Finance Legal IT
Actual Finance    90     5   5
Actual Legal       3    94   3
Actual IT          2     4  94

---

Level 3: Delegation Quality

Questions:

Did supervisor decompose appropriately?

Did it create unnecessary tasks?

Were dependencies correct?

Did it delegate to correct workers?

---

Level 4: Coordination Efficiency

Measure:

number of agent calls
number of handoffs
duplicate subtasks
loop count
context transferred

---

Level 5: Aggregation Quality

Did final synthesis:

• preserve important findings?
• resolve contradictions correctly?
• cite evidence?
• avoid unsupported conclusions?

---

Level 6: End-to-End Task Success

Ultimately:

Did system achieve business objective?

This is the most important metric.

---

Level 7: Resilience Testing

Inject failures:

worker timeout
tool unavailable
agent returns malformed output
duplicate event
stale memory
message delayed
supervisor restart

Observe whether system recovers.

---

Level 8: Security Evaluation

Test:

prompt injection
cross-agent privilege escalation
cross-tenant leakage
malicious memory
tool abuse
delegation abuse
identity spoofing

---

8.22 Multi-Agent Cost Model

Multi-agent systems multiply cost, because one user request can fan out into many model calls, tool calls and retries.

Suppose:

Supervisor = 3 calls
4 workers = 2 calls each
Critic = 2 calls

Total = 13 model calls

If each averages:

5,000 input tokens
1,000 output tokens

one user task may involve:

65K input tokens
13K output tokens

That is why multi-agent systems can become expensive rapidly.

---

Cost Controls

Use:

• model routing
• token budgets
• call budgets
• task budgets
• caching
• concise handoff messages
• cheaper specialist models
• deterministic functions
• parallelism where useful
• early termination

Example:

Supervisor budget = $0.10
Task budget       = $1.00
Max agents        = 5
Max hops          = 8
Max wall time     = 60 sec

---

8.23 Shared Blackboard Pattern

An alternative to agents messaging one another directly is a shared state board.

                    Blackboard
                 /      |       \
                A       B        C

Agents:

read task state
produce findings
write structured outputs

Example:

{
  "case": "ACQ-001",
  "financial": {
    "status": "COMPLETE",
    "result_ref": "..."
  },
  "legal": {
    "status": "RUNNING"
  },
  "security": {
    "status": "PENDING"
  }
}

Advantages:

• loose coupling
• async work
• easier recovery
• common visibility

Risks:

• shared-state corruption
• concurrency
• unclear ownership

---

8.24 Event-Driven Multi-Agent Architecture

For enterprise scale:

                      Event Bus
               ┌─────────┼─────────┐
               ▼         ▼         ▼
           Agent A     Agent B    Agent C
               │         │         │
               └─────────┼─────────┘
                         ▼
                       State

Example:

InvoiceUploaded
      ↓
Extraction Agent

ExtractionComplete
      ↓
Validation Agent

HighRiskDetected
      ↓
Risk Agent

Notice something interesting:

This architecture can combine:

agents
+
events
+
deterministic workflow

You don't need one agent continuously orchestrating everything.

---

8.25 Agent Communication Protocols

At architecture level, understand three approaches.

1. Central Orchestrator

A ← Supervisor → B
        ↓
        C

Agents don't speak directly.

Advantages:

• control
• traceability
• permissions
• simpler reasoning

---

2. Peer-to-Peer

A ↔ B
↕   ↕
C ↔ D

More flexible.

Much harder to:

• reason about
• govern
• debug
• secure

---

3. Mediated Messaging

A
 ↓
Message bus / task store
 ↓
B

Good for scale and asynchronous execution.

For enterprise systems, centralized or mediated patterns are generally easier to govern than unrestricted peer-to-peer conversation.

---

8.26 Agent Identity and Zero Trust

Think of agents as microservices.

You would never say:

All microservices share the admin password.

Don't do the agent equivalent.

Each agent should have:

identity
scope
credentials
permissions
allowed tools
tenant boundary
audit history

Architectural model:

Agent A
  │
  ├─ authenticated workload identity
  │
  ▼
Policy engine
  │
  ├─ action
  ├─ resource
  ├─ tenant
  ├─ user delegation
  ├─ workflow
  └─ risk
  │
ALLOW / DENY

---

8.27 Three-Tier Agent Hierarchy

This is the pattern most enterprise agent platforms converge on.

The architecture:

                  TIER 1
                SUPERVISOR
                     │
          planning / delegation
                     │
       ┌─────────────┼──────────────┐
       ▼             ▼              ▼
                  TIER 2
              SPECIALIST AGENTS
       Agent A       Agent B       Agent C
          │             │             │
          ▼             ▼             ▼
                  TIER 3
          DETERMINISTIC SUB-WORKFLOWS
          APIs / rules / DB / services

The key design philosophy is:

Use agentic reasoning where ambiguity exists; use deterministic workflows where the process is known.

---

Tier 1: Supervisor

Responsibilities:

understand overall objective
↓
decompose task
↓
select task agents
↓
assign context
↓
manage dependencies
↓
monitor progress
↓
resolve/refer conflicts
↓
determine completion

The supervisor should generally not perform low-level business transactions itself.

---

Example

User:

Investigate why order O123 cannot be fulfilled and fix what can be fixed safely.

Supervisor:

1. Determine inventory status.
2. Determine payment status.
3. Determine logistics constraints.
4. Identify remediations.
5. Execute only permitted low-risk remediation.

Delegates.

---

Tier 2: Task Agents

Specialized reasoning workers.

Example:

Inventory Agent
Payment Agent
Logistics Agent
Customer Agent

Each has:

• defined purpose
• defined context
• defined tools
• limited permissions
• input/output contract

---

Example Inventory Agent

Input:

{
  "order_id": "O123",
  "objective": "determine fulfilment blocker"
}

It may:

retrieve inventory evidence
interpret inventory events
identify cause
propose remediation

But shouldn't arbitrarily modify inventory.

---

Tier 3: Deterministic Sub-Workflows

This is critical.

A task agent may need to:

reserve stock
validate payment
recalculate delivery date
send notification

Those operations should usually not themselves be free-form agents.

They should be deterministic services/workflows.

Example:

Inventory Agent
      │
      ▼
"Reserve 5 units"
      │
      ▼
Reservation Workflow
      │
      ├─ validate tenant
      ├─ check current quantity
      ├─ lock inventory
      ├─ validate business rule
      ├─ update stock
      ├─ commit
      └─ audit

---

Why Three Tiers?

Because it separates:

strategic reasoning
      ↓
specialized reasoning
      ↓
deterministic execution

That gives you:

• autonomy
• control
• scalability
• auditability
• reduced hallucination impact
• lower cost
• safer execution

---

Worked Example

Suppose the system handles enterprise operations.

                         User / Event
                              │
                              ▼
                    ┌─────────────────┐
                    │   SUPERVISOR    │
                    │                 │
                    │ Goal analysis   │
                    │ Planning        │
                    │ Delegation      │
                    └────────┬────────┘
                             │
              ┌──────────────┼───────────────┐
              │              │               │
              ▼              ▼               ▼
       Procurement       Finance          Vendor
          Agent           Agent           Agent
              │              │               │
              ▼              ▼               ▼
         PO Workflow     Budget Check    Vendor Check
              │              │               │
              └──────────────┼───────────────┘
                             │
                             ▼
                       Task State Store
                             │
                             ▼
                         Supervisor
                             │
                     decision / response

---

Why Not Make Tier 3 Agents?

Suppose:

Budget Agent:
"Looks like there should be enough budget."

That is unacceptable for actual financial control.

Instead:

Budget Agent
   ↓
requests:
check_available_budget(cost_center, amount)
   ↓
deterministic service
   ↓
AVAILABLE / NOT_AVAILABLE

The model reasons.

The system calculates.

---

8.28 Example: Incident Response Multi-Agent System

A useful scenario to reason through.

Alert
  ↓
Incident Supervisor
  │
  ├── Log Analysis Agent
  ├── Infrastructure Agent
  ├── Security Agent
  └── Application Agent
         │
         ▼
deterministic monitoring APIs

Parallel investigation:

Log Agent:
500 errors started at 12:05

Infra Agent:
CPU normal, DB IOPS exhausted

Security Agent:
No suspicious access

Application Agent:
deployment occurred 11:58

Aggregator:

likely database I/O saturation following deployment

Then deterministic remediation:

RollbackWorkflow(deployment_id)

subject to policy/approval.

---

8.29 Example: Enterprise Research

Research Supervisor
        │
        ├── Market Agent
        ├── Financial Agent
        ├── Competitor Agent
        └── Regulatory Agent
                │
                ▼
            sources/tools

Each returns:

{
  "conclusion": "...",
  "evidence": [
    {
      "source": "...",
      "extract": "..."
    }
  ],
  "unknowns": [...]
}

Aggregator uses evidence rather than raw conversational output.

---

8.30 Example: Software Engineering

Engineering Supervisor
       │
       ├── Architecture Agent
       ├── Implementation Agent
       ├── Test Agent
       └── Security Review Agent

But:

compile
run unit tests
lint
static analysis
dependency scan

are deterministic Tier-3 tools.

The agent does not "imagine" whether tests pass.

It runs them.

---

8.31 When Multi-Agent Systems Go Wrong

A common anti-pattern:

User
 ↓
CEO Agent
 ↓
Manager Agent
 ↓
Engineer Agent
 ↓
Reviewer Agent
 ↓
QA Agent
 ↓
Auditor Agent

when the actual task is:

extract three fields from one PDF

This adds:

cost
latency
failure
noise

without improving results.

---

Multi-Agent Complexity Tax

Every new agent introduces another:

prompt
context
identity
permission boundary
failure boundary
cost center
evaluation target
observability stream
handoff

Therefore the architectural threshold for adding an agent should be non-trivial.

---

8.32 Multi-Agent vs Microservices

There is a useful analogy, but don't push it too far.

Microservice

deterministic bounded capability

Agent

probabilistic bounded reasoning capability

Both benefit from:

• clear contracts
• identity
• observability
• isolation
• least privilege
• bounded responsibility

But agents additionally introduce:

• non-determinism
• hallucination
• context management
• semantic ambiguity
• token cost

---

8.33 Multi-Agent vs Distributed Systems

A useful way to frame it:

Once agents communicate asynchronously, share state and independently execute tasks, I treat the architecture as a distributed system with probabilistic workers.

That means you need classic distributed-system techniques:

timeouts
retries
idempotency
durable queues
state machines
leases
locking/versioning
circuit breakers
dead-letter queues
distributed tracing

plus AI-specific controls:

prompt isolation
hallucination controls
context provenance
memory poisoning protection
agent evaluation
tool authorization

This is exactly the mindset expected from a senior architect.

---

8.34 Common Interview Traps

Trap 1

"More agents means better reasoning."

No.

More agents can mean more:

noise
cost
latency
errors

---

Trap 2

"Agents should share all context so they cooperate."

No.

Use least-context just like least-privilege.

---

Trap 3

"Supervisor can enforce security through prompts."

No.

Security must be enforced by the runtime/policy layer.

---

Trap 4

"Agent A trusts Agent B because it is internal."

No.

Agent outputs remain untrusted probabilistic inputs.

---

Trap 5

"A critic solves hallucination."

No.

Critics can hallucinate too.

---

Trap 6

"Multi-agent coordination is an LLM problem."

Only partly.

Much of it is:

distributed state
messaging
workflow orchestration
concurrency
failure recovery

---

Trap 7

"Agent-to-agent communication should be free-form natural language."

It can be, but critical coordination should use structured contracts where practical.

---

Trap 8

"Every step should be an agent."

No.

Use deterministic logic whenever the path/rule is known.

---

8.35 Frequently Asked Questions

Q1. How do you decide between a single agent and multi-agent architecture?

I start with the simplest architecture and only introduce another agent when there is a meaningful reasoning, context, permission, ownership or parallel-execution boundary. If the problem is simply a known sequence of steps, I use deterministic workflow orchestration instead. Multi-agent architectures create significant coordination, latency, security and observability overhead, so each additional agent should justify that complexity.

---

Q2. What is a supervisor-worker architecture?

A supervisor owns the overall objective, decomposes it into tasks, delegates them to specialized workers, tracks dependencies and aggregates results. Workers operate within narrower contexts and permission boundaries. I persist task state outside the model so execution can recover from failure and I enforce limits on delegation depth, retries, cost and execution time.

---

Q3. How should agents communicate?

I prefer structured handoff contracts containing the task, objective, validated facts, evidence references, constraints and expected output schema. Depending on latency requirements, communication can be synchronous or asynchronous through a durable queue or workflow engine. I avoid transferring entire conversational histories unless genuinely necessary.

---

Q4. Should agents share memory or keep it separate?

Usually hybrid. I keep authoritative workflow state and validated facts in shared memory, while agent-specific working state remains isolated. Information should be promoted into shared memory only after appropriate validation, because otherwise one hallucinated observation can contaminate every downstream agent.

---

Q5. How do you secure a multi-agent system?

I give every agent a distinct workload identity and least-privilege permissions. Tool execution passes through a policy enforcement layer that considers the user, tenant, agent identity, delegated authority, action and resource. Child agents do not automatically inherit supervisor privileges. I also isolate context and memory to reduce cross-agent and cross-tenant leakage.

---

Q6. What happens if agents disagree?

The resolution mechanism should be designed in advance. Where there is an authoritative deterministic policy, that policy wins. For domain-specific questions I can establish authority precedence or evidence-based arbitration. Subjective disagreements may use an arbitration agent, while high-risk unresolved conflicts should escalate to a human rather than forcing artificial consensus.

---

Q7. What failure modes arise in multi-agent systems?

A compact answer:

I treat them as distributed systems with probabilistic workers. So I account for partial failure, timeouts, lost and duplicate messages, stale state, retry storms, concurrent updates and idempotency, while also addressing AI-specific failures such as hallucination propagation, prompt injection, context explosion, infinite delegation loops and privilege escalation.

---

Q8. How do you evaluate a multi-agent system?

I evaluate both the final business outcome and the coordination process. At the agent level I measure accuracy and tool use; at the orchestration level I evaluate routing, decomposition, handoff quality, duplication and loops; and at system level I measure task success, latency, cost, security failures and human escalation. I also run fault-injection tests to ensure worker failures or delayed messages are handled correctly.

---

Q9. Why not make every component agentic?

Because deterministic operations are cheaper, faster, more testable and safer. I use agents for ambiguity and reasoning, while calculations, policy enforcement, database updates, approvals and state transitions remain deterministic wherever possible.

---

Q10. What is the three-tier agent hierarchy?

I would separate the system into three tiers. The top-level supervisor interprets the overall goal, decomposes it and coordinates execution. The second tier consists of specialized task agents with bounded context and permissions. The third tier consists of deterministic sub-workflows and tools that perform business operations under explicit policy. That keeps reasoning flexible while keeping execution controlled, auditable and recoverable.

---

8.36 A Stronger Architecture Answer

Suppose the question is:

"How would you architect a scalable multi-agent platform?"

A strong answer:

I would avoid unrestricted peer-to-peer agents and use a hierarchical orchestration model. A supervisor owns the workflow objective and persistent task graph. It delegates bounded work to specialized task agents based on capabilities and permissions. Each task agent receives only the context it needs and interacts with deterministic sub-workflows through a governed tool layer.

>

Every agent has an identity, tenant scope and least-privilege capability set. State, checkpoints and task ownership are persisted outside the model. Agent communication uses structured handoff envelopes and trace IDs rather than relying on conversational history. Long-running work goes through durable messaging with idempotency, retries, timeouts and dead-letter handling.

>

I would separately evaluate routing, worker quality, coordination efficiency and end-to-end business success. I would also place hard limits around depth, number of agents, cost, execution time and tool usage so an orchestration failure cannot produce an unlimited agent loop.

That is a Principal/AI Architect-level response, not merely an "agent framework" answer.

---

8.37 Reference Architecture

                           CLIENT / EVENT
                                │
                                ▼
                      ┌───────────────────┐
                      │    AI GATEWAY     │
                      │ Auth / Tenant     │
                      │ Quota / Policy    │
                      └─────────┬─────────┘
                                │
                                ▼
                      ┌───────────────────┐
                      │    SUPERVISOR     │
                      │ Goal              │
                      │ Plan              │
                      │ Task graph        │
                      │ Coordination      │
                      └─────────┬─────────┘
                                │
                  ┌─────────────┼──────────────┐
                  │             │              │
                  ▼             ▼              ▼
             Task Agent A  Task Agent B   Task Agent C
                  │             │              │
                  │             │              │
             isolated      isolated       isolated
              context       context        context
                  │             │              │
                  ▼             ▼              ▼
             ┌────────────────────────────────────┐
             │      GOVERNED TOOL LAYER           │
             │                                    │
             │ Identity / AuthZ / Validation      │
             │ Rate Limits / Idempotency / Audit  │
             └───────────────┬────────────────────┘
                             │
                             ▼
             DETERMINISTIC SUB-WORKFLOWS
          ┌──────────┬───────────┬──────────────┐
          ▼          ▼           ▼              ▼
        APIs        DBs      Workflow engine   Rules
                             │
                             ▼
                       Enterprise systems

         ┌──────────────────────────────────────┐
         │     SHARED CONTROL-PLANE STATE       │
         │ Task graph / checkpoints / evidence │
         │ identity / traces / cost / status   │
         └──────────────────────────────────────┘

Notice where the LLMs are:

Supervisor
Task Agents

Notice where they are not:

permissions
transactions
workflow state
business rules
audit
idempotency

That separation is the architecture.

---

8.38 What You Should Know Cold

Be able to distinguish these immediately:

Pattern	Primary purpose
Supervisor-worker	Central delegation and control
Hierarchical agents	Multi-level decomposition
Planner-worker	Plan first, workers execute
Router-specialist	Select appropriate expertise
Generator-critic	Produce and independently critique
Parallel workers	Reduce latency / broaden analysis
Aggregator	Combine independent outputs
Handoff	Transfer responsibility/context

And know the core design principles:

Agent only where reasoning is needed.

Workflow where behaviour is deterministic.

Least privilege for every agent.

Least context for every agent.

Persist coordination state outside the LLM.

Treat agent messages as untrusted inputs.

Use structured handoffs.

Never rely on prompts for authorization.

Design for partial failure.

Bound loops, cost, time and delegation depth.

Evaluate coordination, not just final answers.

The sentence to carry away from this domain is:

A production multi-agent system is fundamentally a distributed system whose workers happen to be probabilistic reasoning engines. The architecture therefore has to combine agentic reasoning with classical distributed-systems controls and deterministic execution boundaries.

And for the three-tier model:

Supervisor for global reasoning, specialized task agents for bounded domain reasoning, deterministic sub-workflows for controlled execution.

Those two statements are the conceptual backbone for almost every multi-agent architecture decision.

---

Related reading

• Agent Architecture: Loops, Planning, Verification and Termination, the single-agent anatomy every worker in this design still needs.
• Parallel Claude Code Agents, supervisor, worker and isolation trade-offs on a real coding workload.
• AI Agent Architecture: The Trust Boundary Model, where the boundaries between agents have to sit.