Agent Orchestration: Frameworks, Durable Execution and Framework-Independent Design

An agent framework is a runtime choice, not an architecture. LangChain, LangGraph and AutoGen supply execution primitives such as state, checkpoints and interrupts; the architecture decides which parts of the system are allowed to know the framework exists. Agent orchestration is the layer that turns those primitives into something an enterprise can audit, resume and change.

At architect level, the question is:

How do you turn model calls, tools, state, policies, human approvals, retries and deterministic business logic into a controllable execution system?

The framework is an implementation choice. The orchestration architecture is the important thing.

A useful mental model is:

             ┌─────────────────────────────┐
             │      Agent Application       │
             └──────────────┬──────────────┘
                            │
                 ┌──────────▼──────────┐
                 │ Orchestration Layer │
                 │                     │
                 │ DAG / graph         │
                 │ routing             │
                 │ state transitions   │
                 │ retries             │
                 │ HITL                │
                 │ checkpoints         │
                 │ failure policy      │
                 └──────────┬──────────┘
                            │
          ┌─────────────────┼───────────────────┐
          │                 │                   │
     ┌────▼────┐       ┌────▼────┐        ┌────▼────┐
     │ Agents  │       │ Tools   │        │Services │
     └────┬────┘       └─────────┘        └─────────┘
          │
     ┌────▼─────┐
     │ LLMs     │
     └──────────┘

       State / Memory / Events / Audit
                  ↓
             Persistence

The strongest architecture is usually one where business orchestration belongs to your platform, while the agent framework remains replaceable.

---

9.1 Agent Framework vs Runtime vs Harness vs Workflow Engine: What Is the Difference?

These terms are frequently mixed together.

Agent framework

Provides abstractions for things such as:

models
prompts
tools
agents
messages
structured output
memory interfaces
tool invocation
agent loops

Example:

LangChain.

LangChain currently describes itself as the higher-level agent framework, while LangGraph provides the lower-level orchestration/runtime underneath it. (Docs by LangChain)

---

Agent runtime / orchestration runtime

Controls execution:

Which step runs next?
What state does it receive?
What happens if it fails?
Where is state persisted?
Can execution pause?
Can execution resume?
Can work happen concurrently?

Example:

LangGraph.

LangGraph explicitly focuses on stateful, long-running agents with persistence, durable execution and HITL. (GitHub)

---

Agent harness

An opinionated implementation around an agent.

For example:

planning
filesystem access
subagents
context management
tool handling
prompt conventions
termination

A harness may itself sit on an orchestration runtime.

A harness in production: the runtime layer that hands a bounded agent the workflow state, the unresolved exceptions and a restricted tool set inside a procurement document pipeline.

---

Workflow engine

A workflow engine is broader than an agent framework.

Examples conceptually:

Temporal
AWS Step Functions
Camunda
Airflow
Dagster
custom workflow runtime

Its primary concern is reliable process execution rather than LLM reasoning.

A production system may therefore use:

Temporal
   ↓
Business workflow
   ↓
Agent task
   ↓
LangGraph
   ↓
LLM + tools

instead of forcing the entire business process into an agent framework.

This distinction becomes important for long-running workflows lasting:

minutes
hours
days
weeks

---

9.2 What Is LangChain Good For?

LangChain historically became popular because it provided reusable abstractions around:

LLMs
prompts
tools
retrievers
document loaders
vector stores
agents
structured outputs
middleware

The modern architecture is simpler to understand as:

LangChain
    │
    │ high-level agent API
    ▼
LangGraph
    │
    │ execution/runtime
    ▼
Models / tools / storage

LangChain 1.x uses create_agent() as its standard agent construction mechanism, and those agents execute on LangGraph underneath. (Docs by LangChain)

---

Why LangChain exists

Without a framework, every provider has slightly different APIs:

openai.chat(...)
anthropic.messages(...)
gemini.generate(...)

Tools may also have different representations.

LangChain gives a common conceptual interface:

Model
Prompt
Message
Tool
Retriever
Agent
Middleware
StructuredOutput

So instead of application code directly depending on provider APIs:

Application
   │
   ├── OpenAI-specific code
   ├── Anthropic-specific code
   └── Gemini-specific code

you get:

Application
     │
 LangChain abstraction
     │
 ┌───┼────────┐
 │   │        │
OpenAI Claude Gemini

---

Where LangChain is useful

It is particularly valuable when you want:

Rapid model integration

Switch or experiment with providers.

Tool abstraction

Represent functions as callable tools.

Structured output

Map model responses into typed structures.

Retrieval integration

Connect:

vector stores
retrievers
rerankers
document stores

Standard agent loops

For example:

Think
  ↓
Choose tool
  ↓
Call tool
  ↓
Observe
  ↓
Repeat

LangChain describes an agent fundamentally as a model using tools iteratively until it reaches completion. (Docs by LangChain)

---

Where LangChain becomes dangerous architecturally

The danger is not LangChain itself.

The danger is letting application architecture become:

from langchain_everything import *

until your domain logic, persistence and orchestration become framework constructs.

For example:

BAD

Customer cancellation logic
        ↓
LangChain prompt
        ↓
LangChain agent
        ↓
LangChain tool
        ↓
LangChain memory
        ↓
LangChain callback

Now changing frameworks means rewriting the application.

Instead:

GOOD

Cancellation Service
        │
Orchestration interface
        │
Agent runtime adapter
        │
LangChain / LangGraph

Your core application should know:

execute_task()
call_model()
invoke_tool()
save_checkpoint()
request_approval()

rather than knowing framework internals everywhere.

---

9.3 What Is LangGraph?

LangGraph is far more important from an AI architect perspective.

Its fundamental idea is:

Agent execution can be represented as a stateful graph.

Instead of an uncontrolled loop:

LLM → tool → LLM → tool → LLM → ???

you create:

             ┌────────────┐
             │ Understand │
             └─────┬──────┘
                   │
             ┌─────▼─────┐
             │   Route   │
             └─────┬─────┘
                   │
           ┌───────┴────────┐
           │                │
      ┌────▼────┐      ┌────▼────┐
      │ Billing │      │ Support │
      └────┬────┘      └────┬────┘
           │                │
           └───────┬────────┘
                   ▼
               Validate
                   │
                   ▼
                 END

LangGraph's current documentation describes it as a low-level orchestration/runtime layer for long-running, stateful agents. (Docs by LangChain)

---

9.4 Graph State

The graph state is the data travelling through execution.

Example:

class AgentState(TypedDict):
    request_id: str
    tenant_id: str
    messages: list
    intent: str
    customer_id: str
    retrieved_data: dict
    risk_score: float
    tool_results: list
    approval_status: str
    final_response: str

Think of it as:

The current durable truth of this workflow execution.

Example lifecycle:

Initial state

{
  request: "Refund invoice 829",
  customer_id: null,
  amount: null
}

        ↓ identify invoice

{
  request: "...",
  customer_id: 182,
  amount: 47000
}

        ↓ policy check

{
  ...
  requires_approval: true
}

        ↓ approval

{
  ...
  approval_status: "approved"
}

        ↓ refund

{
  ...
  refund_id: "RF7822"
}

This is much stronger than storing everything only in conversation messages.

---

9.5 Nodes

A node is an executable step.

Conceptually:

Node = function(state) → state update

Example:

def validate_customer(state):
    customer = customer_service.get(state["customer_id"])

    return {
        "customer_status": customer.status
    }

A node may contain:

deterministic code
LLM call
agent
tool invocation
RAG retrieval
API call
validation
human approval
subgraph

That distinction matters.

Not every node should be an LLM.

A strong enterprise graph may look like:

[Validate input]         deterministic
       ↓
[Classify intent]        LLM
       ↓
[Fetch account]          deterministic API
       ↓
[Analyse account]        LLM
       ↓
[Risk policy]            deterministic rules
       ↓
[Human approval]         external
       ↓
[Execute transaction]    deterministic API

That is much safer than:

One giant autonomous agent
        ↓
"Figure everything out"

---

9.6 Edges

Edges define transitions.

A → B

means:

After A executes, B executes.

Example:

START
  ↓
parse_request
  ↓
retrieve_context
  ↓
generate_answer
  ↓
END

Edges make the workflow explicit.

This is valuable because the graph becomes:

inspectable
testable
observable
auditable
versionable

---

9.7 Conditional Edges

Conditional edges implement branching.

Example:

              risk_check
                  │
           ┌──────┴──────┐
           │             │
       risk < 50      risk >= 50
           │             │
           ▼             ▼
        execute      human_review
                         │
                         ▼
                       execute

Pseudocode:

def route_risk(state):
    if state["risk_score"] >= 50:
        return "approval"
    return "execute"

The important architecture point is:

Use deterministic routing whenever deterministic rules exist.

Don't ask an LLM:

"Should this ₹5 million payment require approval?"

if corporate policy already says:

> ₹1 million → CFO approval

Use:

if payment.amount > CFO_LIMIT:
    return "cfo_approval"

LLMs should resolve ambiguity.

Rules should enforce policy.

---

9.8 Why Does Checkpointing Matter for AI Agents?

Checkpointing means saving the execution state at defined points.

LangGraph checkpointers persist graph state as checkpoints associated with execution threads; these checkpoints underpin fault recovery, HITL and resumable execution. (Docs by LangChain)

Example:

START

   ↓

Node A
   ↓
CHECKPOINT 1

   ↓

Node B
   ↓
CHECKPOINT 2

   ↓

Node C
   ↓
CRASH

Without checkpointing:

START again

With checkpointing:

resume from CHECKPOINT 2

---

Why checkpointing matters for agents

Agents frequently depend on expensive or irreversible operations:

LLM call: ₹2
API query
LLM call: ₹4
generate document
send external request
human waits three hours
database mutation

You don't want:

server restart
     ↓
repeat everything

Even worse:

charge credit card
      ↓
crash
      ↓
restart
      ↓
charge credit card again

Therefore checkpointing must work together with:

idempotency
execution IDs
tool invocation IDs
deduplication
transaction boundaries

---

9.9 Persistence

Persistence is where checkpoints are stored, so that workflow state outlives the process that produced it.

Checkpointing answers:

What do we save?

Persistence answers:

Where and for how long do we save it?

Typical production architecture:

Agent Runtime
     │
     ├── Redis
     │      transient state
     │
     ├── PostgreSQL
     │      durable execution state
     │
     ├── Object Store
     │      large artifacts
     │
     └── Event Store
            audit / replay

Do not assume framework persistence solves enterprise persistence.

You still need decisions around:

tenant isolation
encryption
retention
deletion
PII
audit
backup
recovery
schema migration
state migration
execution versioning

---

9.10 Interrupt / Resume

An interrupt pauses a running workflow at a defined point, persists its state and waits for external input, typically a human decision, before resuming.

An interrupt means:

Execute
  ↓
reach decision point
  ↓
PAUSE
  ↓
persist state
  ↓
wait
  ↓
receive external signal
  ↓
RESUME

LangGraph interrupts are explicitly designed for this model: interrupt() pauses execution, persists state through the persistence layer and allows execution to continue once external input arrives. (Docs by LangChain)

Example:

Agent proposes:

Delete 742 IAM keys
        ↓
INTERRUPT
        ↓
Security administrator reviews
        ↓
Approve 721
Reject 21
        ↓
RESUME
        ↓
Execute approved actions

This is far superior to keeping a process/thread alive waiting for the human.

---

9.11 How Human-in-the-Loop (HITL) Works in Agent Workflows

HITL is an architectural control boundary.

Common forms:

Approval

Agent proposes action
      ↓
Human approve / reject

Editing

Agent drafts action
      ↓
Human modifies parameters
      ↓
Agent continues

Escalation

confidence < threshold
        ↓
human analyst

Exception handling

workflow fails
      ↓
human operator resolves
      ↓
resume

Policy intervention

risk engine flags execution
      ↓
security review

Because LangGraph HITL builds on persisted interrupts/checkpoints, an approval can survive browser refreshes, server restarts and long waits instead of requiring an in-memory session. (Docs by LangChain)

---

Critical HITL principle

Do not merely persist:

{
  "approved": true
}

Persist:

{
  "decision": "approved",
  "reviewer_id": "USR-812",
  "reviewer_role": "FinanceManager",
  "timestamp": "...",
  "execution_id": "...",
  "workflow_version": "2.4.1",
  "proposed_action_hash": "...",
  "comment": "...",
  "modified_parameters": {}
}

Otherwise you have human interaction, but not enterprise auditability.

---

9.12 What Is Durable Execution for AI Agents?

Durable execution means:

A workflow's logical execution survives process failure and can continue without losing already completed progress.

Example:

10:00 analyse request
10:02 call ERP
10:04 request approval
17:30 manager approves
17:31 workflow resumes
17:32 server dies
17:33 another worker resumes
17:34 transaction completes

To the business, that should appear as one logical execution.

LangGraph implements durability through checkpointed graph state and supports recovery from failures and long-lived interruptions. (Docs by LangChain)

But there is a crucial nuance.

Durability does not automatically make external side effects safe.

Suppose:

Node:
   transfer_money()
   checkpoint()

If this happens:

transfer_money()
       ↓
money moves
       ↓
CRASH before checkpoint

the runtime may retry the node.

Therefore:

Durable orchestration
        +
Idempotent side effects
        =
Reliable execution

Use:

idempotency_key = execution_id + node_id + action_id

with external systems wherever possible.

---

9.13 LangGraph Mental Model

The shape to hold in mind:

STATE
  |
  v
NODE
  |
  v
EDGE
  |
  +---- conditional routing
  |
  v
NODE
  |
  v
CHECKPOINT
  |
  +---- interrupt
  |
  +---- failure
  |
  +---- resume
  |
  v
END

The graph is effectively an agent-aware state machine / execution graph.

---

9.14 Is AutoGen Still Worth Using?

AutoGen became influential because it made multi-agent collaboration a first-class abstraction.

Instead of:

User
 ↓
Agent
 ↓
Tools

you could build:

              Supervisor
             /     |     \
            /      |      \
       Research   Coder   Reviewer
            \      |      /
             \     |     /
              Aggregator

AutoGen developed concepts around:

conversable agents
group chats
agent teams
event-driven agents
code execution
distributed runtimes
agent communication
multi-agent patterns

The later AutoGen architecture separated Core, AgentChat and Extensions; AutoGen's stable documentation also includes distributed agent-runtime capabilities such as gRPC worker runtimes. (Microsoft GitHub)

---

Important 2026 AutoGen update

It is worth knowing both the history and the current position:

AutoGen
   ↓
major Microsoft agent research/framework
   ↓
concepts carried forward
   ↓
Microsoft Agent Framework

As of August 2026, Microsoft's AutoGen repository states that AutoGen is in maintenance mode, with new users directed toward Microsoft Agent Framework. Microsoft describes Agent Framework as the direct successor combining ideas from AutoGen and Semantic Kernel with graph-oriented workflow orchestration. (GitHub)

So if the question is:

"Would you start a new enterprise system on AutoGen?"

A stronger answer today is:

"I'd understand AutoGen because many existing systems use it and its multi-agent abstractions shaped the ecosystem, but for a greenfield Microsoft-oriented implementation I'd evaluate Microsoft Agent Framework because Microsoft now identifies it as AutoGen's successor. More importantly, I would keep our domain orchestration behind framework-independent interfaces."

---

9.15 AutoGen vs LangGraph

The historical mental distinction is approximately:

LangGraph
   ↓
stateful graph execution

AutoGen
   ↓
agent-to-agent interaction

Not absolute, but useful.

LangGraph thinking

What is the workflow state?

What node executes next?

How does execution resume?

Where is the checkpoint?

Which edge should be taken?

AutoGen thinking

Which agent speaks?

Who receives the message?

Which agent should solve this subproblem?

How should agents collaborate?

When should the team terminate?

Today these areas increasingly overlap across frameworks.

---

9.16 Framework-Independent Agent Architecture

This is one of the most important items in this entire section.

An enterprise architecture should ideally look like:

┌────────────────────────────────────────────┐
│                Application                  │
├────────────────────────────────────────────┤
│          Domain / Business Services         │
├────────────────────────────────────────────┤
│          Agent Orchestration API            │
│                                             │
│ execute(workflow)                           │
│ resume(execution_id)                        │
│ checkpoint()                                │
│ invoke_agent()                              │
│ invoke_tool()                               │
│ request_approval()                          │
├────────────────────────────────────────────┤
│            Runtime Adapter                  │
├──────────────┬──────────────┬───────────────┤
│ LangGraph    │ MS Agent FW  │ Custom        │
└──────────────┴──────────────┴───────────────┘

Do the same for models:

ModelGateway.generate()

         ↓

OpenAI
Anthropic
Gemini
Bedrock
Azure OpenAI
local model

And tools:

ToolRegistry.invoke(tool_id, input)

              ↓

AWS plugin
SAP plugin
ServiceNow plugin
GitHub plugin
DB plugin

---

What should NOT be framework dependent

Ideally these belong to your platform:

agent identity
permissions
tenant boundary
tool contracts
workflow definitions
execution IDs
audit events
approval records
model policies
budgets
security controls
state schema
memory schema
observability
evaluation
failure taxonomy
business rules

The framework should execute the architecture.

It should not become the architecture.

---

9.17 Framework Selection Trade-Offs

Choose an agent framework against orchestration requirements, not popularity: determinism, statefulness, multi-agent complexity, human workflows, integration and operating model.

Do not choose a framework because:

"Everyone uses LangChain."

Evaluate architectural requirements.

Dimension 1: Determinism

Ask:

How much of the workflow is predetermined?

Highly deterministic:

A → B → C → approval → D

may fit a workflow engine or graph runtime very well.

Highly dynamic:

goal
 ↓
agent decides plan/tools/subagents

needs richer agent abstractions.

---

Dimension 2: Statefulness

Do you need:

one-shot execution

or:

multi-hour workflow
checkpoints
approval
recovery
resume

Long-running agent systems place much greater emphasis on durable state.

---

Dimension 3: Multi-agent complexity

Single agent + tools:

LangChain-style abstraction may suffice.

Complex multi-agent coordination:

supervisor
workers
handoffs
shared state
parallel agents
aggregation

requires richer orchestration.

---

Dimension 4: Human workflows

If approval is central:

pause
persist
audit
resume

must be first-class architecture concerns.

---

Dimension 5: Enterprise integration

Evaluate:

observability
OpenTelemetry
identity
RBAC
secrets
networking
model providers
MCP
deployment
storage
governance

---

Dimension 6: Operational model

Ask:

Who operates it?
How do we recover failures?
How do we replay?
How do we inspect execution?
How do we migrate running workflows?
How do we roll back?

Many frameworks look excellent at:

agent.run()

but enterprise architecture begins at:

agent.run() failed at 03:42 after executing 17 side effects.

---

9.18 How to Avoid Agent Framework Lock-In

Framework lock-in happens when business logic, state, tools or observability can only be expressed through one framework's objects.

There are several forms.

API lock-in

Your entire codebase imports framework types.

LangGraphState
LangGraphMessage
LangGraphTool

everywhere.

---

State lock-in

Persisted state uses framework-specific serialized objects.

Now millions of executions depend on them.

---

Tool lock-in

Business tools are defined only through framework decorators.

Example:

@some_framework_tool
def cancel_invoice(...):

Your domain capability has become dependent on orchestration tooling.

Better:

class CancelInvoice:
    def execute(input: CancelInvoiceInput):
        ...

Then:

LangChain adapter
MCP adapter
REST adapter
internal API adapter

wrap the same capability.

---

Workflow lock-in

The business process exists only as imperative framework code.

graph.add_node(...)
graph.add_conditional_edges(...)

with no external canonical representation.

This becomes particularly problematic if you have hundreds of workflows.

---

Observability lock-in

Your logs and tracing exist only in a framework-specific observability product.

Maintain your own:

execution_id
trace_id
workflow_id
workflow_version
agent_id
tool_id
tenant_id

so telemetry remains portable.

---

9.19 Anti-Corruption Layer for Agent Frameworks

A good enterprise pattern is:

Domain
  │
  │ framework-neutral
  ▼
Agent Runtime Port
  │
  ▼
LangGraph Adapter

For example:

class WorkflowRuntime:
    def start(self, workflow, input):
        ...

    def resume(self, execution_id, signal):
        ...

    def cancel(self, execution_id):
        ...

    def get_state(self, execution_id):
        ...

Your application talks to that interface.

The adapter talks to LangGraph.

That drastically reduces migration cost.

---

9.20 Declarative Orchestration

Imperative orchestration:

graph.add_node("analyse", analyse)
graph.add_node("review", review)
graph.add_edge("analyse", "review")

Declarative orchestration:

workflow:
  id: invoice_review

nodes:

  analyse:
    type: agent
    agent: invoice_analyser

  risk:
    type: service
    service: risk_engine

  approval:
    type: human_approval

edges:

  - from: analyse
    to: risk

  - from: risk
    to: approval
    when: risk_score > 70

The workflow becomes data rather than application code.

---

Why declarative orchestration matters

It enables:

version control
visualization
validation
policy enforcement
workflow catalogs
tenant customization
diffing
approval
rollback
static analysis
automated deployment

You can now ask:

Which workflows call the production-delete tool?

Which workflows permit GPT-X?

Which workflows lack human approval before financial actions?

Which workflows use deprecated plugin v2?

without analyzing arbitrary Python.

This becomes enormously valuable at enterprise scale.

---

9.21 YAML-Defined Workflows

For an enterprise agent platform, I would treat YAML as the declarative playbook definition, not as executable arbitrary code.

Example:

apiVersion: platform.example.com/v1
kind: Playbook

metadata:
  name: cloud-key-remediation
  version: 2.3.1

input:
  schema: CloudKeyRemediationInput@1

nodes:

  discover:
    type: agent
    agent: key-discovery-agent@2

  classify:
    type: agent
    agent: key-risk-agent@1

  approval:
    type: human
    policy: security-admin

  rotate:
    type: tool
    tool: aws.key.rotate@3

  verify:
    type: workflow
    workflow: key-verification@2

edges:

  - from: discover
    to: classify

  - from: classify
    to: approval
    when: risk >= HIGH

  - from: classify
    to: rotate
    when: risk < HIGH

  - from: approval
    to: rotate
    when: decision == APPROVED

  - from: rotate
    to: verify

failurePolicy:
  retry:
    attempts: 3

  onFailure: ESCALATE

The runtime compiles this into an executable graph.

YAML
 ↓
Parser
 ↓
Schema validation
 ↓
Policy validation
 ↓
DAG compiler
 ↓
Executable DAG
 ↓
Runtime

---

Do NOT put arbitrary Python in YAML

Bad:

condition: |
  import boto3
  if foo...

You've now turned YAML into an insecure programming language.

Prefer bounded expressions:

when:
  all:
    - field: risk_score
      operator: gte
      value: 80

    - field: environment
      operator: eq
      value: production

The runtime evaluates only permitted operators.

---

9.22 Versioned Execution DAGs

This is critical.

Suppose you deploy:

Playbook v1

and an execution starts.

Execution E100
Workflow v1

A → B → APPROVAL → C

While the human is waiting, you deploy:

Workflow v2

A → B → SECURITY_REVIEW → APPROVAL → C

Which version should E100 resume against?

Normally:

The exact version it started with.

Therefore execution must bind:

{
  "execution_id": "E100",
  "playbook": "cloud-key-remediation",
  "playbook_version": "1.7.2",
  "compiled_dag_hash": "sha256:..."
}

Do not simply resolve:

cloud-key-remediation → latest

during resume.

---

Immutable execution definition

A very strong architecture is:

Playbook source
      ↓
Compile
      ↓
Immutable DAG artifact
      ↓
SHA-256 digest
      ↓
Execution references digest

Example:

Execution
   |
   +-- workflow_id = KEY_ROTATION
   +-- version = 2.1.4
   +-- dag_hash = 9c728...

Now you can prove exactly what logic executed.

---

Why this matters

Without immutable DAG versions:

Monday:
Agent proposed action under policy A

Tuesday:
workflow changed

Wednesday:
Human approves

System resumes under policy B

That is an audit nightmare.

---

9.23 Input Contracts

Every workflow should expose a typed contract.

Bad:

{
  "data": "whatever"
}

Better:

{
  "tenant_id": "TEN-92",
  "cloud_account_id": "AWS-271",
  "environment": "production",
  "resource_ids": [
    "..."
  ],
  "requested_by": "USR-92"
}

Schema:

{
  "type": "object",
  "required": [
    "tenant_id",
    "cloud_account_id"
  ]
}

At the architecture level:

Caller
  ↓
API validation
  ↓
Playbook input contract
  ↓
Authorization
  ↓
DAG execution

---

Why contracts are essential for agents

LLMs naturally encourage loose interfaces:

"Pass the model some context."

Enterprise systems require:

typed
validated
versioned
authorized
traceable

interfaces.

Apply contracts to:

workflow input
node input
node output
tool input
tool output
agent structured output
event payload
human approval response

---

9.24 Contracts Between Nodes

Example:

Discovery Agent
      │
      │ KeyInventory@v2
      ▼
Risk Agent
      │
      │ RiskAssessment@v3
      ▼
Approval

The agents should not exchange arbitrary prose if machines need to act on the result.

Bad:

"These keys seem risky and probably should be rotated."

Good:

{
  "resource_id": "AKIA...",
  "risk": "HIGH",
  "risk_score": 87,
  "reasons": [
    "age_gt_180_days",
    "owner_missing"
  ],
  "recommended_action": "ROTATE"
}

Natural language can accompany the structure.

It should not replace it.

---

9.25 Failure Policies

Every node should have a defined failure strategy.

Possible policies:

RETRY
SKIP
FALLBACK
ESCALATE
PAUSE
COMPENSATE
ABORT
DEAD_LETTER

Example:

failurePolicy:

  timeout: 30s

  retry:
    attempts: 3
    backoff: exponential
    initialDelay: 2s

  retryOn:
    - RATE_LIMIT
    - TIMEOUT
    - TEMPORARY_UNAVAILABLE

  doNotRetryOn:
    - AUTHORIZATION_DENIED
    - INVALID_INPUT

  onExhausted:
    action: ESCALATE

---

Failure classification matters

Do not treat:

everything = Exception

Define categories:

TRANSIENT

RATE_LIMIT
TIMEOUT
NETWORK_FAILURE
PROVIDER_UNAVAILABLE

usually retryable.

PERMANENT

INVALID_INPUT
RESOURCE_NOT_FOUND
UNSUPPORTED_OPERATION

usually not retryable.

SECURITY

AUTHENTICATION_FAILURE
AUTHORIZATION_DENIED
POLICY_DENIED

should usually stop/escalate rather than retry blindly.

AGENTIC

INVALID_STRUCTURE
LOW_CONFIDENCE
HALLUCINATED_RESOURCE
VERIFICATION_FAILURE

may require regeneration, verifier or human intervention.

---

9.26 Compensation

Distributed agent workflows rarely have true ACID transactions.

Example:

Create user
   ↓
Allocate licence
   ↓
Create mailbox
   ↓
Add CRM account
   ↓
FAIL

You cannot rollback this with:

ROLLBACK;

Instead use compensating actions:

CRM creation fails
       ↓
remove mailbox
       ↓
release licence
       ↓
disable user

This is the Saga pattern applied to agent/workflow orchestration.

An agent framework does not remove distributed-systems architecture.

---

9.27 Semantic Versioning of Playbooks

Treat agent playbooks as software artifacts.

Example:

2.4.7

MAJOR.MINOR.PATCH

PATCH

No contract or behavioural compatibility break.

Example:

prompt improvement
logging improvement
timeout tuning

2.4.6 → 2.4.7

---

MINOR

Backward-compatible capability addition.

Example:

add optional verification stage
support new cloud provider
add optional output field

2.4.7 → 2.5.0

---

MAJOR

Breaking behaviour or contract.

Example:

input schema changed
approval semantics changed
tool contract changed
state format incompatible
workflow outcome meaning changed

2.5.0 → 3.0.0

---

But prompts make semantic versioning tricky

Traditional software:

same input
+
same binary
=
mostly deterministic behaviour

LLM execution:

same input
+
same prompt
+
same model
≠
guaranteed same response

Therefore the platform's version identity should capture more than YAML.

For example:

playbook version
agent version
prompt version
model policy version
tool version
schema version
plugin version

Execution metadata:

{
  "playbook": "2.4.1",
  "agent": "key-risk@3.2.0",
  "prompt": "risk-analysis@7",
  "model_policy": "security-high-reasoning@2",
  "tool": "aws-key-manager@4.1",
  "input_schema": "KeyInput@3"
}

That gives much stronger reproducibility.

---

9.28 Plugin-Isolated Domains

This is particularly important for an enterprise agent platform.

Do not create one giant runtime containing:

AWS tools
HR tools
Finance tools
CRM tools
database tools
email tools
ServiceNow tools
Kubernetes tools

with every agent able to discover everything.

Instead:

                    PLATFORM Runtime
                          │
          ┌───────────────┼───────────────┐
          │               │               │
      AWS Plugin      Finance Plugin    HR Plugin
          │               │               │
      AWS APIs          ERP APIs          HRMS

Each plugin domain defines:

tools
credentials
schemas
permissions
policies
rate limits
network access
audit rules

---

Why plugin isolation matters

Suppose an HR agent is compromised by prompt injection.

Without isolation:

HR Agent
  ↓
tool registry
  ↓
AWSTerminateInstance

That should never even be possible.

With plugin isolation:

HR Agent
  ↓
HR plugin boundary
  ↓
allowed:
   search_employee
   create_leave_request

not visible:
   rotate_aws_key
   execute_sql
   refund_payment

Security is stronger when dangerous capabilities are absent, not merely accompanied by:

"Please don't use this tool."

---

9.29 Plugin Architecture

Conceptually:

Plugin Manifest
      │
      ├── identity
      ├── version
      ├── tools
      ├── schemas
      ├── permissions
      ├── secrets
      ├── network policy
      └── runtime policy

Example:

plugin:
  id: aws-key-management
  version: 3.2.1

permissions:
  required:
    - iam:ListAccessKeys
    - iam:GetAccessKeyLastUsed

tools:
  - discover_keys
  - inspect_key
  - rotate_key

network:
  outbound:
    - iam.amazonaws.com

secrets:
  - aws-role

isolation:
  runtime: container

The platform can then reason about:

Agent
 ↓
Role
 ↓
Allowed plugin
 ↓
Allowed tool
 ↓
Allowed resource
 ↓
Policy

rather than:

Agent has access to all registered Python functions.

---

9.30 Declarative Playbook Compilation

The complete design I would use is:

                   PLAYBOOK REPOSITORY

                   workflow.yaml
                        │
                        ▼
                Schema Validator
                        │
                        ▼
                Contract Resolver
                        │
                        ▼
                 Plugin Resolver
                        │
                        ▼
                 Policy Validator
                        │
                        ▼
                   DAG Compiler
                        │
                        ▼
               Immutable DAG Artifact
                        │
                        ▼
                  Version Registry
                        │
                        ▼
                  Execution Runtime
             ┌──────────┼──────────┐
             │          │          │
           State     Events    Checkpoints
             │          │          │
             └──────────┼──────────┘
                        ▼
                    Audit Store

The compiler should catch problems before deployment:

unknown node
circular dependency
missing plugin
invalid schema
invalid tool
unsupported version
unreachable node
missing terminal state
illegal permission
undefined output
missing failure policy

This is far safer than discovering the problem while an LLM is operating production infrastructure.

---

9.31 Versioned Execution Architecture

At runtime:

Request
   ↓
Resolve playbook
   ↓
Playbook v3.7.2
   ↓
Resolve immutable DAG
   ↓
Create execution E-829182
   ↓
Persist:

tenant
execution ID
workflow version
DAG hash
agent versions
plugin versions
model policy
input contract

Then:

DAG Runtime
     │
     ├── Node 1
     │      ↓
     │   checkpoint
     │
     ├── Node 2
     │      ↓
     │   checkpoint
     │
     ├── approval
     │      ↓
     │   suspend
     │
     ├── resume event
     │
     └── Node 3

That becomes a proper enterprise agent execution plane.

---

9.32 Framework vs Platform Responsibility

A useful separation for an agent platform is:

PLATFORM owns:

Playbooks
DAG definitions
Versions
Contracts
Agent identities
Plugin identities
Tool permissions
Model policy
Tenant policy
Execution metadata
Audit
Failure policy
Approval policy
Observability
Evaluation metadata

while:

LangGraph / other runtime owns:

node scheduling
graph traversal
checkpoint mechanics
interrupt mechanics
parallel execution primitives
runtime execution

And:

LLM provider owns:

inference

This makes the system a platform rather than:

"A wrapper around LangGraph."

---

9.33 Framework Selection Architecture

Think of framework selection as layers.

             BUSINESS APPLICATIONS
                      │
                 PLATFORM API
                      │
            PLAYBOOK / AGENT MODEL
                      │
            ORCHESTRATION ABSTRACTION
                      │
        ┌─────────────┼──────────────┐
        │             │              │
     LangGraph   Microsoft AF     Custom
        │             │              │
        └─────────────┼──────────────┘
                      │
                 MODEL GATEWAY
                      │
       ┌──────────────┼───────────────┐
       │              │               │
    OpenAI         Claude          Gemini

Now technology can evolve independently at different layers.

---

9.34 Where Framework Independence Stops

Do not over-engineer portability.

Trying to support:

LangGraph
AutoGen
CrewAI
Semantic Kernel
Temporal
Step Functions
10 future frameworks

from day one may produce an abstraction so generic it becomes useless.

Instead isolate the stable concepts:

Execution
State
Node
Transition
Agent
Tool
Contract
Checkpoint
Event
Approval
Failure

Then allow framework-specific adapters underneath.

Architectural independence means:

Changing the framework should not require rewriting the business domain.

It does not mean:

Every framework must be interchangeable with one configuration flag.

---

9.35 Production Failure Modes

This is where architecture reviews usually get deeper.

1. Workflow changed while suspended

Mitigation:

immutable execution DAG

2. Node reruns after crash and repeats side effect

Mitigation:

idempotency keys

3. Schema changes break old checkpoints

Mitigation:

versioned state schemas
migration functions

4. Retry storm

Mitigation:

bounded retries
exponential backoff
jitter
circuit breakers

5. Poison message repeatedly crashes execution

Mitigation:

dead-letter queue
quarantine
operator inspection

6. Agent uses unauthorized plugin

Mitigation:

runtime authorization
plugin isolation
capability allowlists

7. Human approves one action but resumed workflow executes another

Mitigation:

Approval should reference:

action ID
action digest
parameters
workflow version

8. Model changes behaviour after provider update

Mitigation:

model registry
version/policy recording
evaluation gates
canary deployment

9. Infinite agent loop

Mitigation:

step budget
token budget
time budget
tool budget
termination conditions

10. Framework upgrade changes runtime behaviour

Mitigation:

adapter layer
contract tests
workflow regression suite
pinned dependencies
canary runtime

---

9.36 Testing Agent Orchestration

Traditional unit tests are insufficient.

You need several layers.

Node tests

given state X
node produces state Y

Routing tests

risk=20 → execute

risk=90 → approval

Contract tests

tool schemas
agent structured outputs
plugin interfaces

Replay tests

Take historical execution:

input
checkpoint
tool outputs

and replay against new workflow/runtime versions.

Failure injection

Simulate:

LLM timeout
429
database outage
tool failure
checkpoint failure
worker crash
approval timeout
invalid model output

Behavioural evaluations

For LLM nodes:

accuracy
policy compliance
tool choice
groundedness
risk

---

9.37 Observability

Every node execution should ideally create something like:

{
  "execution_id": "E829",
  "workflow": "key-remediation",
  "workflow_version": "3.2.0",
  "node": "risk_analysis",
  "attempt": 2,
  "agent": "risk-agent",
  "agent_version": "2.1",
  "model": "...",
  "tool_calls": [],
  "tokens": 4821,
  "latency_ms": 3840,
  "cost": 0.08,
  "status": "SUCCESS"
}

The tracing hierarchy becomes:

Workflow execution
    │
    ├── Node
    │    ├── Agent call
    │    │     ├── LLM call
    │    │     └── Tool call
    │    │
    │    └── validation
    │
    ├── Node
    │
    └── Approval

Then you can answer:

Why did execution fail?

Which model caused the latency?

Which tool consumed the most time?

How many retries occurred?

What did the human approve?

Which workflow version was executing?

What was the total cost?

---

9.38 LangChain vs LangGraph vs AutoGen: Which Should You Use?

Concern	LangChain	LangGraph	AutoGen
Primary mental model	Agent/tool abstractions	Stateful execution graph	Collaborative agents
Model integration	Strong	Usually via LangChain/model APIs	Strong
Tool abstraction	Strong	Supported through nodes/agents	Strong
Explicit workflow control	Moderate	Strong	Moderate/strong
Stateful graphs	Through LangGraph	Core capability	Different abstraction
Checkpoint/resume	Through LangGraph	Core capability	Architecture dependent
HITL	Through LangGraph	Strong	Supported patterns
Multi-agent	Supported	Strong custom graphs	Historically a major strength
Long-running workflow	Via LangGraph	Strong	Possible
Best mental use	Agent construction	Production orchestration	Multi-agent collaboration/research
2026 status	Active	Active	Maintenance mode; successor is Microsoft Agent Framework

LangGraph's current positioning emphasizes durable execution, persistence and HITL, while Microsoft's repository now explicitly identifies AutoGen as maintenance-mode and points greenfield users to Microsoft Agent Framework. (Docs by LangChain)

---

9.39 The Architecture Decision, Stated Plainly

Suppose the question is:

"Would you use LangChain, LangGraph or build your own framework?"

A strong answer would be:

"I wouldn't start with the framework. I'd first classify the orchestration requirements: whether execution is deterministic or agentic, whether it is long-running, whether we need human approvals, checkpoint/resume, parallel workers, multi-agent coordination, durable side effects and strict auditability.

>

For simple tool-using agents, a high-level framework such as LangChain may be sufficient. For complex stateful orchestration where I need explicit control of transitions, persistence, interrupts and recovery, I'd consider LangGraph or another durable orchestration layer.

>

I would avoid putting our domain model directly into framework objects. Our workflow definitions, state contracts, tool contracts, identities, permissions and audit model should remain platform-owned, with a framework adapter underneath. That allows us to change orchestration technology without rewriting the business application."

That answer demonstrates architecture, rather than framework familiarity.

---

9.40 A Platform Architecture Position

For an enterprise agent orchestration platform, the architecture I would defend is:

                     PLATFORM CONTROL PLANE

              ┌─────────────────────────────┐
              │     Playbook Registry       │
              │ YAML + semantic versions    │
              └──────────────┬──────────────┘
                             │
              ┌──────────────▼──────────────┐
              │       DAG Compiler          │
              │ contracts / validation      │
              │ permission checks           │
              └──────────────┬──────────────┘
                             │
              ┌──────────────▼──────────────┐
              │ Immutable DAG Registry      │
              └──────────────┬──────────────┘
                             │
                             │
                     PLATFORM DATA PLANE
                             │
              ┌──────────────▼──────────────┐
              │    Execution Runtime        │
              │                             │
              │ state                       │
              │ checkpoints                 │
              │ retries                     │
              │ interrupts                  │
              │ scheduling                  │
              └──────────────┬──────────────┘
                             │
            ┌────────────────┼──────────────────┐
            │                │                  │
       Agent Runtime     Plugin Runtime    HITL Service
            │                │                  │
            ▼                ▼                  ▼
          Models          Enterprise        Humans
                           Systems

Supporting systems:

Execution Store
State Store
Checkpoint Store
Artifact Store
Event Bus
Audit Store
Secrets Manager
Policy Engine
Identity Service
Observability Platform
Evaluation Platform

The key principle is:

The platform owns the execution semantics; a framework implements some of those semantics.

Therefore:

PLATFORM ≠ LangGraph application

PLATFORM = enterprise agent orchestration platform
             ↓
          may use
             ↓
         LangGraph

---

9.41 Five Distinctions to Remember

These five distinctions prevent a lot of confused architecture discussions:

1. Agent framework ≠ agent architecture
2. State ≠ conversation history
3. Checkpointing ≠ idempotency
4. Durable execution ≠ safe side effects
5. Framework portability ≠ lowest-common-denominator abstraction

And for a platform built on declarative playbooks:

6. YAML workflow ≠ runtime execution
7. Playbook version ≠ execution version
8. Human approval ≠ authorization
9. Tool registration ≠ permission to execute
10. Multi-agent collaboration ≠ uncontrolled agent conversation

---

9.42 Architect-Level Mental Model

The full chain is:

User / Event
      │
      ▼
Input Contract
      │
      ▼
Playbook Resolver
      │
      ▼
Immutable Versioned DAG
      │
      ▼
Execution Instance
      │
      ├── State
      ├── Budget
      ├── Identity
      ├── Permissions
      └── Policy
      │
      ▼
Orchestration Runtime
      │
 ┌────┼───────────────┐
 │    │               │
 ▼    ▼               ▼
Agent Deterministic  HITL
Node     Node
 │        │
 ▼        ▼
LLM      Plugin
 │        │
 └────┬───┘
      ▼
Verification
      │
      ▼
Checkpoint
      │
      ├── Next node
      ├── Retry
      ├── Pause
      ├── Compensate
      └── Terminate
      │
      ▼
Result + Audit Trail

That is the level at which you want to understand Agent Frameworks & Orchestration.

The framework question then becomes comparatively small:

Which runtime gives us the execution primitives we need without forcing our enterprise domain model to become framework-specific?

That is the architect's question.

---

9.43 Frequently Asked Questions

What is the difference between LangChain and LangGraph?

LangChain is the higher-level agent framework: model integration, tool abstractions, structured output and standard agent loops. LangGraph is the lower-level orchestration runtime underneath it, built around explicit graph state, nodes, conditional edges, checkpointing, interrupts and durable execution. Use LangChain to construct agents quickly; use LangGraph when you need explicit control over long-running, stateful execution.

Should a new enterprise system start on AutoGen?

Usually not. Microsoft's AutoGen repository now marks it as in maintenance mode and points new users to Microsoft Agent Framework. AutoGen is still worth understanding because many existing systems use it, but a greenfield Microsoft-oriented build should evaluate Agent Framework, behind framework-independent interfaces.

What is durable execution for AI agents?

Durable execution separates a workflow's progress from the process running it. State is checkpointed at defined boundaries, so an agent can pause for human approval, survive a crash or a deployment, and resume from the last checkpoint. It does not make side effects safe on its own: writes still need idempotency keys so a resumed step does not repeat a payment or a ticket.

How do you avoid agent framework lock-in?

Keep workflow definitions, state contracts, tool contracts, identities, permissions and the audit model platform-owned, and put the framework behind an adapter. Portability does not mean the lowest common denominator: expose framework-specific capabilities through extensions rather than banning them.

Is human approval the same as authorization?

No. A human approving a step confirms intent; it does not grant the agent permission to execute. The tool call still passes through identity, policy and resource authorization after approval.

---

Related reading

• Agent State and Memory Architecture, the state model that checkpointing and resume depend on.
• When AI Becomes Common, the Advantage Moves to How Work Is Orchestrated, why the orchestration layer is where the advantage sits.
• API Contracts in Microservices, the contract discipline that node-to-node contracts borrow from.