Agent frameworks

This post is authored by Aryan Kargwal, PhD at PolyMTL

What is an AI agent framework?

Choosing an AI agent framework is less like choosing a single library and more like deciding which parts of the agent runtime your team wants to own.

A customer-support agent, coding agent, and multi-agent research system may all use model calls and tools, but they require different forms of orchestration, state, permissions, recovery, deployment, and evaluation. A framework that works well for a short tool-calling loop may be a poor fit for a workflow that must pause for approval, survive process restarts, or coordinate several agents over multiple hours.

An AI agent framework provides reusable abstractions for building and operating goal-directed systems that use models, tools, state, and control logic. Depending on the product, those abstractions may include agent loops, tool registration, handoffs, graph execution, session state, durable checkpoints, memory, human approval, deployment, tracing, and evaluation.

No framework removes the need to design the agent itself. Your application still needs a bounded job, clear tool contracts, appropriate permissions, stop conditions, success criteria, and a way to inspect and evaluate what happened during each run. The previous chapter, what are AI agents, covers those architectural decisions in detail.

Frameworks, SDKs, runtimes, harnesses, and platforms

The products commonly described as “AI agent frameworks” do not all occupy the same layer of the stack.

These categories overlap, and they are moving. A framework may include an orchestration engine, an SDK may grow into a broader harness, and several vendors added harness layers during 2026. The useful question is not whether a product calls itself a framework. The useful question is which architectural responsibilities it takes off your plate and which responsibilities remain in your application code. Our explainer on why harnesses are replacing agent frameworks covers this shift, and agent orchestration explained separates the runtime layer from the framework layer.

Quick answer: There is no single best AI agent framework. Choose based on the control model, state requirements, tool surface, deployment environment, programming language, and failure modes of the agent you are building.

AI agent framework comparison

For a hands-on comparison of how several of these frameworks implement the same orchestrator-worker pattern, see our practical comparison of common agent frameworks.

How to choose an AI agent framework

1. Decide how explicit the control flow must be

Frameworks differ most in how the next action is selected.

• Model-directed loops let the model decide whether to call a tool, delegate, continue, or finish. They are quick to build but can be harder to constrain.
• Graph-based systems make nodes, branches, retries, and termination paths explicit. They require more setup but provide a clearer execution model.
• Event-driven systems let steps emit and consume typed events. They work well when several asynchronous activities may run or converge.
• Deterministic workflows encode the sequence in code and reserve model judgment for specific steps.
• Hybrid systems combine fixed business logic with bounded model decisions. This is the most common pattern for production agents.

Choose the least open-ended control model that still supports the behavior your application requires. A fixed approval step, authorization check, or financial transaction should remain deterministic even when the surrounding workflow is agentic. The agent orchestration definition covers how these models differ in practice.

2. Separate state, checkpoints, memory, and retrieval

“Memory” is often used to describe several different capabilities:

• Working context is the information available to the model during the current step.
• Session state stores values and history across related interactions.
• Checkpoints preserve execution progress so that a workflow can pause, recover, or resume.
• Long-term memory stores information that may be reused across sessions, such as user preferences or previous decisions.
• Retrieval queries an external knowledge source rather than preserving everything in the conversation history.

A framework may support one of these capabilities without supporting the others. Evaluate each one separately, especially when the agent must run for a long time or resume after a process failure. Our guides to AI memory and context management in agent harnesses go deeper on the trade-offs, and durable execution explains what it takes to survive a restart.

3. Evaluate the tool and permission model

Tool support involves more than registering a Python or TypeScript function. Review how the framework handles:

• Typed tool schemas and argument validation
• Authentication and user-specific authorization
• Timeouts, retries, fallbacks, and structured errors
• Human approval before sensitive actions
• Sandboxing for code, shell, browser, and file operations
• MCP servers and other external tool protocols
• Tool-call tracing and auditability

A large tool catalog does not automatically make an agent more capable. Expose only the tools and skills required for the current job, then measure whether the agent selects the correct tool and supplies valid arguments. See how to evaluate tool-calling agents.

4. Choose the right multi-agent pattern

Multi-agent support can mean several different things:

• A manager delegates tasks to specialized workers.
• Agents hand control to one another.
• Agents operate as tools called by a central controller.
• A graph routes work through several agent nodes.
• A swarm lets agents transfer work more dynamically.
• Independent workers execute in parallel and return results to an aggregator.

Do not add agents merely to divide a prompt into personas. Multiple agents increase the number of model calls, handoffs, state transitions, and potential coordination failures. Use them when specialization, parallel execution, isolation, or separate permissions create a measurable advantage.

5. Consider provider and infrastructure portability

Model portability matters most when model calls, tools, state, and business logic remain loosely coupled. A framework may claim support for several providers while still exposing provider-specific behavior through tool calling, structured output, built-in search, computer use, or session APIs.

Before relying on portability, run the complete agent against the providers you expect to use. Compare tool behavior, structured-output reliability, context handling, latency, and cost rather than checking whether the provider appears in a configuration list. When we tested seven models under the same harness, behavior varied enough that a model swap had to be treated as a change to the whole system.

6. Match the framework to your team

Language and operational fit often matter more than the number of built-in features.

• Python teams may prefer LangGraph, Pydantic AI, CrewAI, OpenAI Agents, Strands, or LlamaIndex.
• TypeScript teams may prefer Mastra, LangGraph.js, OpenAI Agents for JavaScript, or the Strands TypeScript SDK.
• Microsoft teams may benefit from Agent Framework’s .NET and Azure integration.
• Google Cloud teams may benefit from ADK’s integration with Gemini, Vertex AI, and A2A.
• AWS teams may combine Strands or another framework with Bedrock AgentCore.

Also review deployment, debugging, upgrade, and testing workflows. A framework that produces an impressive prototype but does not fit the team’s production environment will create more work later.

7. Require tracing and evaluation hooks

Framework-native logs can show that an action occurred, but production debugging usually requires a portable trace that connects model calls, retrieval, tools, handoffs, errors, approvals, latency, tokens, and outcomes.

Prefer frameworks that can emit OpenTelemetry-compatible traces or that have an OpenInference integration. This keeps the observability layer separate from the orchestration layer and makes it easier to compare agent implementations without losing historical data. Our write-up on instrumenting agents in any framework explains how one telemetry standard makes cross-framework comparison possible.

LangGraph

LangGraph is a low-level orchestration runtime for stateful, long-running agents. Developers model the workflow as nodes connected by edges, with shared state determining what each node reads, writes, and does next.

The graph can combine deterministic application code with model-directed decisions. Conditional edges handle branches, checkpoints preserve execution state, and human-in-the-loop interrupts can pause the graph until a person approves or modifies the next action.

LangGraph is a strong choice when the workflow needs explicit control over retries, branches, state transitions, recovery, or termination. It works particularly well for business processes in which some paths must remain deterministic while other steps require model judgment. For a deeper introduction, see our overview of LangGraph.

Best for: Complex stateful workflows, long-running agents, approval flows, and applications that need explicit execution paths.

Watch for: LangGraph is intentionally low level. Your team still needs to design state schemas, reducers, retry behavior, tool contracts, and termination conditions. A graph can make control flow visible without making the behavior inside each node correct.

How to trace LangGraph agents with Arize AX

OpenAI Agents SDK

The OpenAI Agents SDK provides a compact set of primitives for building agents in Python and TypeScript. Its core concepts include agents, tools, handoffs, guardrails, sessions, and a runner that manages the model-and-tool loop.

Developers can use handoffs to transfer control between agents, expose agents as tools to a central manager, or orchestrate multiple agents directly in application code. Code-based orchestration can also run independent agents concurrently before combining their results.

The SDK is OpenAI-first and integrates closely with OpenAI’s model and tool capabilities. It can also be extended to other model providers, although teams should test provider-specific behavior before assuming that every agent capability will remain identical.

Best for: Teams standardizing on OpenAI that want a readable, lightweight runtime without adopting a full graph framework.

Watch for: Complex state machines, durable workflows, and explicit business processes generally require additional application code or another orchestration layer. Portability should be tested at the complete-agent level.

How to trace the OpenAI Agents SDK with Arize AX

Google Agent Development Kit

Google’s Agent Development Kit, or ADK, is a code-first framework for building, evaluating, and deploying agents across several languages, including Python, Java, Go, TypeScript, and Kotlin. It supports model-directed agents, deterministic workflow agents, graph workflows, dynamic workflows, tools, sessions, memory, multi-agent composition, and human interaction.

ADK provides predefined sequential, loop, and parallel workflow patterns. ADK 2.0 added a structured workflow runtime with graph-based workflows and dynamic workflows, which let you express complex branching and iteration using ordinary language constructs instead of static routing tables, with automatic checkpointing so a resumed run skips work that already succeeded. Agents can communicate through hierarchical composition, delegation, shared state, or the Agent2Agent protocol. Tools can be attached directly, exposed through MCP, or connected to Google Cloud services.

Although ADK is designed to support different models and deployment environments, it is especially useful for teams already working with Gemini, Vertex AI, Google Cloud infrastructure, multimodal interaction, or A2A-based systems.

Best for: Google Cloud environments, multimodal agents, multi-agent systems, A2A interoperability, and teams that need both deterministic and model-directed orchestration.

Watch for: ADK supports several control patterns, so teams should document which layer owns routing, state, and termination. Avoid treating every multi-agent interaction as an unbounded model decision.

How to trace Google ADK agents with Arize AX

See also: tracing, evaluation, and observability for Google ADK.

 

Microsoft Agent Framework

Microsoft Agent Framework is Microsoft’s current framework for building agents and graph-based workflows. It is the direct successor to AutoGen and Semantic Kernel, built by the same teams, and it reached 1.0 general availability in April 2026. It combines AutoGen’s agent abstractions with Semantic Kernel’s enterprise features, including session-based state management, type safety, middleware, and telemetry.

The framework separates several levels of abstraction. Developers can build a relatively simple agent, use the Agent Harness that adds planning, context compaction, file access, approvals, and operational controls, or define a graph workflow with typed routing, checkpointing, and human-in-the-loop steps.

This range makes Agent Framework useful for teams that need to begin with a focused agent and later introduce more explicit workflow control without moving to an unrelated stack.

Best for: Microsoft and Azure environments, enterprise applications, typed workflows, and teams migrating from AutoGen or Semantic Kernel.

Watch for: Existing AutoGen implementations should not be described as though they are identical to Agent Framework. Semantic Kernel and AutoGen are now in maintenance mode, so plan the migration rather than deferring it indefinitely, and review guidance before changing orchestration, state, middleware, and tracing.

How to trace Microsoft Agent Framework with Arize AX

CrewAI

CrewAI provides two related abstractions: Crews and Flows.

A Crew organizes agents into roles with defined goals, tools, and responsibilities. Depending on the selected process, tasks may run sequentially, through a hierarchical manager, or through a hybrid pattern. A Flow provides a more explicit, stateful workflow in which events, routers, and steps determine how work progresses.

This distinction matters. A role-based Crew is useful when separate agents genuinely benefit from different instructions, tools, permissions, or areas of expertise. A Flow is useful when the surrounding business process needs clearer sequencing, persistence, or branching.

Best for: Research pipelines, business processes, and multi-agent workflows where role specialization and delegation improve the result.

Watch for: Manager agents and repeated delegation can add cost, latency, and coordination failures. Measure whether additional agents improve task success rather than assuming that a larger crew is more capable.

How to trace CrewAI agents with Arize AX

See also: how to set up CrewAI observability.

Mastra

Mastra is a TypeScript framework for building agents and AI-powered applications. It combines agents, typed workflows, tools, memory, retrieval, storage, server capabilities, observability, evaluation, and a harness layer in one integrated ecosystem.

Mastra workflows can define typed inputs and outputs, branches, retries, suspension, resumption, and error handling. Its agent abstractions support more open-ended model-directed behavior, while the workflow layer provides explicit control for steps that should remain predictable. Mastra’s harness layer, introduced in mid-2026 and now exposed as the AgentController class, wraps the agent loop with persistent sessions, thread management, switchable modes, tool-approval policies, subagent delegation, and observational memory.

This combination makes Mastra a strong option for TypeScript teams that want to build the agent and the surrounding application in the same language and framework.

Best for: TypeScript product and platform teams, full-stack AI applications, and systems that need both agents and typed workflows.

Watch for: Mastra covers a broad portion of the application stack and its harness APIs are still moving, including the rename from Harness to AgentController. Review how tightly your deployment, storage, memory, and evaluation layers should depend on one framework, pin versions deliberately, and keep business-specific tools portable where practical.

How to trace Mastra agents and workflows with Arize AX

Pydantic AI

Pydantic AI is a Python framework for building production agents with strong typing, structured outputs, dependency injection, model-provider flexibility, and OpenTelemetry instrumentation.

Developers can begin with a single typed agent and then introduce delegation, programmatic handoffs, graph-based control, durable execution, human approval, or a broader agent harness. Pydantic models make tool arguments, application dependencies, and agent outputs easier to validate at runtime and during testing.

Durability is worth understanding precisely. Pydantic AI does not implement its own durable engine. It attaches a durability capability to an agent and routes model requests, tool calls, and MCP communication through an external platform such as Temporal, DBOS, Prefect, or Restate, and a run is only durable when it executes inside that platform’s workflow.

Pydantic AI is particularly attractive for Python teams that already use Pydantic and want agent behavior to fit familiar engineering patterns rather than relying primarily on prompts and untyped dictionaries.

Best for: Typed Python applications, structured outputs, testable tools, dependency injection, and teams that want model-provider flexibility.

Watch for: Type validation can prove that an output has the expected shape, but it cannot prove that the content is correct. Semantic evaluation, trajectory inspection, and production monitoring remain necessary. Durable execution also adds an external dependency that your team must operate.

How to trace Pydantic AI agents with Arize AX

Strands Agents

Strands Agents is a lightweight SDK for building model-driven agents in Python and TypeScript. Tools can be defined as functions, models can be selected through supported providers, and the SDK manages the loop between model responses and tool execution.

Strands also includes multi-agent patterns for more structured applications. Graph and Swarm are built-in orchestrators: Graph runs agents as nodes in a directed graph with edges, conditions, and parallel execution, while Swarm lets agents hand off to one another so the path is decided at runtime. Workflow is a pattern you implement in code rather than a built-in orchestrator, although the Python strands-agents-toolspackage ships a ready-made workflow tool that handles task dependencies and parallel execution for you.

The small core makes Strands easy to embed in an existing service, while the optional orchestration patterns let teams introduce structure as the agent becomes more complex.

Best for: Lightweight Python or TypeScript agents, AWS environments, provider-flexible model loops, and systems that may grow into graphs or swarms.

Watch for: A model-directed loop can become difficult to reason about as the tool surface grows. Use explicit Graph or Swarm patterns when branches, dependencies, or termination rules need to be auditable. The Python and TypeScript SDKs also differ in meaningful ways, including dependency-resolution and failure semantics in Graph, so validate behavior in the language you actually ship.

How to trace Strands Agents with Arize AX

LlamaIndex Workflows

LlamaIndex Workflows is an event-driven framework for orchestrating agents and multi-step AI applications. Developers define steps that consume and emit typed events, while a shared context object carries state and services across the run.

Steps can trigger other steps, emit several events, wait for a collection of results, or coordinate asynchronous work. This model supports retrieval, tool use, model calls, branching, concurrency, and multi-agent composition without requiring every workflow to be expressed as a fixed graph.

LlamaIndex remains especially strong for data and retrieval use cases. Workflows can combine indexed documents, vector search, structured data, tools, and model reasoning in one execution path. See our introduction to LlamaIndex Workflows for cyclical agents.

Best for: Agentic RAG, document processing, knowledge assistants, research systems, and event-driven applications that depend heavily on internal data.

Watch for: Retrieval quality and agent quality should be measured separately. A well-orchestrated workflow can still fail because it retrieved the wrong documents, applied a poor filter, or assembled misleading context.

How to trace LlamaIndex Workflows with Arize AX

Claude Agent SDK

The Claude Agent SDK, renamed from the Claude Code SDK in late 2025, exposes the runtime and capabilities used by Claude Code for building custom agents. It includes built-in file and shell tools, MCP integration, hooks, permissions, sessions, structured outputs, and subagents.

This makes it closer to an agent harness than a minimal tool-calling library. An agent can inspect files, edit code, execute commands, use custom tools, apply permission policies, delegate to subagents, and continue work across a managed session. Lifecycle hooks such as PreToolUse and PostToolUse are the main policy surface, and permission modes control how much the agent can do without asking.

The SDK is useful for coding and research agents that need to act directly within files, repositories, terminals, or other tool-rich environments.

Best for: Claude-native coding agents, research agents, file-oriented workflows, and applications that need hooks, permissions, MCP tools, or subagents.

Watch for: File, shell, browser, and code-execution tools create substantial security and audit requirements. Use scoped permissions, sandboxing, approval gates, and complete tool traces. Treat permissive modes that bypass approval prompts as a deliberate, reviewed decision.

How to trace the Claude Agent SDK with Arize AX

Amazon Bedrock AgentCore

Amazon Bedrock AgentCore should be evaluated separately from the frameworks above. It is a managed platform for deploying and operating agents built with Strands, LangGraph, CrewAI, custom code, and other frameworks.

AgentCore includes services for runtime execution, identity, memory, tool gateways, browser use, code execution, observability, evaluation, and other production concerns. Teams can use the application framework that fits their control model while relying on AgentCore for selected infrastructure and governance capabilities.

Best for: Teams that want to run agents on managed AWS infrastructure with integrated identity, memory, tool access, observability, and operational controls.

Watch for: Choosing AgentCore does not choose the agent’s orchestration model. Evaluate the application framework and managed infrastructure as separate architectural decisions.

How to observe agents running on Amazon Bedrock AgentCore with Arize AX

See also: operationalizing AI agents at scale with Bedrock AgentCore observability.

Which AI agent framework is best for your use case?

This table should narrow the shortlist, not make the final decision. The best framework is the one that performs reliably on your tasks, fits your operating environment, and exposes enough telemetry to explain failures.

How to evaluate an AI agent framework

A feature matrix tells you what a framework claims to support. An experiment tells you how the complete agent behaves under the conditions your application will encounter. This is the same discipline an evaluation harness provides for models, applied to the whole agent.

1. Build a representative dataset. Include ordinary tasks, difficult cases, ambiguous requests, malformed tool inputs, permission failures, API timeouts, and cases that should escalate to a person.
2. Hold the rest of the system constant. Use the same model, prompts, tools, tool descriptions, data, and success criteria where the frameworks allow it.
3. Trace the complete run. Capture model calls, retrievals, tool selections, arguments, outputs, errors, retries, handoffs, approvals, latency, tokens, cost, and final outcomes.
4. Evaluate outcomes and trajectories. Measure whether the task succeeded as well as whether the agent followed required steps, selected appropriate tools, supplied valid arguments, respected policies, and stopped at the right time.
5. Test recovery. Inject tool failures, incomplete data, timeouts, invalid responses, and interrupted sessions. Verify whether the framework can retry, fall back, checkpoint, resume, or escalate correctly.
6. Run repeated trials. Agent behavior is probabilistic. Compare distributions and failure rates rather than selecting a framework based on one successful demo.
7. Compare operational fit. Review deployment complexity, upgrade risk, security, debugging experience, provider portability, and the effort required to add observability and evaluation.

Metrics for comparing agent frameworks

Our guide to agent evaluation metrics covers how to define and instrument each of these, and why token costs alone do not tell you if your AI is working explains why cost per successful task is the more useful denominator.

Arize AX can capture framework traces through OpenInference and OpenTelemetry, evaluate spans, traces, trajectories, and sessions, create datasets from production failures, and compare complete agent changes through experiments.

• Get started with tracing in Arize AX
• Trace agent trajectories
• Get started with evaluations
• Run full-agent experiments
• Automate investigations with AX Agent Improvement Loop

Teams looking for an open-source workflow can use Arize Phoenix for tracing, evaluation, datasets, and experiments. If you are operating a harness rather than a hand-built agent, see tracing, evaluating, and improving agent harnesses.

AI agent framework FAQ

Frequently asked questions

What is the best open-source AI agent framework?

There is no universal winner. LangGraph is a strong option for explicit stateful graphs, CrewAI for role-based agent teams, Mastra for TypeScript applications, Pydantic AI for typed Python systems, Strands for lightweight model-driven agents, and LlamaIndex Workflows for retrieval-heavy event-driven applications. Evaluate the shortlisted frameworks on your own task dataset.

What is the best Python AI agent framework?

LangGraph, Pydantic AI, OpenAI Agents SDK, CrewAI, Strands, Google ADK, Microsoft Agent Framework, and LlamaIndex Workflows all support substantial Python workflows. The best choice depends on whether you need graph control, type safety, role-based agents, provider-native capabilities, retrieval, or a lightweight loop.

What is the best TypeScript AI agent framework?

Mastra is designed around an integrated TypeScript agent stack. LangGraph, OpenAI Agents SDK, and Strands also provide TypeScript options. Compare their orchestration, state, deployment, and observability models rather than choosing by language alone.

What is the best framework for multi-agent systems?

The answer depends on the coordination pattern. CrewAI emphasizes role-based crews, LangGraph provides explicit graph control, Google ADK and Microsoft Agent Framework support structured multi-agent workflows, and Strands includes built-in Graph and Swarm orchestrators. Use multiple agents only when specialization, parallelism, isolation, or separate permissions improve the result.

Is AutoGen still maintained?

AutoGen is now in maintenance mode. Microsoft Agent Framework is its direct successor, built by the AutoGen and Semantic Kernel teams, and it reached 1.0 general availability in April 2026. Existing AutoGen projects continue to receive critical fixes, but new feature work goes to Agent Framework, and Microsoft publishes a dedicated migration guide because the move is a port rather than a configuration change.

What is the difference between an agent framework and an agent harness?

A framework provides reusable primitives for agents, tools, state, and orchestration. A harness provides a more complete operating environment around the model, including the execution loop, context management, tools, permissions, persistence, recovery, hooks, and often subagents. Some products now span both categories.

Do I need an AI agent framework?

No. A small agent with one or two tools can be built directly with a model API and application code. A framework becomes more useful when the system needs durable state, several tools, branches, retries, handoffs, human approval, multi-agent coordination, or standardized tracing.

Do AI agents need memory?

Not every agent needs long-term memory. A stateless or single-run agent may only require the current request and tool results. Longer workflows may need session state or checkpoints, while personalized applications may need durable memory. Retrieval should usually remain separate from conversation history.

>How can I reduce framework lock-in?

Keep business logic, tool implementations, data access, and success criteria separate from framework-specific orchestration. Use portable telemetry such as OpenTelemetry and OpenInference, retain access to your datasets and traces, and test provider portability with complete agent runs rather than isolated model calls.

Next, explore agent observability to see how these frameworks are instrumented in production, or continue with the complete guide to AI agent tracing and evaluation to learn how to instrument trajectories, create evaluators, build datasets, run experiments, and improve agent behavior in production.