# How Should Enterprises Actually Scale Agentic AI Beyond Pilot Projects in 2026?

Savannah Jenkins · September 23, 2026

> What Scaling Enterprise Agentic AI Actually Means Scaling enterprise agentic AI means moving from isolated demonstrations to dependable systems that...

## What Scaling Enterprise Agentic AI Actually Means

Scaling enterprise agentic AI means moving from isolated demonstrations to dependable systems that complete bounded business work across many teams, applications, and environments. An agent is not simply a chatbot with a longer prompt: it can interpret a goal, select tools, call enterprise systems, make intermediate decisions, and return a result or request approval. The scaling problem therefore concerns reliability, permissions, data access, evaluation, operations, and economics, not only model quality. As of September 2026, the discussion has shifted from whether agents can perform tasks to whether organizations can operate them repeatedly without creating an unmanageable surface area of failures. This shift is visible in the release of agent frameworks, enterprise platforms, and security products referenced in current research, including work from BCG, McKinsey, Bain, Deloitte, IBM, Intel, and HBR.

**Also worth reading:** [How do enterprises actually enforce policies for autonomous AI agents in production?](https://agustin-otegui.com/knowledge/how_do_enterprises_actually_enforce_policies_for_autonomous_ai_agents_in_production.php) · [How do enterprises actually execute enterprise AI agent cost optimization without destroying model performance?](https://agustin-otegui.com/knowledge/how_do_enterprises_actually_execute_enterprise_ai_agent_cost_optimization_without_destroying_model_performance.php) · [How Should Modern Enterprises Architect Their Agentic Orchestration Strategies to Ensure Long-Term Scalability?](https://agustin-otegui.com/knowledge/how_should_modern_enterprises_architect_their_agentic_orchestration_strategies_to_ensure_long-term_scalability.php)

A useful definition of production scale is a system that handles a meaningful volume of requests while meeting explicit service levels, stays within approved data boundaries, and produces an auditable record of its actions. A pilot may achieve all of these for 20 users but fail when 2,000 users invoke it against the same ERP instance. Scale is also organizational: the business needs a platform that different departments can configure without waiting for a central AI team to rewrite every workflow. This explains why architectures based on stable enterprise backends and controlled agent interfaces are often more practical than replacing ERP or core systems with autonomous software.

## Why Enterprise Agentic AI Is Harder Than Ordinary AI Deployment

Traditional enterprise software usually has a defined interface, a fixed transaction path, and a human who decides whether an action is allowed. Agentic systems introduce probabilistic decisions between the user and those systems, so the same request can take different paths depending on the model, available tools, retrieved context, and previous actions. That variability does not make production deployment impossible, but it changes where engineering effort belongs. Teams must test decisions and tool selection, not just response wording, and they must distinguish a harmless incorrect answer from an incorrect payment, contract change, or clinical recommendation.

Several structural factors make scaling harder. Enterprises frequently have fragmented permissions, duplicated data, inconsistent APIs, and legacy workflows that were never designed for machine callers. Regulated industries add documentation, approval, retention, and monitoring requirements, which is why BCG has focused on AI agents in regulated settings. The market is also geographically concentrated: the research context reports that 75% of registered AI company offices in the United Kingdom are located in London, so talent and vendor capacity may not be evenly distributed. Finally, agent frameworks are still developing quickly. A framework that is excellent for experimentation may not provide durable identity management, tenancy, policy enforcement, or versioned observability.

The practical consequence is that model selection is only one architectural decision. Cost, latency, tool reliability, security, and recovery behavior often determine whether a project can scale. Teams that evaluate only benchmark scores can build a system that impresses in a lab and becomes uneconomic or unsafe in production.

## The Reference Architecture: A Controlled Platform Around Stable Systems

A scalable reference architecture normally places the agent behind a governed runtime rather than allowing a model to connect directly to every enterprise service. The runtime owns identity, session state, tool permissions, rate limits, logging, approval gates, and failure handling. It exposes a small set of typed tools to the model, and each tool is backed by a conventional service or API. This preserves the ERP as a stable system of record while allowing agents to act as an interface for selected tasks, an approach specifically discussed in research on ERP and agentic systems.

The architecture should also separate planning from execution. A planner interprets the request and proposes a plan, while a policy engine decides which steps may run automatically, which need human approval, and which are prohibited. Data retrieval should use approved retrieval services and access filters inherited from the user's permissions, rather than sending broad documents to a model and asking it to decide what is visible. A durable workflow engine should record each step, retry safe operations, and pause when a required approval is missing. This design makes the agent a coordinator of existing controls instead of a new authority that bypasses them.

At the infrastructure level, teams need model routing, caching where appropriate, a queue for long-running work, and independent observability for technical and business outcomes. The model is only one replaceable component; tool contracts, policy rules, and evaluation data are often more important for long-term operation. Intel's 2026 Hot Chips architecture discussions and McKinsey's work on foundations for agentic AI at scale both point toward systems thinking rather than a single-model strategy, although neither should be read as proof that one hardware or platform design fits every enterprise.

## A Practical Progression From Pilot to Production

The first practical step is to choose a workflow with measurable value, bounded authority, and a reversible outcome. Claims processing, internal IT triage, customer-service resolution, or research preparation may be easier than autonomous procurement or regulated clinical decisions. Define what the agent may do, what it must never do, and who owns the final outcome. Establish a baseline for cycle time, handling cost, error rate, escalation rate, and human satisfaction before connecting live tools. Without a baseline, a project can appear successful because users are more forgiving during a demonstration than during daily operations.

Next, build a thin production slice with real authentication, logging, and a limited user group. Test against adversarial and mundane cases, including missing permissions, stale records, duplicate requests, tool timeouts, contradictory instructions, and prompt injection embedded in retrieved documents. A useful initial threshold is to require zero unauthorized actions and near-zero duplicate financial or irreversible operations, even if the system initially completes only a small percentage of requests without human help. Expand the scope only after the team can explain failures, assign them to a model, tool, data, or process category, and demonstrate that a change improved the relevant metric.

The operating model matters as much as the technical rollout. Assign named owners for the agent, its tools, its policies, its evaluation suite, and its business process. Review the tool inventory monthly during the first year, because a harmless integration can become dangerous if its permissions or data classification changes. Re-run regression evaluations whenever a model, prompt, retrieval index, or upstream API is updated. This staged approach is slower than announcing a universal agent platform, but it produces evidence that supports expansion and gives finance and risk teams a credible basis for approving further investment.

## Comparing Architectural Choices

There is no single best way to scale enterprise agentic AI. The main choice is usually between extending existing platforms, adopting a managed agent service, building a governed runtime, or using open frameworks directly. The comparison below reflects operational trade-offs rather than a universal ranking.

| Feature | Extend an existing enterprise platform | Use a managed agent service | Build a governed runtime | Adopt an open framework directly |
| --- | --- | --- | --- | --- |
| Time to first bounded workflow | Moderate to fast | Fast | Slow to moderate | Fast for prototypes |
| Control over permissions and audit | High if the platform exposes the right controls | Variable by contract and product | Very high | Low to moderate until custom work is added |
| Model flexibility | Moderate to high | High | High | High |
| Operational burden for the enterprise | Lower | Lower initially, higher at scale and volume | High initially, then predictable | Highest |
| Best fit | Organizations with strong cloud and workflow tooling | Teams needing rapid deployment and vendor support | Regulated or complex enterprises | Developers testing ideas and custom components |
| Main failure mode | Platform features do not map cleanly to the workflow | Lock-in, opaque limits, or unclear data handling | Overbuilding before demand is proven | Prototype architecture becomes production architecture by accident |

Managed services can reduce infrastructure work, but enterprises should confirm where data is processed, which regions are supported, how prompts and tool traces are retained, and whether usage can be capped by department. Building a runtime from open components provides control, but it shifts responsibility for upgrades, security, and incident response to the buyer. Open-source engines and frameworks can be useful components; they are not equivalent to a complete enterprise platform. A hybrid approach is common: a commercial runtime for identity, monitoring, and governance, with model providers and specialized research tools behind replaceable interfaces.

## Evaluation, Security, and Reliability Thresholds

Evaluation must cover the complete task, not just the final answer. A system that produces a fluent summary but calls the wrong customer record has failed. Measure task completion, correct tool selection, argument validity, policy compliance, latency, cost per successful outcome, escalation frequency, and recovery rate. Segment results by user role, data sensitivity, language, and task difficulty, because an aggregate accuracy figure can hide serious failures in a small but important group. For regulated workflows, retain the prompt, retrieved context, tool inputs, tool outputs, approvals, model version, and final decision in an auditable format.

Security controls should assume that agent instructions can be influenced by untrusted content. Retrieved documents, web pages, and user messages must be treated as data, not as higher-priority commands. Use allowlisted tools, least-privilege credentials, short-lived tokens, network restrictions, and server-side authorization checks. Dapto's prompt-and-response firewall positioning and Deloitte's API governance work for agentic AI illustrate the market's recognition that a model gateway alone is insufficient. Governance has to cover APIs, data flows, tool behavior, and the human actions that follow an agent's recommendation.

Reliability targets should be expressed as service levels rather than vague commitments. For example, a research assistant might target 95% successful retrieval within 30 seconds, while a payment workflow might require 99.9% valid tool execution and mandatory approval for every transfer above a defined amount. These are design examples, not universal standards; actual thresholds depend on the consequence of failure. Conduct failure-injection tests for unavailable APIs, partial data, model timeouts, and conflicting records, then verify that the system stops safely rather than repeatedly taking an action. Scale should be paused when a release creates an unexplained increase in unauthorized access, duplicate side effects, or unreviewed changes to high-impact data.

## Cost, Pricing, and the Economics of Volume

Agentic systems are usually more expensive per interaction than a single model response because they may retrieve documents, call several tools, run multiple reasoning steps, and retry after a failure. The relevant cost is cost per successful business outcome, not price per token. A low-cost model that requires three human escalations may be more expensive than a larger model that completes the task correctly, while a highly capable model used for simple classification may waste budget. Measure the full cost of infrastructure, integration maintenance, evaluation, human review, security, incident response, and vendor support.

Pricing structures vary. Managed platforms commonly charge by model input and output, tool calls, workflow executions, storage, or a combination. Open-source engines may have no license fee but still require engineering, hosting, observability, and security investment. Cloud infrastructure costs can rise sharply with long-running agents, particularly if they poll systems or enter retry loops. Set departmental budgets, per-workflow limits, maximum execution steps, and alerts before broad rollout. Mistral AI's February 2026 partnership with Accenture, with financial terms undisclosed, shows that enterprise deployment is often a negotiated program rather than a simple per-seat purchase; that makes transparent unit economics and exit options especially important.

Do not assume that larger context windows or more agents automatically reduce cost. Context adds latency and may increase exposure to irrelevant instructions, while multiple agents can duplicate retrieval and create coordination overhead. Use smaller models for routing and classification, larger models for ambiguous cases, and deterministic code for calculations and policy checks. Revisit routing after real usage data is available, because the cheapest architecture is the one that meets the required quality with the least unnecessary computation.

## Common Mistakes and When to Act

The most common mistake is treating a successful demonstration as evidence of enterprise readiness. Another is designing around autonomous ambition before establishing observability, permissions, and a clear rollback mechanism. Teams also frequently underestimate integration work, particularly when an agent must interact with legacy systems that lack reliable APIs. A third error is allowing each business unit to create its own agent framework, producing inconsistent prompts, duplicated data access, and no common incident response. Finally, organizations often expand user access before defining what constitutes acceptable failure.

A useful rule is to act quickly on bounded, reversible workflows, but slowly on irreversible or legally consequential ones. Move from 10 users to 100 only when quality, cost, and support metrics remain stable. Move from 100 to 1,000 only when identity, tenancy, monitoring, and incident procedures have been tested under realistic load. The urgency may come from competitors or a deadline, but urgency is not a substitute for controls. The 2026 pattern reported by McKinsey, PwC, and other consultancies is a transition from experimentation to enterprise impact, yet the transition remains uneven and many projects will be constrained by data and process readiness rather than by model capability.

The decision to scale should be based on evidence from at least several representative workflows, not a single benchmark. Ask whether the agent improves a business metric, whether users understand its limits, and whether the organization can explain every material action it takes. If those answers are yes, expand gradually. If they are not, narrow the agent's authority, add human approval, or return to workflow redesign before increasing volume.

## A Durable Path Forward

The most defensible enterprise strategy is a governed agent operating over stable systems, with replaceable models and carefully selected tools. Begin with a business process where value can be measured and errors can be contained, then invest in identity, policy, evaluation, logging, and recovery before broad autonomy. Treat agents as a new class of software integration, not as a replacement for the enterprise architecture already responsible for records, controls, and accountability.

This path also explains why ERP systems are likely to remain important. Agents can provide a more natural interface for users, but the ERP still needs to hold authoritative data and enforce financial or operational rules. Similarly, human-agent collaboration should be designed as a normal operating mode rather than a temporary failure state. HBR's concept of an elastic enterprise and current work on API governance both point toward flexible participation between people and automated systems, with policy determining when autonomy increases or decreases.

By September 2026, the practical question is not whether an agent can imitate an expert. It is whether an enterprise can make the agent's actions sufficiently narrow, observable, affordable, and recoverable to trust at scale. Organizations that answer that question with measured architecture and governance will be better positioned to adopt new models and frameworks without rebuilding the entire system each time the technology changes.

## Quick answers

### What is the safest first workflow for enterprise agentic AI?

Start with a bounded workflow that has measurable value, reversible actions, and clear human ownership. Internal research, IT triage, or customer-support preparation can be easier to test than payments, hiring, or regulated clinical decisions. Permissions, logging, and escalation should be implemented before the agent receives broad access.

### How many users should an agentic AI pilot include?

There is no universal number, but a small pilot of roughly 10 to 25 representative users is often enough to expose initial usability and integration problems before wider deployment. Expansion to hundreds or thousands should depend on stable task-success, error, latency, cost, and support metrics. The threshold is operational confidence, not user count itself.

### Do open-source agent frameworks scale better than managed platforms?

Not automatically. Open frameworks can provide flexibility and reduce vendor dependence, but they usually require the buyer to build identity, policy, observability, tenancy, and incident response. Managed platforms can accelerate deployment, although they may introduce usage limits, data-processing questions, and lock-in.

### How should enterprises measure agentic AI performance?

Measure task completion, correct tool use, authorization compliance, latency, cost per successful outcome, escalation rate, and recovery from failures. Evaluate these metrics by user role, data sensitivity, language, and task difficulty rather than relying on one aggregate accuracy score. Auditable traces should connect the final result to the model, tools, data, and approvals used.

### When should a company move from human approval to autonomous execution?

Increase autonomy only after repeated testing shows that low-risk actions remain accurate and policy-compliant under realistic operating conditions. Keep approval for irreversible, financial, regulated, or high-impact decisions until the organization has evidence and an effective exception process. Autonomy should be a graduated operating decision, not a binary launch switch.

Canonical: https://agustin-otegui.com/knowledge/how_should_enterprises_actually_scale_agentic_ai_beyond_pilot_projects_in_2026.php
Markdown: https://agustin-otegui.com/knowledge/how_should_enterprises_actually_scale_agentic_ai_beyond_pilot_projects_in_2026.php/index.md