Enterprise teams keep adding more AI retries.
That is understandable. Long-running responses, tool calls, browser actions, remote connectors, and local inference queues all create more ways for a run to stall or look ambiguous.
But one retry policy is not enough.
Current platform docs already show that AI failures arrive in different classes, and those classes need different responses. Some requests are malformed and should fail closed. Some exceed size or output limits and need a route change, not another attempt. Some hit rate limits and should wait for a retry window. Some overload a shared service. Some lose transport while work may still be alive. Some are explicitly cancelled. Some return partial or incomplete output that looks fluent enough to slip through review.
That is why enterprise AI needs a failure-class contract.
A failure-class contract defines how the workflow names common failure states, which ones are retryable, which ones require resume or reconciliation, which ones need rerouting or segmentation, and which ones must stop for human review before any further action happens.
Why this matters now
The AI stack is becoming more stateful, not less.
Hosted APIs expose longer-lived response objects and more tool activity. MCP transports now document resumability and redelivery. Official error guides distinguish between request shape problems, rate limiting, overload, timeouts, and mid-stream failures. Private AI adds its own queue pressure, watchdog restarts, proxy disconnects, and partial local recovery.
That means enterprise reliability is no longer just about whether the model answered.
It is about whether the workflow can tell the difference between do not retry, retry later, resume from cursor, reroute with smaller scope, and stop because side effects may already have happened.
If those states are collapsed into one generic error handler, teams get the worst mix of behaviors: repeated invalid requests, duplicated downstream work, abandoned live runs, silent partial outputs, and operators who cannot explain what the system is waiting for.
The docs already show why failure class is a route contract
1. OpenAI separates input-fit problems, truncation policy, and incomplete output state
The current OpenAI OpenAPI specification exposes a dedicated /responses/input_tokens endpoint that returns input token counts for a request before execution.
The same specification documents a truncation setting for responses. In auto mode, if the input exceeds the context window, the system truncates by dropping items from the beginning of the conversation. In disabled mode, which the spec marks as the default, the request fails with a 400 instead.
The spec also defines incomplete_details for responses, with a documented reason of max_output_tokens.
Those are already different failure classes. An over-limit request with truncation disabled is not the same as a valid request that ended incomplete because the output ceiling was too low. Neither one should be handled like a transient infrastructure error.
2. Anthropic distinguishes invalid requests, conflicts, size limits, rate limits, overload, and timeouts
Anthropic's current API error guide is explicit. It documents:
- 400 invalid_request_error for request format or content problems,
- 409 conflict_error where the client should resolve the resource conflict and then retry,
- 413 request_too_large when the payload exceeds endpoint size limits,
- 429 rate_limit_error when the account hits rate limits,
- 500 api_error for internal failures where exponential backoff is appropriate,
- 504 timeout_error for request processing timeouts, and
- 529 overloaded_error when the API is temporarily overloaded.
The same docs say Anthropic's SDKs automatically retry transient failures such as connection errors, rate limits, and 5xx errors with exponential backoff, while also warning that when receiving server-sent events, an error can occur after the API already returned HTTP 200.
That means the transport code alone does not tell you the recovery class. A 200 stream can still end in error. A 413 should change the request shape, not repeat it. A 429 should honor a wait window. A 529 says shared capacity is stressed, which is different from a malformed request or a business-state conflict.
3. Anthropic rate-limit docs add explicit wait semantics
Anthropic's current rate-limit docs say a 429 response includes a retry-after header that tells clients how long to wait before trying again. The same page warns that sharp traffic increases can trigger acceleration limits and that usage should ramp gradually.
That is another reason failure class needs to be named explicitly. A rate-limit class is not just "the model failed." It is a scheduling signal. The right response is queue management, backoff, and maybe rerouting to another approved lane, not blind replay against the same constrained path.
4. MCP transport docs say disconnection is not cancellation
The current MCP transports documentation says disconnection may occur at any time due to network conditions and should not be interpreted as the client cancelling its request. To cancel, the client should explicitly send an MCP cancellation notification.
The same docs say servers may make streams resumable, and that a client can reconnect with Last-Event-ID so the server may replay messages that would have been sent after the broken connection.
That is a major operational distinction. A broken transport may mean the run is still alive. If the enterprise treats every disconnect as a dead run and starts over, it can create duplicate work, duplicate tool execution, or conflicting downstream state.
5. MCP lifecycle docs make timeout handling an explicit protocol concern
The current MCP lifecycle specification says implementations should establish timeouts for requests to prevent hung connections and resource exhaustion. When a request has not received a success or error response within the timeout period, the sender should issue a cancellation notification and stop waiting. The same guidance says timeout behavior may consider progress notifications, but systems should still enforce a maximum timeout.
That means waiting forever, retrying immediately, and cancelling explicitly are different control moves. If the workflow does not record which one happened, later operators cannot tell whether the run finished, stalled, resumed elsewhere, or was abandoned halfway through.
What breaks when failure class stays implicit
1. Teams blindly retry deterministic failures
If the request is malformed, too large, or structurally over the route limit, another attempt under the same conditions usually reproduces the same failure with extra latency and confusion.
2. Rate-limit problems get misdiagnosed as model-quality problems
A route that needs backoff, queue discipline, or traffic smoothing can get blamed for being unreliable when the real issue is load policy.
3. Disconnects trigger duplicate work
If the client loses transport but the run or tool execution continues, restarting without reconciliation can duplicate consequential actions or create conflicting state.
4. Incomplete output gets treated like completion
A response that stopped at an output ceiling or dropped context through truncation can still look polished enough to pass shallow review. That does not make it complete.
5. Private AI adds invisible ambiguity
Local inference can improve privacy and cost control. It does not eliminate queue saturation, watchdog restarts, reverse-proxy disconnects, or partial downstream writes. The hosting location changes. The recovery math stays.
What a failure-class contract should include
A useful contract does not need dozens of states. It needs enough classes to pick the right next move consistently.
1. Request-invalid class
Malformed input, unsupported parameters, payload-too-large cases, and fail-closed over-limit requests belong here. Default action: reject, fix, or segment the request. Do not auto-retry unchanged.
2. Incomplete-result class
Valid request, incomplete output. Typical causes include low output ceilings, forced stop conditions, or route-level truncation choices. Default action: inspect completion state, adjust the route or budget, and rerun deliberately if approved.
3. Capacity-constrained class
Rate limits, overload, queue exhaustion, or acceleration limits belong here. Default action: honor retry windows, back off, and consider rerouting only if the fallback contract preserves review scope.
4. Transport-broken but run-state-unknown class
Disconnects, dropped SSE streams, or broken client sessions while the server may still be processing belong here. Default action: reconcile or resume before creating a new run.
5. Explicitly-cancelled or timed-out class
These are termination states, not merely missing output. Default action: record who or what cancelled, whether side effects may have landed, and whether compensation or reconciliation is required.
6. Side-effect-uncertain class
If tools, messages, records, or external writes may have partially executed, treat the event as a business-state problem, not just a compute problem. Default action: verify downstream state before any replay.
Questions buyers and operators should ask
- Which AI failure states are considered retryable on this exact route?
- Which failures require reconciliation or resume before any new run is created?
- Do rate-limit and overload responses honor explicit backoff windows?
- Can the workflow distinguish invalid input, incomplete output, overload, disconnect, and explicit cancellation?
- How are mid-stream errors after HTTP 200 captured and surfaced?
- What evidence is preserved when a run times out, is cancelled, or loses transport?
- When side effects may already have happened, does the system verify downstream state before retrying?
If those answers are vague, the workflow may still be fine for casual chat. It is not ready to anchor enterprise automation that has to survive load, ambiguity, and real business consequences.
The commercial takeaway
Enterprises do not get dependable AI by adding more retries.
They get dependable AI by deciding which failures are safe to retry, which ones require backoff, which ones require resume or reconciliation, and which ones must stop until a human can inspect what already happened.
Current OpenAI, Anthropic, and MCP docs already show the state space. The durable advantage will come from teams that turn those states into an explicit failure-class contract for each approved route.
That is how recovery stops being guesswork and starts becoming an operating standard.
Related reads
- AI Agents Need Idempotency Before Autonomy: The Enterprise Guide to Safe Retries
- Disconnects Are Not Cancellations: Why Enterprise AI Needs Run-State Reconciliation
- Enterprise AI Needs a Stop-Reason Contract: Why Valid Output Is Not Always a Complete Result
- AI Agents Need Dead-Letter Queues: The Recovery Lane for Stuck Enterprise Automation
- AI Workflow Services