Azure AI Outage Lessons for Testing Agent Retry Budgets.

What happened during the Azure AI outage?

Microsoft reported intermittent failures and increased latency in Azure OpenAI Service, Foundry Agent Service and related Azure AI services in Sweden Central on 29 September 2026. The reported customer impact ran from 10:03 to 15:58 UTC. Some affected requests returned HTTP 500 or 503 errors.

Microsoft traced the disruption to a deployment and caching defects that drove traffic to an internal service to roughly 20 times normal. Slower connections and retries compounded the pressure. Health checks then triggered repeated restarts, reducing the capacity available to serve requests. Microsoft changed the health check to stop the restart loop, and availability recovered.

That account describes Microsoft’s infrastructure. The exercise below applies a lesson from the review to a fictional customer workflow. It is independent of Microsoft’s incident investigation.

• Microsoft Azure post-incident review for Sweden Central, 29 September 2026 ↗

Count attempts across the whole job

Imagine a service desk uses an agent to classify incoming requests and assign each ticket to a queue. During intermittent model failures, the model client, workflow runner and queue worker each have their own retry policy. If each layer can make three total attempts for the same operation, nested retries could generate up to 27 model calls from one ticket. That is an illustrative upper bound, assuming each outer attempt runs through all inner attempts.

In staging, send a small batch of synthetic tickets and simulate slow responses, temporary errors and occasional successes. For one exercise, agree on a budget of three total model attempts and 30 seconds of elapsed time per ticket. Count the original attempt, automatic client retries and attempts after redelivery. Choose production limits with the workflow owner based on the customer deadline and dependency capacity.

Check the ticket during degradation and after recovery

The pass criteria should describe what a customer or staff member can verify. For this exercise, each ticket should end up routed once or visibly assigned to manual triage. A retry limit is only half the test if a ticket can disappear when that limit is reached.

1. Verify the shared budget

   Inspect model calls and elapsed time for each synthetic ticket, including jobs redelivered by the queue. Once either limit is reached, check that the workflow stops automatic attempts for that job.

2. Verify the handover

   Confirm the manual queue received the ticket with its original reference and pending status. Check that the customer-facing message reflects the handover without claiming an assignment that has yet to happen.

3. Verify which work can resume

   Restore the simulated model service while a staff member handles one ticket and another job is delivered twice. Check the current ticket state before writing an assignment. Resume eligible work at an agreed rate and confirm staff decisions survive the replay.

What VisiFlo would check with your team

Start with one workflow and agree on the deadline, retry budget and fallback owner. VisiFlo can help turn those rules into staging scenarios and compare the model calls, queue records and final ticket state with the expected outcome. If an integration hides its retries or the queue cannot expose a delivery record, the report can identify the evidence gap and the instrumentation needed to verify it.

A useful release decision comes down to three observable results. Attempts stay within the agreed limits, unresolved tickets reach their owner, and recovered jobs leave each ticket in the correct state. After changes to a client library or queue policy, rerun those same checks.