Recovering from Tool Failures: Retries, Fallbacks, and Graceful Degradation
Production patterns for when agent Actions time out, return errors, or partially succeed.
Tool failures are normal in agent systems. Networks blip, inventories change, sessions expire. Reliability comes from how the agent and your backends respond—not from pretending every Action always succeeds.
Classify Failures Before You Retry
Not every error should be retried.
| Class | Examples | Agent response |
|---|---|---|
| Transient | Timeouts, 503, rate limits | Retry with backoff, then explain delay |
| Deterministic | Validation errors, not found | Fix parameters or ask the user |
| Auth | Expired session / access token | Stop; ask user to re-authenticate |
| Business rule | Not refundable, already cancelled | Explain policy; offer alternative Action |
| Partial | One of three writes succeeded | Compensate or surface incomplete state |
Blind retries on business-rule failures create duplicate side effects and angry users.
Make Backend Errors Actionable
Structure Action responses so agents can branch:
{
"success": false,
"error": "INSUFFICIENT_INVENTORY",
"message": "Only 2 units available, requested 5",
"suggestedAction": "adjustQuantity",
"availableQuantity": 2
}
Mirror those codes in system instructions:
If INSUFFICIENT_INVENTORY: offer to adjust quantity or suggest alternatives.
If AUTH_EXPIRED: do not retry; ask the user to sign in again.
If RATE_LIMITED: wait and retry once, then apologize for the delay.
Retry Policies That Respect Side Effects
Safe automatic retries:
- Idempotent reads (
get_order,search_products) - Explicitly idempotent writes (keyed by idempotency key)
Unsafe without care:
- Charges, emails, ticket creation
For writes, prefer idempotency keys generated per user intent:
POST /api/refunds
Idempotency-Key: run_abc123:create_refund:ORD-100
If the agent retries after a timeout, your backend returns the original result instead of double-refunding.
Fallback Strategies
When the primary Action fails:
- Degrade to read-only help — explain status without mutating
- Alternative Action — store credit instead of refund when policy blocks cash
- Human escalation — HITL or handoff when automation is stuck
- Cached / partial answer — use last known data and label uncertainty
Fallbacks should be explicit in instructions so the model does not invent unsafe workarounds.
Partial Success and Compensation
Multi-step runs fail in the middle. Design compensating Actions or clear user messaging:
- "I reserved inventory but payment failed—I've released the hold."
- Or call
release_holdautomatically whencapture_paymentfails
Traces should show both the failed step and the compensation so operators can audit.
Monitor Failure Patterns
Use Tool Monitoring and Traces to spot systemic issues:
- One Action dominating failure counts
- Spikes after a deployment
- Specific error codes trending up
A rising AUTH_EXPIRED rate often means embed or session TTL is too short for typical conversation length—not a model bug.
User Communication
Users tolerate delays; they do not tolerate silence or false success.
- Acknowledge retries ("Still checking inventory…")
- Never claim a write succeeded without a successful Action response
- Offer a next step when automation stops
Conclusion
Tool failure recovery is product behavior: classify errors, retry only what is safe, structure backend responses, compensate partial writes, and watch Tool Monitoring for systemic drift. Agents that fail well feel reliable even when the underlying systems occasionally do not.
