Voice and Multichannel Agents: One Brain Across Phone, Chat, and Inbox
How to design agents that work across voice, WhatsApp, Slack, and email without fragmenting tools, auth, or governance.
Users do not care that your agent was designed for a web widget. They will call, message on WhatsApp, ping Slack, or email support. Multichannel agents share one policy brain—Actions, Environments, HITL—while adapting presentation to each channel's constraints.
Shared Core, Channel Adapters
Keep a single App-level agent configuration for:
- System instructions and business rules
- Actions / MCP tools
- Auth forwarding patterns
- HITL and Tool Monitoring
Put channel-specific logic in adapters:
- Voice — speech-to-text, text-to-speech, barge-in, shorter turns
- WhatsApp / SMS — concise messages, media limits, async delays
- Slack — thread awareness, workspace identity mapping
- Email — longer form, subject lines, quoted history
- Web / embed — Screen Observer, richer UI, embed tokens
The agent should call the same get_order Action whether the user spoke or typed.
Voice-Specific Design
Voice amplifies latency and ambiguity.
- Prefer confirmations for critical parameters ("Refund forty dollars—is that right?")
- Keep HITL approvals on a parallel operator UI; do not leave callers in dead air
- Stream partial responses when possible so silence does not feel like failure
- Design fallbacks when ASR is low-confidence: spell IDs, offer SMS follow-up
Voice style rules:
- Sentences under ~20 words when confirming Actions
- Read back order IDs digit by digit
- If unsure, ask one clarifying question—never two at once
Identity Across Channels
Map channel identities to your user model before calling privileged Actions:
- Slack user → employee SSO subject
- WhatsApp phone → customer account (with verification)
- Email address → ticket requester, not automatic account takeover
- Voice caller ID → soft signal only; verify with MFA or account questions
Forward the resulting session or OAuth access token into Action calls. Channel presence is not authorization.
Continuity and Handoff
People switch channels mid-journey. Persist:
- Conversation / case ID
- Verified entities (order ID, account ID)
- Pending HITL approvals
- Last successful Action outcomes
A Slack thread should be able to resume what started as a voice call without re-collecting everything—while still re-checking auth.
Channel-Appropriate Tooling
Some Actions are awkward on some channels:
- Rich Screen Observer flows belong on web embeds
- Long document review fits email or Slack, not voice
- One-tap approvals work in Slack; voice needs an operator console
Disable or rewrite tool descriptions per channel adapter if needed, without forking the entire agent.
Observability Across Channels
Tag Traces with channel=voice|whatsapp|slack|email|web so Tool Monitoring can separate issues. A spike in timeouts on voice may be ASR/TTS latency; the same Action on web might be fine.
Compliance Notes
Voice and messaging often fall under recording consent, marketing opt-in, and data residency rules. Store channel consent flags with the session and gate outbound Actions (SMS campaigns, recorded calls) accordingly.
Conclusion
Multichannel agents succeed when the brain is shared and the mouthpieces differ. Keep Actions, Environments, and governance centralized; adapt turn-taking, identity binding, and UX per channel. Users experience one capable assistant—whether they speak, tap, or type.
