Voice agent quickstart: Vapi + Anthropic Claude

Voice agents got real in 2026. Vapi's abstraction is clean enough that you can ship a genuinely-usable phone agent in an hour without touching WebRTC yourself.

The architecture

Twilio phone number → Vapi (WebRTC + ASR + TTS + turn-taking) → Claude Sonnet 5 (reasoning) → back to Vapi → back to Twilio

Step 1 — Provision a Vapi assistant

curl -X POST https://api.vapi.ai/assistant \
  -H 'Authorization: Bearer $VAPI_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "Support Agent",
    "model": { "provider": "anthropic", "model": "claude-sonnet-5", "systemPrompt": "You are a support agent for Acme Corp..." },
    "voice": { "provider": "elevenlabs", "voiceId": "..." },
    "transcriber": { "provider": "deepgram", "model": "nova-3", "language": "en" }
  }'

Step 2 — Attach a phone number

Vapi buys the Twilio number for you:

curl -X POST https://api.vapi.ai/phone-number \
  -H 'Authorization: Bearer $VAPI_KEY' \
  -d '{ "provider": "twilio", "assistantId": "..." }'

Ninety seconds later the number is live and answers to your assistant.

Step 3 — Add tool use

Give the assistant tools to look up an order status, book a callback, etc. Vapi's tool format matches Anthropic's — you can copy your existing Claude tool definitions.

Where this bites

• Turn-taking latency is the whole game. Deepgram Nova-3 + Claude Sonnet 5 + ElevenLabs together is ~1.2s end-of-user-speech to first-audio. That's *just* below the "feels like a bot" threshold.
• Barge-in (interrupting the bot mid-sentence) works but occasionally misfires. Add a 300ms silence threshold.
• Sonnet 5 is smart enough to hallucinate a policy your business doesn't have. Ground everything in retrieval; do not rely on the system prompt alone.