- Problem
- A barbershop or HVAC company loses the call that comes in while they are working. Voicemail is where appointments go to die.
- My role
- Designed and built the real-time voice pipeline and the calendar booking flow end to end.
- Stack
- JavaScript · Node.js · Twilio · Deepgram · GPT-4 · Cartesia · Google Calendar API
- Outcome
- Working prototype that answers a call, holds a conversation, checks availability and books the appointment into Google Calendar.
- 01
Dials the business number
- 02
Answers; streams call audio over WebSocket
- 03
Streaming speech-to-text with interim results
- 04
Decides the reply; calls calendar tools
Google Calendar API: free/busy, create event
- 05
Text-to-speech, streamed in chunks
- 06
Plays audio back to the caller
Problem
Small service businesses — barbershops, HVAC, landscaping — live on inbound calls and are usually too busy doing the work to answer them. A missed call is a missed booking, and most callers do not leave voicemail or call back.
Answering services hand over a message instead of a booked slot.
What I built
The receptionist answers, greets the caller in the business's voice, finds out what they need, offers open times, and books the appointment while they are still on the line.
Under the hood it is a streaming pipeline: Twilio delivers call audio over a media stream, Deepgram transcribes it as the caller speaks, GPT-4 decides what to say and when to call the calendar tool, Cartesia turns the reply into speech, and the audio goes back down the Twilio stream. Google Calendar is the source of truth for availability and the destination for the booking.
- Twilio Programmable Voice
- Inbound number; media streams over WebSocket
- Deepgram
- Streaming speech-to-text with interim results
- GPT-4
- Conversation policy with calendar tools: check availability, create event
- Cartesia
- Low-latency text-to-speech, streamed back in chunks
- Google Calendar API
- Free/busy lookup and event creation
- Node.js orchestrator
- One WebSocket session per call holding transcript, state and turn-taking
Technical decisions
- 01
Stream every stage.
A phone call has a latency budget of about a second before silence feels broken. Waiting for a full transcript, a full LLM reply and a full audio file would be three to five seconds. Streaming STT with interim results, starting TTS on the first sentence, and chunking audio back to Twilio keeps the round trip near the budget.
- 02
Let the model call the calendar as a tool rather than parse its text.
Availability lookups and bookings need exact dates and durations. Tool calls give structured arguments the code can validate, and the model gets the real availability back instead of guessing.
- 03
Handle barge-in.
Callers interrupt. When Deepgram returns new speech while audio is playing, the orchestrator stops the TTS stream and re-plans; without that the agent talks over people.
- 04
Fail to a human-shaped fallback.
If the model can't resolve a time, the agent takes a name and number for the owner to call back, so no call ends in a dead end.
Outcome
- Working prototype: answers, converses, checks availability and books into Google Calendar.
- Try it: call +1 (888) 360-5105 and book an appointment.
- Built in 2025; the latency and barge-in work carried directly into how I think about agent loops in Chirp.
Stack
- JavaScript
- Node.js orchestrator
- Node.js
- One WebSocket session per call holding transcript, state and turn-taking
- Twilio
- Inbound telephony and media streams
- Speech (Deepgram STT, Cartesia TTS)
- Streaming transcription in, chunked synthesis out
- GPT-4
- Conversation policy with calendar tools
- Agentic workflow design
- Tool-calling loop over availability and booking, with fallbacks
- Google Calendar API
- Free/busy lookup and event creation