Artificial Intelligence · Architecture · WebRTC

FreeSWITCH as a Dumb Switch: A Database-Driven, ESL-Controlled Architecture for AI Phone Platforms

By 海边捡石子 · Oct 21, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

A team that bolts an AI bot into the dialplan will rewrite call code when switching ASR vendors, edit XML to add extensions, drop calls during handoffs, and mismatch CDRs. This architecture keeps FreeSWITCH stable by pushing configuration into a database, routing decisions into an ESL program, and AI behind one protocol, which is the difference between a demo and a phone system you can operate.

Summary

FreeSWITCH handles only signaling, media, transcoding, and command execution in this design. All configuration moves into a database served through mod_xml_curl; routes and extensions apply on save, while gateways and ACLs need an explicit publish step. An ESL control engine parks every call first and routes it by number, schedule, and agent state, so inbound, outbound, and transfers share one state machine. Events for a single call are hashed to one worker to preserve ordering, and Redis state carries TTLs so a disconnected ESL can't leave permanent dirty data.

AI connects through a gateway that turns call audio into a unified WebSocket protocol. Vendors can integrate as full bots that bring their own VAD, ASR, LLM, and TTS, or as component adapters that the platform orchestrates. The protocol handles session start/end, PCM audio, DTMF, playback-finished events, transcripts, cancellation via uuid_break, and transfer/hangup actions. Community-edition audio streaming is store-then-play, so the recommended MVP pattern is short sentence-by-sentence playback; end-to-end latency is estimated at 1-3 seconds.

For browser agents, the platform uses SIP over WSS with SIP.js or JsSIP rather than mod_verto, temporary SIP passwords tied to the login session, and a second business WebSocket for agent state. An inbound call is assigned only when SIP registration, business connection, and idle state all hold, with screen pop before ringing. The CDR section distinguishes duration from billsec and warns that on inbound calls the customer leg's billsec includes IVR/bot/queue time. The security checklist starts with FreeSWITCH's default demo extensions 1000-1019 and password 1234, which scanners will find on a public IP.

Takeaways

— FreeSWITCH is scoped to signaling, media, transcoding, and executing commands; routing and business decisions live outside it.

— mod_xml_curl pulls directory, dialplan, and configuration from an HTTP service; missing records must return <result status="not found"/> instead of empty content.

— Routes and extensions apply immediately on save; gateways and ACLs require an explicit publish step using sofia profile external rescan, killgw, or reloadacl.

— Every call parks first, and an ESL control engine routes it after CHANNEL_PARK; park_timeout (20 seconds in the example) prevents stuck channels if the engine fails.

— Call events are hashed by call_id into N fixed worker queues so one call is always processed serially; if queues are bounded, overflow applies backpressure or drops non-critical events.

— Redis call state must carry a TTL; after ESL reconnect, the engine reconciles with show channels and writes an abnormal-end CDR for state FreeSWITCH no longer has.

— Audio reaches AI through mod_audio_stream as 8k mono PCM over WebSocket; the community edition plays back server-returned audio from a file, so true bidirectional streaming is commercial-only.

— The AI gateway protocol includes session.start/session.end, binary PCM, dtmf, playback.finished, response.audio, transcript.final, response.cancel, action.transfer, and action.hangup.

— Interruption triggers uuid_break when user speech lasts 200-400ms (tunable) or non-empty text is recognized; backchannels like 'uh-huh' can be configured not to interrupt.

— End-to-end latency from end-of-speech detection through ASR, LLM first token, and TTS first packet is roughly 1-3 seconds; using an 8k phone ASR model helps because the line is narrowband.

— Browser agents use SIP over WSS with SIP.js or JsSIP, a temporary SIP password stored in Redis for the login session, and a separate business WebSocket for agent state; calls are assigned only when SIP registration, business connection, and idle status all hold.

— duration is from call start to end including ringing; billsec is from answer to end. On inbound calls, the customer leg's billsec includes IVR/bot/queue time, so agent talk time must come from the agent leg.

— FreeSWITCH default config creates demo extensions 1000-1019 with password 1234; on a public IP these get scanned quickly, so the checklist includes changing passwords, deleting static demo users, restricting international dialing, and keeping 30 days of consent recordings for outbound calls.

Conclusions

Treating FreeSWITCH as a dumb executor is what makes the rest of the design work: once configuration and decisions leave the dialplan, changing an ASR vendor or adding an agent becomes a data or API change, not a telephony change.

Hashing every call event to one worker is a deliberate trade: it sacrifices parallelism within a call to guarantee ordering, which matters more in telephony than throughput.

The community-edition audio limitation shapes the whole AI interface: because streaming playback isn't available, the protocol optimizes for short, sentence-level audio turns and prompt cancellation rather than full-duplex conversation.

The security section is unusually concrete because FreeSWITCH's default demo users are a known attack surface; a public IP plus password 1234 means scanners can register and make paid outbound calls before operators notice.

Concepts & terms

ESL (Event Socket Library)

FreeSWITCH's control interface. An inbound ESL connection streams call events and accepts commands, so an external program can own routing and call decisions instead of putting them in the dialplan.

mod_xml_curl

A FreeSWITCH module that fetches directory, dialplan, and configuration via HTTP requests to a service, replacing static XML files with database-driven configuration.

park

A FreeSWITCH application that holds a call in place. This design parks every call and lets an ESL control engine decide whether to route to a bot, IVR, or human agent.

mod_audio_stream

A FreeSWITCH module that pushes call audio as PCM frames over WebSocket to an external service, commonly an AI bot; in the community edition the bot's audio response is returned as base64 and written to a file before playback.

uuid_break

A FreeSWITCH command that interrupts media operations on a channel. It is used to stop TTS playback immediately when the caller starts speaking.

billsec vs. duration

In FreeSWITCH CDRs, duration is the total time from call initiation to hangup, including ringing, while billsec is the billed time from answer to hangup. On inbound calls, the customer leg's billsec includes IVR, bot, and queue time.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗