LiveKit & Frameworks

Frameworks handle the hard parts: audio transport, turn-taking, STT/LLM/TTS orchestration, and telephony integration. LiveKit Agents is one of the leading open-source options.

LiveKit Agents

Open-source framework for building real-time voice (and video) AI agents. Handles WebRTC transport, audio routing, turn-taking, plugin system for STT/LLM/TTS, and SIP/telephony integration.

SIP/Phone

LiveKit Server

Agent Worker

STT Plugin

LLM Plugin

TTS Plugin

python

# Install LiveKit Agents SDK
pip install livekit-agents livekit-plugins-openai livekit-plugins-silero

# Minimal voice agent with LiveKit (Agents 1.0+ API)
from livekit import agents
from livekit.agents import AgentServer, AgentSession, Agent, room_io
from livekit.plugins import silero

class Assistant(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are a helpful voice AI assistant. Keep responses concise."
        )

server = AgentServer()

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: agents.JobContext):
    session = AgentSession(
        stt="deepgram/nova-3",           # Speech-to-text
        llm="openai/gpt-4.1-mini",       # Brain
        tts="cartesia/sonic-3",          # Text-to-speech
        vad=silero.VAD.load(),           # Voice activity detection
    )
    await session.start(room=ctx.room, agent=Assistant())
    await session.generate_reply(
        instructions="Greet the user and offer your assistance."
    )

if __name__ == "__main__":
    agents.cli.run_app(server)

Other Open-Source Frameworks

LiveKit isn't the only open-source option. These frameworks offer similar capabilities with different tradeoffs around architecture and ecosystem.

Vocode

Open-source library for building voice agents. Supports multiple STT/LLM/TTS backends. Good Python SDK with telephony support (Twilio, Vonage).

Pipecat

Open-source framework from Daily.co. Focuses on real-time AI pipelines. Clean abstractions for STT→LLM→TTS chains. WebRTC native.

Have a suggestion for a framework to include here? Email us at [email protected]

Commercial Voice Agent Platforms

Managed platforms that handle infrastructure, orchestration, and telephony for you. Faster to deploy but less control over the underlying stack.

These companies are not sponsors or affiliated with this training or Call Center Village. They're listed for educational awareness only.
That being said, if any of you are reading this — Call Center Village is always looking for sponsors!

Vapi

Commercial voice agent platform. Handles infrastructure, telephony, and orchestration. Quick to deploy but less control over the underlying stack.

Bland.ai

Commercial platform specifically for phone call agents. Provides phone numbers, handles scaling. Enterprise focused.

ElevenLabs Conversational AI

End-to-end voice agent platform from ElevenLabs. Combines their industry-leading TTS with built-in STT, LLM orchestration, and telephony — all managed.

Retell AI

Voice agent platform focused on production deployments. Handles orchestration, latency optimization, and telephony integration out of the box.

Voiceflow

Conversation design platform for building and deploying voice and chat agents. Visual builder with integrations for telephony and messaging channels.

Have a suggestion for a platform to include here? Email us at [email protected]

Telephony: Open-Source Infrastructure

Self-hosted telephony infrastructure for full control over your voice pipeline. These tools handle SIP signaling, media routing, and call management — all running on your own servers.

LiveKit SIP

LiveKit's built-in SIP connector. Bridges phone calls directly into LiveKit rooms.

FreeSWITCH

Open-source telephony platform. Self-hosted SIP server. Full control but complex setup.

Asterisk

The original open-source PBX. Mature, widely deployed, huge community. Powers many call centers and IVR systems.

Kamailio

High-performance open-source SIP proxy and server. Handles routing, load balancing, and SIP signaling at scale.

RTPEngine

Open-source media proxy for RTP traffic. Often paired with Kamailio to handle audio/video relay, transcoding, and NAT traversal.

OpenSIPS

Open-source SIP server focused on speed and flexibility. Used for large-scale VoIP routing, load balancing, and fraud detection.

Have a suggestion for a telephony tool to include here? Email us at [email protected]

Telephony API Providers

Cloud services that provide phone numbers, SIP trunks, and programmable voice APIs. These handle the carrier side so you don't have to manage your own telecom infrastructure.

Twilio

Most popular. Provides phone numbers, SIP trunks, and programmable voice API. Huge ecosystem and documentation.

Telnyx

Developer-focused alternative to Twilio. Competitive pricing, SIP trunk support, global coverage.

Plivo

Cloud communications platform with voice and SMS APIs. Competitive pricing and global carrier network.

Vonage

Programmable voice and SIP trunking APIs. Strong international coverage and WebRTC support.

Bandwidth

Direct carrier with APIs. Owns its own network infrastructure, which can mean lower latency and better pricing at scale.

SignalWire

Founded by the creators of FreeSWITCH. Cloud telephony with a developer-friendly API and strong open-source roots.

Have a suggestion for a telephony provider to include here? Email us at [email protected]

Check Your Understanding

Question 1 / 1

What's the main advantage of LiveKit over building a custom WebRTC solution?

Loading…