AI agents in production
ChatsX: one inbox for Facebook, Instagram and WhatsApp, with agents on the hot path.
A unified inbox built on Meta Graph API webhooks, sustaining 10,000 requests per second at peak and 50,000 concurrent WebSocket connections, with LLM agents doing auto response, conversation summarisation and intent routing.
The backend under load
The Django, DRF and Go core sustains 10,000 requests per second at peak. p95 latency came down from 900ms to 220ms through query rewrites, multi layer caching and PgBouncer connection pooling.
Underneath it, fault tolerant async pipelines process 300,000 background jobs with dead letter queues, idempotent retries, priority routing and admin facing observability.
The realtime rebuild
The WebSocket layer was re architected on Django Channels and Daphne to carry 50,000 concurrent connections. Response time went from around 5,000ms to around 100ms, a 96 percent cut.
Managed cloud load balancing and autoscaling were replaced with a self built Caddy stack: a custom load balancer, autoscaler and process manager. Lower DevOps spend, no vendor lock in, and deploys stayed zero downtime.
Agents that actually run in production
Planner, executor and reviewer graphs with tool calling, structured outputs and eval harnesses. Built on LangGraph and CrewAI against the Claude and OpenAI APIs, automating support triage end to end.
- Auto response drafted and sent on high confidence intents
- Conversation summarisation so an operator picks up mid thread
- Intent routing with a confidence score attached to every classification
- Eval harnesses, so a prompt change is measured rather than eyeballed
Stack