Case study, prototype

Reception System

An AI phone handler prototype for small businesses. Answers calls, extracts intent, books appointments, using a fine-tuned LLM that can run locally with no per-call cloud fees. View the code

Python FastAPI Faster Whisper spaCy NER Phi-3 / Llama 3.2 (QLoRA) Piper TTS Twilio UWE Enterprise Scholarship
Prototype
No live client deployment
Local
Fine-tuned LLM inference
111
Automated test checks

The Problem

Small clinics and SMEs miss calls constantly: receptionists are expensive, call-centre services are impersonal, and most AI phone tools run on a cloud subscription that doesn't suit businesses handling sensitive data. A dental clinic can't route patient calls through a third-party cloud LLM without GDPR implications.

The ask: answer calls, extract the reason, and book the appointment, with a model that can run on hardware the business already owns, no data leaving the building at inference time.

Architecture

1
Inbound call
Twilio captures the call and streams audio to the backend
2
Faster-Whisper
CPU-optimised STT transcribes the caller in real time
3
spaCy NER
Extracts DATE, TIME, SERVICE, PERSON entities from the transcript
4
Fine-tuned LLM
Phi-3 mini / Llama 3.2 3B, QLoRA fine-tuned on 600 synthetic call samples, outputs a structured action
5
Pydantic schema
Validates the action as one of 6 typed cases before anything is confirmed
6
Piper TTS
Reads the booked slot back to the caller

Key Challenges

Outcome

A working prototype, not a live deployment: no paying client yet. Validated with 111 automated pipeline checks (booking, cancellations, edge cases, profanity handling) and benchmarked on both GPU (fine-tuned, 480 synthetic samples) and CPU-only (quantised, no GPU at inference) conditions. A separate app is currently collecting real caller audio to test against genuine speech before any commercial pilot.

Built under the UWE Enterprise Scholarship (£1,000 award), part of an MSc group project. Plan is to productise the online version as a SaaS for SMEs; the fully offline variant likely won't fit that subscription model given its zero-cloud design. Full code, eval results and test suite on GitHub

Back to workNext: dissertation