On-Premise Speech AI Receptionist

I need a fully local, speech-to-speech AI receptionist that can receive phone calls on Windows and or Linux machines without relying on any external cloud calls/subscriptions. The assistant must be able to listen to callers, understand their intent, speak back naturally, and—most importantly—handle two core tasks for me:

• Answer customer inquiries in real time for the serivces i provide
• Schedule appointments directly into a calendar or booking system

design considerations,
hardware requirements
as well as training and setup guidance (I will build the hardware myself and learn and understand the software design).

A smooth pipeline—speech-to-text, intent recognition, response generation, and text-to-speech—is essential; you’re free to choose the stack (e.g., Whisper, Vosk, local LLMs, Rasa, Piper-TTS, etc.) so long as everything runs offline on both Windows and Linux.

Deliverables
• Source code and install scripts for the complete local stack
• An easy way for me to add or edit FAQs and booking rules
• Demonstration of the system running on a Windows workstation and a Linux SBC/VM, showing it can field the inquiries above and book an appointment successfully

Once tested on my side, I’ll handle enclosure, mics, and speakers; you just ensure the software is sturdy, latency is low, and voices sound professional.

preffer someone who has does this before for self hosting AI gents and local LLMs
explain and provide tutorials