Techniques That Actually Improved Accuracy When Running Faster-Whisper as STT for Real-Time Voice AI (Avatars/Voicebots)
While running faster-whisper as the STT component in a real-time voice interaction system (avatar/voicebot), I’ve compiled practical accuracy-improvement techniques that showed real-world effectiveness. These are not batch-style single-shot transcriptions, but insights specifically for integrating into conversational systems.
1. initial_prompt — The Most Effective Yet Most Misused Feature
Whisper’s initial_prompt acts as a lexical bias for the decoder by providing the "transcription that came just before this audio." When speech is ambiguous, the model tends to "fall back" to spellings present in the prompt. This is the most powerful countermeasure against misrecognizing proper nouns (e.g., "社員数" → "シャインズ").
Common Mistake: Passing a Dictionary of Proper Nouns
