WhisperSTT brings fast, private, offline Speech To Text to Unreal Engine, built on the open source whisper.cpp inference engine. All transcription happens on device no cloud calls, no API keys, no per request cost, and no player audio ever leaves the machine.

This is offline, on device Speech To Text for Unreal Engine, powered by whisper.cpp

Documentation

Key Features

• Push To Talk recording start and stop recording on demand and receive a transcription.

• Continuous Listening : Built in voice activity detection (VAD) automatically detects when a player starts and stops speaking, with no manual start/stop required.

• Multilingual : Transcribe in any language whisper.cpp supports, auto detect the spoken language per utterance, or translate straight to English, all with a multilingual model.

• Windows and Android : Same Blueprint API on desktop and mobile, including on device microphone permission handling on Android.

• Fully Asynchronous : Transcription runs on a background thread; the game thread is never blocked.

• Fuzzy Speech Matching : Compare recognized speech against a list of expected phrases (dialogue choices, voice commands) with word level, order, and typo tolerant matching including scripts without spaces, like Chinese and Japanese.

• Blueprint First API : Every feature is exposed as a Blueprint callable function or event; no C++ required to use the plugin.

• Configurable : Model, language, translation, and voice activity detection (VAD) sensitivity are editable from Project Settings, and tunable at runtime.

Use Cases

• Voice driven dialogue systems and NPC conversations

• Hands free voice commands for accessibility or immersive gameplay

• Voice controlled UI navigation

• Multilingual games that need speech input without per request cloud costs

• Prototyping conversational AI features without a cloud dependency

How It Works

Audio is captured from the microphone and, in continuous mode, passed through a lightweight voice activity detector that watches RMS energy across consecutive frames to decide when a speech segment starts and ends. Once a segment is ready, it's resampled, silence-trimmed, and handed to whisper.cpp on a background thread pool. The result is broadcast back on the game thread through the On Speech Recognized Blueprint delegate, so gameplay code never has to worry about threading. On Android, the model is loaded from the packaged .pak through Unreal's file layer rather than a filesystem path, and thread count is capped to avoid landing work on efficiency cores.

Dependencies / Requirements

This plugin requires the following Unreal Engine plugins to be enabled:

• Audio Capture : Required for microphone audio capture and speech recognition functionality.

• Android Permission : Required when using the plugin on Android to request and manage microphone recording permission.