Voice Dialog lets your UE5 characters listen, think, and speak in real time.

🎙️ Built for Volcengine, this plugin connects realtime ASR, Volcengine Ark LLM streaming, TTS, and Real-Time Dialogue (RTD) into one simple Blueprint-friendly workflow. Add Voice Dialog Client and Fx Audio Capture to your Actor, enter your own Volcengine API credentials, choose Pipeline A (ASR → LLM → TTS) or Pipeline B (RTD end-to-end WebSocket), configure your models and voice, then start building realtime AI voice conversations.

✨ Simple Blueprint workflow

The plugin uses Unreal Engine's AudioCapture plugin for microphone input and exposes clear events for recognized speech, assistant text deltas, sentence boundaries, dialog turn completion, PCM audio chunks, playback finished, UI state changes, and errors. It is suitable for NPC dialog, AI assistants, virtual characters, and digital human projects.

singleton MetaHumanAudioToFace ready

Realtime PCM16-LE stream events (OnAssistantAudioStreamChunk / OnAssistantAudioStreamEnded) can be connected to MetaHumanAudioToFaceRuntime, allowing generated speech to drive MetaHuman facial animation and lip-sync. Set Audio Output Mode to Blueprint Stream Only or Auto Play And Stream, then wire chunks into BeginAudioStream, FeedAudioStream, and EndAudioStream — or use the built-in URTDStreamingPcmPlayer playback API. MetaHumanAudioToFace is a separate product and is not included with this plugin.

🎚️ Configurable voices and models

Supports system prompts, dialog style (bot name, voice type, RTD model), Ark Direct Model or Agent Bot backends, new/legacy console auth modes, pipeline A/B switching, barge-in via InterruptCurrentTurn, early barge-in detection with volume ducking, and per-turn snapshots. Pipeline B offers full-duplex mic passthrough; Android builds include AEC-aware capture.

🎬 Demo video

Digital Human Demo

Note: This video shows a broader digital-human project, not the plugin’s full feature set. Voice Dialog is used in this demo only for realtime AI voice conversation — other visuals, animation, and gameplay are outside the scope of this plugin.

📘 Documentation

English: English documentation (PDF)

中文: Chinese documentation (PDF)

🔑 API key setup guide

Volcengine API Key Application Guide (Chinese PDF)

🎧 Voice & model reference

Configure voice IDs, models, and languages in the plugin settings / Blueprint. Default RTD example: zh_female_vv_jupiter_bigtts · model 1.2.1.1 (O2.0). Full voice/model catalogs are covered in the documentation above.

Requires your own Volcengine account, API credentials (ASR, TTS, Ark LLM, and/or Real-Time Dialogue), internet access, and Unreal Engine's AudioCapture plugin for microphone input. Inject secrets at runtime via the component or Blueprint — do not store keys in DefaultEngine.ini. API usage may incur separate charges from Volcengine. Android requires RECORD_AUDIO permission.