Description
ℹ️ Note: The images with plugin examples and the demo project were created using the Runtime Audio Importer and/or Runtime Text To Speech / Runtime AI Chatbot Integrator / Runtime Local LLM plugins. So, to follow these examples, you will need to install these plugins as well. However, you can also implement your own audio or text input solution without using them.
🗣️ Three Advanced Models for Every Project Need!
Bring your MetaHuman and custom characters to life with zero-latency, real-time lip sync! Now featuring three quality models to suit your project requirements:
🌟 Mood-Enabled Realistic Model - Emotion-aware facial animation for MetaHuman characters with 12 different moods
🌟 Realistic Model - Enhanced visual fidelity specifically for MetaHuman characters with more natural mouth movements (81 facial controls)
⚡️ Standard Model - Broad compatibility with MetaHumans and custom characters (14 visemes)
Transform your digital characters with seamless, real-time lip synchronization that works completely offline and cross-platform! Watch as your characters respond naturally to speech input, creating immersive and believable conversations with minimal setup.
Demo projects:
📘 Demo projects documentation & setup guide
AI Conversational NPC Demo (Full AI Workflow) – speech recognition + AI chatbot + TTS + lip sync, with a fully modular provider system
Requires this plugin + Runtime Audio Importer + Speech Recognizer, optional: AI Chatbot + Text To Speech + Runtime Local LLM.
Basic Lip Sync Demo – simpler workflows: microphone input, audio files, TTS
Requires this plugin + Runtime Audio Importer, optional: Text To Speech + AI Chatbot
ℹ️Standard model requires a small extension plugin (see here).
Quick links:
📌 Plugin Support & Custom Development: [email protected] (tailored solutions for teams & organizations)
Latest video tutorials:
High-Quality (Realistic Model) Tutorials:
Standard Model Tutorials:
Basic Setup:
🚀 Key features:
Real-time Lip Sync from microphone input and any other audio sources
Emotional Expression Control with 12 different moods and configurable intensity
Pixel Streaming microphone support - enable live lip sync from browser-based input!
Offline and Native Processing - no internet connection required
Cross-platform Compatibility: Windows, Mac, iOS, Linux, Android, Meta Quest
Optimized for real-time performance on all platforms
Works with both MetaHuman and custom characters (Daz Genesis 8/9, Reallusion CC3/CC4, Mixamo, FACS, ARKit, custom morph targets)
Live microphone input or Captured audio playback
Synthesized speech
From audio file/buffer
Custom audio source (including streaming)