LiteRT-LM is a local, on-device LLM runtime and AI Agent plugin for Unreal Engine 5.8. Run compatible .litertlm text models on the user's hardware GPU through Blueprint or C++ without sending prompts to a cloud service. It is designed for offline AI, private NPC dialogue, Gemma-based game agents, streaming text generation, and structured tool calling.

QUICK FACTS

• Unreal Engine 5.8.

• Win64 and Android ARM64.

• Blueprint and C++ APIs.

• Local, strict GPU text inference; no silent CPU fallback.

• Streaming and non-streaming generation.

• Multiple independent Agent memories sharing one loaded model/runtime.

• No cloud account, API key, Python, or Bazel required by consumer projects.

• Compatible model files are selected by the user and are not bundled.

START HERE

1. Enable the plugin.

2. Open Project Settings > Plugins > LiteRT-LM and select a compatible .litertlm model.

3. For one conversation, use Quick Chat and call Ask or Ask Streaming.

4. For NPCs, persistent characters, or parallel histories, create one Agent per conversation and keep it alive while its memory is needed.

WHAT YOU CAN BUILD

• Offline or privacy-sensitive NPC dialogue and game AI.

• Blueprint AI assistants and local text-generation interfaces.

• Multiple Agents with independent canonical memory while sharing one model.

• Structured tool requests whose names and JSON arguments are validated and executed by your game.

• Win64 games and Android applications with verified model packaging and extraction.

RUNTIME AND API

Quick Chat is the shortest single-conversation path. Agent objects provide independent histories for multi-NPC and persistent-character scenarios. Requests share one model and are coordinated by a serial inference queue, avoiding accidental per-Agent model duplication.

The plugin includes streaming completion, cancellation and error events; token and timing metrics; runtime diagnostics; model download and verification helpers; memory import/export; Blueprint-callable APIs; public Unreal C++ headers; Native SDK access; an MCP gateway; and a stable C ABI boundary.

IMPORTANT REQUIREMENTS

• Inference is strict hardware-GPU only. Unsupported or unverified GPU paths fail explicitly.

• Text models are supported; Vision and Audio are not currently exposed.

• Android support depends on device GPU, driver, and available memory.

• The plugin includes the required third-party runtime libraries for supported targets, but not model weights.

DOCUMENTATION AND DEMO

Documentation: https://winyunq.github.io/LiteRT-LM-Unreal/

Blueprint quick start: https://winyunq.github.io/LiteRT-LM-Unreal/GUIDE_QUICK_START.html

Multiple conversations: https://winyunq.github.io/LiteRT-LM-Unreal/GUIDE_MULTIPLE_CONVERSATIONS.html

C++ guide: https://winyunq.github.io/LiteRT-LM-Unreal/GUIDE_CPP.html

Demo release: https://github.com/winyunq/LiteRTDemo/releases/tag/v5.1.0

Demo source: https://github.com/winyunq/LiteRTDemo

Support: [email protected]

LiteRT-LM Discord: https://discord.gg/3DN45KgWXj