Overview

Unreal AI Integration Platform (UAIP) is a plugin that exposes the Unreal Engine Editor and Runtime as a structured API for AI agents to drive via the Model Context Protocol (MCP). AI tools like Claude Code, Codex CLI, Cursor, Windsurf, and GitHub Copilot can connect and issue semantic commands — no coordinate clicks, no brittle UI scripting.

UAIP makes it possible for an AI agent to:

- Observe the editor through screenshots and JSON state dumps

- Act on assets, Blueprints, materials, levels, sequences, and more through 540+ semantic commands

- Verify its own work by reading captures back, closing the "edit → run → check" loop without human intervention

- Execute ordered batches of commands as scenarios with retries, timeouts, and value splicing between steps

Key Features

- 540+ native semantic commands spanning Editor, Runtime, and Observation domains

- 190+ bridges to UE 5.8's official Toolset — combined ~730+ commands accessible from one API

- Visual & structural observation — viewport captures, Slate widget tree dumps, world state, editor state, returned as consistent artifacts

- Scenario execution — submit ordered command batches with per-step timeout / retry / abort policy and value splicing between steps

- Multi-transport — MCP, HTTP, WebSocket, and CLI all hit the same dispatcher

- Per-session Capability gating + process-wide SafetyPolicy for fine-grained access control

- Crash recovery and automatic editor lifecycle management

- Self-verifying mutation loops — the AI reads its own screenshots back through the MCP Read tool to confirm what it just did

- Cross-asset graph editing — Blueprint / Material / Niagara / AnimBlueprint / ControlRig / Sequencer / PCG / MetaSound / BehaviorTree / StateTree / Dataflow / EQS / CommonConversation

Supported AI Agents

- Claude Code (Anthropic)

- Codex CLI (OpenAI)

- Cursor

- Windsurf

- GitHub Copilot

- Any MCP-compatible client

Use Cases

- Agent-driven editor automation — let the AI build levels, edit Blueprints, set up materials, and place sequencer cues from natural language

- Self-verifying mutation loops — mutate → capture → describe → adjust without human approval at every step

- Automation Test execution — run UE Automation Tests and analyze results from the AI side

- PIE soak-test scenarios — automated playtests with checkpoint captures

- AI-assisted prototyping — rapidly iterate on level layouts, character setups, and gameplay logic with the AI as a pair programmer

- Documentation and knowledge extraction — dump editor state, world state, and asset structures for AI analysis

Relationship to UE 5.8 Toolset

UE 5.8 introduced the official Toolset integration. UAIP includes 190+ bridges to Toolset — anything Epic exposes through Toolset is callable from UAIP. On top of that, UAIP adds 540+ native commands covering ground Toolset doesn't reach: cross-asset graph editing, structural dumps (Slate tree, world state), scenario execution, PIE lifecycle, and crash recovery. UAIP also supports UE 5.7, which is not covered by Toolset.

Safety & Governance

UAIP applies per-command authorization in three layers:

1. Per-session Capability — each command declares required capabilities, granted per session

2. Process-wide SafetyPolicy — coarse switches for read-only mode, denied commands, log dump permission, keyboard input permission

3. Route-specific opt-in — scenario execution, HTTP transport, and other routes can be enabled or disabled independently

This lets you let an AI agent loose on a real project without watching every step.

Architecture

- All commands return a consistent response shape: `{Success, Data, Artifacts, ErrorCode?, ErrorMessage?}`

- Artifacts (PNG / JSON / Log / Report / Bundle) are stored per-session and accessed via stable IDs

- Capability and SafetyPolicy are evaluated before dispatch; rejected calls return clear error codes

- The MCP bridge handles editor lifecycle, crash recovery, and artifact streaming automatically

What's Included

- UAIP plugin (Source + Binaries) for UE 5.7 and 5.8

- 540+ semantic commands + 190+ Toolset bridges

- Comprehensive English and Japanese documentation

- HTTP / WebSocket / CLI transports for non-MCP integrations

- Capability and SafetyPolicy configuration via `DefaultUAIP.ini`

- Sample scenarios and use case examples in the documentation

Compatibility

- Unreal Engine: 5.7 and 5.8

- Platform: Windows (Win64) only for 1.0. Linux/macOS planned for future versions.

- Python 3.10+ (only when using the separately-distributed MCP Bridge)

Documentation & Support

- Official Trailer: https://youtu.be/o-33jgYLF0A

- Full documentation (English / Japanese): https://github.com/Naotsun19B/UnrealAIIntegrationPlatform-Document

- GitHub repository (issues): https://github.com/Naotsun19B/UnrealAIIntegrationPlatform-Document/issues

- Demo build (feature-limited, free): https://github.com/Naotsun19B/UnrealAIIntegrationPlatform-Document/releases?q=Demo

- MCP Bridge (free, MIT, required for MCP integration): https://github.com/Naotsun19B/UnrealAIIntegrationPlatform-Document/releases?q=MCPBridge

A free demo build is also available on GitHub Releases. It covers observation, PIE control, UI automation, and scenario execution (about 60 commands) and is the easiest way to evaluate UAIP before purchase. Captures in the demo build carry a "UAIP Demo" watermark.