Unreal Engine 5 · C++ · PyTorch · ONNX · Multiplayer Networking
AI Multiplay Latency Predictor
An end-to-end UE5, PyTorch, ONNX, and NNE prototype for learned multiplayer vehicle latency compensation.
Problem
This project focused on multiplayer vehicle synchronization, where network latency can make fast-moving actors appear in the wrong position or orientation.
- In a racing-game-style environment, delayed network updates can create visible movement error because vehicle motion changes quickly over time.
- Constant-velocity prediction can work for simple linear motion, but it becomes less reliable during non-linear movement such as drifting and sharp turning.
- The goal was to test whether a learned movement-delta predictor could reduce visual position error inside a real-time Unreal Engine multiplayer simulation.
Approach & Implementation
I led a 3-member team and built the pipeline connecting Unreal vehicle-motion data, a learned movement predictor, and runtime latency compensation.
- Implemented Unreal C++ feature extraction for vehicle-motion logs, calculating velocity, acceleration, angular velocity, angular acceleration, velocity × delta time, and angular velocity × delta time, then exporting them as CSV training data with target position and rotation deltas.
- Trained a PyTorch DNN to predict movement deltas instead of absolute positions. The model used min-max feature scaling, residual connections, and GLU layers, and was validated with RMSE before export to ONNX.
- Integrated the ONNX model into UE5 using NNE (Neural Network Engine) and connected asynchronous inference output to the network-smoothing flow so the simulated vehicle could use predicted deltas for visual latency compensation.
Result
The project produced a UE5 prototype applying AI-based motion prediction to multiplayer latency compensation.
- Inspired by Nexon’s NDC22 talk on machine-learning-based position prediction for KartRider: Drift.
- Compared constant-velocity and DNN-based prediction under non-linear vehicle movement; the DNN path followed the actual trajectory more closely in the project’s result visualization.
- Connected Unreal-side data collection, PyTorch training, ONNX export, UE5 NNE inference, and runtime smoothing into one working prototype.