license: mit
language:
- en
pipeline_tag: image-to-video
library_name: diffusers
base_model:
- Skywork/SkyReels-V2-I2V-1.3B-540P
Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model
📝 Overview
Matrix-Game-2.0（1.8B） is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion
✨ Key Features
- 🚀 Feature 1: Real-Time Distillation Efficient few-step diffusion for streaming video synthesis at 25 FPS, producing minute-level, high-fidelity videos across complex environments with ultra-fast speed.
- 🖱️ Feature 2: Precise Action Injection A mouse/keyboard-to-frame module that embeds user inputs as direct interactions, enabling frame-level control and dynamic response in generated videos.
- 🎬 Feature 3: Massive Interactive Data Pipeline A scalable production system for Unreal Engine & GTA5 that generates ~1350 hours of high-quality interactive video data, covering diverse scenes with frame-level realism.
🔥 Latest Updates
- [2025-08] 🎉 Initial release of Matrix-Game-2.0 Model
Model Overview
Matrix-Game-2.0（1.8B） is derived from the Wan. By removing the text branch and adding action modules, the model predicts next frames only from visual contents and corresponding actions.
📈 Performance Comparison
GameWorld Score Benchmark Comparison
| Model | Image Quality ↑ | Aesthetic Quality ↑ | Temporal Cons. ↑ | Motion Smooth. ↑ | Keyboard Acc. ↑ | Mouse Acc. ↑ | Object Cons. | Scenario Cons. |
|---|---|---|---|---|---|---|---|---|
| Oasis | 0.27 | 0.27 | 0.82 | 0.99 | 0.73 | 0.56 | 0.18 | 0.84 |
| Ours | 0.61 | 0.50 | 0.94 | 0.98 | 0.91 | 0.95 | 0.64 | 0.80 |
Metric Descriptions:
Image Quality / Aesthetic: Visual fidelity and perceptual appeal of generated frames
Temporal Consistency / Motion Smoothness: Temporal coherence and smoothness between frames
Keyboard Accuracy / Mouse Accuracy: Accuracy in following user control signals
Object Consistency: Geometric stability and consistency of objects over time
Scenario Consistency: Scenario consistency over time
Please check our GameWorld benchmark for detailed implementation.
🚀 Quick Start
# clone the repository:
git clone xxx
cd Matrix-Game-2.0
# install dependencies:
pip install -r requirements.txt
# inference
bash xxx.sh
⭐ Acknowledgements
We would like to express our gratitude to:
- Diffusers for their excellent diffusion model framework
- SkyReels-V2 for their strong base model
- Self-Forcing for their excellent work
- MineRL for their excellent gym framework
- Video-Pre-Training for their accurate Inverse Dynamics Model
- GameFactory for their idea of action control module
We are grateful to the broader research community for their open exploration and contributions to the field of interactive world generation.
📎 Citation
If you find this project useful, please cite our paper:
xxx