Skywork/Matrix-Game-2.0

Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model

📝 Overview

Matrix-Game-2.0（1.8B） is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion

✨ Key Features

• 🚀 Feature 1: Real-Time Distillation Efficient ​​few-step diffusion​​ for streaming video synthesis at ​​25 FPS​​, producing ​​minute-level, high-fidelity videos​​ across complex environments with ultra-fast speed.
• 🖱️ Feature 2: Precise Action Injection A ​​mouse/keyboard-to-frame​​ module that embeds user inputs as direct interactions, enabling frame-level control and dynamic response in generated videos.
• 🎬 Feature 3: Massive Interactive Data Pipeline A scalable production system for ​​Unreal Engine & GTA5​​ that generates ​​~1200 hours​​ of high-quality interactive video data, covering diverse scenes with frame-level realism.

🔥 Latest Updates

• [2025-08] 🎉 Initial release of Matrix-Game-2.0 Model

Model Overview

Matrix-Game-2.0（1.8B） is derived from the Wan. By removing the text branch and adding action modules, the model predicts next frames only from visual contents and corresponding actions.

📈 Performance Comparison

GameWorld Score Benchmark Comparison

Model	Image Quality ↑	Aesthetic Quality ↑	Temporal Cons. ↑	Motion Smooth. ↑	Keyboard Acc. ↑	Mouse Acc. ↑	Object Cons.	Scenario Cons.
Oasis	0.27	0.27	0.82	0.99	0.73	0.56	0.18	0.84
Ours	0.61	0.50	0.94	0.98	0.91	0.95	0.64	0.80

Metric Descriptions:

• Image Quality / Aesthetic: Visual fidelity and perceptual appeal of generated frames

• Temporal Consistency / Motion Smoothness: Temporal coherence and smoothness between frames

• Keyboard Accuracy / Mouse Accuracy: Accuracy in following user control signals

• Object Consistency: Geometric stability and consistency of objects over time

• Scenario Consistency: Scenario consistency over time

  Please check our GameWorld benchmark for detailed implementation.

🚀 Quick Start

# clone the repository:
git clone https://github.com/SkyworkAI/Matrix-Game.git
cd Matrix-Game/Matrix-Game-2

# install apex and FlashAttention
# Our project also depends on [FlashAttention](https://github.com/Dao-AILab/flash-attention)
# install dependencies:
pip install -r requirements.txt
python setup.py develop

# inference
python inference.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --img_path {path-to-the-input-image} \
    --output_folder outputs \
    --num_output_frames 150 \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}
# inference streaming
python inference_streaming.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --output_folder outputs \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}

⭐ Acknowledgements

We would like to express our gratitude to:

• Diffusers for their excellent diffusion model framework
• SkyReels-V2 for their strong base model
• Self-Forcing for their excellent work
• MineRL for their excellent gym framework
• Video-Pre-Training for their accurate Inverse Dynamics Model
• GameFactory for their idea of action control module

We are grateful to the broader research community for their open exploration and contributions to the field of interactive world generation.

📎 Citation

If you find this project useful, please cite our paper: