|
|
--- |
|
|
library_name: diffusers |
|
|
license: mit |
|
|
pipeline_tag: text-to-video |
|
|
tags: |
|
|
- video-generation |
|
|
--- |
|
|
|
|
|
# DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video Generation |
|
|
|
|
|
This repository hosts the Dual-Expert Consistency Model (DCM) as presented in the paper [Dual-Expert Consistency Model for Efficient and High-Quality Video Generation](https://huggingface.co/papers/2506.03123). DCM addresses the challenge of applying Consistency Models to video diffusion, which often leads to temporal inconsistency and loss of detail. By using a dual-expert approach, DCM achieves state-of-the-art visual quality with significantly reduced sampling steps. |
|
|
|
|
|
For more information, please refer to the project's [Github repository](https://github.com/Vchitect/DCM). |
|
|
|
|
|
## Usage |
|
|
|
|
|
You can use this model with the `diffusers` library. Make sure you have `diffusers`, `transformers`, `torch`, `accelerate`, and `imageio` (with `imageio-ffmpeg` for MP4/GIF saving) installed. |
|
|
|
|
|
```bash |
|
|
pip install diffusers transformers torch accelerate imageio[ffmpeg] |
|
|
``` |
|
|
|
|
|
Here is a quick example to generate a video: |
|
|
|
|
|
```python |
|
|
from diffusers import DiffusionPipeline |
|
|
import torch |
|
|
import imageio |
|
|
|
|
|
# Load the pipeline |
|
|
# The custom_pipeline argument is necessary because the pipeline class (WanPipeline) |
|
|
# is defined within the repository and not part of the standard diffusers library. |
|
|
pipe = DiffusionPipeline.from_pretrained("Vchitect/DCM", torch_dtype=torch.float16, custom_pipeline="Vchitect/DCM", trust_remote_code=True) |
|
|
pipe.to("cuda") |
|
|
|
|
|
# Define the prompt and generation parameters |
|
|
prompt = "A futuristic car driving through a neon-lit city at night" |
|
|
generator = torch.Generator(device="cuda").manual_seed(0) # for reproducibility |
|
|
|
|
|
# Generate video frames |
|
|
video_frames = pipe( |
|
|
prompt=prompt, |
|
|
num_frames=16, # number of frames to generate |
|
|
num_inference_steps=4, # DCM excels at efficient generation in few steps |
|
|
guidance_scale=7.5, # Classifier-free guidance scale |
|
|
generator=generator, |
|
|
).frames[0] # Assuming the output is a list containing one video (list of frames) |
|
|
|
|
|
# Save the generated video |
|
|
output_path = "generated_video.gif" # You can change this to .mp4 if imageio[ffmpeg] is properly set up |
|
|
imageio.mimsave(output_path, video_frames, fps=8) # frames per second |
|
|
print(f"Video saved to {output_path}") |
|
|
``` |
