Hugging Face's logo

alibaba-pai
/
Wan2.2-Fun-A14B-Control-Camera

Text-to-Video
Wan2.2
Diffusers
Safetensors
English
Chinese
video
video-generation
Model card Files
xet
Community
1
bubbliiiing commited on
Commit
de56f7f
·
1 Parent(s): 1b29a35

Update Weights

Browse files
Files changed (6) hide show
  1. README.md +62 -3
  2. README_en.md +64 -6
  3. high_noise_model/config.json +1 -1
  4. high_noise_model/diffusion_pytorch_model.safetensors +3 -0
  5. low_noise_model/config.json +1 -1
  6. low_noise_model/diffusion_pytorch_model.safetensors +3 -0
README.md CHANGED
@@ -33,8 +33,9 @@ tags:
33
 
34
  | 名称 | 存储空间 | Hugging Face | Model Scope | 描述 |
35
  |--|--|--|--|--|
36
- | Wan2.2-Fun-A14B-InP | 47.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-InP) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-InP) | Wan2.2-Fun-14B文图生视频权重，以多分辨率训练，支持首尾图预测。 |
37
- | Wan2.2-Fun-A14B-Control | 47.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control)| Wan2.2-Fun-14B视频控制权重，支持不同的控制条件，如Canny、Depth、Pose、MLSD等，同时支持使用轨迹控制。支持多分辨率（512，768，1024）的视频预测，支持多分辨率（512，768，1024）的视频预测，以81帧、每秒16帧进行训练，支持多语言预测 |
 
38
 
39
  # 视频作品
40
 
@@ -137,6 +138,65 @@ Generic Control Video (Canny, Pose, Depth, etc.) and Trajectory Control:
137
  </tr>
138
  </table>
139
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
140
  # 快速启动
141
  ### 1. 云使用: AliyunDSW/Docker
142
  #### a. 通过阿里云 DSW
@@ -287,4 +347,3 @@ webui支持文生视频、图生视频、视频生视频和普通控制生视频
287
 
288
  # 许可证
289
  本项目采用 [Apache License (Version 2.0)](https://github.com/modelscope/modelscope/blob/master/LICENSE).
290
-
 
33
 
34
  | 名称 | 存储空间 | Hugging Face | Model Scope | 描述 |
35
  |--|--|--|--|--|
36
+ | Wan2.2-Fun-A14B-InP | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-InP) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-InP) | Wan2.2-Fun-14B文图生视频权重，以多分辨率训练，支持首尾图预测。 |
37
+ | Wan2.2-Fun-A14B-Control | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control)| Wan2.2-Fun-14B视频控制权重，支持不同的控制条件，如Canny、Depth、Pose、MLSD等，同时支持使用轨迹控制。支持多分辨率（512，768，1024）的视频预测，支持多分辨率（512，768，1024）的视频预测，以81帧、每秒16帧进行训练，支持多语言预测 |
38
+ | Wan2.2-Fun-A14B-Control-Camera | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control-Camera) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control-Camera)| Wan2.2-Fun-14B相机镜头控制权重。支持多分辨率（512，768，1024）的视频预测，支持多分辨率（512，768，1024）的视频预测，以81帧、每秒16帧进行训练，支持多语言预测 |
39
 
40
  # 视频作品
41
 
 
138
  </tr>
139
  </table>
140
 
141
+ ### Wan2.2-Fun-A14B-Control-Camera
142
+
143
+ <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
144
+ <tr>
145
+ <td>
146
+ Pan Up
147
+ </td>
148
+ <td>
149
+ Pan Left
150
+ </td>
151
+ <td>
152
+ Pan Right
153
+ </td>
154
+ <td>
155
+ Zoom In
156
+ </td>
157
+ <tr>
158
+ <td>
159
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Up.mp4" width="100%" controls autoplay loop></video>
160
+ </td>
161
+ <td>
162
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Left.mp4" width="100%" controls autoplay loop></video>
163
+ </td>
164
+ <td>
165
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Right.mp4" width="100%" controls autoplay loop></video>
166
+ </td>
167
+ <td>
168
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Zoom_In.mp4" width="100%" controls autoplay loop></video>
169
+ </td>
170
+ <tr>
171
+ <td>
172
+ Pan Down
173
+ </td>
174
+ <td>
175
+ Pan Up + Pan Left
176
+ </td>
177
+ <td>
178
+ Pan Up + Pan Right
179
+ </td>
180
+ <td>
181
+ Zoom Out
182
+ </td>
183
+ <tr>
184
+ <td>
185
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Down.mp4" width="100%" controls autoplay loop></video>
186
+ </td>
187
+ <td>
188
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_UL.mp4" width="100%" controls autoplay loop></video>
189
+ </td>
190
+ <td>
191
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_UR.mp4" width="100%" controls autoplay loop></video>
192
+ </td>
193
+ <td>
194
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Zoom_Out.mp4" width="100%" controls autoplay loop></video>
195
+ </td>
196
+ </tr>
197
+ </table>
198
+
199
+
200
  # 快速启动
201
  ### 1. 云使用: AliyunDSW/Docker
202
  #### a. 通过阿里云 DSW
 
347
 
348
  # 许可证
349
  本项目采用 [Apache License (Version 2.0)](https://github.com/modelscope/modelscope/blob/master/LICENSE).
 
README_en.md CHANGED
@@ -30,12 +30,13 @@ tags:
30
  - [Reference](#reference)
31
  - [License](#license)
32
 
33
- # 模型地址
34
 
35
- | 名称 | 存储空间 | Hugging Face | Model Scope | 描述 |
36
  |--|--|--|--|--|
37
- | Wan2.2-Fun-A14B-InP | 47.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-InP) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-InP) | Wan2.2-Fun-14B text-to-video generation weights, trained at multiple resolutions, supports start-end image prediction. |
38
- | Wan2.2-Fun-A14B-Control | 47.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control)| Wan2.2-Fun-14B video control weights, supporting various control conditions such as Canny, Depth, Pose, MLSD, etc., and trajectory control. Supports multi-resolution (512, 768, 1024) video prediction at 81 frames, trained at 16 frames per second, with multilingual prediction support. |
 
39
 
40
  # Video Result
41
 
@@ -138,6 +139,64 @@ Generic Control Video (Canny, Pose, Depth, etc.) and Trajectory Control:
138
  </tr>
139
  </table>
140
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
  # Quick Start
142
  ### 1. Cloud usage: AliyunDSW/Docker
143
  #### a. From AliyunDSW
@@ -287,5 +346,4 @@ The web UI supports text-to-video, image-to-video, video-to-video, and controlle
287
  - CameraCtrl: https://github.com/hehao13/CameraCtrl
288
 
289
  # License
290
- This project is licensed under the [Apache License (Version 2.0)](https://github.com/modelscope/modelscope/blob/master/LICENSE).
291
-
 
30
  - [Reference](#reference)
31
  - [License](#license)
32
 
33
+ # Model zoo
34
 
35
+ | Name | Storage Size | Hugging Face | Model Scope | Description |
36
  |--|--|--|--|--|
37
+ | Wan2.2-Fun-A14B-InP | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-InP) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-InP) | Wan2.2-Fun-14B text-to-video generation weights, trained at multiple resolutions, supports start-end image prediction. |
38
+ | Wan2.2-Fun-A14B-Control | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control)| Wan2.2-Fun-14B video control weights, supporting various control conditions such as Canny, Depth, Pose, MLSD, etc., and trajectory control. Supports multi-resolution (512, 768, 1024) video prediction at 81 frames, trained at 16 frames per second, with multilingual prediction support. |
39
+ | Wan2.2-Fun-A14B-Control-Camera | 64.0 GB | [🤗Link](https://huggingface.co/alibaba-pai/Wan2.2-Fun-A14B-Control-Camera) | [😄Link](https://modelscope.cn/models/PAI/Wan2.2-Fun-A14B-Control-Camera)| Wan2.2-Fun-14B camera lens control weights. Supports multi-resolution (512, 768, 1024) video prediction, trained with 81 frames at 16 FPS, supports multilingual prediction. |
40
 
41
  # Video Result
42
 
 
139
  </tr>
140
  </table>
141
 
142
+ ### Wan2.2-Fun-A14B-Control-Camera
143
+
144
+ <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
145
+ <tr>
146
+ <td>
147
+ Pan Up
148
+ </td>
149
+ <td>
150
+ Pan Left
151
+ </td>
152
+ <td>
153
+ Pan Right
154
+ </td>
155
+ <td>
156
+ Zoom In
157
+ </td>
158
+ <tr>
159
+ <td>
160
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Up.mp4" width="100%" controls autoplay loop></video>
161
+ </td>
162
+ <td>
163
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Left.mp4" width="100%" controls autoplay loop></video>
164
+ </td>
165
+ <td>
166
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Right.mp4" width="100%" controls autoplay loop></video>
167
+ </td>
168
+ <td>
169
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Zoom_In.mp4" width="100%" controls autoplay loop></video>
170
+ </td>
171
+ <tr>
172
+ <td>
173
+ Pan Down
174
+ </td>
175
+ <td>
176
+ Pan Up + Pan Left
177
+ </td>
178
+ <td>
179
+ Pan Up + Pan Right
180
+ </td>
181
+ <td>
182
+ Zoom Out
183
+ </td>
184
+ <tr>
185
+ <td>
186
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_Down.mp4" width="100%" controls autoplay loop></video>
187
+ </td>
188
+ <td>
189
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_UL.mp4" width="100%" controls autoplay loop></video>
190
+ </td>
191
+ <td>
192
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Pan_UR.mp4" width="100%" controls autoplay loop></video>
193
+ </td>
194
+ <td>
195
+ <video src="https://pai-aigc-photog.oss-cn-hangzhou.aliyuncs.com/wan_fun/asset_Wan2_2/v1.0/Zoom_Out.mp4" width="100%" controls autoplay loop></video>
196
+ </td>
197
+ </tr>
198
+ </table>
199
+
200
  # Quick Start
201
  ### 1. Cloud usage: AliyunDSW/Docker
202
  #### a. From AliyunDSW
 
346
  - CameraCtrl: https://github.com/hehao13/CameraCtrl
347
 
348
  # License
349
+ This project is licensed under the [Apache License (Version 2.0)](https://github.com/modelscope/modelscope/blob/master/LICENSE).
 
high_noise_model/config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "_class_name": "Wan2_2Transformer3DModel",
3
  "_diffusers_version": "0.32.2",
4
- "add_control_adapter": false,
5
  "add_ref_conv": false,
6
  "cross_attn_norm": true,
7
  "cross_attn_type": "cross_attn",
 
1
  {
2
  "_class_name": "Wan2_2Transformer3DModel",
3
  "_diffusers_version": "0.32.2",
4
+ "add_control_adapter": true,
5
  "add_ref_conv": false,
6
  "cross_attn_norm": true,
7
  "cross_attn_type": "cross_attn",
high_noise_model/diffusion_pytorch_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b242a310d161460d3ecf8430d4f8ee01398bebc3925a479c267b71318e8e30ad
3
+ size 29584580312
low_noise_model/config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "_class_name": "Wan2_2Transformer3DModel",
3
  "_diffusers_version": "0.32.2",
4
- "add_control_adapter": false,
5
  "add_ref_conv": false,
6
  "cross_attn_norm": true,
7
  "cross_attn_type": "cross_attn",
 
1
  {
2
  "_class_name": "Wan2_2Transformer3DModel",
3
  "_diffusers_version": "0.32.2",
4
+ "add_control_adapter": true,
5
  "add_ref_conv": false,
6
  "cross_attn_norm": true,
7
  "cross_attn_type": "cross_attn",
low_noise_model/diffusion_pytorch_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:277a7b0374f9d0d5ba0e769fa9648687301cfff8b32d32c9b57d122dbea24c02
3
+ size 29584580312