1. Motion Control & Style Transfer with Wan2.1 Fun ControlNet

Wan 2.1 Fun ControlNet is a cutting-edge AI model developed by Alibaba Pal, specifically designed for video generation by transferring instant style . It builds upon the Wan 2.1 framework and introduces two powerful models: Fun Control and Inpaint. These models enable precise motion control, style transfer across frames, and frame-by-frame enhancements all while being optimized for local PCs.

The Fun Control model takes motion cues from reference videos (like Instagram/TikTok dance clips) and mimics them in your AI-generated content. 

Meanwhile, the Inpaint model lets you modify existing video frames with pinpoint accuracy, making it perfect for refining details or adding creative touches. 

It gives you advantages like Precision Motion Control, Consistent Style Transfer on low resource usage with detailed Advanced Customization.

Installation

1. Setup the Wan model and workflow (Official by ComfyUI)if you haven’t done yet.

update comfyui manager

2. Update ComfyUI from the manager section by selecting “Update ComfyUI” option. Use “git pull” command using cmd if you found any error while updating it.

download wan fun controlnet model

3. Download the Wan1.3 billion model (diffusion_pytorch_model.safetensors) from wan’s hungging face repository. after downloading save the model inside your “ComfyUI/models/diffusion_models” folder.

Other relevant models (Clip,VAE) already present into the Comfyui if you already using the wan’s Txt To video/Image To Video workflows. So, downloading these are not required.

Kijai’s Wan setup is also one of the best alternative. Just use the Kijai’s Wan basic workflow and then download the Wan Fun variant from Kijai’s repository.

You can also download and use the Wan Fun 14B GGUF variant by City96 for faster generation but make sure you are using the GGUF setup already explained in our Wan GGUF tutorial section.

Workflow

The Wan Fun workflow (Wan2.1-fun-controlnet-workflow.json) can be downloaded from our Hugging face repository.

Tags

Related Posts