Installing Control Net in Stable Diffusion 1.5: A Step-by-Step Guide

 complete guide to control net

So, here the waiting is over because a new model released
officially by Stability Ai which is a changing moment for the image
generation community.

We have experienced and used a couple of months, including the released Control Net which gives an extra edge over others and helps much for AI lovers in the field of image generation.

Features:

  • Supported for Stable Diffusion 1.5
  • For Experimental and Production ready
  • Supports Automatic1111 and ComfyUI
  • Easy plug and play

Install ControlNet in Automatic1111:

1. First of all you need to download and install the Controlnet models from Hugging Face

Download ControlNet from hugging face

2. Now, it will be helpful if you download each model available on the control net from the Hugging Face space platform. This helps us to work on every aspect of the control net and we can enjoy the full advantage of the Controlnet. But this is optional and depends on your requirements.

controlnet files download

3. To download it you need to go to their hugging face space. Here, the files with .pth extensions are the models and the similar name files with .yaml extensions are the respective model’s configurations. 

https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main

 If you head over the control net models of Hugging Face, you will see files that end with the letter “e” which means these are used for experimental purposes and the other control net models with the letter “p” are for production-ready which can be deployed on the server for standalone applications and files with “sd15” denote that these are the Stable Diffusion 1.5 version base model.

save controlnet files

 4. After downloading these, you can move the downloaded files into the respective path folder for working.

5. Move to your stable diffusion root folder.

stable-diffusion-webuiwebuiextensionssd-webui-controlnetmodels

6. Save your files inside the “models” folder.

restart automatic1111 to load controlnet

7. Now go back to Automatic1111 and click on the “Refresh” button to take effect and let all the models load on the Automatic1111 control net section.

Controlnet installed

8. Enjoy the power of ControlNet in image generation. 

Install ControlNet in ComfyUI:


comfyui manager github repo

0. First of all, we see how to install the manager tab which we usually see on ComfyUI presented over the right panel. For installing the ComfyUI Manager go to the github repo link. and just click the copy button to copy the cloning link.

clone repo using command prompt

1. Open the Command prompt by typing Window key + R and copy the following command to clone the respective repository:

git clone https://github.com/ltdrdata/ComfyUI-Manager.git

comfyui installation folder

If you get zip downloaded files, make sure you use a zip file extractor(like 7zip or WinRar software) to unzip the downloaded files. Now after cloning, just copy and paste the downloaded folder to the ComfyUI installation root folder. If you have the portable version, you have “ComfyUI-windows-portable” folder.

ComfyUI ControlNet Manager tab

2. After installing, just restart the ComfyUI to see the change effect. You will see a MANAGER tab like this presented in the above image. Click on the Manager tab.

ComfyUI ControlNet  Install nodes

3. A new small pop-up with multiple options will appear. Just click on Install Custom Nodes.

ComfyUI ControlNet Install preprocessors

4. Again a new window will appear. Now on the rightmost side of the window, you will a search box. Just search for “CONTROL NET” and click the Search button which will be available in the top right corner. You will see various preprocessors related to Control Net.

5. Now click the Install button presented over to the right of the dashboard to install all the Control net-related preprocessors.

6. Now,  done. Just close by click the Close button and Restart the Comyfy UI because it doesn’t show the changes automatically.

ComfyUI ControlNet Installed preprocessors access

7. Now open it again and right click on the dashboard of the Comfy UI and move to the following option:    

Add Node > Control Net PreProcessors >

 So Inside this option, you will get access to all the preprocessors that we have installed recently.

clone hugging face controlnet lora

8. Now, move to the Hugging Face portal and install the Stable diffusion XL version model released by Stability AI. Here, select the link which is for Control – Net LORA (But actually this is not the LORA that you know in the field of stable diffusion). 

These are small and are more memory efficient.  You have to clone the repository.

comfyui installation folder

9. Place the downloaded file into the ComfyUI/models/ControlNet folder.


after putting all folder

10. Here is the result after putting all the downloaded files into the ComfyUI installation folder.

Comfy UI controlnet feature

Now let’s see what these Control Net Preprocessors
can do.

If you are loading the recently installed preprocessors for the first time, this will take some time in the background to download some required files which need to run the preprocessors.

Comfy UI controlnet feature

You can use some features to add some
depth or sharpness to the images as well.

Conclusion: 

Control net is the most effective model that we can use as an extension on ComfyUI for making the image more depth, open pose, face swapping, consistent image, etc. It works on the concept of neural network mechanisms for pre-trained diffusion models.