ComfyUI: A Comprehensive Guide from Novice to Expert

We will cover all the basic nodes that are widely used by the community for art generation in detail with examples. While moving to the basic first install ComfyUI into your machine.

1. CheckPointLoader: This is one of the common nodes. Adds this by right-clicking on canvas then

Alternatively, you can do this using the search option by left double click on Canvas, search “checkpoint” and selecting “Load checkpoint” option provided.

This node gets an input in the form of a checkpoint. The model checkpoints get stored inside the “ComfyUI_windows_portablemodelscheckpoints” folder. So, whenever you want to use

any Stable Diffusion base models you should always store that into that directory.

It has three outputs i.e. “Clip“, “VAE” and “Model” which we discuss further.

Now, here sometimes you get a checkpoint with “deprecated” which means it’s no longer used by the ComfyUI.

2. Primitive Node: To create this node, right click on checkpoint loader> Click ckpt_name to input > Drag this out > Add Node>Utils>Primitive

Basically, this node name influences the value of the checkpoint you have currently selected.

The “Control Auto Generate” option is used to change the checkpoints multiple times automatically whenever you generate an art. However, it’s experimental and not used in normal use cases.

3. CheckPointLoader(Model output): Generally, it goes to the KSampler node. To connect to Ksmapler, just click and drag to the canvas area to open a dropdown menu where you can select “KSampler“.

4. CheckPointLoader(Clip): Clip(Contrastive Language-Image Pre-Training) gets connected to the CLIP Text Encode (Prompt) box where you have to put your positive /negative prompt.

Actually, Clip takes a positive/negative input and using the Tokenization technique breaks it into multiple tokens which are again converted into numbers(in machine learning it’s called Conditioning) because machines cannot understand words so it process in only numbers.

For example: If you input “a man” then this gets converted into “1 boy” This is what Tokenization is.

5. CheckPointLoader(VAE): VAE(Variational Auto Encoder) connect to the VAE Decode box.

Now, this is optional -You can also load individual nodes by double left-clicking on canvas for the Load VAE, Load Clip, and UNET Loader which actually combine to form “Load checkpoint“.

So, whenever you try to load your desired Stable Diffusion models in the “.safetensors” or “.ckpt” extension these need to be loaded on the “Load checkpoint” node. And these models usually comprise UNET, Clip, and VAE.

This means the load checkpoints are responsible for de-assembling all three functions (Unet, Clip, VAE) for the workflow pipeline.

Let’s say we have any image in pixel form (255,255,6) in RGB formation values. but, here the situation is machines can’t understand the images like the human eyes do.

So, the image pixel values get converted to binary numbers. Now in the machine learning world, when we train a model there is a difference between what we expect and what the result is. Actually here

the human can differentiate with the image gesture, quality, etc. but the model uses a percentage of Loss.

Usually, this is the way to train any machine learning model. However, if we talk about these image models this is not the optimal way to train it. So, StabilityAI has worked on a solution called Diffusion where an image that is in pixel form gets converted into Latent form or vice-versa using the VAE (Encoder/Decoder).

Here, one thing we need to mention is in the process of conversion from pixel to Latent or vice versa it loses the minute amount of image data. So, for example you are working on a big project and there are lots of nodes connected to each other you will lose a lot of information during these conversions.

So, you have to balance between what your requirement is and how many conversions you want to add for getting the optimal result.

6. Ksampler: The Ksampler has various inputs with a single output. Actually, this is responsible for the sampling process with a bunch of steps. With each step, the denoising of the image gets performed, and each time we get a better image from the earlier one.

Now, we will discuss a lot of options in Ksampler. All has been explained in a detailed simple manner:-

• Seed: It’s normally the initial point where the random value is generated for any particular generated image. After the first generation, if you set its randomness to fixed, the model will generate the same style of image. This is one of the techniques to achieve consistency. Its default value is 0. The minimum value is 0 and the maximum value is in hexadecimal(0xffffffffffffffff) in actual its 1844674407370955. You must set your seed value between this specified range only.

• Steps: It refers to the inference steps means the number of steps the diffusion mechanism needs to generate an intermediate latent image sample processed in latent space. In each step, the image is processed from noising and denoising attempts to get the perfect output. The default is 20, the minimum value is 0 and the maximum is 10000.

• CFG: CFG(Classifier Free Guidance scale) It takes the float(decimal) values. Its default value is 8.0, minimum is 0.0 and  100.0 is the maximum value. It basically helps the diffusion models to follow and adds influence with the prompts. But, if you go further higher values you will get a totally different image perspective. So, usually closer to 7-8 is best to get the demanding results.

• Sampler Name: It’s the type of algorithm for noising and denoising the latent image to achieve optimal performance. Many of the popular algorithms are DPM++, Euler, Euler A, etc.

Sampler Type	Relative Speed
Euler	Fast
Euler a	Fast
Heun	Medium
LMS	Fast
LMS Karras	Fast
DDIM	Fast
PLMS	Fast
DPM2	Medium
DPM2 a	Medium
DPM2 Karras	Medium
DPM2 a Karras	Medium
DPM++ 2S a	Medium
DPM++ 2S a Karras	Medium
DPM++ 2M	Fast
DPM++ 2M Karras	Fast
DPM++ SDE	Medium
DPM++ SDE Karras	Medium
DPM fast	Fast
DPM adaptive	Slow
UniPC	Fast

• Scheduler: It’s the Ksampler’s Scheduler for scheduling techniques.

• Positive conditioning: The positive prompt we used to generate AI Art.

• Negative conditioning: It’s the negative prompt that we want don’t want in Image generation.

• Noise Scheduler: It generally controls how much noise you have in the image it should be in each step.

• Denoise factor: This is basically used when you are working in an “image-to-image” workflow. So, during conversion from latent to pixel and vice-versa it detects how much percentage we want to keep and change. If the denoising factor is 1 means 100% we don’t want to keep any image but when we give 0.5 means 50% we want to change our image with 50% of as it is.

7. Ksampler Advanced: This node is rarely used for depth workflow where we need many sampling steps with a diffusion latency mechanism. For example: with SDXL1.0 base models we use the refiners model as well.

Here, we need to use first Ksampler for loading SDXL1.0 base models and the later one with the refiner model.

Its to take in mind that every diffusion model behaves different with its sampler method, its steps, CFG scale, positive, negative conditionings etc.

So, you need to take it into consideration while downloading any diffusion models from Hugging Face or CivitAI. To gather more relevant information you should check their description section to leverage the maximum potential of the required model.

We explained the basic workflow and nodes that are normally used to generate images in the text-to-image process. You can look further and download extra workflows from our Hugging Face repository where we share multiple workflows with different nodes enabled.