Speed Up Image Generation with SDXL Lightning

The all new improved model  Stable Diffusion XL (SDXL) lightning version has been released by the ByteDance which will going to blow your mind. This model is capable of generating 1024 by 1024 pixels of image in just 2 Steps.

Yes, only 2 inference Steps!

The model has been distilled from the popular Stable Diffusion XL base1.0 model. This means you get faster inference without effecting the actual output.

We observed people are struggling with Stable Diffusion XL running locally due to VRAM shortage. So, if you are costing an arm and a leg then this model will be a game changer. 

Feature:

– Faster generation with low inference Steps

– Simple Plug and play

– Optimized and Light Weight

– Requires Low VRAM

– Generates high resolution images

– Better than SDXL Turbo and SD1.5

– LoRA support

Speed Testing:

Lets test how much detailed and faster generation we get:

Example1:

Prompt: a beautiful girl, highly detailed, 8k

This took 209 milliseconds to generate 1024×1024 pixels art. Quite impressive and incredible.

Example2:

Prompt: a beautiful queen, highly detailed, 8k

This took 237 milliseconds to generate this art. Really amazing!

Always use Euler sampler/DDIM or DPM++ and Scheduler to sgm_uniform setup while using this model.

Now, let’s jump into the installation section and see in which type of WebUI you can install it and take the benefit of faster generation.

Installing in ComfyUI:

1. First, move to installation page of Hugging Face and click on “Files and version” section.

2. All types of checkpoints are available. The 1-Step, 2-Step, 4-Step and 8-Step models. Its recommended to use 2-Step /4-Step or 8-Step, but the 1-Step model is for experimental purpose. Actually these are optional. If you want to generate image using 2 Steps then download only 2-Step model. For using all the steps then you need download all the steps models and you can setup all in ComfyUI and use any of them as required.

3. After downloading, save it into “/ComfyUI/models/checkpoints” folder.

4. Next is to Download the ComfyUI workflow and do setup.

5. If you want to use the LoRA features, then download the files that have “lora” keyword as shown on the above image and put them into “/ComfyUI/models/loras” folder.

6. Now, Download the ComfyUI workflow and do setup for LoRA.

If you face any error on loading the workflows like missing nodes then you can go to ComfyUI manager and click “Install Missing Custom Nodes“. 

Installing in Automatic1111:

Currently officially no support has been provided for this WebUI.

But, you can use  models like Dreamshaper XL lightning, Juggernaut XL lightning that is fine tuned on Stable Diffusion XL 1.0 base model. You can download these model checkpoints from CIVITAI or Hugging Face. Here, we have illustrated to download the models using Hugging Face platform.

1. For downloading Dreamshaper XL lightning move to the official page of Hugging Face and click on “Files and version” section.

2. Download the safetensors files provided for this model. Here, two files have been provided. Download any one of them as required and save it into the “stable-diffusion-webui/models/Stable-diffusion/” folder.

If you want to use the  Juggernaut XL lightning then, go to the official page of Hugging face and click on “Files and version” tab.

Next is to download the safetensor file which uses 4-Step sampling.

3. After downloading move the files as usual into “stable-diffusion-webui/models/Stable-diffusion/” folder.

4. Restart your Automatic1111 to take effect and you will get your lightning models into the checkpoints. 

5. Now for generating image using this model use the recommended settings:

Resolution: 832 by 1216

Sampler: DPM++ SDE

Steps: 4-6

CFG scale:1-2

Prompt: Should not be much longer

Conclusion:

Stable Diffusion XL lightning has been improved a lot as compared to other faster Stable Diffusion models. The main aspect of this model is that its inference time is lesser which makes more effective in generating images. Its light and a distilled version of Stable diffusion 1.0 base model generating faster without effecting the result.