Instant Style Captured: Photomaker V1/V2

Don’t want to go into those complicated stuff of style generation. Try Photomaker models to generate your image with your desired style. The model has been created by TencentArc. This provides you with features like the IP Adapter, with simple-to-use methods. 

Source: TencentArc’s Hugging Face

For further research, you can access the research paper. Photomaker can be integrated with ControlNet, T2I-Adapter, and IP-Adapter by implementing customized nodes in ComfyUI and with customized extension in ForgeUI. Let’s checkout the installation process.

Installation:

1. First do the ComfyUI installation, or just update it from the ComfyUI manager if you already have.

2. Just move to ComfyUI manager, and search from the list “ComfyPhotomakerPlus” (labeled by shimizu)and click on the install/update button.

Alternative:

Move to the folder “ComfyUI/custom_nodes” and type “cmd” on the address bar to open the command prompt. Into the command prompt type this command to do the installation:

git clone https://github.com/shiimizu/ComfyUI-PhotoMaker-Plus.git

3. Install the dependencies, if you did not yet:

(a) Move to onnxruntime official page and do the installation process.

(b) Insightface- Open the command prompt and just copy the command:

pip install insightface

Make sure that its using Insight Face in the background, so its only used for research purpose only and not for commercial one.

For ComfyUI portable users, open the command prompt and copy the command:

python_embeded/python.exe -m pip install insightface

PhotomakerV1

PhotomakerV2

4. Download the PhotomakerV1 or PhotomakerV2 model (you can download both) from the Hugging Face repository. Move inside “ComfyUI/models/photomaker” and store the downloaded models.

Create new folder “photomaker” if you don’t have inside “ComfyUI/models/” and then save the models. 

5. Restart ComfyUI to take effect.

Workflow Explanation:

1. Load the workflow for V1 /V2, which is available into “ComfyUI/custom_nodes/ComfyUI-PhotoMaker-Plus/examples” folder. Alternatively, it can be downloaded from Github. For illustration, we have shown the V2 workflow. 

2. Load your Stable Diffusion XL(SDXL) based model into “Load checkpoints” node. We have used Juggernaut XL as the checkpoint because it works best with the human faces. 

Set your relevant dimensions from the “empty latent” node supported for checkpoints. We set the image resolution to 1024 by 1024. 

Reference Image

3. Load your reference image into the “Load image” node. We uploaded the reference image. Configure the KSampler as instructed for your SDXL models.

You can download any SDXL models from CivitAi and get the relevant settings from its description section. 

4. Put your prompt style into “Photomaker Encode Plus” node. Click “Queue prompt” to generate. 

PhotomakerV1 output

PhotomakerV2 output

PhotomakerV2 output

PhotomakerV2 output

Prompt: instagram photo, portrait photo of a woman img, night life, professional phtography, uhd, 32k

Negative Prompt: text, watermark

You can try our random prompt generator for different art style specifically built for Stable Diffusion models.

Recommended Settings we used:

Sample Steps: 40

CFG Scale: 5

Here all the outputs are generated with PhotomakerV2 in 1024by 1024 resolution.

Now, we tested with different reference image.

Reference Image

PhotomakerV1

PhotomkaerV2

After using both the models(V1/V2), the results was pretty clear that PhotomakerV2 is more effective as compared to the older one. 

Photomaker V2 released ! 😁

Install it Now…. 😃#photomaker #stablediffusion
Create your own Style with your AI influencer…😍
.
.
.
Workflow Available (included) 👇😬 pic.twitter.com/7dg8p9n6uK

— Stable Diffusion Tutorials (@SD_Tutorial) August 2, 2024

After testing, we concluded that Photomaker V2 is more powerful than V1 in terms of face identification. We posted all the comparisons with different style in out X(twitter) handle.

Some recommended tips:

1. Make sure to use the trigger word, you want to customize with. For instance: man img or woman img or girl img in your positive prompts to get the optimized results.

2. Use the recommended settings for best results.

3. Upload more reference images for better face identification.

4. Set the style strength value between 30-50 for realistic image generation. The larger value the less face detection it will be and vice-versa but the stylize will be more influenced. 

5. For lower-end GPU users, the number of images can be set to lower with lower sampling steps for faster generation but will impact face identity.

Conclusion:

Use Photomaker for instant stylized image generation if you don’t want to go into the complexity of IP-Adapter. But, for optimized output, it’s usually seen in the community that usage of IP-Adpater provides you multiple advantages with professional editing.