Flux Support #1176
Replies: 27 comments 18 replies
-
|
I assumed that I need to download the node Anyline of TheMistoAI
And x-flux-comfyui of the Xlabs-AI
|
Beta Was this translation helpful? Give feedback.
-
|
no, you don't need either of those, just the nodes in https://github.com/Acly/krita-ai-diffusion/wiki/ComfyUI-Setup extra nodes are only required for GGUF (I should add that to the instructions) |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for that, i really appreciate that much work |
Beta Was this translation helpful? Give feedback.
-
|
Is it possible to use the --fast flag from Comfy in the plugin too? I have a 40x series card. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for this. I have two Flux questions:
Thank you |
Beta Was this translation helpful? Give feedback.
-
Not yet in the release. Added here: 13e613f |
Beta Was this translation helpful? Give feedback.
-
|
run into generation error, seems to be missing clip. I have all my clips as I am using OK in Comfyui but is it mixing teh T5 up?
followed by a lot of block and shape size mismatch log pre error:
I have in the clip folder: clip_g.safetensors Do I need to rename t5 base or t5xxl_fp8_e4m3fn.safetensors to t5.safetensors |
Beta Was this translation helpful? Give feedback.
-
|
It picks the first thing with "t5" in the name. Try deleting/moving/renaming t5_base.safetensors so it will use one of the other two. (What is t5_base used for?) |
Beta Was this translation helpful? Give feedback.
-
|
With that name change (either file or in resources) |
Beta Was this translation helpful? Give feedback.
-
|
I swapped the name in resources.py as test and now it is generating:
|
Beta Was this translation helpful? Give feedback.
-
There are models with custom trained CLIP L TEs. One example: Accompanying custom TE found here: |
Beta Was this translation helpful? Give feedback.
-
|
Flux controlnet inpainting(just find this don't know, if it helps make flux model more usable): |
Beta Was this translation helpful? Give feedback.
-
|
Implemented in 1ca5ed9 Results are better than without the Inpaint CN, but not great. SDXL still works better in general. But it's an alpha model, there is hope an improved version won't be too far off. |
Beta Was this translation helpful? Give feedback.
-
|
Added instructions on how to install the inpaint model in the main post. From version 1.25.0, it will be used when there's a selection and strength is at 100%. |
Beta Was this translation helpful? Give feedback.
-
|
I try to add controlnet to flux but seems it has not yet been installed in Interstice cloud. Hope it can be added soon. |
Beta Was this translation helpful? Give feedback.
-
|
I'm trying and something is wrong at the moment |
Beta Was this translation helpful? Give feedback.
-
|
how did you load gguf? |
Beta Was this translation helpful? Give feedback.
-
|
ok, ok work for me |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I have tested flux-Canny-ControlNet-v3. Safetensors and MistoLine (mistoline-flux. dev-v1), and both of them have poor image quality, which cannot compare to XL's ControlNet. Do you have any recommended Flux ControlNet for testing? |
Beta Was this translation helpful? Give feedback.
-
|
Use low strength to avoid poor image quality. But they don't follow very closely (even with high strength). It took 6+ months for great ControlNet to come out for XL. I think we just have to wait. |
Beta Was this translation helpful? Give feedback.
-
|
Hello Acly @Acly ,
|
Beta Was this translation helpful? Give feedback.
-
|
Hi syl1130, how did you make the Union Pro work in Krita ai diffusion? |
Beta Was this translation helpful? Give feedback.
-
|
Hello Acly @Acly , I have added translations for the text added in version 1.24.0. |
Beta Was this translation helpful? Give feedback.
-
|
I have install MistoLine ,but i could not find in krita,but I can find canndy or depth |
Beta Was this translation helpful? Give feedback.
-
|
Got an error which is strange that flux model recognized as SD 1.5 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, BNB-NF4 models is working? |
Beta Was this translation helpful? Give feedback.
-
|
Hello Acly @Acly , I have added translations for the text added in version 1.25.0. |
Beta Was this translation helpful? Give feedback.
-
|
is support for bnb nf4 possible? |
Beta Was this translation helpful? Give feedback.
-
|
Updated the guidelines to refer to the beta version of the inpainting model. Better delete/move the old (Alpha) version of the model to make sure it uses the new one. |
Beta Was this translation helpful? Give feedback.
-
|
"GPU memory usage: 27GB" - ¡Ay, caramba! |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Acly , About using Flux.1-dev-Controlnet-Upscaler. Thank you |
Beta Was this translation helpful? Give feedback.
-
|
Yes, best rename it to |
Beta Was this translation helpful? Give feedback.
-
|
I kinda feel like I have to ask. Is there any specific reasoning why there aren't some opt in for experimental features list where you can straight up install the different Flux models from. The main reason why I have that question is because there are a few common things that you straight up need the Flux models to fix even if you are using an SD model for a specific style like hands and feet's if you don't want to sit down and redraw them by hand. The two main benefits that I can see from it is that you guys will be able to know that all the users have at least the provided base configuration and the second is that you can attach FAQ plus link to this thread. Also, if I'm not completely incorrect you could probably gray out the FP16 models for anyone that rock 4GB of VRAM. Also, stuff like this would probably be unnecessary to ask if you just got the correct configuration |
Beta Was this translation helpful? Give feedback.
-
|
I'd recommend switching to using this control net union: https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro |
Beta Was this translation helpful? Give feedback.
-
|
I choose option 1 and just dropped Flux in models/checkpoints |
Beta Was this translation helpful? Give feedback.
-
|
people seem to like this pixelwave flux finetune https://www.reddit.com/r/StableDiffusion/comments/1gcbn4t/comment/ltswixq/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button Are there any other fine tunes that you have been looking at? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Please can we get this :) |
Beta Was this translation helpful? Give feedback.
-
|
There's Flux1 dev NF4 model for low VRAM (6-12Gb) devices. |
Beta Was this translation helpful? Give feedback.
-
|
Did you manage to launch it? |
Beta Was this translation helpful? Give feedback.
and others
-
Flux: Current State
Plugin version: 1.25.0
Experimental Support
Flux ecosystem is very active, but a lot of extension models are early beta versions or simply don't exist. Please keep that in mind and prefer to comment about models here rather than opening issues.
Model Download
There are various options to get started with Flux, they differ in complexity and hardware requirements.
Option 1: Full Checkpoint
The easiest way is to download a full checkpoint which includes everything in one package. Choose any of the following models:
ComfyUI/models/checkpointsComfyUI/models/checkpointsThe models must be placed into the
ComfyUI/models/checkpointsfolder. Then you will be able to choose them from the Checkpoint list in the Style settings.16 GB VRAM recommended, but it will work with less.
Option 2: Separate Diffusion Model
Many distributions of Flux contain only the diffusion model. In order to use the model you also need the text encoders and image auto-encoder, as separate files. Because all variants of Flux share the same text/image encoders, this gives you more flexibility and saves disk space.
Install the GGUF custom node
This is only needed if you have a custom ComfyUI install and want to use GGUF files. Please install city96/ComfyUI-GGUF. You can also find them in ComfyUI Manager.
Download the Text Encoders
Clip-L (clip_l.safetensors) →
ComfyUI/models/clipT5-XXL (t5xxl_fp8_e4m3fn.safetensors) →
ComfyUI/models/clipYou can also alternatively use T5 quantizations from city96/t5-v1_1-xxl-encoder-gguf -
Q5_K_Mis a good trade-off.Download the VAE
Flux AE (ae.safetensors) →
ComfyUI/models/vaeDownload a Diffusion Model
Pick one, depending on your available VRAM and desired quality:
These files need to go into the
ComfyUI/models/diffusion_modelsfolder!The plugin will show them in the same place as regular checkpoints and you can use them in the same way, as long as you have all prerequisite models.
Inpaint (Fill, Expand, Add/Remove Object)
Inpainting requires the following ControlNet model:
ComfyUI/models/controlnetPlease rename the file to
FLUX.1-dev-Controlnet-Inpainting-Beta.safetensors. Only works with Flux [dev].If you don't want to use the inpaint model, use the "Generate (Custom)" option and uncheck "Seamless".
ControlNet
The following models are supported:
ComfyUI/models/controlnetComfyUI/models/controlnetComfyUI/models/controlnetComfyUI/models/controlnet(rename!)They are meant to be used with the "dev" version of Flux and don't work well with "schnell". They're also not available on cloud for now.
Common Issues
The Flux workload is not installed
Try restarting Krita if you copied in a new model while it was running
Text encoder or VAE not found
Make sure models are in the correct folder and have the correct name. Otherwise they cannot be found.
Please check client.log file for more info.
Future Work
Beta Was this translation helpful? Give feedback.
All reactions