Feature Announcement Thread

[mcmonkey4eva]

This is a dedicated thread you can follow where new features will be announced. This will only be used for the biggest features.

You can find the previous Alpha/Beta Stability AI era feature announcement thread at Stability-AI/StableSwarmUI#11

You'll also find more news on Discord at https://discord.gg/q2y38cqjNw - the announcements channel there has smaller updates.

The releases page https://github.com/mcmonkeyprojects/SwarmUI/releases will contain bulk summaries of updates in more detail than here

And, of course, if you're very obsessive, you can watch every individual git commit https://github.com/mcmonkeyprojects/SwarmUI/commits/master/

[mcmonkey4eva]

SD3 Controlnets (eg https://huggingface.co/InstantX) are now supported in Swarm! (thanks of course to comfy doing the actual hard work on the internals) They use the new ControlNetApplySD3 node internally, and will be autodetected by the model architecture handler (because InstantX published in diffusers format ie no coherent metadata by default).

(I recommend if you download the instantx models, to rename the files when you save them and then edit metadata so they're not all the helplessly blank diffusion_pytorch_model.safetensors). If you name the canny model with canny in the title and the pose model with pose in the title, the preferred preprocessor will be autodetected from the name (otherwise of course just select preprocessor manually).

Note that instantx's models are the first ever controlnets for SD3, ie not going to be the best ones.

[mcmonkey4eva]

Swarm dropdown parameters now have human-friendly names rather than the raw comfy IDs.

For example, here's the sampler list now vs before:

For example dpmpp_2m is now labeled clearly as DPM++ 2M, and has the hint-text 2nd Order Multi-Step

the list is a bit chonkier now, but hopefully it's now an intelligible text wall vs previously being a spasm of meaningless letters

You can still happily filter-search by the raw ID or the clean name

Or here's the Aspect Ratio list:

Previous it had just the numbers, without the hints

[mcmonkey4eva]

Black Forest Labs' new Flux.1 model is now supported in Swarm! https://blackforestlabs.ai/announcing-black-forest-labs/

See docs at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#black-forest-labs-flux1-models for specific usage info

[mcmonkey4eva]

The Swarm Image Editor now has a Paintbucket tool, and an integrated SAM2 autosegmenter button, so you can for example do something like this:

[mcmonkey4eva]

New: bulk civitai metadata scanner utility

This (slightly hacky but sufficient) bulk scanner lets you update your model's metadata from civitai. Useful for if you have a lot of externally downloaded models (ie not using the swarm model downloader that autosyncs civit metadata while downloading). Has a bunch of configuration options for which models to scan / how to apply updates. To work fully, this will autocalculate the hash of any models that don't have a stored hash - be warned this is a slow process. Also, the metadata updates themselves can be slow. So, for large model folders, this will take a while (but only has to be ran once and then it's saved for all models you have in that folder, and then all those models are near-instant if you rerun over them).

ps, civitai now supports Swarm image metadata when you upload images!

[mcmonkey4eva]

New: extensions manager tab

lets you install Swarm extensions, update them, etc.

Also, works has been done to make writing extensions more approaching, including easier ways to support Comfy node pack install requirements, and the Dynamic Thresholding Extension is updated to be a good model of how to write a good extension

[mcmonkey4eva]

• Added support for AliMama Inpainting Flux Controlnet https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta
  -> Download the model, save in controlnet model folder, probably rename it to something less dumb than its default filename
  -> refresh swarm, find it in the Controlnets models tab, click Edit Metadata and change its architecture to Flux.1 ControlNet - AliMama Inpaint
  -> use the image editor (or any init image + mask) with max creativity
  -> make sure controlnet input group is enabled with the model selected, don't use a preprocessor
  -> mask your image, type your prompt, enjoy
  -> Note: it is very very vram hungry. overloads on 4090 in fp8, barely fits with a q4 gguf
  -> Note: it is a bit seed dependent, the model is marked beta, you'll have to try a bit to make it do what you want

[mcmonkey4eva]

Genmo Mochi 1 (Text-To-Video) Models are now natively supported in SwarmUI!

When you select a Text-To-Video model, a new Text To Video param group will appear on the left

Note that currently even on a 4090 you can barely generate a short 25 frame video, as the VAE requires more VRAM than anyone has. Edit: This is fixed now, VAE Tiling is supported!

See Model Support doc for details: https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#genmo-mochi-1-text2video

[mcmonkey4eva]

Lightricks LTX Video is fully natively supported in SwarmUI for both Text-To-Video and Image-To-Video, see notes at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#lightricks-ltx-video

[mcmonkey4eva]

BFL's "Flux Tools" (Redux to mix images, Canny/Depth special models as almost-controlnets-but-different, Fill aka inpaint model) are fully supported in SwarmUI, including some special hack parameters to make Redux more powerful, see notes at https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Model%20Support.md#flux1-tools

[mcmonkey4eva]

New UI feature: confused about the more complex prompt syntax inputs like Segment or Regional prompts? Be confused no more, there's now helper UIs to guide you through constructing perfect inputs every time!