Small Improvements and Random Feature Idea List #550
Description
This issue is a list of various small things that could be improved, and random things that can be added.
The "small things" part is basically here to serve as a grab-bag of things for people looking to contribute to the project.
If you're wondering "but these are small and easy, why don't you just do them yourself?", two reasons:
- 1: I want to encourage contribution. Swarm is free and open source, I don't get paid for this. I don't want to develop this project all alone, and having easy things to fix is a great way for new contributors to get started.
- 2: Even small things take time. Finding where in the code to do it, editing it, testing it, testing for possible regressions, making sure the choices made about how to do it were right, etc. It takes time, and when you add up all the things to do... there's more things to do than time in the day for 1 person.
The "random ideas" part is things which aren't important enough to warrant a feature request on their own, but might be interesting.
ComfyUI Updates
- Comfy now has
/api/routing prefix. Swarm should use this - Comfy now natively includes
SaveImageWebsocketnode, this should replace the SwarmSaveImageWS (keep the swarm node for backwards compat, but stop generating it) - PixArt Alpha/Sigma is native supported in comfy now (these models suck but probably worth supporting just on principle)
- port more of the SwarmKSampler features to work with SamplerCustomAdvanced
Debugging and wotnot
- detect zip/ckpt header when safetensors fails to load and give a better error message for wrong file extensions
- model 'view raw metadata' button and a way to clear non-modelspec metadata easily in UI
- network diagnostics in resource usage section of the Server tab - eg total number of connections, etc.
UI Improvements
- refresh button should disable+reenable when refresh ends, as a visual indicator. Maybe a loading symbol?
- put cascade latent compression behind a cascade feature lock and move to regular Sampling (ie match flux guidance handling)
- param group naming parens, eg
Init Image (Img2Img)with that little gray-out effect that I have in dropdown selectors - "Create New Wildcard" or Preset should prefix current folder path in the name box
- loras in a preset should add not replace
- preset names should be filtered to use filename-compatible IDs internally (eg no wonky slashes)
- Path settings change should force a client modellist refresh just to avoid confusion
- prompt "+" button should have a "Add Image Prompt" maybe
- update options: way to update to latest release vs latest dev (via git calls)
- more control over comfy & comfy node updates. Button to trigger an update check manually (next to the swarm button?) a swarm-managed node listing tab, etc.
Internal Improvements
- comfyuicustomworkflow param should work just like raw workflow for parameter autogeneration - setup and run custom workflow via API #383
- don't emit "//cid=" syntax in output prompt (to avoid original_prompt when there's no change for segments)
- figure out more direct video file management internals, eg directly embedding metadata to animated webp, mp4, etc. files rather than a separated json
Features to test
- test https://github.com/instantX-research/Regional-Prompting-FLUX (are there decent comfy nodes for it?)
- IPAdapter flux? https://github.com/Shakker-Labs/ComfyUI-IPAdapter-Flux (doesn't use the normal nodes which is annoying but might be worth the effort to support if the models are good?) (see also https://www.reddit.com/r/StableDiffusion/comments/1gx1p37/instantxflux1devipadapter_hugging_face/ )
- pulid flux? https://github.com/sipie800/ComfyUI-PuLID-Flux-Enhanced or https://github.com/lldacing/ComfyUI_PuLID_Flux_ll/tree/main
- "recomposite color correct" to make flux fill work well -- line up pixels, select those with mask=0, ensure there's more than a minimum pixel count, and then measure what color correction is needed, take the average, and apply it to the full image -- see https://discord.com/channels/1243166023859961988/1243185862234210389/1327390619068530729 for my attempt at this. Unfortunately flux vae is dumdum inconsistent so even that's not good enough. probably need a full multiaxis gradient for the color shift. Might be a smarter way to do it? idk
- SVDQuant? https://github.com/mit-han-lab/nunchaku/tree/main/comfyui