Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significantly improved img2img quality by scaling masked area before inference #715

Merged
merged 4 commits into from
Sep 6, 2022
Merged

Conversation

TingTingin
Copy link
Contributor

With this experimental addition you can increase the quality of img2img generations by pre-scaling the masked area

E.G
prompt: elon musk face
This is the selection mask
image

This is the image generated from using regenerate only masked area
00589-50_k_lms_745571120_0 00_elon_musk_face

This is the image generated with the new resize and regenerate only masked area
00596-50_k_lms_3320840799_0 00_elon_musk_face

As you can see the quality is much improved

Potential Improvements

  • Upscaling the image using realesrgan or another upscaler yields even better results as of now the image just uses LANCZOS scaling
  • The mask cropper currently "zooms in" on the mask then extends a *square from the left-most side of the image that encapsulates the entire mask it would be better if it was centered would also potentially help with *blending
  • Because of the crop on the image certain context is lost of the overall image it would be nice if there was a Context setting that dictated how much to expand the sampled area (at the cost of quality) to increase coherency
  • the code currently uses a global variable returned_info to pass some data around would be better if it was handled more efficiently
  • *square I was under the impression that you could send any 64 multiple dimensioned image to get it generated so the initial code wouldn't use squares but instead would use whatever shape would fit while having a 64 multiple dimension on both sides up to the max width/height that was specified in the ui but I kept getting an error so I had to change the code to always create a square instead not sure what I did wrong
  • *blending The blending needs to improve as of right now it takes the generated image applies the mask with a blur and call it a blur but there's a few issues
    • when you blur an edge you need to have enough "room" or else the blur will go over the edge and give a hard line essentially the mask blur should also increase the size or "room" the mask has to accommodate the blur more appropriately and of course also make the mask is the same size as the image its getting applied to at the end

@hlky hlky changed the base branch from master to old-dev September 6, 2022 23:31
@hlky hlky merged commit f86905e into Sygil-Dev:old-dev Sep 6, 2022
hlky added a commit that referenced this pull request Sep 6, 2022
* Update README.md

* Add sampler name to metadata (#695)

Co-authored-by: EliEron <example@example.com>

* old-dev-merge

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>
hlky added a commit that referenced this pull request Sep 7, 2022
* #715 #699 #698 #663 #625 #617 #611 #604 (#716)

* Update README.md

* Add sampler name to metadata (#695)

Co-authored-by: EliEron <example@example.com>

* old-dev-merge

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>

* img2img-fix (#717)

* Revert "img2img-fix (#717)"

This reverts commit 70d4b1c.

* img2img fixes

* Revert "img2img fixes"

This reverts commit e66eddc.

* Revert "Revert "img2img-fix (#717)""

This reverts commit bf08b61.

* img2img fixed

* feat: bring back Crop mode, formatting

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>
Co-authored-by: Thomas Mello <work.mello@gmail.com>
@Cikmo Cikmo mentioned this pull request Sep 7, 2022
TinyBeeman pushed a commit to TinyBeeman/stable-diffusion-webui-tiny that referenced this pull request Sep 8, 2022
* Sygil-Dev#715 Sygil-Dev#699 Sygil-Dev#698 Sygil-Dev#663 Sygil-Dev#625 Sygil-Dev#617 Sygil-Dev#611 Sygil-Dev#604 (Sygil-Dev#716)

* Update README.md

* Add sampler name to metadata (Sygil-Dev#695)

Co-authored-by: EliEron <example@example.com>

* old-dev-merge

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>

* img2img-fix (Sygil-Dev#717)

* Revert "img2img-fix (Sygil-Dev#717)"

This reverts commit 70d4b1c.

* img2img fixes

* Revert "img2img fixes"

This reverts commit e66eddc.

* Revert "Revert "img2img-fix (Sygil-Dev#717)""

This reverts commit bf08b61.

* img2img fixed

* feat: bring back Crop mode, formatting

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>
Co-authored-by: Thomas Mello <work.mello@gmail.com>
hlky added a commit that referenced this pull request Sep 8, 2022
* #715 #699 #698 #663 #625 #617 #611 #604 (#716)

* Update README.md

* Add sampler name to metadata (#695)

Co-authored-by: EliEron <example@example.com>

* old-dev-merge

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>

* img2img-fix (#717)

* Revert "img2img-fix (#717)"

This reverts commit 70d4b1c.

* img2img fixes

* Revert "img2img fixes"

This reverts commit e66eddc.

* Revert "Revert "img2img-fix (#717)""

This reverts commit bf08b61.

* img2img fixed

* - Removed duplicated calls to save_sample.
- Change variables and arguments to be more self-explanatory and easier to understand what they do.

* Moved streamlit files to their proper location, before they were incorrectly added to the repository root folder.

* Added retry dependency for the streamlit version.

* Added .cmd file for easy running and updating the streamlit version of the UI.

* Removed duplicated entry for streamlit on the environment.yaml file.

* Removed some unnecessary lines from the the webui_streamlit.cmd file.

* add gfpgan folder to gitignore, auto gen by imglab

* added placeholder text similar to gradio

* added auto conversion for 4 channel PNG to RGB

* fix: regex escape characters

* Update Readme links to sd-webui when appropriate (#781)

* Update link to sd-webui when appropriate

* added LDSR instruction per devilismyfriend guide

* fix: stack overflow during recursion call (#784)

* Added option to set default sampler name from config file, will be useful for those wanting to change the default sampler and  have it persist even when closing the UI and opening it again.

* Added try and except block to handle basic errors like StopException which is raised by streamlit when you hit the stop button and KeyError which happens also when stopping the generation because it tries to check the model at the end which is not loaded at that time, this can be ignored and so thats the reason for the exception.

* separate css to external file

* Added "git pull" and "git stash" to the commands run by the cmd scripts when launching the UI, this should make it so people who use it can automatically update the code from the repo and be up to date without manually using those commands everytime.

* resolve conflict with master

Co-authored-by: EliEron <subanimehd@gmail.com>
Co-authored-by: EliEron <example@example.com>
Co-authored-by: ZeroCool <ZeroCool940711@users.noreply.github.com>
Co-authored-by: ZeroCool940711 <alejandrogilelias940711@gmail.com>
Co-authored-by: Hafiidz <3688500+Hafiidz@users.noreply.github.com>
Co-authored-by: Thomas Mello <work.mello@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants