Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make union_inpaint preprocessor #3035

Open
light-and-ray opened this issue Aug 12, 2024 · 2 comments
Open

Make union_inpaint preprocessor #3035

light-and-ray opened this issue Aug 12, 2024 · 2 comments

Comments

@light-and-ray
Copy link

light-and-ray commented Aug 12, 2024

          > Inpaint: Does not work right now

I can get something. Just use "None" preprocessor + image where black pixels mean mask. This comfy workflow do the same

00008-1835962484-a face of cat
ComfyUI_00008_

You can get masking workflow from metadata

But there is still a black contour, it happens because:
Resize Mode
Just Resize
Crop and Resize
Resize and Fill

Originally posted by @light-and-ray in #2998 (comment)

@light-and-ray
Copy link
Author

So the best way I think is to create a union_inpaint preprocessor, which pastes black pixels under mask after image resized

@light-and-ray light-and-ray changed the title Make union inpaint preprocessor Make union_inpaint preprocessor Aug 12, 2024
@light-and-ray
Copy link
Author

I've given up 😔

class PreprocessorInpaintUnion(Preprocessor):
    def __init__(self):
        super().__init__(name="inpaint_union")
        self.tags = ["Inpaint"]
        self.slider_resolution = PreprocessorParameter(visible=False)
        self.sorting_priority = 50
        self.accepts_mask = True
        self.requires_mask = True

    def __call__(
        self,
        input_image,
        resolution,
        slider_1=None,
        slider_2=None,
        slider_3=None,
        **kwargs
    ):
        input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
        input_image = input_image[:, :, :3]

        return Preprocessor.Result(
            value=input_image,
            display_images=input_image[None, :, :, :],
        )

This preprocessor gives a good image which is included as display_images, but the processed image is a total garbage
00026-505827856-empty

If I pass preprocessed image into None preprocessor, it works, but with a black contour

class PreprocessorInpaintUnion(Preprocessor):
    def __init__(self):
        super().__init__(name="inpaint_union")
        self.tags = ["Inpaint"]
        self.slider_resolution = PreprocessorParameter(visible=False)
        self.sorting_priority = 50
        self.accepts_mask = True
        self.requires_mask = True

    def __call__(
        self,
        input_image,
        resolution,
        slider_1=None,
        slider_2=None,
        slider_3=None,
        **kwargs
    ):
        # input_image[:, :, :3][input_image[:, :, 3]!= 0] = 0
        # input_image = input_image[:, :, :3]

        image = Image.fromarray(
                np.ascontiguousarray(input_image[:, :, :3].clip(0, 255).astype(np.uint8)).copy()
            )
        mask = Image.fromarray(
                np.ascontiguousarray(input_image[:, :, 3].clip(0, 255).astype(np.uint8)).copy()
            )
        black_image = Image.new('RGB', image.size, (0, 0, 0))
        image.paste(black_image, mask=mask)
        masked_image = HWC3(np.asarray(image).astype(np.uint8))

        return Preprocessor.Result(
            value=masked_image,
            display_images=masked_image[None, :, :, :],
        )

In this code I tried to make the same in pillow, and the result is better. But still worse then I would pass it into None preprocessor. And also there is a contour, although I've even set pixel perfect and resolution in call is 1024p

00027-3205936420-empty

I have no idea how to work with numpy images even with llama as assistant 😔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant