diff --git a/README.MD b/README.MD index 2ab41449..de0028f2 100644 --- a/README.MD +++ b/README.MD @@ -80,8 +80,9 @@ When this error has occurred, please check the network environment. ## Update **If the dependency package error after updating, please reinstall the relevant dependency packages.
+* [Ultra](#Ultra) nodes added the option to run ```VitMatte``` on the CUDA device, resulting in a 5-fold increase in running speed. * Commit [QueueStop](#QueueStop) node, used to terminate the queue operation. -* Optimize performance of the ```vitmate``` method for Ultra nodes when processing large-size image. +* Optimize performance of the ```VitMate``` method for [Ultra](#Ultra) nodes when processing large-size image. * [CropByMaskV2](#CropByMaskV2) add option to round the cutting size by multiples. * Commit [CheckMask](#CheckMask) node, it detect whether the mask contains sufficient effective areas. Commit [HSVValue](#HSVValue) node, it convert color values to HSV values. * [BooleanOperatorV2](#BooleanOperatorV2), [NumberCalculatorV2](#NumberCalculatorV2), [Integer](#Integer), [Float](#Float), [Boolean](#Boolean) nodes add string output to output the value as a string for use with [SwitchCase](#SwitchCase). @@ -125,7 +126,7 @@ Part of the code for BlendMode V2 is from [Virtuoso Nodes for ComfyUI](https://g * [CreateGradientMask](#CreateGradientMask) node add ```center``` option. * Commit [GetColorToneV2](#GetColorToneV2) node, can select the main and average colors for the background or body. * Commit [ImageRewardFilter](#ImageRewardFilter) node, can filter out poor quality pictures. -* Ultra nodes add ```VITMatte(local)``` method, You can choose this method to avoid accessing huggingface.co if you have already downloaded the model before. +* [Ultra](#Ultra) nodes add ```VITMatte(local)``` method, You can choose this method to avoid accessing huggingface.co if you have already downloaded the model before. * Commit [HDR Effect](#HDR) node，it enhances the dynamic range and visual appeal of input images. this node is repackage of [HDR Effects (SuperBeasts.AI)](https://github.com/SuperBeastsAI/ComfyUI-SuperBeasts). * Commit [CropBoxResolve](#CropBoxResolve) node. * Commit [BiRefNetUltra](#BiRefNetUltra) node, it using the BiRefNet model to remove background has better recognition ability and ultra-high edge details. @@ -133,7 +134,7 @@ Part of the code for BlendMode V2 is from [Virtuoso Nodes for ComfyUI](https://g * Commit [ImageHub](#ImageHub) node, supports up to 9 sets of Image and Mask switching output, and supports random output. * Commit [TextJoin](#TextJoin) node. * Commit [PromptEmbellish](#PromptEmbellish) node. it output polished prompt words, and support inputting images as references. -* Ultra nodes have been fully upgraded to V2 version, with the addition of VITMatte edge processing method, which is suitable for handling semi transparent areas. Include [MaskEdgeUltraDetailV2](#MaskEdgeUltraDetailV2), [SegmentAnythingUltraV2](#SegmentAnythingUltraV2), [RmBgUltraV2](#RmBgUltraV2) and [PersonMaskUltraV2](#PersonMaskUltraV2) nodes. +* [Ultra](#Ultra) nodes have been fully upgraded to V2 version, with the addition of VITMatte edge processing method, which is suitable for handling semi transparent areas. Include [MaskEdgeUltraDetailV2](#MaskEdgeUltraDetailV2), [SegmentAnythingUltraV2](#SegmentAnythingUltraV2), [RmBgUltraV2](#RmBgUltraV2) and [PersonMaskUltraV2](#PersonMaskUltraV2) nodes. * Commit [Color of Shadow & Highlight](#Highlight) node, it can adjust the color of the dark and bright parts separately. Commit [Shadow & Highlight Mask](#Shadow) node, it can output mask for dark and bright areas. * Commit [CropByMaskV2](#CropByMaskV2) node, On the basis of the original node, it supports ```crop_box``` input, making it convenient to cut layers of the same size. * Commit [SimpleTextImage](#SimpleTextImage) node, it generate simple typesetting images and masks from text. This node references some of the functionalities and code of [ZHO-ZHO-ZHO/ComfyUI-Text_Image-Composite](https://github.com/ZHO-ZHO-ZHO/ComfyUI-Text_Image-Composite). @@ -1359,8 +1360,25 @@ Output: * x: The x-coordinate of the top left corner position. * y: The y-coordinate of the top left corner position. + +## Ultra节点组 +![image](image/ultra_nodes.jpg) +Nodes that use ultra fine edge masking processing methods, the latest version of nodes includes: SegmentAnythingUltraV2, RmBgUltraV2, BiRefNetUltra, PersonMaskUltraV2, SegformerB2ClothesUltra and MaskEdgeUltraDetailV2. +There are three edge processing methods for these nodes: +* ```PyMatting``` optimizes the edges of the mask by using a closed form matching to mask trimap. +* ```GuideFilter``` uses opencv guidedfilter to feather edges based on color similarity, and performs best when edges have strong color separation. +The code for the above two methods is from the [ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters) in spacepxl's Alpha Matte, thanks to the original author. +* ```VitMatte``` uses the transformer vit model for high-quality edge processing, preserving edge details and even generating semi transparent masks. +Note: When running for the first time, you need to download the vitmate model file and wait for the automatic download to complete. If the download cannot be completed, you can run the command ```huggingface-cli download hustvl/vitmatte-small-composition-1k``` to manually download. +After successfully downloading the model, you can use ```VITMatte(local)``` without accessing the network. +* VitMatte's options: ```device``` set whether to use CUDA for vitimate operations, which is about 5 times faster than CPU. ```max_megapixels``` set the maximum image size for vitmate operation, and oversized images will be reduced in size. For 16G VRAM, it is recommended to set it to 3. + +The following figure is an example of the difference in output between three methods. +![image](image/mask_edge_ultra_detail_v2_example.jpg) + + ### SegmentAnythingUltra -Improvements to [ComfyUI Segment Anything](https://github.com/storyicon/comfyui_segment_anything), combined with the Alpha Matte node of [ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters) in spacepxl, result in more detailed edges for masks, thanks to the original author. +Improvements to [ComfyUI Segment Anything](https://github.com/storyicon/comfyui_segment_anything), thanks to the original author. *Please refer to the installation of ComfyUI Segment Anything to install the model. If ComfyUI Segment Anything has been correctly installed, you can skip this step. * From [here](https://huggingface.co/bert-base-uncased/tree/main) download the config.json，model.safetensors，tokenizer_config.json，tokenizer.json and vocab.txt 5 files to ```ComfyUI/models/bert-base-uncased``` folder. @@ -1394,7 +1412,8 @@ On the basis of SegmentAnythingUltra, the following changes have been made: * detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. * detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. * detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. - +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. ### RemBgUltra Remove background. compared to the similar background removal nodes, this node has ultra-high edge details. @@ -1421,8 +1440,8 @@ On the basis of RemBgUltra, the following changes have been made: * detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. * detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. * detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. - - +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. ### BiRefNetUltra Using the BiRefNet model to remove background has better recognition ability and ultra-high edge details. @@ -1440,7 +1459,8 @@ Node options: * black_point: Edge black sampling threshold. * white_point: Edge white sampling threshold. * process_detail: Set to false here will skip edge processing to save runtime. - +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. ### PersonMaskUltra Generate masks for portrait's face, hair, body skin, clothing, or accessories. Compared to the previous A Person Mask Generator node, this node has ultra-high edge details. @@ -1470,7 +1490,9 @@ On the basis of PersonMaskUltra, the following changes have been made: * detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. * detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. * detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. - +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. +* ### SegformerB2ClothesUltra ![image](image/segformer_ultra_example.jpg) Generate masks for characters' faces, hair, arms, legs, and clothing, mainly used for segmenting clothing. @@ -1502,7 +1524,36 @@ Node Options: * black_point: Edge black sampling threshold. * white_point: Edge white sampling threshold. * process_detail: Set to false here will skip edge processing to save runtime. +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. + + +### MaskEdgeUltraDetail +Process rough masks to ultra fine edges. +This node combines the Alpha Matte and the Guided Filter Alpha nodes functions of Spacepxl's [ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters), thanks to the original author. +![image](image/mask_edge_ultra_detail_example.jpg) +Node options: +![image](image/mask_edge_ultra_detail_node.jpg) +* method: Provide two methods for edge processing: PyMatting and OpenCV-GuidedFilter. PyMatching has a slower processing speed, but for video, it is recommended to use this method to obtain smoother mask sequences. +* mask_grow: Mask expansion amplitude. positive values expand outward, while negative values contract inward. For rougher masks, negative values are usually used to shrink their edges for better results. +* fix_gap: Repair the gaps in the mask. if obvious gaps in the mask, increase this value appropriately. +* fix_threshold: The threshold of fix_gap. +* detail_range: Edge detail range. +* black_point: Edge black sampling threshold. +* white_point: Edge white sampling threshold. + +### MaskEdgeUltraDetailV2 +The V2 upgraded version of MaskEdgeUltraDetail has added the VITMatte edge processing method.(Note: Images larger than 2K in size using this method will consume huge memory) +This method is suitable for handling semi transparent areas. + +On the basis of MaskEdgeUltraDetail, the following changes have been made: +![image](image/mask_edge_ultra_detail_v2_node.jpg) +* method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. +* edge_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. +* edge_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. ### YoloV8Detect Use the YoloV8 model to detect faces, hand box areas, or character segmentation. Supports the output of the selected number of channels. @@ -1600,34 +1651,6 @@ Node options: * fix_threshold: The threshold for fix_gap. * main_subject_detect: Setting this to True will enable subject detection, ignoring differences outside of the subject. - -### MaskEdgeUltraDetail -Process rough masks to ultra fine edges. -This node combines the Alpha Matte and the Guided Filter Alpha nodes functions of Spacepxl's [ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters), thanks to the original author. -![image](image/mask_edge_ultra_detail_example.jpg) - -Node options: -![image](image/mask_edge_ultra_detail_node.jpg) -* method: Provide two methods for edge processing: PyMatting and OpenCV-GuidedFilter. PyMatching has a slower processing speed, but for video, it is recommended to use this method to obtain smoother mask sequences. -* mask_grow: Mask expansion amplitude. positive values expand outward, while negative values contract inward. For rougher masks, negative values are usually used to shrink their edges for better results. -* fix_gap: Repair the gaps in the mask. if obvious gaps in the mask, increase this value appropriately. -* fix_threshold: The threshold of fix_gap. -* detail_range: Edge detail range. -* black_point: Edge black sampling threshold. -* white_point: Edge white sampling threshold. - -### MaskEdgeUltraDetailV2 -The V2 upgraded version of MaskEdgeUltraDetail has added the VITMatte edge processing method.(Note: Images larger than 2K in size using this method will consume huge memory) -This method is suitable for handling semi transparent areas. The following figure is an example of the difference in output between three methods. -![image](image/mask_edge_ultra_detail_v2_example.jpg) - -On the basis of MaskEdgeUltraDetail, the following changes have been made: -![image](image/mask_edge_ultra_detail_v2_node.jpg) -* method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. -* edge_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. -* edge_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. - - ### MaskGrow Grow and shrink edges and blur the mask ![image](image/mask_grow_example.jpg) diff --git a/README_CN.MD b/README_CN.MD index 461ca3ce..b1971026 100644 --- a/README_CN.MD +++ b/README_CN.MD @@ -80,8 +80,9 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git ## 更新说明 **如果本插件更新后出现依赖包错误，请重新安装相关依赖包。 +* [Ultra](#Ultra) 节点增加```VitMatte```方法在CUDA设备运行选项，运行速度提升5倍。 * 添加 [QueueStop](#QueueStop) 节点，用于停止队列。 -* 优化Ultra节点的```vitmatte```方法在处理大尺寸图片时的性能。 +* 优化[Ultra](#Ultra)节点的```VitMatte```方法在处理大尺寸图片时的性能。 * [CropByMaskV2](#CropByMaskV2) 增加裁切尺寸按倍数取整选项。 * 添加 [CheckMask](#CheckMask) 节点, 用于检测遮罩是否包含足够的有效区域。 * 添加 [HSVValue](#HSVValue) 节点, 用于转换色值为HSV值。 @@ -126,7 +127,7 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git * [CreateGradientMask](#CreateGradientMask) 节点增加 ```center``` 选项。 * 添加 [GetColorToneV2](#GetColorToneV2) 节点, 可选择背景或主体的主色和平均色。 * 添加[ImageRewardFilter](#ImageRewardFilter) 节点, 可过滤掉品质欠佳的图片。 -* Ultra 节点增加VITMatte(local)方法，如果之前已经下载了模型，可选择此方法避免访问huggingface.co。 +* [Ultra](#Ultra) 节点增加VITMatte(local)方法，如果之前已经下载了模型，可选择此方法避免访问huggingface.co。 * 添加 [HDR Effect](#HDR) 节点，增强图片动态范围。这个节点是[HDR Effects (SuperBeasts.AI)](https://github.com/SuperBeastsAI/ComfyUI-SuperBeasts)的重新封装。感谢原作者。 * 添加 [CropBoxResolve](#CropBoxResolve) 节点。 * 添加 [BiRefNetUltra](#BiRefNetUltra) 节点, 使用BiRefNet模型去除背景，有更好的识别能力，同时具有超高的边缘细节。 @@ -134,7 +135,7 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git * 添加 [ImageHub](#ImageHub) 节点，支持最多9组Image和Mask切换，支持随机输出。 * 添加 [TextJoin](#TextJoin) 节点。 * 添加 [PromptEmbellish](#PromptEmbellish) 节点, 对简单的提示词润色，支持图片输入参考，支持中文输入。 -* Ultra 节点全面升级到V2版本，增加了VITMatte边缘处理方法，此方法适合处理半透明区域。包括 [MaskEdgeUltraDetailV2](#MaskEdgeUltraDetailV2), [SegmentAnythingUltraV2](#SegmentAnythingUltraV2), [RmBgUltraV2](#RmBgUltraV2) 以及 [PersonMaskUltraV2](#PersonMaskUltraV2) 节点。 +* [Ultra](#Ultra) 节点全面升级到V2版本，增加了VITMatte边缘处理方法，此方法适合处理半透明区域。包括 [MaskEdgeUltraDetailV2](#MaskEdgeUltraDetailV2), [SegmentAnythingUltraV2](#SegmentAnythingUltraV2), [RmBgUltraV2](#RmBgUltraV2) 以及 [PersonMaskUltraV2](#PersonMaskUltraV2) 节点。 * 添加 [Color of Shadow & Highlight](#Highlight) 节点，可对暗部和亮部分别进行色彩调整。添加 [Shadow & Highlight Mask](#Shadow) 节点, 可输出暗部和亮部的遮罩。 * 添加 [CropByMaskV2](#CropByMaskV2) 节点，在原节点基础上支持```crop_box```输入，方便裁切相同尺寸的图层。 * 添加 [SimpleTextImage](#SimpleTextImage) 节点。从文字生成简单排版的图片以及遮罩。这个节点参考了[ZHO-ZHO-ZHO/ComfyUI-Text_Image-Composite](https://github.com/ZHO-ZHO-ZHO/ComfyUI-Text_Image-Composite)的部分功能和代码。 @@ -1345,9 +1346,24 @@ mask为可选输入项，如果这里输入遮罩，将作用于输出结果。 * x: 左上角位置x坐标输出。 * y: 左上角位置y坐标输出。 -### SegmentAnythingUltra -对[ComfyUI Segment Anything](https://github.com/storyicon/comfyui_segment_anything)的改进，结合了spacepxl的[ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters)的Alpha Matte节点，使遮罩有更具细节的边缘，感谢原作者。 +## Ultra节点组 +![image](image/ultra_nodes.jpg) +一组使用了超精细边缘遮罩处理方法的节点，最新版节点包括SegmentAnythingUltraV2, RmBgUltraV2, BiRefNetUltra, PersonMaskUltraV2, SegformerB2ClothesUltra 和 MaskEdgeUltraDetailV2。 +这些节点有3种边缘处理方法: +* ```PyMatting``` 通过遮罩 trimap, 对遮罩进行closed-form matting优化边缘。 +* ```GuideFilter``` 使用 opencv guidedfilter 根据颜色相似度对边缘进行羽化，对于边缘具有很强的颜色分离时效果最佳。 +以上两种方法的代码来着spacepxl的[ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters)的Alpha Matte节点，感谢原作者。 +* ```VitMatte``` 使用transfromer vit模型进行高质量的边缘处理，保留边缘细节，甚至可以生成半透明遮罩。 +注：首次运行时需要下载vitmatte模型文件，等待自动下载完成即可。如果无法完成下载，可运行命令```huggingface-cli download hustvl/vitmatte-small-composition-1k```手动下载模型。 +模型成功下载之后可以使用VITMatte(local)无需访问网络。 +* VitMatte的选项:```device``` 设置是否使用cuda进行vitmatte运算，cuda运算速度比cpu快5倍左右。```max_megapixels```设置vitmatte运算的最大图片尺寸，超大的图片将缩小处理。对于16G显存建议设置为3。 + +下图为三种方法输出区别的示例。 +![image](image/mask_edge_ultra_detail_v2_example.jpg) + +### SegmentAnythingUltra +对[ComfyUI Segment Anything](https://github.com/storyicon/comfyui_segment_anything)的改进，使遮罩有更具细节的边缘，感谢原作者。 *请参照ComfyUI Segment Anything的安装方法安装模型。如果已经正确安装了ComfyUI Segment Anything，可跳过此步骤。 * 从 [这里](https://huggingface.co/bert-base-uncased/tree/main) 下载 config.json，model.safetensors，tokenizer_config.json，tokenizer.json 和 vocab.txt 5个文件到 ```ComfyUI/models/bert-base-uncased```文件夹。 * 下载 [GroundingDINO_SwinT_OGC config file](https://huggingface.co/ShilongLiu/GroundingDINO/resolve/main/GroundingDINO_SwinT_OGC.cfg.py), [GroundingDINO_SwinT_OGC model](https://huggingface.co/ShilongLiu/GroundingDINO/resolve/main/groundingdino_swint_ogc.pth), @@ -1371,18 +1387,17 @@ mask为可选输入项，如果这里输入遮罩，将作用于输出结果。 * process_detail: 此处设为False将跳过边缘处理以节省运行时间。 * prompt: SAM的prompt输入。 - ### SegmentAnythingUltraV2 -SegmentAnythingUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:超过2K尺寸的图片使用此方法将占用大量内存) +SegmentAnythingUltra的V2升级版，增加了VITMatte边缘处理方法。 ![image](image/ultra_v2_nodes_example.jpg) 在SegmentAnythingUltra的基础上做了如下改变: ![image](image/segment_anything_ultra_v2_node.jpg) -* detail_method: 边缘处理方法。提供了VITMatte, VITMatte(local), PyMatting, GuidedFilter。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 +* detail_method: 边缘处理方法。提供了VITMatte, VITMatte(local), PyMatting, GuidedFilter。 * detail_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 * detail_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 - - +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 ### RemBgUltra 去除背景。与类似的背景移除节点相比，这个节点具有超高的边缘细节。 @@ -1401,13 +1416,15 @@ SegmentAnythingUltra的V2升级版，增加了VITMatte边缘处理方法。(注 * process_detail: 此处设为False将跳过边缘处理以节省运行时间。 ### RmBgUltraV2 -RemBgUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:超过2K尺寸的图片使用此方法将占用大量内存) +RemBgUltra的V2升级版，增加了VITMatte边缘处理方法。 在RemBgUltra的基础上做了如下改变: ![image](image/rmbg_ultra_v2_node.jpg) * detail_method: 边缘处理方法。提供了VITMatte, VITMatte(local), PyMatting, GuidedFilter。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 * detail_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 * detail_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 ### BiRefNetUltra 使用BiRefNet模型去除背景，有更好的识别能力，同时具有超高的边缘细节。 @@ -1425,7 +1442,8 @@ RemBgUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:超过2 * black_point: 边缘黑色采样阈值。 * white_point: 边缘黑色采样阈值。 * process_detail: 此处设为False将跳过边缘处理以节省运行时间。 - +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 ### PersonMaskUltra 为人物生成脸、头发、身体皮肤、衣服或配饰的遮罩。与之前的A Person Mask Generator节点相比，这个节点具有超高的边缘细节。 @@ -1447,21 +1465,21 @@ RemBgUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:超过2 * white_point: 边缘黑色采样阈值。 * process_detail: 此处设为False将跳过边缘处理以节省运行时间。 - ### PersonMaskUltraV2 -PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:超过2K尺寸的图片使用此方法将占用大量内存) +PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。 在PersonMaskUltra的基础上做了如下改变: ![image](image/person_mask_ultra_v2_node.jpg) * detail_method: 边缘处理方法。提供了VITMatte, VITMatte(local), PyMatting, GuidedFilter。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 * detail_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 * detail_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 - +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 ### SegformerB2ClothesUltra ![image](image/segformer_ultra_example.jpg) 为人物生成脸、头发、手臂、腿以及服饰的遮罩，主要用于分割服装。模型分割代码来自[StartHua](https://github.com/StartHua/Comfyui_segformer_b2_clothes)，感谢原作者。 -与comfyui_segformer_b2_clothes节点相比，这个节点具有超高的边缘细节。(注意:生成边缘超过2K尺寸的图片使用VITMatte方法将占用大量内存) +与comfyui_segformer_b2_clothes节点相比，这个节点具有超高的边缘细节。 *从[https://huggingface.co/mattmdjaga/segformer_b2_clothes](https://huggingface.co/mattmdjaga/segformer_b2_clothes)下载全部文件至```ComfyUI/models/segformer_b2_clothes```文件夹。 @@ -1488,7 +1506,32 @@ PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。(注意: * black_point: 边缘黑色采样阈值。 * white_point: 边缘黑色采样阈值。 * process_detail: 此处设为False将跳过边缘处理以节省运行时间。 +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 +### MaskEdgeUltraDetail +处理较粗糙的遮罩使其获得超精细边缘。 +![image](image/mask_edge_ultra_detail_example.jpg) + +节点选项说明: +![image](image/mask_edge_ultra_detail_node.jpg) +* method: 提供PyMatting和OpenCV-GuidedFilter两种方法处理边缘。PyMatting处理速度较慢，但是对于视频，建议使用这种方法获得更平滑的遮罩序列。 +* mask_grow: 遮罩扩张幅度。正值是向外扩张，负值是向内收缩。对于较粗糙的遮罩，通常使用负值使其边缘收缩以获得更好的效果。 +* fix_gap: 修补遮罩中的空隙。如果遮罩中有比较明显的空隙，适当调高此数值。 +* fix_threshold: 修补遮罩的阈值。 +* detail_range: 边缘细节范围。 +* black_point: 边缘黑色采样阈值。 +* white_point: 边缘黑色采样阈值。 + +### MaskEdgeUltraDetailV2 +MaskEdgeUltraDetail的V2升级版，增加了VITMatte边缘处理方法，此方法适合处理半透明区域。 +在MaskEdgeUltraDetail的基础上做了如下改变: +![image](image/mask_edge_ultra_detail_v2_node.jpg) +* method: 边缘处理方法。增加了VITMatte和VITMatte(local)方法。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 +* edge_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 +* edge_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 ### YoloV8Detect 使用YoloV8模型检测人脸、手部box区域，或者人物分割。支持输出所选择数量的通道。 @@ -1589,34 +1632,6 @@ PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。(注意: * main_subject_detect: 此项设为True将开启主体侦测，忽略主体之外的差异。 -### MaskEdgeUltraDetail -处理较粗糙的遮罩使其获得超精细边缘。 -本节点结合了spacepxl的[ComfyUI-Image-Filters](https://github.com/spacepxl/ComfyUI-Image-Filters)的Alpha Matte和Guided Filter Alpha两者功能，感谢原作者。 -![image](image/mask_edge_ultra_detail_example.jpg) - -节点选项说明: -![image](image/mask_edge_ultra_detail_node.jpg) -* method: 提供PyMatting和OpenCV-GuidedFilter两种方法处理边缘。PyMatting处理速度较慢，但是对于视频，建议使用这种方法获得更平滑的遮罩序列。 -* mask_grow: 遮罩扩张幅度。正值是向外扩张，负值是向内收缩。对于较粗糙的遮罩，通常使用负值使其边缘收缩以获得更好的效果。 -* fix_gap: 修补遮罩中的空隙。如果遮罩中有比较明显的空隙，适当调高此数值。 -* fix_threshold: 修补遮罩的阈值。 -* detail_range: 边缘细节范围。 -* black_point: 边缘黑色采样阈值。 -* white_point: 边缘黑色采样阈值。 - -### MaskEdgeUltraDetailV2 -MaskEdgeUltraDetail的V2升级版，增加了VITMatte边缘处理方法，此方法适合处理半透明区域。(注意:超过2K尺寸的图片使用此方法将占用大量内存) - -下图为三种方法输出区别的示例。 -![image](image/mask_edge_ultra_detail_v2_example.jpg) - -在MaskEdgeUltraDetail的基础上做了如下改变: -![image](image/mask_edge_ultra_detail_v2_node.jpg) -* method: 边缘处理方法。增加了VITMatte和VITMatte(local)方法。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 -* edge_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 -* edge_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 - - ### MaskGrow 对mask进行扩张收缩边缘和模糊处理 ![image](image/mask_grow_example.jpg) diff --git a/image/birefnet_ultra_node.jpg b/image/birefnet_ultra_node.jpg index 73c9f1be..b5f69f97 100644 Binary files a/image/birefnet_ultra_node.jpg and b/image/birefnet_ultra_node.jpg differ diff --git a/image/mask_edge_ultra_detail_v2_node.jpg b/image/mask_edge_ultra_detail_v2_node.jpg index aece2d24..3d06041a 100644 Binary files a/image/mask_edge_ultra_detail_v2_node.jpg and b/image/mask_edge_ultra_detail_v2_node.jpg differ diff --git a/image/person_mask_ultra_v2_node.jpg b/image/person_mask_ultra_v2_node.jpg index 25ae115c..ec2ce542 100644 Binary files a/image/person_mask_ultra_v2_node.jpg and b/image/person_mask_ultra_v2_node.jpg differ diff --git a/image/rmbg_ultra_v2_node.jpg b/image/rmbg_ultra_v2_node.jpg index 4610e4dc..98c55c77 100644 Binary files a/image/rmbg_ultra_v2_node.jpg and b/image/rmbg_ultra_v2_node.jpg differ diff --git a/image/segformer_ultra_node.jpg b/image/segformer_ultra_node.jpg index c22aeed1..1232a927 100644 Binary files a/image/segformer_ultra_node.jpg and b/image/segformer_ultra_node.jpg differ diff --git a/image/segment_anything_ultra_node.jpg b/image/segment_anything_ultra_node.jpg index 6abb2e0b..a50adb11 100644 Binary files a/image/segment_anything_ultra_node.jpg and b/image/segment_anything_ultra_node.jpg differ diff --git a/image/ultra_nodes.jpg b/image/ultra_nodes.jpg new file mode 100644 index 00000000..4c5e16ea Binary files /dev/null and b/image/ultra_nodes.jpg differ diff --git a/py/birefnet_ultra.py b/py/birefnet_ultra.py index 827c4549..07cc28be 100644 --- a/py/birefnet_ultra.py +++ b/py/birefnet_ultra.py @@ -8,7 +8,7 @@ class BiRefNetUltra: def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - + device_list = ['cuda','cpu'] return { "required": { "image": ("IMAGE",), @@ -18,6 +18,8 @@ def INPUT_TYPES(cls): "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), }, "optional": { } @@ -29,7 +31,7 @@ def INPUT_TYPES(cls): CATEGORY = '😺dzNodes/LayerMask' def birefnet_ultra(self, image, detail_method, detail_erode, detail_dilate, - black_point, white_point, process_detail): + black_point, white_point, process_detail, device, max_megapixels): ret_images = [] ret_masks = [] @@ -58,7 +60,7 @@ def birefnet_ultra(self, image, detail_method, detail_erode, detail_dilate, _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = tensor2pil(_mask) diff --git a/py/imagefunc.py b/py/imagefunc.py index de0abe6a..30151265 100644 --- a/py/imagefunc.py +++ b/py/imagefunc.py @@ -1366,7 +1366,6 @@ def load_RMBG_model(): return net - def RMBG(image:Image) -> Image: rmbgmodel = load_RMBG_model() w, h = image.size @@ -1385,6 +1384,39 @@ def RMBG(image:Image) -> Image: _mask = torch.from_numpy(np.squeeze(im_array).astype(np.float32)) return tensor2pil(_mask) +def guided_filter_alpha(image:torch.Tensor, mask:torch.Tensor, filter_radius:int) -> torch.Tensor: + sigma = 0.15 + d = filter_radius + 1 + mask = pil2tensor(tensor2pil(mask).convert('RGB')) + if not bool(d % 2): + d += 1 + s = sigma / 10 + i_dup = copy.deepcopy(image.cpu().numpy()) + a_dup = copy.deepcopy(mask.cpu().numpy()) + for index, image in enumerate(i_dup): + alpha_work = a_dup[index] + i_dup[index] = guidedFilter(image, alpha_work, d, s) + return torch.from_numpy(i_dup) + +#pymatting edge detail +def mask_edge_detail(image:torch.Tensor, mask:torch.Tensor, detail_range:int=8, black_point:float=0.01, white_point:float=0.99) -> torch.Tensor: + from pymatting import fix_trimap, estimate_alpha_cf + d = detail_range * 5 + 1 + mask = pil2tensor(tensor2pil(mask).convert('RGB')) + if not bool(d % 2): + d += 1 + i_dup = copy.deepcopy(image.cpu().numpy().astype(np.float64)) + a_dup = copy.deepcopy(mask.cpu().numpy().astype(np.float64)) + for index, img in enumerate(i_dup): + trimap = a_dup[index][:, :, 0] # convert to single channel + if detail_range > 0: + trimap = cv2.GaussianBlur(trimap, (d, d), 0) + trimap = fix_trimap(trimap, black_point, white_point) + alpha = estimate_alpha_cf(img, trimap, laplacian_kwargs={"epsilon": 1e-6}, + cg_kwargs={"maxiter": 500}) + a_dup[index] = np.stack([alpha, alpha, alpha], axis=-1) # convert back to rgb + return torch.from_numpy(a_dup.astype(np.float32)) + class VITMatteModel: def __init__(self,model,processor): self.model = model @@ -1397,31 +1429,45 @@ def load_VITMatte_model(model_name:str, local_files_only:bool=False) -> object: vitmatte = VITMatteModel(model, processor) return vitmatte -def generate_VITMatte(image:Image, trimap:Image, local_files_only:bool=False) -> Image: +def generate_VITMatte(image:Image, trimap:Image, local_files_only:bool=False, device:str="cpu", max_megapixels:float=2.0) -> Image: if image.mode != 'RGB': image = image.convert('RGB') if trimap.mode != 'L': trimap = trimap.convert('L') - max_size = 2048 + max_megapixels *= 1048576 width, height = image.size ratio = width / height - target_width = math.sqrt(ratio * max_size * max_size) + target_width = math.sqrt(ratio * max_megapixels) target_height = target_width / ratio target_width = int(target_width) target_height = int(target_height) - log(f"vitmatte image size {width}x{height} too large, resize to {target_width}x{target_height} for processing.") - if width * height > max_size * max_size: + if width * height > max_megapixels: image = image.resize((target_width, target_height), Image.BILINEAR) trimap = trimap.resize((target_width, target_height), Image.BILINEAR) + log(f"vitmatte image size {width}x{height} too large, resize to {target_width}x{target_height} for processing.") model_name = "hustvl/vitmatte-small-composition-1k" + if device=="cpu": + device = torch.device('cpu') + else: + if torch.cuda.is_available(): + device = torch.device('cuda') + else: + log("vitmatte device is set to cuda, but not available, using cpu instead.") + device = torch.device('cpu') vit_matte_model = load_VITMatte_model(model_name=model_name, local_files_only=local_files_only) + vit_matte_model.model.to(device) + log(f"vitmatte processing, image size = {image.width}x{image.height}, device = {device}.") inputs = vit_matte_model.processor(images=image, trimaps=trimap, return_tensors="pt") with torch.no_grad(): + inputs = {k: v.to(device) for k, v in inputs.items()} predictions = vit_matte_model.model(**inputs).alphas + if torch.cuda.is_available(): + torch.cuda.empty_cache() + torch.cuda.ipc_collect() mask = tensor2pil(predictions).convert('L') mask = mask.crop( (0, 0, image.width, image.height)) # remove padding that the prediction appends (works in 32px tiles) - if width * height > max_size * max_size: + if width * height > max_megapixels: mask = mask.resize((width, height), Image.BILINEAR) return mask @@ -1464,38 +1510,6 @@ def get_a_person_mask_generator_model_path() -> str: wget.download(model_url, model_file_path) return model_file_path -def mask_edge_detail(image:torch.Tensor, mask:torch.Tensor, detail_range:int=8, black_point:float=0.01, white_point:float=0.99) -> torch.Tensor: - from pymatting import fix_trimap, estimate_alpha_cf - d = detail_range * 5 + 1 - mask = pil2tensor(tensor2pil(mask).convert('RGB')) - if not bool(d % 2): - d += 1 - i_dup = copy.deepcopy(image.cpu().numpy().astype(np.float64)) - a_dup = copy.deepcopy(mask.cpu().numpy().astype(np.float64)) - for index, img in enumerate(i_dup): - trimap = a_dup[index][:, :, 0] # convert to single channel - if detail_range > 0: - trimap = cv2.GaussianBlur(trimap, (d, d), 0) - trimap = fix_trimap(trimap, black_point, white_point) - alpha = estimate_alpha_cf(img, trimap, laplacian_kwargs={"epsilon": 1e-6}, - cg_kwargs={"maxiter": 500}) - a_dup[index] = np.stack([alpha, alpha, alpha], axis=-1) # convert back to rgb - return torch.from_numpy(a_dup.astype(np.float32)) - -def guided_filter_alpha(image:torch.Tensor, mask:torch.Tensor, filter_radius:int) -> torch.Tensor: - sigma = 0.15 - d = filter_radius + 1 - mask = pil2tensor(tensor2pil(mask).convert('RGB')) - if not bool(d % 2): - d += 1 - s = sigma / 10 - i_dup = copy.deepcopy(image.cpu().numpy()) - a_dup = copy.deepcopy(mask.cpu().numpy()) - for index, image in enumerate(i_dup): - alpha_work = a_dup[index] - i_dup[index] = guidedFilter(image, alpha_work, d, s) - return torch.from_numpy(i_dup) - def mask_fix(images:torch.Tensor, radius:int, fill_holes:int, white_threshold:float, extra_clip:float) -> torch.Tensor: d = radius * 2 + 1 i_dup = copy.deepcopy(images.cpu().numpy()) diff --git a/py/mask_edge_ultra_detail_v2.py b/py/mask_edge_ultra_detail_v2.py index 8140e8cf..a38ed8d1 100644 --- a/py/mask_edge_ultra_detail_v2.py +++ b/py/mask_edge_ultra_detail_v2.py @@ -10,7 +10,7 @@ def __init__(self): def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - + device_list = ['cuda','cpu'] return { "required": { "image": ("IMAGE",), @@ -23,6 +23,8 @@ def INPUT_TYPES(cls): "edte_dilate": ("INT", {"default": 6, "min": 1, "max": 255, "step": 1}), "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), }, "optional": { } @@ -34,7 +36,7 @@ def INPUT_TYPES(cls): CATEGORY = '😺dzNodes/LayerMask' def mask_edge_ultra_detail_v2(self, image, mask, method, mask_grow, fix_gap, fix_threshold, - edge_erode, edte_dilate, black_point, white_point,): + edge_erode, edte_dilate, black_point, white_point, device, max_megapixels,): ret_images = [] ret_masks = [] l_images = [] @@ -72,7 +74,7 @@ def mask_edge_ultra_detail_v2(self, image, mask, method, mask_grow, fix_gap, fix _mask = tensor2pil(mask_edge_detail(_image, _mask, detail_range//8, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, edge_erode, edte_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) ret_image = RGB2RGBA(orig_image, _mask.convert('L')) diff --git a/py/person_mask_ultra_v2.py b/py/person_mask_ultra_v2.py index bd7de6c8..bbd4b893 100644 --- a/py/person_mask_ultra_v2.py +++ b/py/person_mask_ultra_v2.py @@ -18,7 +18,7 @@ def __init__(self): def INPUT_TYPES(self): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - + device_list = ['cuda','cpu'] return { "required": { @@ -36,6 +36,8 @@ def INPUT_TYPES(self): "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), }, "optional": { @@ -64,7 +66,7 @@ def get_mediapipe_image(self, image: Image): def person_mask_ultra_v2(self, images, face, hair, body, clothes, accessories, background, confidence, detail_method, detail_erode, detail_dilate, - black_point, white_point, process_detail): + black_point, white_point, process_detail, device, max_megapixels,): import mediapipe as mp a_person_mask_generator_model_path = get_a_person_mask_generator_model_path() @@ -151,7 +153,7 @@ def person_mask_ultra_v2(self, images, face, hair, body, clothes, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = mask2image(_mask) diff --git a/py/rmbg_ultra_v2.py b/py/rmbg_ultra_v2.py index 9bae40c2..f071177e 100644 --- a/py/rmbg_ultra_v2.py +++ b/py/rmbg_ultra_v2.py @@ -10,7 +10,7 @@ def __init__(self): def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - + device_list = ['cuda','cpu'] return { "required": { "image": ("IMAGE",), @@ -20,6 +20,8 @@ def INPUT_TYPES(cls): "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), }, "optional": { } @@ -31,7 +33,7 @@ def INPUT_TYPES(cls): CATEGORY = '😺dzNodes/LayerMask' def rmbg_ultra_v2(self, image, detail_method, detail_erode, detail_dilate, - black_point, white_point, process_detail): + black_point, white_point, process_detail, device, max_megapixels): ret_images = [] ret_masks = [] @@ -56,7 +58,7 @@ def rmbg_ultra_v2(self, image, detail_method, detail_erode, detail_dilate, _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = mask2image(_mask) diff --git a/py/segformer_ultra.py b/py/segformer_ultra.py index 7cdd5506..dcc3d3c9 100644 --- a/py/segformer_ultra.py +++ b/py/segformer_ultra.py @@ -40,6 +40,7 @@ def __init__(self): @classmethod def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] + device_list = ['cuda','cpu'] return {"required": { "image":("IMAGE",), @@ -62,9 +63,11 @@ def INPUT_TYPES(cls): "detail_method": (method_list,), "detail_erode": ("INT", {"default": 12, "min": 1, "max": 255, "step": 1}), "detail_dilate": ("INT", {"default": 6, "min": 1, "max": 255, "step": 1}), - "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), + "black_point": ("FLOAT", {"default": 0.15, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), } } @@ -76,7 +79,7 @@ def INPUT_TYPES(cls): def segformer_ultra(self, image, face, hat, hair, sunglass, upper_clothes, skirt, pants, dress, belt, shoe, left_leg, right_leg, left_arm, right_arm, bag, scarf, detail_method, - detail_erode, detail_dilate, black_point, white_point, process_detail + detail_erode, detail_dilate, black_point, white_point, process_detail, device, max_megapixels, ): ret_images = [] @@ -144,7 +147,7 @@ def segformer_ultra(self, image, _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = mask2image(_mask) diff --git a/py/segment_anything_ultra_v2.py b/py/segment_anything_ultra_v2.py index 5db8e91b..9b60f13a 100644 --- a/py/segment_anything_ultra_v2.py +++ b/py/segment_anything_ultra_v2.py @@ -16,7 +16,7 @@ def __init__(self): def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - + device_list = ['cuda','cpu'] return { "required": { "image": ("IMAGE",), @@ -30,6 +30,8 @@ def INPUT_TYPES(cls): "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), "prompt": ("STRING", {"default": "subject"}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), }, "optional": { } @@ -42,8 +44,9 @@ def INPUT_TYPES(cls): def segment_anything_ultra_v2(self, image, sam_model, grounding_dino_model, threshold, detail_method, detail_erode, detail_dilate, - black_point, white_point, process_detail, - prompt, ): + black_point, white_point, process_detail, prompt, + device, max_megapixels + ): global SAM_MODEL global DINO_MODEL global previous_sam_model @@ -81,7 +84,7 @@ def segment_anything_ultra_v2(self, image, sam_model, grounding_dino_model, thre _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(_image, _trimap, local_files_only=local_files_only) + _mask = generate_VITMatte(_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = mask2image(_mask) diff --git a/pyproject.toml b/pyproject.toml index c4f65988..4f940c92 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "comfyui_layerstyle" description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress." -version = "1.0.10" +version = "1.0.11" license = "MIT" dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "opencv-contrib-python", "pymatting", "segment_anything", "timm", "addict", "yapf", "colour-science", "wget", "mediapipe", "loguru", "typer_config", "fastapi", "rich", "google-generativeai", "diffusers", "omegaconf", "tqdm", "transformers", "kornia", "image-reward", "ultralytics", "blend_modes", "blind-watermark", "qrcode", "pyzbar", "psd-tools"]