Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mat1 and mat2 must have the same dtype, but got BFloat16 and Float #577

Open
hjj-lmx opened this issue Feb 21, 2025 · 0 comments
Open

mat1 and mat2 must have the same dtype, but got BFloat16 and Float #577

hjj-lmx opened this issue Feb 21, 2025 · 0 comments

Comments

@hjj-lmx
Copy link

hjj-lmx commented Feb 21, 2025

# 初始化推理状态
    inference_state = sam2_model.video_predictor.init_state(video_path=input_video_path)

    # 传入的是多帧的mask数组
    for mask_object in mask_objects:
        mask_byte = down_file.url_to_byte(mask_object.get("mask_url"))
        image_data = io.BytesIO(mask_byte)
        mask_image = Image.open(image_data).convert('L')  # 转换为灰度图
        transform = transforms.Compose([
            transforms.ToTensor(),
            transforms.Lambda(lambda x: (x > 0.5).float())
        ])
        mask = transform(mask_image)
        mask = mask.squeeze(0)
        sam2_model.video_predictor.add_new_mask(inference_state,
                                                mask_object.get("frame_idx"),
                                                mask_object.get("frame_idx"),
                                                mask)
    # 传播提示获取整个视频的masklet
    video_segments = {}
    for out_frame_idx, out_obj_ids, out_mask_logits in sam2_model.video_predictor.propagate_in_video(
            inference_state):
        video_segments[out_frame_idx] = {
            out_obj_id: (out_mask_logits[i] > 0.0).cpu().numpy()
            for i, out_obj_id in enumerate(out_obj_ids)
        }

mask_objects的长度是1的适合能正常运行,大于1就报错

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant