How to overlay 3D cam heatmap on the 3D image and visualize it slice by slice?

[guyucowboy]

Hi, frgfm.
Could you provide a example to overlay 3D cam on the 3D image and visualize slice by slice?
Thanks for your sharing. I found the code is helpful. But I do not know how to use the code for visualize a 3D cube image with shape (32,32,32). I want to visualize the result slice by slice. Then, 3D cam heatmap can be saved in 2D ways.
Thank you very much!
Best regards.
Yu Gu

[frgfm]

Hello @guyucowboy 👋

My apologies for the late reply!
This is a very interesting question, I'll try to answer as best as I can:

• beforehand, I have to say: I'm not a data visualization expert so don't take my opinion for the everlasting truth
• common image related-visualization leverage 2 spatial dimensions and colors to "render" the values: in the case of a CAM of an image, the CAM ends up with values in the range [0, 1] across 2 spatial dimensions. So we use colors to map the values to a meaningful palet and that's the magic

import torch
import matplotlib.pyplot as plt

# Let's take a random cam
cam_2d = torch.rand((32, 32))
# Matplotlib will automatically select a colormap to render the values
plt.imshow(cam_2d.numpy()); plt.show()

• Ok so this was for 2D tensors, now you asked about 3D spatial tensors, which brings a new difficulty. Matplotlib can render 3D plots, but there is a data visualization question here: say we can plot the spatial structures, how do we render the activation this time? If we only use RGB, considering it's a cube filled with values, you won't see anything but the outside. In the end, I can see a few possibilities:
  • use a fixed value for RGB, and map the data value to the "alpha" channel with translate to transparency
  • use a fixed alpha value (lower than 0.5 so that you can see through the cube), and map the data value with RGB (using a colormap)
  • use both RGB & alpha to render the values
  • non-static option: animate the plot to visualize the slice

I'll illustrate all three, feel free to pick the option that suits you best!

import torch
import matplotlib.pyplot as plt
from matplotlib.cm import get_cmap

# Let's take a random cam
cam_3d = torch.rand((32, 32, 32))

ax = plt.subplot(111, projection='3d')
# Initialize the color with RGBA
colors = np.empty((*cam_3d.shape, 4))
# Fixed RGB (grey), Alpha as values
colors[..., :3] = .5
# Alpha = 1 means no transparency, so I rescale down the values
colors[..., -1] = 0.5 * cam_3d.numpy()
ax.voxels(np.ones(cam_3d.shape, dtype=bool), facecolors=colors); plt.show()

# Now with alpha fixed
ax = plt.subplot(111, projection='3d')
# Fixed Alpha, RGB as values (using a colormap)
colors[..., -1] = .5
# Pick a colormap here https://matplotlib.org/stable/tutorials/colors/colormaps.html
# I personally use jet for TorchCAM
cmap = get_cmap('jet')
colors[..., :3] = cmap(cam_3d.numpy())[..., :-1]
ax.voxels(np.ones(cam_3d.shape, dtype=bool), facecolors=colors); plt.show()

# Finally with RGBA to display values
ax = plt.subplot(111, projection='3d')
# Combine both previous iterations
colors[..., -1] = 0.5 * cam_3d.numpy()
cmap = get_cmap('jet')
colors[..., :3] = cmap(cam_3d.numpy())[..., :-1]
ax.voxels(np.ones(cam_3d.shape, dtype=bool), facecolors=colors); plt.show()

Then the choice is all a matter of taste & use cases!

Final option I had in mind: animate along a given axis.
And here is how to "visualize by slice" along the first axis:

import torch
import matplotlib.pyplot as plt
import matplotlib.animation as animation

# Let's take a random cam
cam_3d = torch.rand((32, 32, 32))
npy_cam = cam_3d.numpy()

fig, ax = plt.subplots()

img = ax.imshow(npy_cam[0, ...])

# For the animation (imagine a GIF), we have to set the starting point
def reset():
    img.set_data(npy_cam[0, ...])
    return (img,)
# Now we defined how frame are defined
def animate(idx):
    img.set_data(npy_cam[idx, ...])
    return (img,)

anim = animation.FuncAnimation(fig, animate, init_func=reset, frames=cam_3d.shape[0], interval=20, blit=True)
plt.show()

which produces this:

I hope this helps, let me know if this isn't clear :)

[guyucowboy]

Hi, frgfm.
Thanks for your reply and great helps !
Now, I know how to render the existing 3D cam heatmap on the 3D image with your great helps!
But I still do not know how to generate the 3D cam for the 3D cnn.
I did it like this as follows. The model is a 3D Resnet CNN. But there should be someting wrong in the code.

def cam_visualization_3Dcube(model, test_loader):
    model.eval()
    for batch_idx, (inputs, targets) in enumerate(test_loader):
        inputs, targets = inputs.cuda(), targets.cuda()  # sizes of input: [1,1,32,32,32], # [batch, channel, Z, Y, X]
        cam_extractor = GradCAM(model, target_layer='layer4', input_shape=(32, 32, 32) )  
        out = model(inputs)
        activation_map = cam_extractor(out.squeeze(0).argmax().item(), out)  
        activation_map_numpy = activation_map[0].cpu().numpy()

When I debug my code, I found activation_map = {list}<class 'list'>: [tensor ([[[nan]]], device = 'cuda:0')]
I found in the core.py, line 169-174.

            if self._relu:
                cam = F.relu(cam, inplace=True) 
                # in my code, cam here is cam={Tensor} ([[[4.2340]]],device ='cuda:0')          
            # Normalize the CAM
            if normalized:
                cam = self._normalize(cam)
                # in my code, cam here is cam={Tensor} ([[[nan]]],device ='cuda:0')      

In addition, after executing the code, weights = self._get_weights(class_idx, scores). I found 'weights' is a list with a single Tensor in my code. Then for loop for weight, activation in zip(weights, self.hook_a): is executed only once.

In my code, I found cam became nan after normalized.

Then, I set normalized as False in the core.py in line 137.
def __call__(self, class_idx: int, scores: Optional[Tensor] = None, normalized: bool = False) -> List[Tensor]:

I found activation_map = {list}<class 'list'>: [tensor ([[[4.2340]]], device = 'cuda:0')].
activation_map is a single value rather than a matrix.

There should be something wrong in the function cam_visualization_3Dcube. But I do not know how to generate 3D cam for 3D CNN.

Thanks for your great helps! I am looking forward to your reply.
Best regards
Yu Gu

[frgfm]

Oh I misunderstood your problem then!

It's very hard for me to debug this without having the full code 😅
You mention some line numbers but I don't have access to the code, so I can't do much, unfortunately. Could you either:

• share your code in a github repo, or in a Colab notebook?
• share the minimal snippet to reproduce the bug here + the full traceback of the error (not only the final message but the entire tracing of the error)

Best

[guyucowboy]

Hi, frgfm.
Thank you very much for your reply! Sorry to reply you so late! I have put my data (including a small data set and several .py files) into colab to run. But when I planned to share the entire program with you, I found that only .ipynb files can be shared without data. Since I am not proficient in programming so much, I have not uploaded codes on GitHub instead of forking others’ projects until now. Thus, I am not very familiar with the process of uploading the codes with using git bash. Therefore, I transferred the relevant code to Google Drive (https://drive.google.com/drive/folders/1Bo-27ZnRti3UMaGuvltkVmdNFAkkBkjO?usp=sharing). Or, if necessary, I email you the code (about 200M). Thank you very much! Looking forward to your reply!

PS:

1. I ran the collect_env.py file you mentioned, and here is my environment:
   TorchCAM version: N/A
   PyTorch version: 1.8.0
   OS: Microsoft Windows 10
   Python version: 3.7.10
   Is CUDA available: Yes
   CUDA runtime version: 11.1.74
   GPU models and configuration: GPU 0: GeForce RTX 3090
   Nvidia driver version: 457.85
   cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\cudnn_ops_train64_8.dll

2. Here, I will introduce you the detailed information about how to run this project.
   In short, you only need to run ‘cam_3dcube.py’. Other folders, such as ‘data’ folder, or ‘dataloader.py’, etc. , provide support for running ‘cam_3dcube.py’. You only need to make the location of each folder in the project unchanged, the program can run.
   The result of the print function in the project would be "activation_map_numpy: [[[nan]]]" and "activation_map_numpy.shape: (1, 1, 1)"
   Please help me to solve why this happens. Thank you very much!
   I am looking forward to your reply.
   Best regards.
   Yu Gu

[frgfm]

Thanks for that!

You mentioned a colab, but this is a link to a google drive, right? (a colab would be this for instance: https://colab.research.google.com/github/frgfm/notebooks/blob/main/torch-cam/quicktour.ipynb)

About your environment, it looks like you haven't installed torcham 🤔 Perhaps you only copied or cloned the repo but didn't install the project?

You mentioned that the error was happening on line 137 of your script. But that would be the inference of the model?
Either way, could you paste the full traceback of you error please? 🙏

[guyucowboy]

Hi frgfm!
Thank you for your reply! I studied the .ipynb file you sent and thought about it again. The main program: ‘cam_3dcube.py’ has a complicated calling relationship with other files, such as ‘dataloader.py’ and ‘transforms.py’, and it does not seem to be concise enough to put them all in one .ipynb. In addition, since my code is a 3d network, the part of the network changed from 2d to 3d needs to be modified by myself, so it cannot downloaded directly from the Internet.

Thank you for your understanding!

Besides, you mentioned uploading the code to GitHub before. I studied it these days and I have successfully uploaded the code to GitHub! The following is a link to my code:https://github.com/Ljiaqii/cam_3d_cube_project/tree/master

And, I wrote a detailed readme.md, which contains the output of the activation_map_numpy is ‘nan’, the shape of the activation_map_numpy.:(1, 1, 1) , the error track of the program, the environment, and the packages used.
Thank you so much for helping me to solve the problem!
Best regards!
Yu Gu

[Ljiaqii]

Thank you for your help, I have learned a lot!

I set batchsize to 1 and put only one image inside, so this should do not matter.

When you have time, I hope we can continue to communicate. Thanks again! Best regards!

[frgfm]

You're welcome! Did it solve your problem to replace this with my suggestion?

[Ljiaqii]

Thanks for your help!

The two modifications you provided are for the second sentence of my code:

activation_map_numpy=activation_map[0].cpu().numpy()

result=overlay_mask(to_pil_image(inputs),to_pil_image(activation_map[0],mode='F'),alpha=0.5)

However, in L126, activation_map_numpy = activation_map[0].cpu().numpy(), the output is already nan.

activation_map_numpy:[[[nan]]]
activation_map_numpy.shape:(1, 1, 1)

Therefore, only changing the second sentence will still not change the output.
(https://github.com/Ljiaqii/cam_3d_cube_project/blob/master/cam_3dcube.py#L126-L129)

Looking forward to continue communicating with you. Best regards!

[frgfm]

The fact that the activation map is of shape (1, 1, 1) is probably indicating that:

• either your input tensor is too small
• you targeted a layer where the image is (1, 1, 1) (not very interesting 😅 )

In your script, you use input of shape (32, 32, 32) so this is the second option. Since the CAMs are normalized by dividing by the spatial variance, which is 0 here (because you only have 1 spatial element), I suggest increasing the input size or target an earlier layer :)

[Ljiaqii]

Thank you for your reply!

I don't quite understand what you mean by this statement: Since the CAMs are normalized by dividing by the spatial variance, which is 0 here (because you only have 1 spatial element). Could you please explain in more detail the meaning of this sentence, it would be greatly appreciated!

When I replaced layer4 with layer1, layer2, layer3, activation_map_numpy no longer outputs nan values! You are right! Thank you very much!

Take the output of layer3 as an example:

activation_map_numpy:[[[0.11956415 0.0753939 ]
[0.41538852 0.3445538 ]]

[[0. 0.20141964]
[0.33995917 1. ]]]
activation_map_numpy.shape:(2, 2, 2)
As for increasing the input size, the input of my main program is [32,32,32], which is not easy to change.

With your previous help, I've managed to animate the display! The code used is the one you provided earlier:

# defining the test model
def cam_visualization_single_cube(model, test_loader):
    model.eval()

    # input[1,1,32,32,32] [batch, channel, Z,Y,X]
    for batch_idx, (inputs, targets) in enumerate(test_loader):
        inputs, targets = inputs.cuda(), targets.cuda()
        # replace layer4 with layer3
        cam_extractor = SmoothGradCAMpp(model, target_layer='layer3',
                                        input_shape=(32, 32, 32))

        out = model(inputs)
        class_idx = out.squeeze(0).squeeze(0).argmax().item()
        activation_map = cam_extractor(class_idx, out)
        # activation_map = cam_extractor(class_idx = 0, scores = out) # class_idx=0 or 1
        activation_map_numpy = activation_map[0].cpu().numpy()

        # The following is modified from your previous response in the discussion.
        fig, ax = plt.subplots()
       # activation_map_numpy is equivalent to npy_cam
        img = ax.imshow(activation_map_numpy[0, ...])

        def reset():
            img.set_data(activation_map_numpy[0, ...])
            return (img,)

        # Now we defined how frame are defined
        def animate(idx):
            img.set_data(activation_map_numpy[idx, ...])  # slice
            return (img,)

        print(f'activation_map_numpy:{activation_map_numpy}')
        print(f'activation_map_numpy.shape:{activation_map_numpy.shape}')

        anim = animation.FuncAnimation(fig, animate, init_func=reset, frames=activation_map_numpy[0].shape, interval=20, blit=True)

        plt.show()
        # add squeeze
        inputs_new = inputs.squeeze()
        # activation_map_new = activation_map_numpy.squeeze()
        print(f'inputs_new.shape:{inputs_new.shape}')
        # replace inputs with inputs_new
        result = overlay_mask(to_pil_image(inputs_new[0]), to_pil_image(activation_map_numpy[0], mode='F'), alpha=0.5)
        # results = [overlay_mask(to_pil_image(img.cpu()), to_pil_image(amap, mode='F'), alpha=0.5) for img, amap in zip(inputs, activation_map)] # 1

        # result = overlay_mask(to_pil_image(inputs[0][0].cpu()), to_pil_image(activation_map[0][0], mode='F'), alpha=0.5) # 2

When I run this code, although it can be visualized, there is still an error, which I will explain in detail in a later discussion.

Now I just want to show a certain slice. For example, when the cube size is 32×32×32, z, y, x, I only want to display the CAM image of this layer when z=5, and overlay it on the original image.
But when I use

# anim = animation.FuncAnimation(fig, animate, init_func=reset, frames=activation_map_numpy[0].shape, interval=20, blit=True)
# plt.show()
inputs_new = inputs.squeeze()
result = overlay_mask(to_pil_image(inputs_new[0]), to_pil_image(activation_map_numpy[0], mode='F'), alpha=0.5)

The program gives the following error:

Traceback (most recent call last):
inputs_new.shape:torch.Size([32, 32, 32])
File "D:/LiuJiaqi/cam_3d_cube_project/cam_3dcube_liu.py", line 174, in
cam_visualization_single_cube(net, test_data_loader)
File "D:/LiuJiaqi/cam_3d_cube_project/cam_3dcube_liu.py", line 154, in cam_visualization_single_cube
result = overlay_mask(to_pil_image(inputs_new[0]), to_pil_image(activation_map_numpy[0], mode='F'), alpha=0.5)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\torchcam\utils.py", line 47, in overlay_mask
overlayed_img = Image.fromarray((alpha * np.asarray(img) + (1 - alpha) * overlay).astype(np.uint8))
ValueError: operands could not be broadcast together with shapes (32,32) (32,32,3)

Process finished with exit code 1

We have tried many methods but still haven't solved this error. If you have a good idea, could you please tell me how to solve it?
Thank you again for your help! Best regards!

[Ljiaqii]

In the previous discussion, in order to slice and display the cube, I # added the following two lines of code:

        anim = animation.FuncAnimation(fig, animate, init_func=reset, frames=activation_map_numpy[0].shape, interval=20, blit=True)

        plt.show()

The following is the error track:

Exception in Tkinter callback
Traceback (most recent call last):
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\tkinter_init_.py", line 749, in callit
func(*args)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\backends_backend_tk.py", line 139, in _on_timer
super()._on_timer()
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\backend_bases.py", line 1216, in _on_timer
ret = func(*args, **kwargs)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\animation.py", line 1477, in _step
still_going = super()._step(*args)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\animation.py", line 1189, in _step
self._draw_next_frame(framedata, self._blit)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\animation.py", line 1208, in _draw_next_frame
self._draw_frame(framedata)
File "E:\anaconda3.8\envs\pytorch1.8_cuda11.1_cudnn8.0.5_py37_ljq_naslung\lib\site-packages\matplotlib\animation.py", line 1776, in _draw_frame
self._drawn_artists = self._func(framedata, *self._args)
File "D:/LiuJiaqi/cam_3d_cube_project/cam_3dcube_liu.py", line 140, in animate
img.set_data(activation_map_numpy[idx, ...]) # slice
IndexError: index 2 is out of bounds for axis 0 with size 2

Thanks again!

[Ljiaqii]

Thank you for your help!
Your last reply gave us a hint, thank you very much!
We have successfully implemented the slice by slice visualization. According to the README, the visualization at layer4 should be the most accurate. However, our program achieves the best effect in layer1 (the cube is cropped centered on the lung nodule, and layer1 can correctly locate the position of the lung nodule); the effects of layer2 and layer3 decrease in turn.
As the picture shows:

Now I have two questions:

1. Is there any problem with the code we implemented?
   (Since to_pil_image can only display 2D numpy, we use the 3D numpy display performed by the show_cam_on_image function)
2. Why does layer1 work best?
   Here is the code we implemented:

def show_cam_on_image(img: np.ndarray,
                      mask: np.ndarray,
                      use_rgb: bool = False,
                      colormap: int = cv2.COLORMAP_JET) -> np.ndarray:
    """ This function overlays the cam mask on the image as an heatmap.
    By default the heatmap is in BGR format.

    :param img: The base image in RGB or BGR format.
    :param mask: The cam mask.
    :param use_rgb: Whether to use an RGB or BGR heatmap, this should be set to True if 'img' is in RGB format.
    :param colormap: The OpenCV colormap to be used.
    :returns: The default image with the cam overlay.
    """
    heatmap = cv2.applyColorMap(np.uint8(255 * mask), colormap)
    if use_rgb:
        heatmap = cv2.cvtColor(heatmap, cv2.COLOR_BGR2RGB)
    heatmap = np.float32(heatmap) / 255
    if np.max(img) > 1:
        raise Exception(
            "The input image should np.float32 in the range [0, 1]")

    # img changed from (32,32) to (32,32,3), used to add to heatmap
    # img is a grayscale image, convert it to a grayscale image in RGB format
    img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)

    cam = heatmap + img
    cam = cam / np.max(cam)
    return np.uint8(255 * cam)

# normalize the array [0-1]
def normalization(data):
    _range =  np.max(data)-np.min(data)
    return (data - np.min(data))/_range

def cam_visualization_single_cube(model, test_loader):
    model.eval()

    # input[1,1,32,32,32] [batch, channel, Z,Y,X]
    for batch_idx, (inputs, targets) in enumerate(test_loader):
        inputs, targets = inputs.cuda(), targets.cuda()
        # target_layer can be layer1, layer2, layer3. Currently layer1 works best
        cam_extractor = SmoothGradCAMpp(model, target_layer='layer1',
                                        input_shape=(32, 32, 32))

        out = model(inputs)
        class_idx = out.squeeze(0).squeeze(0).argmax().item()
        activation_map = cam_extractor(class_idx, out)
        # activation_map = cam_extractor(class_idx = 0, scores = out) # class_idx=0 or 1
        activation_map_numpy = activation_map[0].cpu().numpy()

        print(f'activation_map_numpy:{activation_map_numpy}')
        print(f'activation_map_numpy.shape:{activation_map_numpy.shape}')
        # add squeeze
        inputs_new = inputs.squeeze()
        # activation_map_new = activation_map_numpy.squeeze()
        print(f'inputs_new.shape:{inputs_new.shape}')

        inputs_new_numpy = inputs_new.cpu().numpy()
        # Normalize the input data [0-1], show_cam_on_image requires the input image to be in [0,1]
        inputs_new_numpy_norm = normalization(inputs_new_numpy)
        # inputs_new_numpy = Image.fromarray(inputs_new_numpy)  # Numpy to image class, but 3d numpy will make an error

        resize_factor = inputs_new_numpy.shape[0]/activation_map_numpy[0].shape[0]  # resize_factor:32/2 = 16
        # from scipy.ndimage.interpolation import zoom
        activation_map_numpy_resized = zoom(activation_map_numpy, resize_factor, mode = 'nearest', order = 2)

        for idx in range(inputs_new_numpy.shape[0]):  # Cycle display
            # idx = 0 # first select the 0th layer slice, idx value range: 0-31
            inputs_new_norm_slice =inputs_new_numpy_norm[idx,:,:]  #
            activation_map_numpy_slice = activation_map_numpy_resized[idx, :, :]

            inputs_new_norm_slice_original = cv2.cvtColor(inputs_new_norm_slice, cv2.COLOR_GRAY2RGB)
            # Convert the range from 0-1 to 0-255
            inputs_new_norm_slice_original = np.uint8(255 * inputs_new_norm_slice_original)
            # Realize the display of a CAM and load it on the original image
            # Note that the input here is still inputs_new_norm_slice
            cv2.imwrite(f'cube_{idx}_original.jpg', inputs_new_norm_slice_original)
            cam_image = show_cam_on_image(inputs_new_norm_slice, activation_map_numpy_slice, use_rgb=True)
            # cam_image is RGB encoded whereas "cv2.imwrite" requires BGR encoding.
            cam_image = cv2.cvtColor(cam_image, cv2.COLOR_RGB2BGR)
            # Realize the display of a CAM and load it on the original image
            cv2.imwrite(f'cube_{idx}_with_cam.jpg', cam_image)


        print('finish')

Best regards!

[frgfm]

According to the README, the visualization at layer4 should be the most accurate

Which README are you referring to? 🤔
The choice of the layer is entirely up to you, it just happens that, without fusion, people are usually interested in the last layers for 2D Conv networks. In your case, I would suggest checking the fuse_cams method of LayerCAM over here: https://frgfm.github.io/torch-cam/methods.html#torchcam.methods.LayerCAM

Also, the visualization will only be relevant if your layer's output size is relevant for visualization. For instance, if you have an output of spatial shape (2, 2), there isn't much to display that would be interesting. So in your case, the input shape makes it that layer4 would be (1, 1, 1) if I'm not mistaking. So I'm not surprised you don't get nice visualizations 😅
Layer1 happens to be the layer with the biggest output shape, so I would look for more reasons than this (the rest is entirely up to your architecture and its parameters)

One last thing: I understand it's only for visualization, but in your code, for the normalization part, divide by the range of values instead of the dynamic max. For instance, if this is an RGB or grayscale image, usually it's encoded in uint8, meaning [0, 255]. And in that case, you only have to divide by 255 👍

I hope this helps!

[Ljiaqii]

Hi, thanks for your help! Following the method you provided, I used fuse_cam.
The generated heatmaps are as follows (I picked three of them for reference):

Below is the relevant code, could you please help me see if the implementation is correct? Thanks again! Best regards!

        cam_extractor_all = LayerCAM(model, target_layer=['layer1','layer2','layer3'],  
                                        input_shape=(32, 32, 32))

        out = model(inputs)
        class_idx = out.squeeze(0).squeeze(0).argmax().item()
        # activation_map = cam_extractor(class_idx, out)
        cam_layer_all= cam_extractor_all(class_idx, out)

        # activation_map = cam_extractor(class_idx = 0, scores = out) # class_idx=0 or 1
        # use fuse-cam
        fused_cam = cam_extractor_all.fuse_cams(cam_layer_all)
        # activation_map_numpy= fused_cam[0].cpu().numpy()
        activation_map_numpy = fused_cam.cpu().numpy()

[frgfm]

The choice of the target layers is up to you & experimentation results, so I cannot say for that part but for the rest, it looks good yes 👍

[Ljiaqii]

Thank you for your quick reply, I learned a lot! Best regards!