Multimodal input binary classifier with Saliency

[NullCodex]

❓ Questions and Help

Hi Everyone,

Question:

How can I apply saliency to a dataset composed of categorical and image data?

I am somewhat of a beginner with pytorch and the available resources are just not clicking with my use case. The ultimate goal is for me to plot the saliency of a model, but I am stuck on calculating the gradient. Any help or guidance would be much appreciated.

What I've reviewed:

Multimodal_VQA_Captum_Insights tutorial
BERT tutorials
(These resources all have very different data structures(images/sentences) and are confusing for a beginner to translate to an easier image/categorical dataset)

My issue

Model

Model(
  (label_embedding): Embedding(10, 10)
  (model): Sequential(
    (0): Linear(in_features=1034, out_features=512, bias=True)
    (1): LeakyReLU(negative_slope=0.2, inplace=True)
    (2): Linear(in_features=512, out_features=512, bias=True)
    (3): Dropout(p=0.4, inplace=False)
    (4): LeakyReLU(negative_slope=0.2, inplace=True)
    (5): Linear(in_features=512, out_features=512, bias=True)
    (6): Dropout(p=0.4, inplace=False)
    (7): LeakyReLU(negative_slope=0.2, inplace=True)
    (8): Linear(in_features=512, out_features=1, bias=True)
  )
)

Categorical

The categorical data is just a normal label such as 4

What I tried

saliency = Saliency(model)
grads = saliency.attribute((input, label), target=None)

where input is just a (1, 32, 32) image. I set target=None since it's a binary classifier.

Failure output

One of the differentiated Tensors does not require grad

Since the label is not a float it does not require grad, is there a way i can use the saliency method to capture the grads?

[bilalsal]

Hi @NullCodex

did you try label.requires_grad = True?

[NullCodex]

Yes, but since label is not a float, it says it does not require grad.

[NullCodex]

The label data is for instance tensor([2]).

[bilalsal]

I see. Maybe to narrow down the issue, did you try other Captum algorithms? Like IntegratedGradients?

[NullCodex]

I am pretty sure it's because captum doesn't support grads with integers. I tried other algorithms as well and none of them works. Are there any working examples where tensor([2]) generates grads or require grads.

[NullCodex]

Also are leakyRelu not supported for LRP because when i tried it on my model i got Module type <class 'torch.nn.modules.activation.LeakyReLU'> is not supported.No default rule defined.

[bilalsal]

I am pretty sure it's because captum doesn't support grads with integers. I tried other algorithms as well and none of them works. Are there any working examples where tensor([2]) generates grads or require grads.

Captum relies on PyTorch built-in functions to compute the gradient, and it is true that constants do not require a grad.
I am trying to understand how your model works to help you apply Captum correctly.
Could you share your forward() function?

[NullCodex]

The entire class is below:

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()

        self.label_embedding = nn.Embedding(10, 10)

        self.model = nn.Sequential(
            nn.Linear(10 + int(np.prod((1, 32, 32))), 512),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 512),
            nn.Dropout(0.4),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 512),
            nn.Dropout(0.4),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 1),
        )

    def forward(self, img, labels):
        # Concatenate label embedding and image to produce input
        d_in = torch.cat((img.view(img.size(0), -1), self.label_embedding(labels)), -1)
        validity = self.model(d_in)
        return validity

Right now, I am hacking it so that i change the class to take only the img in forward and fixing the label but i wouldn't be able to do that for deepliftshap for instance since it requires baselines

[bilalsal]

What about changing your forward so that it takes d_in as a single tensor?
Then it will be straightforward for Captum to attribute the model's output to this input tensor.

d_in should be also straightforward to construct outside of forward.

Hope this helps

[bilalsal]

Also are leakyRelu not supported for LRP because when i tried it on my model i got Module type <class 'torch.nn.modules.activation.LeakyReLU'> is not supported.No default rule defined.

Regarding LRP, the current implementation indeed does not readily define a rule for all torch.nn modules, however, it should be straightforward to define one (see #712).

Hope this helps

[NullCodex]

With using d_in the output shape is 1034 rather than (1, 32, 32). What i did to make it visualize the gradient was to only take the first 1024 part since that can be reshaped to (1, 32, 32). In this case this is giving the contribution of grads from the image only is that correct? What can i do about the remaining grad to understand their significance? @bilalsal

[nanohanno]

Also are leakyRelu not supported for LRP because when i tried it on my model i got Module type <class 'torch.nn.modules.activation.LeakyReLU'> is not supported.No default rule defined.

Regarding LRP, the current implementation indeed does not readily define a rule for all torch.nn modules, however, it should be straightforward to define one (see #712).

Hope this helps

I guess the error message is a bit misleading here. Should we change that to something like

f"Module of type {type(layer)} has no rule defined and no default rule exists for this module type."
"Please, set a rule explicitly for this module and assure that it is appropriate for this type of layer."

in

captum/captum/attr/_core/lrp.py

Line 283 in 7300cd8

raise TypeError(

[bilalsal]

Thanks for the good suggestion, @nanohanno!

Do you want to create a PR for this? I can gladly help getting this landed. I can also update the message myself if needed.

[bilalsal]

With using d_in the output shape is 1034 rather than (1, 32, 32). What i did to make it visualize the gradient was to only take the first 1024 part since that can be reshaped to (1, 32, 32). In this case this is giving the contribution of grads from the image only is that correct? What can i do about the remaining grad to understand their significance?

Hi @NullCodex , yes that sounds reasonable: the first 1024 values are attributions of the 32x32 pixels, and the last 10 values are for the embedding of the labels.

[NullCodex]

@bilalsal how should i provide an explanation of the remaining grad? since there are still 10 values.

[bilalsal]

@bilalsal how should i provide an explanation of the remaining grad? since there are still 10 values.

The remaining values represent the attribution of the label embedding: Each value, e.g. the grad for the 1st channel in the 10-dimensional embedding) indicates how likely the respective channel (in the label embedding) contributed to the output. This info could be insightful if you want to shed light into how the learned nn.Embedding impacts the output.

I recommend you do further experimentation and visualization of the remaining grads to get a closer idea about them and whether they can help your analysis.

Hope this helps

[nanohanno]

Thanks for the good suggestion, @nanohanno!

Do you want to create a PR for this? I can gladly help getting this landed. I can also update the message myself if needed.

I am happy to create a PR for it.