Label ignore/filter for torch/nn

Ignore/bypass some inputs according to a specific set of ignore labels. This package allows for generally available, out-of-the-box criterions in torch/nn to be used in situations where a specific label of a certain class/output is needed to be overlooked/ignored when backproping through a network model.

Install

To install the package simply do the following:

git clone https://github.com/farrajota/criterion_filter
cd criterion_filter && luarocks make

Usage

This package consists of two submodules:

• criterion_filter.Single(Criterion, Ignore_Label) wraps an input criterion and filters all indexes which have the same label as in Ignore_Label.
• criterion_filter.Parallel() is a direct refactoring of the nn.ParallelCriterion() container, where :add(Criterion, Weight, Ignore_label) allows you to specify a set of ignore labels for each criterion that you add.

The following code snippets shows how to use the package for cases using single and multiple criterias.

Single criterion

Example 1

--[[ Example using `ClassNLLCriterion`. 
Computes the loss for an input tensor with all allowed targets,
and proceed to compute the loss with a blacklisted label. 
]]
require 'criterion_filter'

-- 1. define model
model = nn.Sequential()
model:add(nn.Linear(10,4))

-- 2. define criterion
nll = nn.ClassNLLCriterion()
criterion = criterion_filter.Single(nll, 4) -- set to ignore class 4

-- 3. define input data 
input = torch.Tensor(5,10):uniform()
target = torch.Tensor(5):random(1,3)

-- 4. compute loss (forward pass only)
-- 4.1. (no labels to be ignored at this point)
output = model:forward(input)
print('Output Tensor:')
print(output)
err1 = criterion:forward(output,target)
print('Error with no ignored labels: ' .. err1)

-- 4.2. set one target label to 0 (to be ignored)
target[5] = 4
err2 = criterion:forward(output,target)
print('Error after setting one target label to 0: ' ..err2)
print('Error equal? ' .. tostring(err1 == err2))

Example 2

--[[ Example using `MSECriterion`.
Computes the loss and gradient for an input tensor + targets,
and proceeds to compute the loss and gradient with blacklisted labels.
]]
require 'criterion_filter'

-- 1. define model
model = nn.Sequential()
model:add(nn.Linear(10,4))

-- 2. define criterion
mse = nn.MSECriterion()
criterion = criterion_filter.Single(mse, {torch.Tensor(4):fill(0), {{1,2,3,4}}}) -- set to ignore class two labels

-- 3. define input data 
input = torch.Tensor(5,10):uniform()
target = torch.Tensor(5,4):uniform()

-- 4. compute loss+gradient (forward+backward pass)
-- 4.1. (no labels to be ignored at this point)
output = model:forward(input)
err1 = criterion:forward(output,target)
gradOutput1 = criterion:backward(output,target):clone()
print(err1)
print(gradOutput1)

-- 4.2. set one target to 0's (to be ignored)
target[1]:fill(0)
err2 = criterion:forward(output,target)
gradOutput2 = criterion:backward(output,target):clone()
print(err2)
print(gradOutput2)

--4.3. set another target to 0's and one to {1,2,3,4} 
target[3]:fill(0)
target[4]:copy(torch.Tensor{1,2,3,4})
err3 = criterion:forward(output,target)
gradOutput3 = criterion:backward(output,target):clone()
print(err3)
print(gradOutput3)

Multiple criterions

Example 1

--[[ Example using multiple criterions (`ClassNLLCriterion` + `ClassNLLCriterion`).
Here, the loss is computed using two equal criterias with equal weigths. 
Check how the loss value differs when it is computed with ignored/filtered/blacklisted labels.
]]
require 'criterion_filter'

-- 1. define model
model = nn.Sequential()
prl = nn.ConcatTable()
prl:add(nn.Linear(10,10))
prl:add(nn.Linear(10,10))
model:add(prl)

-- 2. define criterions
criterion = criterion_filter.Parallel()
criterion:add(nn.ClassNLLCriterion(), 1, 6) -- set different ignore labels
criterion:add(nn.ClassNLLCriterion(), 1, 7) -- set different ignore labels

-- 3. define input data
input = torch.Tensor(5,10):uniform()
target1 = torch.Tensor(5):random(1,10)
target1[1]=1
target2 = torch.Tensor(5):random(1,10)
target2[1]=1

-- 4. compute loss
-- 4.1. without ignore labels
output = model:forward(input)
print(output[1])
print(output[2])
print(target1)
print(target2)
err1 = criterion:forward(output,{target1, target2})
print(err1)

-- 4.2. now with ignored/blacklisted labels
target1[1] = 6
target2[1] = 7 
err2 = criterion:forward(output,{target1, target2})
print(err2)

-- 4.3. flip labels 
target1[1] = 7
target2[1] = 6 
err3 = criterion:forward(output,{target1, target2})
print(err3)

Example 2

--[[ Example using multiple criterions (`ClassNLLCriterion` + `MSECriterion`).
Here, the loss is computed using two different criterias and weigths. 
Check how the loss value differs when it is computed with ignored/filtered/blacklisted labels.
]]
require 'criterion_filter'

-- 1. define model
model = nn.Sequential()
prl = nn.ConcatTable()
prl:add(nn.Linear(10,5))
prl:add(nn.Linear(10,4))
model:add(prl)

-- 2. define criterions
criterion = criterion_filter.Parallel()
criterion:add(nn.ClassNLLCriterion(), 1)  -- no ignore label defined
criterion:add(nn.MSECriterion(), 0.5, torch.Tensor({1,1,1,1})) -- set an ignore label

-- 3. define input data
input = torch.Tensor(5,10)
target1 = torch.Tensor(5):random(1,5)
target2 = torch.Tensor(5,4):uniform()

-- 4. compute loss
-- 4.1. without ignore labels
output = model:forward(input)
err1 = criterion:forward(output, {target1, target2})
print('Error without ignore labels: ' .. err1)

-- 4.2. now with ignored/blacklisted labels
target2[2]:fill(1)
target2[5]:fill(1)
err2 = criterion:forward(output, {target1, target2})
print('Error with ignore labels: ' .. err2)