Feature: support dict-like input for to_numpy and to_device

[sakuranew]

dict like input is possible for user customized dataset or output function,but will get error in current code such as forward_iter
And current to_tensor support dict input, but to_device doesn't support

[sakuranew]

Actually,I overwrite some function to solve this,

    def _forward_iter(self, X, training=False, device='cpu'):
        dataset = self.get_dataset(X)
        iterator = self.get_iterator(dataset, training=training)
        for data in iterator:
            Xi = skorch.dataset.unpack_data(data)[0]
            yp = self.evaluation_step(Xi, training=training)
            yield to_device(yp, device=device)
    def predict(self, X):
        rs = []
        for yp in self._forward_iter(X, training=False):
            if self.predict_func:
                r=self.predict_func(yp)
                rs.append(r)
        return rs

[BenjaminBossan]

Hi, thanks for contributing.

Could you please explain what exactly you need to achieve? I couldn't quite understand that. E.g., if you could give a minimal code sample that fails with the current code but works with your fix, it would be very helpful (this code sample could also be a unit test, which this PR would need to provide eventually).

The main reason why I'm asking is because as you said, to_tensor takes care of dict and so far that was enough for me.

[sakuranew]

        for step, batch in enumerate(epoch_iterator):
            model.train()
            batch = tuple(t.to(args.device) for t in batch)
            inputs = {"input_ids": batch[0], "attention_mask": batch[1], "labels": batch[3]}

If I have different data input format，a list batch need to change the index to get Expected results，this is hard for complex model or input format.
So,I try to use dict input ,and it's meaningful,pytorch offical default_collate function support dict and skorch to_tensor support it

Surely,dict input have no conflicts with to_numpy or to_device in train phrase,but in predict phrase

    def infer(self, x, **fit_params):
        x = to_tensor(x, device=self.device)

    def forward_iter(self, X, training=False, device='cpu'):
        dataset = self.get_dataset(X)
        iterator = self.get_iterator(dataset, training=training)
        for data in iterator:
            Xi = unpack_data(data)[0]
            yp = self.evaluation_step(Xi, training=training)
            yield to_device(yp, device=device)

This code,model input will through to_tensor to cuda,but for model output,it only support list-like format,so if someone have multi output(such as my model have 50+ output for text-to-sql task 😂) and want to use dict to make things easier,this feature will really help them and does't has influence for normally using skorch .

[BenjaminBossan]

Thanks for explaining your use case. I agree that it shouldn't hurt to add these possibilities. Would you please add unit tests for the new additions? That means, tests with dict as input that would currently fail but work with your added changes.

[sakuranew]

Would you please add unit tests for the new additions?

Do you mean write a test function like this?

    @pytest.mark.parametrize('device_from, device_to', [
        ('cpu', 'cpu'),
        ('cpu', 'cuda'),
        ('cuda', 'cpu'),
        ('cuda', 'cuda'),
        (None, None),
    ])
    def test_check_device_tuple_torch_tensor(
            self, to_device, x_tup, device_from, device_to):
        if 'cuda' in (device_from, device_to) and not torch.cuda.is_available():
            pytest.skip()

        prev_devices = [None for _ in range(len(x_tup))]
        if None in (device_from, device_to):
            prev_devices = [x.device.type for x in x_tup]

        x_tup = to_device(x_tup, device=device_from)
        for xi, prev_d in zip(x_tup, prev_devices):
            self.check_device_type(xi, device_from, prev_d)

        x_tup = to_device(x_tup, device=device_to)
        for xi, prev_d in zip(x_tup, prev_devices):
            self.check_device_type(xi, device_to, prev_d)

[BenjaminBossan]

@sakuranew exactly, same for to_numpy.

[sakuranew]

@sakuranew exactly, same for to_numpy.

Sorry,I don't see unittest for to_numpy ,and I don't know how to write a test function
I just learn tuple test in to_device and write a dict test 🤣

[BenjaminBossan]

Thanks for adding the tests. You are right, there are currently no tests for to_numpy (oops), so no need to write a test for that.

When it comes to your current test, I would like to see one change. At the moment, you do:

x_dict = to_device(x_dict, device=device_from)

I.e. you overwrite the x_dict variable. Instead, I would like for you to create a new variable and check that. In addition, you should check that the original x_dict is unchanged.

The reason why it's needed here and not, e.g., when testing tuple, is that dict is mutable. But we don't want to mutate the original input. Therefore, the test needs to make sure that the original input is not affected by to_numpy. You could add a comment that explains this as well.

Apart from that 👍

[sakuranew]

@BenjaminBossan
Hi,Thank you for your correction and advice ,I have learned a lot
Please review again

[BenjaminBossan]

Thanks for making the corrections.

At the moment, some tests fail (see here). The reason is this line:

assert x_dict == original_x_dict

which fails with

RuntimeError: bool value of Tensor with more than one value is ambiguous

So what you could do is check the keys and the values separately. For the values, since they are tensors, you need to call (val0 == val1).all(), otherwise the assertion will fail.

You can also run the tests locally first to make sure they pass (if you don't do that already):

# in skorch root directory
# only your new tests:
py.test -k test_check_device_dict_torch_tensor

# all tests
py.test

This way, you get immediate feedback and don't have to wait for travis CI.

Please ask if you have any questions.

[BenjaminBossan]

Thanks @sakuranew this looks good.

@ottonemo is adding tests for to_numpy in #658 but we can add the test for dicts later.