Distributed SharedVariableSet

[nilesh-c]

In some problems like matrix/tensor factorization, for a specific input, the variables that need to be modified are known beforehand. What if each variable is a vector, there are millions of variables, and storing the whole matrix in SharedVariableSet would take too much memory (say, > 32 GB)?

Provided we divide the input in a specific manner, the weight matrix can also be divided, is there a way to distribute SharedVariableSet and load only the subset that's needed for a specific input chunk?

@zhangyuc how hard would it be to add this feature?