-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
what's reason for making register_module internal? #515
Comments
While I don't like the implicit submodule registration via reflection that we have currently, I don't see exactly what is the problem you are facing. I forgot details of operator overloading in F#, but you could do Sequential Chain(Module a, Module b) => torch.nn.Sequential(a, b);
Sequential Chain(Sequential s, Module m) => torch.nn.Sequential(s.layers.Append(m)); |
the ->> operator implementation calls register_module to bind the submodule (but now it cannot). Also other similar functions e.g. register_parameter, etc. are still public. register_module has been accessible thus far without any adverse affects. Marking it internal needlessly limits future extensibility. |
@fwaris why my |
@fwaris -- the reason I made it internal is that it's not in the Pytorch APIs. If there's a strong reason to have it be public, then it's not a big deal to make it so. |
Also, @fwaris, is your "->>" operator similar to the TorchSharp F# operator override _.forward(input) =
input
--> conv1 --> relu --> conv2 --> relu --> pool1 --> dropout1
--> flatten
--> fc1 --> relu --> dropout2 --> fc2
--> logsm Defined in Tensor.cs: public static Tensor op_MinusMinusGreater(Tensor t, torch.nn.Module m) => m.forward(t); |
@NiklasGustafsson The '->>' operator is inspired by DiffSharp. It binds two modules together (to create joined modules). The --> operator invokes the forward function of a module with a tensor. The two behave differently but can be intermixed. |
The next release will have this change, but it will take a little bit, because that is going to be a huge one (adding support for parameter groups). |
Thanks @NiklasGustafsson. I can work with the older version for now. Nice to know that we are getting parameter group support. Big models are hard to train and need to use many tricks, e.g. different learning rates for different layers, etc. |
I am building a functional wrapper around torchsharp. It uses register_module to compose modules together in a functional style. It is now broken. Please advise as to an appropriate replacement.
Making it internal enforces a very OO view on the models - which is not ideal in my view.
Example:
There are extensions for more complex forward invocations but in general it makes the structure of the model very apparent and its easier to experiment with variations on the base structure during model development.
The text was updated successfully, but these errors were encountered: