Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally put biases on neurons in Converter #150

Open
hunse opened this issue Apr 17, 2020 · 0 comments
Open

Optionally put biases on neurons in Converter #150

hunse opened this issue Apr 17, 2020 · 0 comments

Comments

@hunse
Copy link
Collaborator

hunse commented Apr 17, 2020

In the Keras-to-Loihi example, we currently have biases turned off for all the conv layers. Loihi can handle biases, so it would be nice to not have to do this.

I think the best way to do this would be to have a converter option that moves the biases to the neurons, rather than keeps them as a separate node. With this option on, it would mean that training the network in NengoDL would be slightly different than training in Keras, because we'd have more biases (i.e. each neuron would have a bias). I think this isn't a problem, though. Any network converted from Keras would still be identical if it's not trained at all in NengoDL.

Another option would be to do this automatically if we're in inference_only mode. We talked about this briefly in #137. However, I prefer the additional control of having the explicit converter option (i.e. you can still train the network in NengoDL with individual neuron biases, if you want).

A workaround for this would be to use the converter as-is with use_bias=False on all Keras layers, and then go through the network and turn on trainable for all ensembles (since this would make the biases trainable again). For the purposes of an example, though, that's a little confusing. Also, it doesn't allow training the network in Keras and then converting to Nengo while keeping existing biases on the neurons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant