EltAffine Functionality

[Darwin2011]

Hi, @ducha-aiki

I am trying to figure out BN implementations from the PR you test and there's no bias and shift implemented there.

I also notice that from your experiments, it seems that BN + Affine doesn't improve performance that much from the initial training stages.

And in your Caffe fork, https://github.com/ducha-aiki/caffe, there's another version of bn implementation as Caffe PR 1965, which implements shift and bias.

So may I know why such two operations is dropped in Caffe Upstream you test? Do they even hurt performance? Or what version should I choose to use?

Thanks a lot.

.

[ducha-aiki]

@Darwin2011 the caffe maintainers said that want layers to be smaller building block, so separate EA. EA could be implemented with dummydatalayer in very unelegant way, but with existing layers. So my version was dropped.

However, I am going to PR bias layer and scale layer, which together would EA and maintainers are ok with merging them, when ready.

[Darwin2011]

Thanks a lot for your kind help.