When using size with the toggle, the width and height that are auto calculated are for btn-md. It seems like the issue is because the computeStyle function is being called before the size attributes are applied to the DOM.
Here is sample code to reproduce:
<toggle ng-model="model" size="btn-xs"></toggle>
<button class="btn btn-xs btn-primary">Size of .btn-xs</button>
And here is a plunker showing the issue
https://plnkr.co/edit/ps1b0St62YSLK9e7Mu8n