CPU-only version of torch-scatter installed

[cswinter]

Our poetry build installs the cpu only version of torch-scatter which causes significant slowdown.
With pip, this is done with the --find-links https://github.com/python-poetry/poetry/issues/1391 option, but it appears this functionality is missing in poetry: python-poetry/poetry#1391
There may be a workaround.

[cswinter]

In fact, it doesn't cause that much slowdown. There doesn't seem to be an easy workaround and we want to replace torch-scatter anyway, so we can live with it until that happens.