Improve description of device handling, add array.to_device

[rgommers]

Closes gh-157

[kgryte]

LGTM.

[leofang]

LGTM, but I guess we need someone from TensorFlow to give a green light? IIRC TF allows implicit transfer.

[rgommers]

LGTM, but I guess we need someone from TensorFlow to give a green light? IIRC TF allows implicit transfer.

Right, this PR is meant to be just a clarification based on what we already discussed earlier. Quoting from #39 (comment):

Also, I am not sure we should enforce constraints on where inputs and outputs can be placed for an operation. Such constraints can make it harder to write portable library code where you don't control the inputs and may have to start by copying all inputs to the same device. Tensorflow runtime is allowed to copy inputs to the correct device if needed.

That is a good question, should it be enforced or just recommended? Having device transfers be explicit is usually better (implicit transfers can make for hard to track down performance issues), but perhaps not always.

I think in the end this is an execution rather than a syntax/semantics question, so maybe the "with the convention ..." phrase in this PR should be replaced with a phrase that's more like "strong recommendation".

Cc @edloper, @agarwal-ashish

[agarwal-ashish]

LGTM.

One nit is that the device property on operations likely specifies where the kernel runs, which is typically but not necessarily, the same as the device where outputs are placed.

[rgommers]

One nit is that the device=None property on operations likely specifies where the kernel runs, which is typically but not necessarily, the same as the device where outputs are placed.

Thanks @token. device=None is a keyword for array creation functions, so if it's not specified I think then the device that the array is placed on is "whatever the default device or placement strategy of the library is".

[agarwal-ashish]

One nit is that the device=None property on operations likely specifies where the kernel runs, which is typically but not necessarily, the same as the device where outputs are placed.

Thanks @token. device=None is a keyword for array creation functions, so if it's not specified I think then the device that the array is placed on is "whatever the default device or placement strategy of the library is".

Sorry I meant even when the device is specified, a particular op may choose to output on a different device than where the kernel is executed (e.g. copy, shape, rpc related ops, etc). If this property is restricted to array creation functions only then the two devices will likely match.

[rgommers]

If this property is restricted to array creation functions only then the two devices will likely match.

It is. Thanks for clarifying. Are the cases where an array creation call would still end up on a different device restricted to avoiding an out-of-memory error? If so, we can explicitly add that as a note.

[edloper]

Just to clarify: for libraries that follow this convention, it would be an error to perform an operation with tensors that are not all allocated on the same device?

[leofang]

If self is already on device, do we expect a no-op (returning self) or a copy?

[rgommers]

Good point, should specify that. I'd say no-op.

[leofang]

Let's get this PR in, as no body objects and we don't want to drag indefinitely. We can continue the discussion and pick up any loose ends in the follow-up PR #259.