[Client] Provide common ObjectRefType

[devin-petersohn]

Describe your feature request

We do a lot of defensive type checking in Modin when using Ray to make sure that things are working. It would be very convenient if ClientObjectRef extended ObjectRef so that assertions like assert(isinstance(obj, ray.ObjectRef)) would still work without needing to handle the Client object separately.

[devin-petersohn]

Related to modin-project/modin#2688

[Bhavya6187]

Hey guys, can you verify if this example works with current nightlies - https://docs.ray.io/en/master/modin/index.html#modin-with-the-ray-client?

I run into this issue when I try to load Modin csv's or parquets using ray client. Here's the corresponding issue on Modin's side - modin-project/modin#2688

[ericl]

This turns out to be quite tricky to do, since ObjectRef is a native cdef that depends on the C++ runtime. Would the following workaround work?

ObjectRefType = Union[ray.ObjectRef, ClientObjectRef]

Then you can type check around ObjectRefType instead of ObjectRef directly.

[devin-petersohn]

Yes, that will work for our purposes. Something like:

from ray import ObjectRefType

assert isinstance(myobj, ObjectRefType)

edit: Did you mean from Ray or within our code to add this line?

[ericl]

@devin-petersohn it seems there are some cdef limitations on inheriting from an abstract base class. Would one of the following options work for you in Modin instead?

assert is_object_ref_type(myobj)
assert isinstance(myobj, ObjectRefType.__args__)

In the short term, I'd recommend implementing a custom is_object_ref_type function in Modin that checks if it's either ObjectRef or ClientObjectRef to unblock Ray client users.

In Ray 1.3 we can add either is_object_ref_type as a native API in Ray or the ObjectRefType Union type as above. We could also probably refactor the ObjectRef implementation so a real abstract base class is possible, but this would be a quite large change.

[devin-petersohn]

Sounds good @ericl, we'll do that.

[ericl]

Moving to core bugs for tracking

[pcmoritz]

Would it be possible to only have a single ObjectRef type for both client and the rest of Ray? Certainly from the API side that would be the simplest (no need for is_object_ref_type etc)?

If the implementation requires it, the ObjectRef could have an internal distinction if it is a client ref or a Ray ref.

[ericl]

It's a possibility, we'd have to look into how much refactoring is needed, since ObjectRef is a Cython type.