GH-47358: [Python] IPC and Flight options representation

[rmnskb]

Rationale for this change

Please see #47358

What changes are included in this PR?

Added __repr__ dunder methods to ipc.IpcWriteOptions, ipc.IpcReadOptions and flight.FlightCallOptions

Are these changes tested?

No

Are there any user-facing changes?

Nicer representations of the classes mentioned above

• GitHub Issue: [Python] IPC and Flight options lack a nice repr/str #47358

[rmnskb]

hey @pitrou, I hope you're doing well :)
I've started implementing the __repr__ dunders for the classes that you've mentioned in #47358, however, now I stumbled upon FlightCallOptions's arguments not being convertable back to Python objects after they are set as C objects inside FlightCallOptions.options.
Could you please provide some guidance on what's the better way to tackle this? I've gone through the implementations of underlying objects in both APIs, and couldn't find any methods that would already do that? Thank you :)

[pitrou]

I've started implementing the __repr__ dunders for the classes that you've mentioned in #47358, however, now I stumbled upon FlightCallOptions's arguments not being convertable back to Python objects after they are set as C objects inside FlightCallOptions.options.

I'm not sure what you mean, because if you define the __repr__ in Cython you should be able to access the C-level attributes. You may need to add the necessary API declarations for C++ Flight types.

For example, this change works for me:

diff --git a/python/pyarrow/_flight.pyx b/python/pyarrow/_flight.pyx
index fe2e1b3d67..ca842f35bb 100644
--- a/python/pyarrow/_flight.pyx
+++ b/python/pyarrow/_flight.pyx
@@ -142,6 +142,10 @@ cdef class FlightCallOptions(_Weakrefable):
             return &((<FlightCallOptions> obj).options)
         raise TypeError(f"Expected a FlightCallOptions object, not '{type(obj)}'")
 
+    def __repr__(self):
+        # Draft implementation, should be improved
+        return f'FlightCallOptions(timeout={self.options.timeout.count()})'
+
 
 _CertKeyPair = collections.namedtuple('_CertKeyPair', ['cert', 'key'])
 
diff --git a/python/pyarrow/includes/libarrow_flight.pxd b/python/pyarrow/includes/libarrow_flight.pxd
index b1af6bcb4f..f2b15fb706 100644
--- a/python/pyarrow/includes/libarrow_flight.pxd
+++ b/python/pyarrow/includes/libarrow_flight.pxd
@@ -272,6 +272,7 @@ cdef extern from "arrow/flight/api.h" namespace "arrow" nogil:
 
     cdef cppclass CTimeoutDuration" arrow::flight::TimeoutDuration":
         CTimeoutDuration(double)
+        double count()
 
     cdef cppclass CFlightCallOptions" arrow::flight::FlightCallOptions":
         CFlightCallOptions()

[rmnskb]

I've started implementing the __repr__ dunders for the classes that you've mentioned in #47358, however, now I stumbled upon FlightCallOptions's arguments not being convertable back to Python objects after they are set as C objects inside FlightCallOptions.options.

I'm not sure what you mean, because if you define the __repr__ in Cython you should be able to access the C-level attributes. You may need to add the necessary API declarations for C++ Flight types.

For example, this change works for me:

diff --git a/python/pyarrow/_flight.pyx b/python/pyarrow/_flight.pyx
index fe2e1b3d67..ca842f35bb 100644
--- a/python/pyarrow/_flight.pyx
+++ b/python/pyarrow/_flight.pyx
@@ -142,6 +142,10 @@ cdef class FlightCallOptions(_Weakrefable):
             return &((<FlightCallOptions> obj).options)
         raise TypeError(f"Expected a FlightCallOptions object, not '{type(obj)}'")
 
+    def __repr__(self):
+        # Draft implementation, should be improved
+        return f'FlightCallOptions(timeout={self.options.timeout.count()})'
+
 
 _CertKeyPair = collections.namedtuple('_CertKeyPair', ['cert', 'key'])
 
diff --git a/python/pyarrow/includes/libarrow_flight.pxd b/python/pyarrow/includes/libarrow_flight.pxd
index b1af6bcb4f..f2b15fb706 100644
--- a/python/pyarrow/includes/libarrow_flight.pxd
+++ b/python/pyarrow/includes/libarrow_flight.pxd
@@ -272,6 +272,7 @@ cdef extern from "arrow/flight/api.h" namespace "arrow" nogil:
 
     cdef cppclass CTimeoutDuration" arrow::flight::TimeoutDuration":
         CTimeoutDuration(double)
+        double count()
 
     cdef cppclass CFlightCallOptions" arrow::flight::FlightCallOptions":
         CFlightCallOptions()

Hm, that's interesting, since the same changes have segfaulted for me 🤔
Ok, thanks for the confirmation, I will look into that further.

[rmnskb]

hi @pitrou,

I have implemented __repr__ methods for the aforementioned objects as discussed. For flight.FlightCallOptions, I've decided to go with the straightforward approach by assigning the constructor arguments to private attributes, and then accessing them later. Please let me know what you think :)

[AlenkaF]

Thank you for another contribution @rmnskb !

The overall approach looks good to me. I like that you're only appending the read and write options when they're defined. I think we don’t currently have wrap_options functionality to reconstruct the Python Ipc*Options from self.options.read/write_options, so holding on to the original Python objects makes sense.

One request from me — it would be great to add __repr__ tests, though there aren’t many currently.

One very minor nit: stylistically, I’d consider adding the parentheses inside the class name in the __repr__, like this:

    def __repr__(self):
        return (f"<pyarrow.ipc.IpcReadOptions ("
                ...
                f"included_fields={self.included_fields})>")