-
Notifications
You must be signed in to change notification settings - Fork 21
Description
Search before asking
- I searched in the issues and found nothing similar.
Description
Add a poll() method to the Python LogScanner API to enable incremental, timeout-based record reading, similar to the C++ bindings.
Current situation:
- Python bindings only support
scan()which blocks until all records are fetched - No way to do incremental polling for real-time data consumption
Proposed addition:
# Poll with timeout (milliseconds)
result = log_scanner.poll(timeout_ms=5000)
# Returns PyArrow Table with records (or empty table on timeout)
if result.num_rows > 0:
df = result.to_pandas()
# Process data...Benefits:
- Enables real-time streaming patterns
- Better control over blocking behavior
- Consistency with C++ API surface
Willingness to contribute
- I'm willing to submit a PR!
Metadata
Metadata
Assignees
Labels
No labels