Skip to content

Add blocking poll similar to C++ bindings into python bindings #153

@fresh-borzoni

Description

@fresh-borzoni

Search before asking

  • I searched in the issues and found nothing similar.

Description

Add a poll() method to the Python LogScanner API to enable incremental, timeout-based record reading, similar to the C++ bindings.

Current situation:

  • Python bindings only support scan() which blocks until all records are fetched
  • No way to do incremental polling for real-time data consumption

Proposed addition:

  # Poll with timeout (milliseconds)
  result = log_scanner.poll(timeout_ms=5000)

  # Returns PyArrow Table with records (or empty table on timeout)
  if result.num_rows > 0:
      df = result.to_pandas()
      # Process data...

Benefits:

  • Enables real-time streaming patterns
  • Better control over blocking behavior
  • Consistency with C++ API surface

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions