extract_jsonpath give an error if filters are used in records_jsonpath

[MeltyBot]

Migrated from GitLab: https://gitlab.com/meltano/sdk/-/issues/382

Originally created by @andrey.shalitkin on 2022-05-11 21:59:09

---

Summary

extract_jsonpath give an error if filters are used in records_jsonpath

Steps to reproduce

Add filter records_jsonpath in RESTStream, e.g.
records_jsonpath = '$.somenode[?(@.attribute=="Value")]'

What is the current bug behavior?

JsonPathLexerError: Error on line 1, col 15: Unexpected character: ?

What is the expected correct behavior?

No errors

Possible fixes

The fix is described over here: h2non/jsonpath-ng#8

Here is the jsonpath.py that would fix it:

"""JSONPath helpers."""

from typing import Any, Generator, Union

import jsonpath_ng
import memoization
from jsonpath_ng.ext import parse

def extract_jsonpath(
    expression: str, input: Union[dict, list]
) -> Generator[Any, None, None]:
    """Extract records from an input based on a JSONPath expression.

    Args:
        expression: JSONPath expression to match against the input.
        input: JSON object or array to extract records from.

    Yields:
        Records matched with JSONPath expression.
    """
    compiled_jsonpath = _compile_jsonpath(expression)

    match: jsonpath_ng.DatumInContext
    for match in compiled_jsonpath.find(input):
        yield match.value


@memoization.cached
def _compile_jsonpath(expression: str) -> jsonpath_ng.JSONPath:
    """Parse a JSONPath expression and cache the result.

    Args:
        expression: A string representing a JSONPath expression.

    Returns:
        A compiled JSONPath object.
    """
    return parse(expression)

[MeltyBot]

View 4 previous comments from the original issue on GitLab