Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Elasticsearch synchronization is not always able to read in data from elastic #503

Closed
nagi49000 opened this issue Aug 11, 2024 · 3 comments · Fixed by #505
Closed
Assignees
Labels
bug bug community community Effort - Low Effort - Low Frequency - Daily Frequency - Daily Reach - VeryFew Reach - VeryFew Severity - S3 Severity - S3

Comments

@nagi49000
Copy link
Contributor

nagi49000 commented Aug 11, 2024

Describe the bug
On Elastic 8.14.3, reading docs from elastic with search() or scan() fails with a Python KeyError of "index" not found

To Reproduce
Steps to reproduce the behavior:

  1. Connect to an 8.14.3 elastic instance from memgraph Cypher editor
  2. In memgraph Cypher editor, run a query to get data from an index in that elastic index
CALL elastic_search_serialization.search("<some index in the elastic instance>",  "{\"match_all\": {}}", 1000, 0)
YIELD *
RETURN *;

Expected behavior
Return a valid response from the elasticsearch query without error

Additional context
Add any other context about the problem here.
Fix is straightforward. In the two functions search() and scan() in elastic_search_serialization.py, there is a code block reading (in search)

    for hit in response[HITS][HITS]:
        hit[ID] = hit[_SOURCE][INDEX][ID]
        hit[_SOURCE].pop(INDEX, None)
        hits.append(hit)

and (in scan)

    for item in response:
        item[ID] = item[_SOURCE][INDEX][ID]
        item[_SOURCE].pop(INDEX, None)
        items.append(item)

These blocks can be updated to (in search)

    for hit in response[HITS][HITS]:
        if hit[_SOURCE].get(INDEX, None):
            hit[ID] = hit[_SOURCE][INDEX][ID]
            hit[_SOURCE].pop(INDEX, None)
        hits.append(hit)

and (in scan)

    for item in response:
        if item[_SOURCE].get(INDEX, None):
            item[ID] = item[_SOURCE][INDEX][ID]
            item[_SOURCE].pop(INDEX, None)
        items.append(item)
@nagi49000 nagi49000 added the bug bug label Aug 11, 2024
@gitbuda gitbuda added Severity - S3 Severity - S3 Effort - Low Effort - Low community community Frequency - Daily Frequency - Daily Reach - VeryFew Reach - VeryFew labels Aug 12, 2024
@gitbuda
Copy link
Member

gitbuda commented Aug 12, 2024

Thanks a lot @nagi49000 for reporting!
Since the fix is simple, feel free to post a PR 🙏
Hopefully, the team can pick this up quickly 💪

@nagi49000
Copy link
Contributor Author

Thanks @gitbuda - PR up #505

@gitbuda gitbuda linked a pull request Aug 13, 2024 that will close this issue
19 tasks
@gitbuda
Copy link
Member

gitbuda commented Aug 13, 2024

Fixed by #505

@gitbuda gitbuda closed this as completed Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bug community community Effort - Low Effort - Low Frequency - Daily Frequency - Daily Reach - VeryFew Reach - VeryFew Severity - S3 Severity - S3
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants