You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[x] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When generating embeddings with the LlamaIndexEmbeddingsWrapper a wrong cache hit leads to all embeddings being the same.
This is due to the _generate_cache_key function removing the first argument for methods. However it seems self is not passed here as an argument, hence valid arguments are sliced away here and every cache key becomes the same.
Expected behavior
Distinct embed_text arguments need to result in distinct vectors.
Distinct embed_text arguments need to be cached under distinct cache keys.
Additional context
We noticed this bug when running metrics and receiving implausible results (e.g. SemanticSimilarity always being 1).
This test case did not catch it either, since it's only tested with functions, not methods.
The text was updated successfully, but these errors were encountered:
[x] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
When generating embeddings with the
LlamaIndexEmbeddingsWrapper
a wrong cache hit leads to all embeddings being the same.This is due to the
_generate_cache_key
function removing the first argument for methods. However it seemsself
is not passed here as an argument, hence valid arguments are sliced away here and every cache key becomes the same.Ragas version: 0.2.13
Python version: 3.11.2
Code to Reproduce
see this gist
Error trace
N/A (see assertion in example code)
Expected behavior
Distinct
embed_text
arguments need to result in distinct vectors.Distinct
embed_text
arguments need to be cached under distinct cache keys.Additional context
We noticed this bug when running metrics and receiving implausible results (e.g. SemanticSimilarity always being 1).
This test case did not catch it either, since it's only tested with functions, not methods.
The text was updated successfully, but these errors were encountered: