[BUG] llama_index errors get attached to the wrong span

[mikeldking]

Describe the bug
The bug is that I ran the internal llama_index notebook using a trial cohere key. This causes a rate limit error but unfortunately it's attached to the embedding span, not the reranker span.

To Reproduce
Run the llama_index_tracing_example with cohere and a trial API key

Expected behavior
The errors should be attached to the reranker span.

Environment (please complete the following information):

• OS: MacOS
• Notebook Runtime VS Code
• Browser: Any
• Version 0.0.45

[mikeldking]

@tammy37 also facing similar issues

[mikeldking]

I think there are few problems:

• errors not propagated up the run tree so the root span doesn't error out
• re-ranker errors are not captured on the reranker spans

[axiomofjoy]

This issue is being caused by an inconsistency in the trace_map passed by the LlamaIndex callback system. Most of the time when an error occurs, the exception event is a sibling event that follows the callback event in which the exception occurs. When an error occurs in a re-ranker, however, the exception events are children of the event in which the exception occurred. The diagram below shows the trace_map for the particular trace from the llama_index_tracing_example notebook. As you can see, the embedding node is the preceding node in the tree when doing a BFS, so the exception events get associated with the embedding event by our current code.

[axiomofjoy]

We need to make a change to LlamaIndex to either:

• pass all exception events as children of the events that caused the exception,
• pass error information not as a distinct event from the event that caused the error, but on the on_event_end hook for that event.

[axiomofjoy]

I filed an issue with LlamaIndex here.

[axiomofjoy]

To address this issue further, we need to improve visibility into error status codes at the trace level.