retriever span in llama-index 0.9.5 is now put under a new trace and new callback manager (potentially fragmenting the parent-child graph of the spans)

[RogerHYang]

In llama-index 0.9.5, retriever starts a new trace and new callback manager (of BaseRetriever), instead of just a new span under the same trace and same callback manager of its parent span.

In general, there are too many ways a different callback manager can specified for each portion of a trace. This can lead to the fragmentation of the trace map (i.e. the parent-child graph of the spans), because each callback manager keeps and updates a separate trace map. When phoenix receive the final trace map, it only come from one callback manager, and the trace map may not be a complete version containing all the spans.

Alternatively, we can construct our own trace map using the parent_id argument in on_event_start, but that has the downside of a potential memory leak: we may not know when the trace map can be destroyed.