How to load traces in another project than default?

[remiconnesson]

I thought the name constructor was used for that, but digging into the source code it wasn't

phoenix/src/phoenix/trace/trace_dataset.py

Lines 139 to 140 in 46be32b

	# TODO: This is not used in any meaningful way. Should remove
	self.name = name or f"{GENERATED_DATASET_NAME_PREFIX}{str(self._id)}"

I don't see a way to add the spans to the correct project, is there a way to do so?

phoenix/src/phoenix/session/session.py

Lines 134 to 140 in 46be32b

	self.traces = Traces()
	if trace_dataset:
	for span in trace_dataset.to_spans():
	self.traces.put(span)
	for evaluations in trace_dataset.evaluations:
	for pb_evaluation in encode_evaluations(evaluations):
	self.traces.put(pb_evaluation)

[dosubot[bot]]

Hey there, @remiconnesson! Great to see you diving into the details of Arize Phoenix again. Hope you've been doing well!

Based on the information you've provided and the context from similar threads, it seems like you're trying to assign spans to a specific project other than the default one in Arize Phoenix.

To achieve this, you can use the using_project context manager provided in the projects.py file. This context manager allows you to switch the project for all spans created within its context. Here's how you can use it:

with using_project('your_project_name'):
    # Your code to create spans goes here
    # All spans created in this block will be associated with 'your_project_name'

When you enter the with using_project('your_project_name'): block, the __enter__ method of the using_project class is called. This method wraps the __init__ method of trace.ReadableSpan with a wrapper that overrides the project name in the span's resource attributes to the one you specify. When you exit the block, the __exit__ method is called, which restores the original __init__ method, ensuring that spans created outside the context manager are not affected.

This approach is particularly useful in notebook environments where you might want to dynamically assign spans to different projects for testing or demonstration purposes. However, as noted in the context manager's documentation, this method should not be used in production environments or complex OpenTelemetry setups due to the potential for unexpected behavior.

You can find more details about this in the projects.py file.

As for storing and retrieving traces for each project, the Arize Phoenix repository handles this through the Traces class in the traces.py file. The Traces class has several methods and properties that facilitate the storage and retrieval of traces. You can find more details about this in the traces.py file.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• [ENHANCEMENT] include the project_name in the trace dataset export
• [persistence] scope out how to scale to projects
• src/phoenix/trace/projects.py
• src/phoenix/core/traces.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mikeldking]

Hi @remiconnesson that's correct that there is no way to associate a trace dataset at the current moment to a project. Sorry about that - we can definitely add this but we are actually in the midst of adding actual persistence to the application itself so we probably won't tackle this immediately. Thanks for the feedback - totally think I understand your need and if you file a ticket we will definitely prioritize it and find someone to work on it in the near future.

[remiconnesson]

Nope don't worry, @mikeldking I'll pour everythin in default and add an tag for the project in the metadata or somewhere in the meantime.
Can I ask you where would you recommend to add such a tag?

[mikeldking]

I'd say must in the metadata attribute as a key-value pair. That way you SHOULD be able to filter by it.

[remiconnesson]

I'm unable to access the metadata from the query is that expected behavior? @mikeldking

[axiomofjoy]

Hey @remiconnesson, can you try something like metadata["field-key"] == "field-value"

[remiconnesson]

Amazing that's the way thank you :)