[Bug]: Langsmith - dotted_order must contain at least two parts for child runs for run_id

[MarkRx]

What happened?

An update was made recently to generated the dotted_order field. It does not handle child records properly. The make_dot_order method needs to be updated to handle the case if parent_run_id is sent in. The documentation suggests it may also need to handle the full hierarchy.

Sample arguments:

{
    'model': 'azure/gpt-4o',
    'deployment_id': 'gpt-4o',
    'messages': [{
            'role': 'system',
            'content': '...'
        }, {
            'role': 'user',
            'content': '...'
        }
    ],
    'temperature': 0.2,
    'timeout': 120,
    'api_base': None,
    'metadata': {
        'run_name': 'describe',
        'tags': ['bitbucket_server', 'describe', 'version:0.2.6'],
        'extra': {
            'metadata': {
                'command': 'describe',
                'pr_url': '<redacted>'
            }
        },
        'parent_run_id': '82889dff-18a0-4f99-b371-a62c402983fe'
    }
}

Relevant log output

14:28:49 - LiteLLM:ERROR: langsmith.py:401 - Langsmith HTTP Error: 400 - {"detail":"dotted_order must contain at least two parts for child runs for run_id:ff13eb42-cd5b-4f33-b9a4-aa2d2db19e14 trace_id:ff13eb42-cd5b-4f33-b9a4-aa2d2db19e14 dotted_order:20241209T212844354467Zff13eb42-cd5b-4f33-b9a4-aa2d2db19e14 parent_run_id:82889dff-18a0-4f99-b371-a62c402983fe"}

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.53.1

Twitter / LinkedIn details

No response

@ishaan-jaff

[hnykda]

I really don't love this, but here is a workaround I did when using litellm (you can infer what you have to do in the language of your choice):

import uuid
from litellm import acompletion

from langsmith.run_helpers import get_current_run_tree
from langsmith import traceable

from datetime import datetime, timezone


def get_dotted_order(parent_run_dotted_order: str, current_run_id: str):
    current_time = datetime.now(timezone.utc)
    formatted_time = (
        current_time.strftime("%Y%m%dT%H%M%S") + f"{current_time.microsecond:06d}Z"
    )
    return f"{parent_run_dotted_order}.{formatted_time}{current_run_id}"


def patch_metadata(metadata: dict):
    new_run_id = str(uuid.uuid4())
    run = get_current_run_tree()

    metadata = {
        **metadata,
        "parent_run_id": str(run.id),
        "id": new_run_id,
        "trace_id": str(run.trace_id),
        "dotted_order": get_dotted_order(run.dotted_order, new_run_id),
    }
    return metadata


# @traceable
async def _llm_call(model, messages, **kwargs):
    metadata = kwargs.get("metadata", {})
    metadata = patch_metadata(metadata)

    return await acompletion(
        model=model, messages=messages, metadata=metadata, **kwargs
    )