Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] invocation parameter parsing from a span to playground does not properly surface parsing errors #5198

Closed
Parker-Stafford opened this issue Oct 25, 2024 · 0 comments · Fixed by #5202
Assignees
Labels
bug Something isn't working triage issues that need triage

Comments

@Parker-Stafford
Copy link
Contributor

Describe the bug
When the invocation parameters are unparsable (via zod or json parse) the error is not surfaced up to the playground leading to lossy information without the user knowing

Here are some tests to reproduce the issue
This one will return the following for invocation parameters

[
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "string",
    "path": [],
    "message": "Expected object, received string"
  }
]
 it("should return invocation parameters parsing errors if they are malformed", () => {
    const parsedAttributes = {
      llm: {
        model_name: "gpt-3.5-turbo",
        invocation_parameters: '"invalid"',
      },
    };
    expect(getModelConfigFromAttributes(parsedAttributes)).toEqual({
      modelConfig: {
        modelName: "gpt-3.5-turbo",
        provider: "OPENAI",
        invocationParameters: {},
      },
      parsingErrors: [MODEL_CONFIG_WITH_INVOCATION_PARAMETERS_PARSING_ERROR],
    });
  });

this one will just not have any parsing errors since the json is invalid

 it("should return invocation parameters parsing errors if they are malformed", () => {
    const parsedAttributes = {
      llm: {
        model_name: "gpt-3.5-turbo",
        invocation_parameters: "invalid",
      },
    };
    expect(getModelConfigFromAttributes(parsedAttributes)).toEqual({
      modelConfig: {
        modelName: "gpt-3.5-turbo",
        provider: "OPENAI",
        invocationParameters: {},
      },
      parsingErrors: [MODEL_CONFIG_WITH_INVOCATION_PARAMETERS_PARSING_ERROR],
    });
  });
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage issues that need triage
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants