[FEA]: Update LLMContext to store outputs that are not json serializable #1560
Assignees
Labels
feature request
New feature or request
Comments
This was referenced Mar 11, 2024
rapids-bot bot
pushed a commit
to nv-morpheus/MRC
that referenced
this issue
Apr 3, 2024
… possible, and as pybind11::object otherwise (#455) * Add an optional `unserializable_handler_fn` callback to `mrc::pymrc::cast_from_pyobject` which will be invoked for any unsupported Python object. Allowing for serializing unsupported object. ex: ```cpp pymrc::unserializable_handler_fn_t handler_fn = [](const py::object& source, const std::string& path) { return nlohmann::json(py::cast<float>(source)); }; // decimal.Decimal is not serializable py::object Decimal = py::module_::import("decimal").attr("Decimal"); py::object o = Decimal("1.0"); EXPECT_EQ(pymrc::cast_from_pyobject(o, handler_fn), nlohmann::json(1.0)); ``` * Add `JSONValues` container class which is an immutable container for holding Python values as `nlohmann::json` objects if possible, and as `pybind11::object` otherwise. The container can be copied and moved, but the underlying `nlohmann::json` object is immutable. * Updates `nlohmann_json` from 3.9 to 3.11 for `patch_inplace` method. * Incorporates ideas from @drobison00's PR #417 with three key differences: 1. Changes to the `cast_from_pyobject` are opt-in when `unserializable_handler_fn` is provided, otherwise there is no behavior change to the method. 2. Unserializable objects are stored in `JSONValues` rather than a global cache. 3. Does not rely on parsing the place-holder values. This PR is related to nv-morpheus/Morpheus#1560 Authors: - David Gardner (https://github.com/dagardner-nv) - Devin Robison (https://github.com/drobison00) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #455
rapids-bot bot
pushed a commit
that referenced
this issue
Apr 5, 2024
* Contains breaking changes for the C++ API * No breaking changes for the Python API * Updates `nlohmann_json from` 3.9 to 3.11 for `patch_inplace` method. * Requires nv-morpheus/MRC#455 to be merged first Closes #1560 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #1589
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is this a new feature, an improvement, or a change to existing functionality?
Change
How would you describe the priority of this feature request
Medium
Please provide a clear description of problem this feature solves
Encountered in PR #1539 we would like to store an exception as the output of an LLM Node when the node encounters an error, however exceptions are not json serializable.
Describe your ideal solution
Create a container which has the ability to store either a json output or a
py::objectfor anything not json serializable.Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: