How to disable events' input and output data in the stream events API endpoint

[shaojun]

hi,
Say I have a very simple chain, that the first runnable produced a huge output (like all text from a huge pdf) and then send to LLM for summarize, and finally return summarized content to client side.
The client side are using stream event to get progress running info reported from server for better UX, as I can see, every client side received event has the intput and output data, that would be a perf problem or unnecessary for my case especially for that huge text output:

event: data
data: {"event":"on_chain_end","name":"extract_huge_text_from_upload_pdf_file","run_id":"66bff721-99ba-454b-b883-c5a47c9d209a","tags":["seq:step:1","INFO"],"metadata":{},
"data":{"input":{"filePathIsC:\\huge.pdf.......","output":{"I'm so huge, and client side does not need me at all.........................."

I read the https://python.langchain.com/docs/expression_language/streaming#filtering-events seems the mininal control granularity is runnable.

How can I just disable the data for specific runnable while keep its event? sth like:

my_runnable.with_config({"disable_show_input_output": "true"})

[eyurtsev]

There's a parameter (that we need to rename) on add_routes:

stream_log_name_allow_list: Optional[Sequence[str]] = None

You specify it on the server side and it will only include events from runnables with a matching run name. You may need to name the runnables in your config

my_runnable.with_config({'run_name': 'meow'})

Alternatively, the client itself can provide filtering parameters (take a look at the astream_events API documentation in langchain) -- but only use this strategy for internal tools etc. If it's exposed via a web-service to untrusted users, those users will be able to use the endpoint to ask for additional data

---

Please note that this strategy is also useful to deal with serialization issues if intermediate steps in the overall chain return non-serializable outputs

[shaojun]

I don't want to exclude an event(i believe that can be straight forward done by filter by name or tag from client). I want the event but don't include its input and output data.

[eyurtsev]

There's nothing built-in to do that right now. This is probably a feature that makes sense, but I'll need to think about about the best API for it.

In the meantime, you can do one of:

• Include some placeholder steps in the chain that don't accept inputs and outputs, but can be used to proxy progress.
• use a custom stream implementation that does processing server side on the event stream: e.g., https://github.com/langchain-ai/langserve/blob/main/examples/agent_custom_streaming/server.py#L160

[shaojun]

thanks!
For me this is a must feature in langserve which can best keep client side UX (get all server side progress reported and meanwhile don't get unnessary data).

[rhlarora84]

+1 on this. Filtering support in the stream_events is not going to be helpful here. I don't need inputs to be streamed back as part of the payload which overtime becomes huge