Make request payload size configurable

[MrAta]

Motivation

For scenarios where we have long context lengths (e.g. 32k) and have multiple prompts in a single completion request, the size of the http request payload exceeds the default 2MB that rust json has which fails to send the request to the engine.

Modifications

This PR makes the request payload size configurable by adding a new max_request_payload_size field to the Config struct.

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[ByronHsu]

Can you add a test in

sglang/rust/py_test/test_launch_server.py

Line 175 in 993956c

def test_2_add_and_remove_worker(self):

with a ultra long request and setting the DP to be 1 to test the payload?

[MrAta]

Can you add a test in

sglang/rust/py_test/test_launch_server.py

Line 175 in 993956c

def test_2_add_and_remove_worker(self):

with a ultra long request and setting the DP to be 1 to test the payload?

Ag, added some tests.

[ByronHsu]

Nice bug catch!

[ByronHsu]

@MrAta can you release a new version after this is merged?