Fix pickle error with remote code models in vLLM Ray worker process

[eicherseiji]

Why are these changes needed?

Since #53621, vLLM engines running DeepSeek-V2-Lite and related models that fetch remote code fail with pickle errors, since the engine only registers custom configs to serialize by value (and avoid the pickle error) if transformers_modules can be imported.

# vllm/transformers_utils/config.py
def maybe_register_config_serialize_by_value() -> None:
    try:
        import transformers_modules
    except ImportError:
        # the config does not need trust_remote_code
        return
...

We relied on a call to AutoProcessor.from_pretrained to initialize transformers_modules, so that maybe_register_config_serialize_by_value would execute correctly when AsyncLLM starts. This was removed in #53621. Now we can use init_hf_modules so to accomplish the same result more directly.

We could also fix this upstream with vllm-project/vllm#19510.

Traceback:

(ServeController pid=14535) Traceback (most recent call last):
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/deployment_state.py", line 718, in check_ready
(ServeController pid=14535)     ) = ray.get(self._ready_obj_ref)
(ServeController pid=14535)         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
(ServeController pid=14535)     return fn(*args, **kwargs)
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
(ServeController pid=14535)     return func(*args, **kwargs)
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/_private/worker.py", line 2847, in get
(ServeController pid=14535)     values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
(ServeController pid=14535)                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/_private/worker.py", line 947, in get_objects
(ServeController pid=14535)     raise value.as_instanceof_cause()
(ServeController pid=14535) ray.exceptions.RayTaskError(RuntimeError): ray::ServeReplica:default:LLMDeploymentdeepseek.initialize_and_get_metadata() (pid=32141, ip=10.0.127.215, actor_id=b962327f5b252ba15620da8102000000, repr=<ray.serve._private.replica.ServeReplica:default:LLMDeploymentdeepseek object at 0x7ae2c961c7d0>)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
(ServeController pid=14535)     return self.__get_result()
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
(ServeController pid=14535)     raise self._exception
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 1022, in initialize_and_get_metadata
(ServeController pid=14535)     await self._replica_impl.initialize(deployment_config)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 730, in initialize
(ServeController pid=14535)     raise RuntimeError(traceback.format_exc()) from None
(ServeController pid=14535) RuntimeError: Traceback (most recent call last):
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 703, in initialize
(ServeController pid=14535)     self._user_callable_asgi_app = await asyncio.wrap_future(
(ServeController pid=14535)                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 1442, in initialize_callable
(ServeController pid=14535)     await self._call_func_or_gen(
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 1388, in _call_func_or_gen
(ServeController pid=14535)     result = await result
(ServeController pid=14535)              ^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/llm_server.py", line 442, in __init__
(ServeController pid=14535)     await asyncio.wait_for(self._start_engine(), timeout=ENGINE_START_TIMEOUT_S)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/asyncio/tasks.py", line 489, in wait_for
(ServeController pid=14535)     return fut.result()
(ServeController pid=14535)            ^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/llm_server.py", line 488, in _start_engine
(ServeController pid=14535)     await self.engine.start()
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/vllm/vllm_engine.py", line 280, in start
(ServeController pid=14535)     self.engine = await self._start_engine()
(ServeController pid=14535)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/vllm/vllm_engine.py", line 337, in _start_engine
(ServeController pid=14535)     return await self._start_engine_v1()
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/vllm/vllm_engine.py", line 405, in _start_engine_v1
(ServeController pid=14535)     return self._start_async_llm_engine(
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/llm/_internal/serve/deployments/llm/vllm/vllm_engine.py", line 528, in _start_async_llm_engine
(ServeController pid=14535)     engine = vllm.engine.async_llm_engine.AsyncLLMEngine(
(ServeController pid=14535)              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/vllm/v1/engine/async_llm.py", line 123, in __init__
(ServeController pid=14535)     self.engine_core = core_client_class(
(ServeController pid=14535)                        ^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 734, in __init__
(ServeController pid=14535)     super().__init__(
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/vllm/v1/engine/core_client.py", line 404, in __init__
(ServeController pid=14535)     self.resources.local_engine_manager = CoreEngineProcManager(
(ServeController pid=14535)                                           ^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/site-packages/vllm/v1/utils.py", line 142, in __init__
(ServeController pid=14535)     proc.start()
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/process.py", line 121, in start
(ServeController pid=14535)     self._popen = self._Popen(self)
(ServeController pid=14535)                   ^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/context.py", line 288, in _Popen
(ServeController pid=14535)     return Popen(process_obj)
(ServeController pid=14535)            ^^^^^^^^^^^^^^^^^^
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in __init__
(ServeController pid=14535)     super().__init__(process_obj)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/popen_fork.py", line 19, in __init__
(ServeController pid=14535)     self._launch(process_obj)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 47, in _launch
(ServeController pid=14535)     reduction.dump(process_obj, fp)
(ServeController pid=14535)   File "/home/ray/anaconda3/lib/python3.11/multiprocessing/reduction.py", line 60, in dump
(ServeController pid=14535)     ForkingPickler(file, protocol).dump(obj)
(ServeController pid=14535) _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite.604d5664dddd88a0433dbae533b7fe9472482de0.configuration_deepseek.DeepseekV2Config'>: import of module 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite.604d5664dddd88a0433dbae533b7fe9472482de0.configuration_deepseek' failed

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[kouroshHakha]

just one nit, let's also rerun the release tests on this PR before merging.