[core] Fix incorrect comment (ray-project#51575)

dentiny · Dhakshin Suriakannu · commit 5f161b6fd052 · 2025-03-27T11:39:53.000-07:00
Current implementation is, we create multiple grpc stubs but not TCP connections. I did some benchmark, which shows multiple grpc stub does improvement performance so we cannot simply remove them. Baseline: ray-project#51656 Benchmark result: ```sh ubuntu@hjiang-devbox-pg$ python3 /home/ubuntu/ray/release/release_logs/compare_perf_metrics /home/ubuntu/ray/release/release_logs/base_version /home/ubuntu/ray/release/release_logs/new_version REGRESSION 14.66%: single_client_tasks_and_get_batch (THROUGHPUT) regresses from 5.969306035542166 to 5.094453847363354 (14.66%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 14.51%: tasks_per_second (THROUGHPUT) regresses from 229.36388671973262 to 196.07926002040125 (14.51%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_nodes.json REGRESSION 12.24%: single_client_get_object_containing_10k_refs (THROUGHPUT) regresses from 13.238079759644892 to 11.61748355563387 (12.24%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 6.99%: multi_client_put_calls_Plasma_Store (THROUGHPUT) regresses from 16841.57415006188 to 15663.994106069527 (6.99%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 6.66%: single_client_wait_1k_refs (THROUGHPUT) regresses from 5.0308320360232734 to 4.695580178438465 (6.66%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 3.27%: 1_n_actor_calls_async (THROUGHPUT) regresses from 8271.004793738302 to 8000.250850666334 (3.27%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 2.93%: client__tasks_and_put_batch (THROUGHPUT) regresses from 14039.49342189659 to 13628.021852828302 (2.93%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 2.57%: 1_n_async_actor_calls_async (THROUGHPUT) regresses from 7549.855764807742 to 7355.844212331484 (2.57%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 2.51%: n_n_async_actor_calls_async (THROUGHPUT) regresses from 23671.11885393803 to 23076.0705989556 (2.51%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 2.48%: single_client_tasks_sync (THROUGHPUT) regresses from 952.0269029041373 to 928.4100648169424 (2.48%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 2.27%: n_n_actor_calls_with_arg_async (THROUGHPUT) regresses from 2786.0434704559507 to 2722.7190863593105 (2.27%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 1.89%: client__1_1_actor_calls_sync (THROUGHPUT) regresses from 523.9186869126953 to 514.0347539070235 (1.89%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 1.48%: placement_group_create/removal (THROUGHPUT) regresses from 755.9036396341403 to 744.686404601542 (1.48%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 1.38%: client__1_1_actor_calls_async (THROUGHPUT) regresses from 1055.293613301056 to 1040.6856076865438 (1.38%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 1.21%: client__tasks_and_get_batch (THROUGHPUT) regresses from 0.963509372266668 to 0.9518418662213537 (1.21%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 1.00%: single_client_put_calls_Plasma_Store (THROUGHPUT) regresses from 4998.683273892644 to 4948.940736644676 (1.00%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.95%: n_n_actor_calls_async (THROUGHPUT) regresses from 27023.396892676123 to 26767.1762953109 (0.95%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.94%: single_client_put_gigabytes (THROUGHPUT) regresses from 19.0889649635755 to 18.909259900586363 (0.94%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.72%: 1_1_async_actor_calls_async (THROUGHPUT) regresses from 4642.895287623349 to 4609.41957458445 (0.72%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.71%: single_client_tasks_async (THROUGHPUT) regresses from 7909.367644778592 to 7852.882247928618 (0.71%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.37%: single_client_get_calls_Plasma_Store (THROUGHPUT) regresses from 10536.660461092333 to 10497.924384397613 (0.37%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.31%: client__1_1_actor_calls_concurrent (THROUGHPUT) regresses from 1045.148463125327 to 1041.9241552653498 (0.31%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.14%: 1_1_actor_calls_concurrent (THROUGHPUT) regresses from 5161.152587049948 to 5154.097793024906 (0.14%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 0.08%: client__put_calls (THROUGHPUT) regresses from 774.1865536687783 to 773.580427400101 (0.08%) in /home/ubuntu/ray/release/release_logs/new_version/microbenchmark.json REGRESSION 29.16%: dashboard_p95_latency_ms (LATENCY) regresses from 26.737 to 34.534 (29.16%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_nodes.json REGRESSION 28.52%: dashboard_p99_latency_ms (LATENCY) regresses from 3058.247 to 3930.337 (28.52%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_actors.json REGRESSION 16.62%: stage_3_creation_time (LATENCY) regresses from 1.8721938133239746 to 2.183365821838379 (16.62%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_many_tasks.json REGRESSION 11.56%: dashboard_p99_latency_ms (LATENCY) regresses from 277.091 to 309.116 (11.56%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_pgs.json REGRESSION 9.37%: dashboard_p99_latency_ms (LATENCY) regresses from 77.557 to 84.822 (9.37%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_nodes.json REGRESSION 5.97%: avg_pg_remove_time_ms (LATENCY) regresses from 1.3214836546547326 to 1.4004240105108736 (5.97%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_placement_group.json REGRESSION 5.43%: dashboard_p50_latency_ms (LATENCY) regresses from 5.399 to 5.692 (5.43%) in /home/ubuntu/ray/release/release_logs/new_version/benchmarks/many_nodes.json REGRESSION 3.73%: avg_iteration_time (LATENCY) regresses from 1.1903677296638489 to 1.234775812625885 (3.73%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_dead_actors.json REGRESSION 1.09%: stage_1_avg_iteration_time (LATENCY) regresses from 12.530043482780457 to 12.667147731781006 (1.09%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_many_tasks.json REGRESSION 0.48%: stage_2_avg_iteration_time (LATENCY) regresses from 38.180511474609375 to 38.36462712287903 (0.48%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_many_tasks.json REGRESSION 0.32%: stage_3_time (LATENCY) regresses from 1884.1879193782806 to 1890.2630755901337 (0.32%) in /home/ubuntu/ray/release/release_logs/new_version/stress_tests/stress_test_many_tasks.json REGRESSION 0.05%: 10000_args_time (LATENCY) regresses from 19.075126932999993 to 19.085101717 (0.05%) in /home/ubuntu/ray/release/release_logs/new_version/scalability/single_node.json ``` Reproduction commands: ```sh BUILDKITE_TOKEN=bkua_77ebf3852a34e264e48619dab87574f634e9e4f7 python3 /home/ubuntu/ray/release/release_logs/fetch_release_logs.py new_version e00693c dentiny:hjiang/fix-client-conn BUILDKITE_TOKEN=bkua_77ebf3852a34e264e48619dab87574f634e9e4f7 python3 /home/ubuntu/ray/release/release_logs/fetch_release_logs.py base_version b92dcd1 dentiny:hjiang/baseline-release-benchmark python3 /home/ubuntu/ray/release/release_logs/compare_perf_metrics /home/ubuntu/ray/release/release_logs/base_version /home/ubuntu/ray/release/release_logs/new_version ``` Signed-off-by: dentiny <dentinyhao@gmail.com> Signed-off-by: Dhakshin Suriakannu <d_suriakannu@apple.com>
diff --git a/src/ray/common/ray_config_def.h b/src/ray/common/ray_config_def.h
@@ -916,10 +916,8 @@ RAY_CONFIG(int64_t, py_gcs_connect_timeout_s, 30)
 // Costs an extra RPC.
 // TODO(vitsai): Remove this flag
 RAY_CONFIG(bool, enable_reap_actor_death, true)
-// The number of sockets between object manager.
-// The higher the number the higher throughput of the data
-// transfer it'll be, but it'll also user more sockets and
-// more CPU resources.
+
+// The number of grpc clients between object managers.
 RAY_CONFIG(int, object_manager_client_connection_num, 4)
 
 // The number of object manager thread. By default, it's
