From c8fbf108b749ed4a0f39a7aa6e3ba0bc5a91c3d8 Mon Sep 17 00:00:00 2001 From: Ryan McCormick Date: Fri, 1 Sep 2023 12:03:07 -0700 Subject: [PATCH 1/2] Add notes on IPv6 --- docs/protocol/README.md | 42 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/docs/protocol/README.md b/docs/protocol/README.md index 35e62e837e..b811d7951e 100644 --- a/docs/protocol/README.md +++ b/docs/protocol/README.md @@ -55,6 +55,48 @@ specification](https://github.com/triton-inference-server/common/blob/main/proto is also available. In addition, you can find the GRPC health checking protocol protobuf specification [here](https://github.com/triton-inference-server/common/blob/main/protobuf/health.proto). +## Restricted Protocols + You can configure the Triton endpoints, which implement the protocols, to restrict access to some protocols and to control network settings, please refer to [protocol customization guide](https://github.com/triton-inference-server/server/blob/main/docs/customization_guide/inference_protocols.md#httprest-and-grpc-protocols) for detail. + +## IPv6 + +Assuming your host or [docker config](https://docs.docker.com/config/daemon/ipv6/) +supports IPv6 connections, `tritonserver ` can be configured to use IPv6 +HTTP endpoints as follows: +``` +$ tritonserver ... --http-address ipv6:[::1]& +... +I0215 21:04:11.572305 571 grpc_server.cc:4868] Started GRPCInferenceService at 0.0.0.0:8001 +I0215 21:04:11.572528 571 http_server.cc:3477] Started HTTPService at ipv6:[::1]:8000 +I0215 21:04:11.614167 571 http_server.cc:184] Started Metrics Service at ipv6:[::1]:8002 +``` + +This can be confirmed via `netstat`, for example: +``` +$ netstat -tulpn | grep tritonserver +tcp6 0 0 :::8000 :::* LISTEN 571/tritonserver +tcp6 0 0 :::8001 :::* LISTEN 571/tritonserver +tcp6 0 0 :::8002 :::* LISTEN 571/tritonserver +``` + +And can be tested via `curl`, for example: +``` +$ curl -6 --verbose "http://[::1]:8000/v2/health/ready" +* Trying ::1:8000... +* TCP_NODELAY set +* Connected to ::1 (::1) port 8000 (#0) +> GET /v2/health/ready HTTP/1.1 +> Host: [::1]:8000 +> User-Agent: curl/7.68.0 +> Accept: */* +> +* Mark bundle as not supporting multiuse +< HTTP/1.1 200 OK +< Content-Length: 0 +< Content-Type: text/plain +< +* Connection #0 to host ::1 left intact +``` From 06112305a6f5bdc1f0ead6c439ba18f71b1ce07d Mon Sep 17 00:00:00 2001 From: Ryan McCormick Date: Fri, 1 Sep 2023 12:05:54 -0700 Subject: [PATCH 2/2] Remove extra space --- docs/protocol/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/protocol/README.md b/docs/protocol/README.md index b811d7951e..ddec7fc1d3 100644 --- a/docs/protocol/README.md +++ b/docs/protocol/README.md @@ -64,7 +64,7 @@ to [protocol customization guide](https://github.com/triton-inference-server/ser ## IPv6 Assuming your host or [docker config](https://docs.docker.com/config/daemon/ipv6/) -supports IPv6 connections, `tritonserver ` can be configured to use IPv6 +supports IPv6 connections, `tritonserver` can be configured to use IPv6 HTTP endpoints as follows: ``` $ tritonserver ... --http-address ipv6:[::1]&