[RayJob] Add token authentication support for light weight job submitter

[Future-Outlier]

Why are these changes needed?

1. support RayJob using k8s job mode, light weight job submitter

What changes were made?

When doing job submission, we will do 2 steps in the light weight job submitter, and each steps require http header.

1. send a http request to submit the job
2. establish a web socket connection to tail the job logs

1 is supported here: #4210
2 is supported in this PR, I supported it by adding http header in web socket's dial options.

How was this tested?

cd ray-operator/

IMG=kuberay/operator:nightly-7 make docker-build
kind load docker-image kuberay/operator:nightly-7
helm install kuberay-operator --set image.repository=kuberay/operator --set image.tag=nightly-7 ../helm-chart/kuberay-operator

IMG=kuberay/submitter:1321 make docker-image-rayjob-submitter
kind load docker-image kuberay/submitter:1321

logs from light weight job submitter

apiVersion: ray.io/v1
kind: RayJob
metadata:
  name: rayjob-light-weight-submitter-auth-token-v4
spec:
  submissionMode: "K8sJobMode"
  rayClusterSpec:
    rayVersion: '2.52.0' # should match the Ray version in the image of the containers
    authOptions:
      mode: "token"
    headGroupSpec:
      rayStartParams: {}
      #pod template
      template:
        spec:
          containers:
          - name: ray-head
            image: rayproject/ray:nightly-py311-cpu
            ports:
            - containerPort: 6379
              name: gcs-server
            - containerPort: 8265 # Ray dashboard
              name: dashboard
            - containerPort: 10001
              name: client
            resources:
              limits:
                cpu: "1"
              requests:
                cpu: "200m"
            volumeMounts:
            - mountPath: /home/ray/samples
              name: code-sample
          volumes:
          - name: code-sample
            configMap:
              name: ray-job-code-sample
              items:
              - key: sample_code.py
                path: sample_code.py
    workerGroupSpecs:
    - replicas: 1
      minReplicas: 1
      maxReplicas: 5
      rayStartParams: {}
      template:
        spec:
          containers:
          - name: ray-worker # must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc'
            image: rayproject/ray:nightly-py311-cpu
            resources:
              limits:
                cpu: "1"
              requests:
                cpu: "200m"
  submitterPodTemplate:
    spec:
      restartPolicy: Never
      containers:
      - name: my-custom-rayjob-submitter
        image: kuberay/submitter:1321
        command: ["/submitter"]
        args: ["--runtime-env-json", '{"pip":["requests==2.26.0","pendulum==2.1.2"],"env_vars":{"counter_name":"test_counter"}}', "--", "python", "/home/ray/samples/sample_code.py"]
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-job-code-sample
data:
  sample_code.py: |
    import ray
    import os
    import requests

    ray.init()

    @ray.remote
    class Counter:
        def __init__(self):
            # Used to verify runtimeEnv
            self.name = os.getenv("counter_name")
            assert self.name == "test_counter"
            self.counter = 0

        def inc(self):
            self.counter += 1

        def get_counter(self):
            return "{} got {}".format(self.name, self.counter)

    counter = Counter.remote()

    for _ in range(5):
        ray.get(counter.inc.remote())
        print(ray.get(counter.get_counter.remote()))

    # Verify that the correct runtime env was used for the job.
    assert requests.__version__ == "2.26.0"

Related issue number

#4203

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[Copilot]

Pull Request Overview

This PR adds token authentication support to the lightweight job submitter by enabling it to read and use authentication tokens when communicating with the Ray dashboard and tailing job logs via websockets.

• Token authentication support for job submission via HTTP API
• Token authentication support for log tailing via WebSocket connections
• Environment variable RAY_AUTH_TOKEN integration for retrieving auth tokens

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
ray-operator/rayjob-submitter/rayjob-submitter.go	Adds authToken parameter to TailJobLogs function and configures WebSocket connection with Bearer token authentication header when token is provided
ray-operator/rayjob-submitter/cmd/main.go	Retrieves RAY_AUTH_TOKEN from environment and passes it to both the dashboard client initialization and log tailing function

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[andrewsykim]

nit: Authorization

[andrewsykim]

RAY_AUTH_TOKEN_ENV_VAR

[rueian]

I think hard-coding the name here is acceptable for now, since importing the entire utils package could significantly increase the submitter size.

[win5923]

Can you add TODO here to change x-ray-authorization?
ray-project/ray#58819

[andrewsykim]

I think we can just add it in this PR now

[Future-Outlier]

actually "authorization" is for API clients, SDKs, and x-ray-authorization is for proxy.
in this case, light weight job submitter is using API clients, SDKs.
but they will both work, so its ok to fix this later.