kubernetes_scheduler: add service_account runopt so users can specify per job acls + log fix

[d4l3k]

Fixes #406

This adds a new service_account runopt that sets the service_account_name on the pod spec of each pod.

https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md

It also fixes a small bug when fetching logs w/ _ in the role name.

Test plan:

added unit tests

torchx run --scheduler kubernetes -c service_account=default --wait --log dist.ddp --script dist_app.py -j 1x1

[facebook-github-bot]

@d4l3k has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[codecov]

Codecov Report

Merging #408 (ea963b1) into main (1f1a50f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #408   +/-   ##
=======================================
  Coverage   94.21%   94.21%           
=======================================
  Files          66       66           
  Lines        3681     3684    +3     
=======================================
+ Hits         3468     3471    +3     
  Misses        213      213           

Impacted Files	Coverage Δ
torchx/pipelines/kfp/adapter.py	98.52% <ø> (ø)
torchx/schedulers/kubernetes_scheduler.py	92.78% <100.00%> (+0.11%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f1a50f...ea963b1. Read the comment docs.

[facebook-github-bot]

@d4l3k has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@d4l3k has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.