[discussion] How to set clusterspec

[gaocegege]

Now we set the TF_CONFIG to get the cluster spec in the training code, and it follows the idea in Google Cloud Machine Learning Engine (Cloud ML Engine).

In TensorFlow distributed training docs, it uses CLI arguments instead. We can not support those two ways since we want to hide the service discovery layer from the users, so I think we could discuss which one will be supported. Maybe AI engineers could give us more info.

\cc @DjangoPeng @ScorpioCPH

[jlewi]

TF_CONFIG is a TensorFlow convention that TF APIs like the EstimatorAPI use to get information about the runtime environment and configure the job appropriately.

If users want to use command line arguments, they can write a launcher script that parses TF_CONFIG and sets the command line arguments as needed. This is what we do in the TFCNN example; see https://github.com/kubeflow/kubeflow/blob/master/tf-controller-examples/tf-cnn/launcher.py

I think the problem with command line arguments is that everyone will use slightly different conventions.

[gaocegege]

SGTM, WDYT @DjangoPeng

[gaocegege]

I am closing it since it is stale. If any comments I will reopen it, thanks.