[Fix] Several bugs when using HvdAllToAllEmbedding to train model and save/restore KV files. #1631

Job	Run time
Clang C++ code format	9s
Check that we can build the docs	10m 18s
Bazel code format	10s
Check that the pre-commit works	48s
Check that the source code is compliant	10m 32s
Run the cpu tests in a small python docker image	23m 20s
Valid build files	1m 33s
Yapf Python code format	46s
Notebook format	15s
Notebook lint	16s
Check that the CODEOWNERS is valid	9s
	48m 16s

Provide feedback