Is it possible to use this library for doing additional domain specific pre-training of the Bert weights?

[peregilk]

No description provided.

[kpe]

In principle yes. This would not be much different than the fine-tuning usually done with BERT.
Currently bert-for-tf's does not include the final dense classification layers used in the MLM pre-training task (so they have to be added explicitly in the Keras Model). And if you want to reuse the original weights for those classification layers you might have to manually load the weights or carefully name them (by using the exact same names used in the pre-trained checkpoint).
But, I guess, it should be better, if I add a config parameter controlling the instantiation of the cls/ layers, so that the weights from the pre-trained checkpoint are loaded automatically.
Good point, Thank you, @peregilk!