Aitextgen colab notebook fp16 training

[Blazeolmo]

[katemonster33]

You need to follow the tutorial to compile and install the apex module. You need to comment out lines 32-40 in setup.py inside apex to bypass an error regarding the version of cuda that torch is compiled against, otherwise the pip install command will fail.