Known issues

With some specific versions of Theano and with floatX=float32, it is possible that the training of the neural network that fits the Q-function fails and returns nan. In that case you can try to update to the latest version of Theano or add a flag to set floatX to float64: THEANO_FLAGS='floatX=float64' python run ..