why setting retain_graph=True

[BaohaoLiao]

When you do .backward(), you set retain_graph=True as follows:

minREV/rev.py

Line 252 in 4f32aca

g_Y_1.backward(dY_2, retain_graph=True)

minREV/rev.py

Line 283 in 4f32aca

f_X_2.backward(dY_1, retain_graph=True)

According to this discussion, retain_graph=True will make a backward pass that does not delete intermediary results. The above two backward passes don't share the same sub-network. I think deleting intermediate results will not affect them. And setting retain_graph=False will further save more memory.