Is it normal that running train_line.py renders samples from one mode only?

[saleml]

No description provided.

[josephdviviano]

I likely need to change the default options to enable off policy exploration

[josephdviviano]

Hmm, on my machine, the training isn't perfect (the default options undertrain the policy) but I definitely sample from both modes.

Were your experiments off of master or is this maybe related to changes we made RE: off policy training in the other open pr (#174)?

[saleml]

I'm investigating the issue. Indeed, different behavior on mater and on fix_off_policy.

One thing worth noting is that with the original number of trajectories (1.28e6), sure the samples are from one mode only, but are more accurate.

Edit1: And when running the code for slightly longer on the fix_off_policy branch (3e6 trajectories), I obtain a figure that is similar to the one obtained with master.

[saleml]

In fix_off_policy, I had forgotten a keyword. The problem is fixed in 89c72b5