Creating a gym env with Basilisk - Memory Leak?

[WRetagne]

Hey everyone,

first of all thank you for developing Basilisk! It's an amazing program with so many capabilities!

I am trying to interface Basilisk with openAI Gym and followed https://bitbucket.org/avslab/basilisk-gym-interface/src/master/ closely.
However, it seems like each time I call the simulation like rwAttitudeSimulator.RWAttitudeSimulator(.1, 1.0, self.step_duration) in my reset method the old simulation doesn't get removed from memory. I don't know if this is exactly where the memory leak is coming from. I've tried using tracemalloc to investigate and I see this as a result:

The spacecraft.py calls keep growing with each time I reset my environment. If I just loop through my simulation (and therefore just create rwAttitudeSimulator.RWAttitudeSimulatorr(.1, 1.0, self.step_duration) once) this doesn't happen, but of course then I am not starting from a clean initial state. Have you encountered something similar before? Do you have any idea where the memory leak might come from?
Cheers
Wiebke

[ephraim271]

Hi @WRetagne , were you able to find a fix? We are facing the same issue

[schaubh]

Please note that calling Reset on spacecraft does not restore fully the conditions at the start of the script. In Monte Carlo runs we tend to kill off a BSK simulation and start a new one. The ability to on the fly restore initial setup across all BSK modules would be nice, but we have not rewritten the simulation modules to achieve this. With FSW modules the reset function does typically reset the module to clean initial conditions.

@adam-herrmann-95 or @Mark2000 , any thoughts on this as you have worked with Gym interface to BSK?

[Mark2000]

As an initial note, our Basilisk RL environment has moved to https://github.com/AVSLab/bsk_rl. Depending on what you’re doing, building off of the new generalized environment in this repo could be the easiest route forward.

Our approach (especially with the generalized environment, which is getting the most current development) is to completely delete the Basilisk simulation and any Basilisk components on env.reset and reinstantiate every part. That said, we’re still seeing a very slow memory leak with this environment, though I’ve yet to spend enough time tracing it to blame the leak on Basilisk specifically. Planning to take a closer look in the next month or two, will report back if the root cause is identified.

[WRetagne]

Hey Mark, thanks for the udpate! As a note: I've tried the Basilisk env with both an rllib implementation and a custom RL implementation and the memory leak has happened in both cases. I am also completely deleting and reinstantiating it on every env.reset. Let me know if I can help in any way!

[Mark2000]

How fast of a leak are you seeing? Currently with a moderately complex environment with a single Earth-observing satellite, I can make it 1.5-5M steps before running out of the 3.8 Gb of memory per core (one rolloutworker per core) that we have on our computing cluster.

[WRetagne]

I haven't tracked it exactly, but I can confirm it is a rather slow increase. I have one setup where I run multiple simulations concurrently and it does seem like it's a linear increase then. So if I run two simulations at the same time, then the memory leak is happening twice as fast

[Mark2000]

@WRetagne Found what I believe to be the main source of the memory leak, going to copy-paste some discussion here. In short, if you were reading or logging and reading Basilisk messages with vectors of vectors (such as rotation matrices, etc.), this causes leaks.

Tracked down the memory leak to reading messages that carry a vector of vectors. It appears that the inner vectors stay referenced somewhere. As an example with (using the objgraph package to track allocations), in scenarioCentralBody, repeatedly stepping the sim while accessing PositionVector (added after the first call to ExecuteSimulation)

    for i in range(10):
        if i == 0:
            print("Objects from creation: (type  total  +new)")
            objgraph.show_growth()
        else:
            print("New objects since last step/read: (type  total  +new)")
            objgraph.show_growth()
        scSim.ConfigureStopTime(simulationTime + macros.sec2nano(1000 * i))
        scSim.ExecuteSimulation()
        gravFactory.spiceObject.planetStateOutMsgs[0].read().PositionVector

does not show any new allocations:

New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)
New objects since last step/read: (type  total  +new)

Accessing J20002Pfix, a vector of three vectors, instead of PositionVector shows that three new vectors are added in memory and stay there

New objects since last step/read: (type  total  +new)
list     4685        +3
New objects since last step/read: (type  total  +new)
list     4688        +3
New objects since last step/read: (type  total  +new)
list     4691        +3
New objects since last step/read: (type  total  +new)
list     4694        +3
New objects since last step/read: (type  total  +new)
list     4697        +3
New objects since last step/read: (type  total  +new)
list     4700        +3
New objects since last step/read: (type  total  +new)
list     4703        +3
New objects since last step/read: (type  total  +new)
list     4706        +3

These can be viewed and show that they are in fact the vectors for each row of J20002Pfix .

Some additional data. The first list is emptied, and has the referents in the first picture, the simbaseclass and some listiterator that references all three lists. The second and third have three elements (which are just the values from the rotation matrix) and are only referred to by the same list iterator as the first one.
My hypothesis is that whatever is emptying the first list is intending to delete them all

@juan-g-bonilla did some swig wizardry and found

Hi all, sorry it took a while to answer. Turns out it is SWIG dark magic. In particular, in the typemap for 2 dimensional arrays (defined in swig_conly_data.i) we are not decreasing the references to the inner lists correctly. PyList_New will create an object with a ref counter of 1, PyList_Append to the outer list will add another reference count, which will be removed on outer list de-scope. Thus, we need an additional Py_DECREF to account for the initial ref count and thus have the inner list disappear with the outer list. The fixed typemap would look like:

%typemap(out) type [ANY][ANY] {
    int i, j;
    $result = PyList_New(0);
    for(i=0; i<$1_dim0; i++)
    {
        PyObject *locRow = PyList_New(0);
        for(j=0; j<$1_dim1; j++)
        {
            PyObject *outObject = PyFloat_FromDouble($1[i][j]);
            PyList_Append(locRow, outObject);
        }
        PyList_Append($result, locRow);
        Py_DECREF(locRow);  // <- new line
    }
}

I can do a quick PR with the hotfix. I have tested it briefly and it seems to solve at least this leak. Haven't done a thorough test, however, so take this with a grain of salt.

Will update in this thread once the patch is made and let you know if that accounts for the majority of the leakiness I was suffering from.

[Mark2000]

#447 implemented the fix and decreases memory usage. Still at least one other significant source of leaks that I'm tracking down; will keep reporting back on this thread.