Program hangs

[kexul]

Dear openff developers:
I came across some molecules which made the program hang at generating bespoke QC data for a long time (one day) and it consumes lots of memory (about 15G). But I did not see any backend QC program is running ( in this case I use torchani with GPU, but I did not see any GPU utilization). When I pause the program manually in debug mode, it stopped at the optimizeGeometry function.

 def optimizeGeometry(self):
        """
        High-level optimization loop.
        This allows calcEnergyForce() to be separated from the rest of the codes
        """
        self.calcEnergyForce()
        self.prepareFirstStep()
        while self.state not in [OPT_STATE.CONVERGED, OPT_STATE.FAILED]:
            self.step()
            if self.state == OPT_STATE.NEEDS_EVALUATION:
                self.calcEnergyForce()
                self.evaluateStep()
        if self.state == OPT_STATE.FAILED:
            raise GeomOptNotConvergedError("Optimizer.optimizeGeometry() failed to converge.")
        return self.progress

Here is one of the molecules:
mol25_minimised.zip

I used mostly the same code as the blog post, except xtb was replaced by torchani. Here are the fragments it generated:

[jthorton]

Hi, @kexul thanks for tracking down the GPU issue, did adding this improve the performance? The code section above seems to be from geometric which is the optimiser we use, calcEnergyForce should be calling out to torchani to calculate the forces do you know if its this line that hangs?

[kexul]

did adding this improve the performance?

Nope, I found that torchANI is not the bottleneck. Pure CPU is fast enough.

calcEnergyForce should be calling out to torchani to calculate the forces do you know if its this line that hangs?

Nope, as far as I can see, the program hangs because the converge criterion is not satisfied for many iterations.

[jthorton]

Nope, as far as I can see, the program hangs because the converge criterion is not satisfied for many iterations.

Ah this agrees with what I have seen as well, we do plan on exposing the optimisation settings soon which would allow you to have looser criteria for ANI we have not yet confirmed if this helps, however. We also plan on adding a pre-optimisation stage which might help accelerate the convergence.

[kexul]

Ah this agrees with what I have seen as well, we do plan on exposing the optimisation settings soon which would allow you to have looser criteria for ANI we have not yet confirmed if this helps, however. We also plan on adding a pre-optimisation stage which might help accelerate the convergence.

Thanks for your work 🤗

[SimonBoothroyd]

It seems like this issue has been resolved now, but please feel free to re-open it if this is not the case! 🙂