Confinement is very slow

[jtgilbert]

I haven't dived into the existing confinement code to see what exactly are the bottlenecks, but on a large HUC with high drainage density (10190002) after 10 days it was ~29% done.

While confinement was still only an Arc toolbox, I needed an open source solution for some of my masters work so I coded up an OS version. I dug it up, cleaned it up a tiny it and did some testing and it completed that same HUC in 6 hours. It also produces almost identical value and confining margin outputs when compared with outputs run using the current version. So, I'm going to link the code here for future reference.

[philipbaileynar]

Thanks @jtgilbert

I am drawn to the following sentence in your post:

It also produces almost identical value and confining margin outputs

Are you saying that the latest open source riverscapes tools version of confinement produces nearly identical results to your command line ArcPy version? If so this is extremely encouraging!

Regarding performance... this is nearly always caused by some kind of union operation. We have learned several workarounds to avoid these big union operations, but no one has looked at the new confinement tool in ages. Every time we get close we get pulled back onto VBET or channel area... the two tools directly upstream of confinement.

[jtgilbert]

@philipbaileynar, no it's not ArcPy. It's open source python. I haven't looked closely to see if the algorithm is essentially the same, but the riverscapes version and my own version I wrote independently a while back produce pretty much the same results.

[philipbaileynar]

Thanks. The new riverscapes version does lots of important things like create a project file, use geopackages, organize data as inputs, intermediates and outputs. I suspect the problem is that we have changed the dimensionality of the inputs a lot. For example VBET used to be one polygon, now it is one polygon per level path. And channel area includes waterbodies etc. As I said before I would like to identify if there are any union operations that can be removed or precisely identify the cause of the slowdown. I am encouraged that it is producing meaningful results already!

[MattReimer]

THis is good to hear. Confinement hasn't gotten the same love that RSContext, Taudem, ChannelArea, VBET (with caveats) and BRAT have gotten so it's definitely ripe for an optimization pass.

Like @philipbaileynar said, look for union operations that are taking forever. Vector unions seem to scale exponentially with the density and complexity of the geometry. In the past we've solved this somewhat by being more careful about order of operations or doing things in raster space instead of vector space.