v4 ParticleSet feature discussion

[fluidnumerics-joe]

The ParticleSet.execute method advances all particles in the ParticleSet through time by repeatedly applying a given kernel function (e.g., AdvectionRK4) until a specified endtime or runtime is reached. It also handles:

• Releasing new particles (repeat)
• Writing particle output
• Running interaction kernels (if provided)
• Invoking custom kernels
• Advancing the fieldset time index appropriately

Currently, the time step size dt can be float, timedelta, or np.timedelta64 and can be positive (forward in time) or negative (backward in time)

The duration of the execution in "model-time" is set either by endtime or runtime (both cannot be provided at the same time). The endtime is the model time while runtime is the amount of time in the model that passes relative to the time at the start of ParticleSet.execute

File output frequency is controlled by outputdt. While there is a standard file output for the particle set, users can optionally write Field data to file.

Particle-particle interactions can be handled by setting the pyfunc_inter argument.

While I suspect we will want all of these features in v4 of Parcels, I want to get a discussion started around what features we want to start with so that we can incrementally incorporate some of the changes we've made for time management and evaluation in fields and fieldsets.

I would propose that we initially focus on (pre v4alpha)

• shifting all time data to be timedelta or datetime objects
• invoking existing advection and time stepping kernels
• managing file IO for particle sets

This would cover a good deal of use cases and would allow us to quickly get some demonstrations together for using the new UxGrid and XGrid classes with Field and FieldSet.

Is the list of features above complete ? Are there other items we want to consider for v4alpha ?

[erikvansebille]

Thanks for this write-up, @fluidnumerics-joe. I fully agree with this plan. A few small comments

• Since pset.execute does not control which kernels are advected (that's done in kernel.py, I don't think we need to limit ourselves to advection kernels only. It would be useful to also test e.g. interpolation kernels. But I do agree that we should leave InteractionKernels out of scope for now
• The support for PostIteractionCallbacks can also be postponed for now (and perhaps even be dropped altogether)
• We can also wait with implementing the field.write() support; that will be very different now that we have array under the hood anyways, so will have to be reconsider altogether
• We can also simplify by supporting only one of runtime or endtime. I'd vote for only endtime for now
• Not sure if it simplifies much, but we could also temporarily drop support for verbose progress-bar?

Does that make sense?

[fluidnumerics-joe]

I think this all makes sense. I'll start on a first cut with

• shifting all time data to be timedelta or datetime objects
• invoking arbitrary kernels using existing kernels.py framework (with tests focusing on various combinations of time integrators and interpolation kernels)
• no particle interaction kernels (will open an issue for this when a PR for this issue is created)
• managing file IO for particle sets
• using only endtime (type matches the time dimension type from the FieldSet)

[VeckoTheGecko]

• no particle interaction kernels (will open an issue for this when a PR for this issue is created)

Re-iterating something mentioned by Ryan Abernathey at Scipy 2019, I think it would be good for us to support working with Parcels in the cloud (I think we can really benefit from supporting researchers to horizontally scale out). I think its important to carefully consider how particle-particle interaction is implemented in v4 - so I would prefer that this is discussed first in an issue and when other parts of v4 is more fleshed out.

[fluidnumerics-joe]

what is the connection between particle-particle interaction kernels and working in the cloud ?

[VeckoTheGecko]

That if we have simulations with no particle particle interaction, its quite parallelizable allowing us to scale up horizontally by deploying more instances and partitioning the particleset across the instances.

[erikvansebille]

It's absolutely true that particle interaction hampers parallelisation; but parallelisation itself is a bit more complex than simple spatial partitioning. Because we don't know where particles will go to, the partitioning efficacy will decrease in time, requiring (expensive) rebalancing. This is a big topic that I've been thinking about (but not yet solving) for years...