Add MPI_gaththerV as an intermediate step between LS_root and LS_distributed(bk=cs_grid)

[metdyn]

Description

Trajectory sampler: add locstream_chunk and MPI_gaththerV as an intermediate step between ls_rt and ls_distributed(bk=cs_grid) to save computational time, coauthored with @bena-nasa and @tclune.

Regrid C5760_L2 to 6-hr IODA locstream points (29 platform) takes 68.059 s (HISTORY) with 1200 cores.

A related issue is the timing for MPI_gatherV from LS_distributed (equal number of points) to LS_root using either LEVEL_BY_LEVEL or ALL_IN_ONCE:

Title:  time(history)/s  from  Extdatadriver.x
 
X_Z_M          T/s(3d gatherV)	   T/s(2d gatherV  level by level)
------------------------------------------------
c1440_L137_M1260  69.870     57.276
c1440_L137_M5040  95.762
c5760_L137_M8820  93.140     98.494
 
M=cores on discover
------------------------------------------------
Nobs=1543929

Related Issue

Motivation and Context

How Has This Been Tested?

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Trivial change (affects only documentation or cleanup)

Checklist:

• I have tested this change with a run of GEOSgcm (if non-trivial)
• I have added one of the required labels (0 diff, 0 diff trivial, 0 diff structural, non 0-diff)
• I have updated the CHANGELOG.md accordingly following the style of Keep a Changelog

[tclune]

See inline comments.

[metdyn]

working offline for improvement

[tclune]

See inline commentns.

[metdyn]

See inline commentns.

I think the comment has been resolved (code was revised.)

The old code was revised and I cannot find the corresponding lines any more