Ensemble Algebra Call 06 Dec 2016

MSM usecase

On the way to O(1000) ensembles.

New set of data:

• 100k timesteps, 50 macrostates, 100 ensembles, dt=0.00002: https://github.com/radical-collaboration/adaptive-msm/tree/master/entk/Data/iters1_100ksteps_50macros_100ensemble/plots
• 100k timesteps, 50 macrostates, 100 ensembles, dt=0.0001: https://github.com/radical-collaboration/adaptive-msm/tree/master/entk/Data/iters_1_100k_timesteps_50_macros/plots

EE usecase

New set of data:

All data is available on Stampede at /work/02734/vivek91/expanded_ensemble_data.

• "fixed_80k_3iters_attempt_1" - 80K timesteps per simulation, 3 iterations
• "fixed_80k_5iters_attempt_1" - 80K timesteps per simulation, 5 iterations
• "fixed_80k_10iters_attempt_1" - 80K timesteps per simulation, 10 iterations
• "fixed_100k_5iters_attempt_1" - 100K timesteps per simulation, 5 iterations
• "fixed_100k_10iters_attempt_1" - 100K timesteps per simulation, 10 iterations

Currently, the data of interest (kept at the above location) is about ~350GB. Requested additional space (500-1000GB) on Jetstream to house this data.

Earlier error recreated, files logged and stored.

• Failed task: /work/02734/vivek91/radical.pilot.sandbox/rp.session.js-168-20.jetstream-cloud.org.vivek91.017139.0001-pilot.0000/unit.000471
• Data from previous iterations: /work/02734/vivek91/expanded_ensemble_data/failed_attempt

Paper

• Added pseudographs to the paper. Specifically: graph investigating parameter and resource adaptivity, strong and weak scalability graphs, convergence as a function of walltime
• Tabularized adaptivity

Todos

• Debug error in EE.
• Keep going bigger with MSM experiments.
• Investigate how to design and implement a) resource adaptivity experiments and b) convergence vs walltime experiments
• Reiterate on adaptivity and experiment section, rest might be more straightforward to write.

Meeting notes

• Will be debugging error in EE usecase with Michael
• Look into the MSM papers to check whether the results we got are expected