Execution of scripts in parallel

[EvgeniiChaikin]

Execute scripts in parallel in order to greatly speed up the pipeline

1 CPU

Script Run Time Statistics: 

Script name                               Run time (s)        Fraction of total CPU time
------------------------------------------------------------------------
density_temperature.py                    11.1547             0.017135  
density_temperature_sf_fraction.py        10.7804             0.016560  
density_temperature_metals.py             12.6458             0.019425  
density_temperature_dust.py               13.4180             0.020611  
density_temperature_dust_to_metals.py     12.4652             0.019148  
density_temperature_dust2metal.py         13.6190             0.020920  
density_internal_energy.py                10.7415             0.016500  
density_pressure.py                       10.8351             0.016644  
star_formation_history.py                 25.6979             0.039474  
stellar_mass_evolution.py                 11.0942             0.017042  
HI_mass_evolution.py                      10.6592             0.016373  
H2_mass_evolution.py                      13.1257             0.020162  
metallicity_distribution.py               10.2261             0.015708  
birth_density_distribution.py             10.1476             0.015588  
birth_density_metallicity.py              9.8525              0.015134  
birth_density_redshift.py                 10.2730             0.015780  
birth_metallicity_redshift.py             9.7195              0.014930  
last_SNII_density_distribution.py         10.4075             0.015987  
last_AGN_density_distribution.py          9.2032              0.014137  
gas_metallicity_evolution.py              10.4640             0.016074  
star_metallicity_evolution.py             10.1578             0.015603  
bh_metallicity_evolution.py               10.4073             0.015987  
bh_accretion_evolution.py                 10.7051             0.016444  
bh_mass_evolution.py                      10.4581             0.016065  
bh_masses.py                              9.8481              0.015128  
gas_masses.py                             10.2573             0.015756  
gas_smoothing_lengths.py                  10.0745             0.015475  
number_of_agn_thermal_injections.py       10.1617             0.015609  
max_temperatures.py                       10.2659             0.015769  
max_temperature_redshift.py               10.1438             0.015582  
particle_updates_step_cost.py             28.5435             0.043845  
wallclock_simulation_time.py              30.1190             0.046265  
wallclock_number_of_steps.py              28.9964             0.044541  
simulation_time_number_of_steps.py        30.2862             0.046522  
wallclock_timebin_hist.py                 28.0391             0.043071  
density_temperature_species.py            13.6845             0.021021  
density_species_interp.py                 133.7056            0.205383  
last_SNII_kick_velocity_distribution.py   9.7282              0.014943  
max_SNII_kick_velocities.py               8.8930              0.013660  
Wallclock time                            678.0040            1.041472  
Total CPU time                            651.0055            1.000000 

12 CPUs

Script Run Time Statistics: 

Script name                               Run time (s)        Fraction of total CPU time
------------------------------------------------------------------------
density_temperature.py                    12.9654             0.017813  
density_temperature_sf_fraction.py        13.5427             0.018606  
density_temperature_metals.py             13.5929             0.018675  
density_temperature_dust.py               16.4714             0.022630  
density_temperature_dust_to_metals.py     16.1675             0.022212  
density_temperature_dust2metal.py         15.6576             0.021512  
density_internal_energy.py                13.1701             0.018094  
density_pressure.py                       13.2331             0.018181  
star_formation_history.py                 31.3954             0.043133  
stellar_mass_evolution.py                 12.7456             0.017511  
HI_mass_evolution.py                      12.5751             0.017277  
H2_mass_evolution.py                      12.7520             0.017520  
metallicity_distribution.py               13.1944             0.018127  
birth_density_distribution.py             13.0812             0.017972  
birth_density_metallicity.py              12.1711             0.016721  
birth_density_redshift.py                 12.4436             0.017096  
birth_metallicity_redshift.py             12.2524             0.016833  
last_SNII_density_distribution.py         13.4711             0.018508  
last_AGN_density_distribution.py          11.2991             0.015524  
gas_metallicity_evolution.py              13.2264             0.018171  
star_metallicity_evolution.py             12.3857             0.017016  
bh_metallicity_evolution.py               12.0574             0.016565  
bh_accretion_evolution.py                 12.7389             0.017502  
bh_mass_evolution.py                      12.7086             0.017460  
bh_masses.py                              12.5016             0.017176  
gas_masses.py                             11.8704             0.016308  
gas_smoothing_lengths.py                  12.5460             0.017237  
number_of_agn_thermal_injections.py       11.7799             0.016184  
max_temperatures.py                       12.7415             0.017505  
max_temperature_redshift.py               12.5619             0.017258  
particle_updates_step_cost.py             33.1130             0.045493  
wallclock_simulation_time.py              31.3771             0.043108  
wallclock_number_of_steps.py              29.5474             0.040594  
simulation_time_number_of_steps.py        31.2170             0.042888  
wallclock_timebin_hist.py                 30.9743             0.042555  
density_temperature_species.py            14.9127             0.020488  
density_species_interp.py                 128.3961            0.176400  
last_SNII_kick_velocity_distribution.py   9.5681              0.013145  
max_SNII_kick_velocities.py               9.4641              0.013002  
Wallclock time                            166.2901            0.228461  
Total CPU time                            727.8699            1.000000  

In this example, the increase in the wall clock time is a factor of 4, which is NOW limited by the runtime of a single script density_species_interp.py

[EvgeniiChaikin]

Any comments are welcome!

[JBorrow]

Looks great! You should put the most expensive script first in the list. That way it will begin being executed ASAP and reduce your run-time even more.