Some PyCBC Offline jobs are still using too much memory

[spxiwh]

There was some work in/in advance of O4 to reduce the memory burden of plotting jobs when reading in the [HUGE] HDF_TRIGGER_MERGE files (often done via the coincident files).

However, there are still a small number of jobs that are using a large amount of memory (the following example probably is from a poorly tuned workflow, but you get the idea):

16103666 H    00:30 110     ┣━plot_singles_ID51_ID0017400 [plot_sing..017400.sh]
 ┗━MemoryUsage 60923 MB > RequestMemory 30000 MB.
16103668 H    00:43 110     ┣━plot_hist_ID58_ID0017406 [plot_hist..017406.sh]  
 ┗━MemoryUsage 60923 MB > RequestMemory 8000 MB.
16103670 H    00:30 110     ┣━plot_singles_ID51_ID0017401 [plot_sing..017401.sh]
 ┗━MemoryUsage 60925 MB > RequestMemory 30000 MB.
16103671 H    00:31 110     ┣━plot_snrchi_ID49_ID0017388 [plot_snrc..017388.sh]
 ┗━MemoryUsage 85791 MB > RequestMemory 8000 MB.
16103638 H    00:39  10     ┃  ┣━merge_page_coi..o_ID0_PID1_ID1 [merge_pag..D1_ID1.sh: 5]
 ┗━MemoryUsage 16704 MB > RequestMemory 1980 MB.

So in particular plot_singles, plot_hist, plot_snrchi and page_coi... (whatever that one is!) should be looked at to see if they can have memory footprint reduced. Assigning to Gareth, but others might want to take this on!

[tdent]

Can this be closed?

[spxiwh]

Many of these are still using more memory than they should/could ... Just not enough that it's impossible to run them

[tdent]

OK, can you check if the list of codes involved should be shortened

[GarethCabournDavies]

I think we can close this now, but will let @spxiwh be the one to push the button if agreed

[GarethCabournDavies]

I think this can be closed

To close this out, the PRs I could find to solve this issue in the mentioned files are: plot_singles(#4545), plot_hist (#4545), plot_snrchi (#4729) and page_coinc_snrchi (#4764)