Visibility of "TTree draw XXX%" progress bar

[grzanka]

JSROOT with its partial read from HTTP (see #284) gives a fantastic way to analyse large datafiles in the browser.

There is just one thing which makes it cumbersome for the first-time users of JSROOT. Some of my collegues were confused and unsure what happens when large file is being loaded via URL. The "TTree draw XXX%" progress bar is not easy to spot at first glance (even on a regular laptop screen size, not to mention large external display), take a look at following screenshot:

I am trying to use following URL:

https://root.cern/js/latest/?file=https://s3p.cloud.cyfronet.pl/datarawlv2v3/20231205m3.flat.root&item=channel_0/negative&opt=total_segment_no:peak_index::peak_stop-peak_start%3E2;accum:3000000;hbins:300;drawopt:logz,colz,gridxy,colpal55

which tries to open part of ~470 MB ntuple.

I would expect that in case of loading file from URL the user will be informed that "Loading happens". This might be a popup or modal with clear message in the center of the screen. Also - the progress itself might be better visible if located in some more exposed place. Would it also be possible to display amount of downloaded data (like 1.3 MB / 30 MB) in addition to a percentage ?

[linev]

Yes, log message is very small and one should know where to search for it.
It is intentionally done very small - while JSROOT can be just small part of more complex HTML page
and should not dominate over other HTML elements. But optionally I can provide more prominent elements.

Would it also be possible to display amount of downloaded data (like 1.3 MB / 30 MB)

Currently percentage of processed events from the tree is shown.
I also could try to add size of already requested data.
But count total amount is more complicated - therefore I prefer to keep as it is.

There is one problem with http server you are using.
It does not support multi-part http request. Already in the beginning when reading ROOT file header only first part of the multi-part request is returned. Therefore JSROOT fallback to simple http requests - which is much much slower (~100 times). Therefore drawing evolves very slowly - for me it takes several minutes.

[grzanka]

Yes, log message is very small and one should know where to search for it. It is intentionally done very small - while JSROOT can be just small part of more complex HTML page and should not dominate over other HTML elements. But optionally I can provide more prominent elements.

Thanks ! Maybe there could be an user control on the style, like whether the progress is embedded or is a modal ?

Would it also be possible to display amount of downloaded data (like 1.3 MB / 30 MB)

Currently percentage of processed events from the tree is shown. I also could try to add size of already requested data. But count total amount is more complicated - therefore I prefer to keep as it is.

There is one problem with http server you are using. It does not support multi-part http request. Already in the beginning when reading ROOT file header only first part of the multi-part request is returned. Therefore JSROOT fallback to simple http requests - which is much much slower (~100 times). Therefore drawing evolves very slowly - for me it takes several minutes.

Thanks ! I will investigate that with our S3 filesystem team.

[linev]

I add in dev branch possibility to switch progress message display to modal mode.

Here example how it works with your file.

It is url parameter &progress=modal. It will be part of 7.6.0 release.

[grzanka]

Yes, log message is very small and one should know where to search for it. It is intentionally done very small - while JSROOT can be just small part of more complex HTML page and should not dominate over other HTML elements. But optionally I can provide more prominent elements.

Would it also be possible to display amount of downloaded data (like 1.3 MB / 30 MB)

Currently percentage of processed events from the tree is shown. I also could try to add size of already requested data. But count total amount is more complicated - therefore I prefer to keep as it is.

There is one problem with http server you are using. It does not support multi-part http request. Already in the beginning when reading ROOT file header only first part of the multi-part request is returned. Therefore JSROOT fallback to simple http requests - which is much much slower (~100 times). Therefore drawing evolves very slowly - for me it takes several minutes.

I discussed with our S3 team. Unfortunately we do not have support of multi-part http request. I was advised to decrease number of HTTP requests. This was possible due to tuning basket size in the ROOT file. In the example below:

https://root.cern/js/latest/?file=https://s3p.cloud.cyfronet.pl/datarawlv2v3/20231205m3.flat.root&item=channel_0/negative&opt=total_segment_no:peak_index::peak_stop-peak_start%3E2;accum:3000000;hbins:300;drawopt:logz,colz,gridxy,colpal55

which you've used before the basket size was very small, therefore there were many requests and loading was slow.

With increased basket size (to >1MB size) the plotting works much better, see for example:

https://jsroot.gsi.de/dev/?file=https://s3p.cloud.cyfronet.pl/datarawlv2v4/20231205m2.slim.root&item=channel_0/negative&opt=total_segment_no:peak_start_us::peak_length_ns%3E0.2;accum:3000000;hbins:300;drawopt:logz,colz,gridxy,colpal55&progress=modal

I've enabled modal progress and it's now clearly visible.

Unfortunately when trying to adjust the basket size / number of http requests I've ran into some issues with uproot, being discussed here: scikit-hep/uproot5#1130 (comment) It seems that ROOT doesn't handle nicely very large baskets.

Another advice was to use multiplexing in HTTP2 to reuse the same connection and have multiple streams of data (https://www.cloudflare.com/learning/performance/http2-vs-http1.1/), but I think its far from the topic discussed here.

Finally - I am happy with progress bar, we may close this issue. Will this feature be described as well in the documentation ?

[linev]

I add "&progress=modal" to list of supported options into JSROOT documentation

[linev]

Yes, increasing basket size helps to reduce number of HTTP requests.

It seems that ROOT doesn't handle nicely very large baskets.

To my knowledge, uproot uses own implementation for TTree I/O.
Does you have problem to create and read such tree in ROOT itself?

[grzanka]

Yes, increasing basket size helps to reduce number of HTTP requests.

Do you have some advice on optimal size of HTTP request ?

I am also curious what is the loading time of this case for you (I am pretty close to my computing center now):

https://jsroot.gsi.de/dev/?file=https://s3p.cloud.cyfronet.pl/datarawlv2v4/20231205m2.slim.root&item=channel_0/negative&opt=total_segment_no:peak_start_us::peak_length_ns%3E0.2;accum:3000000;hbins:300;drawopt:logz,colz,gridxy,colpal55&progress=modal

It seems that ROOT doesn't handle nicely very large baskets.
To my knowledge, uproot uses own implementation for TTree I/O. Does you have problem to create and read such tree in ROOT itself?

I haven't tried ROOT itself. uproot seemed the easiest option, as I am simply converting HDF file into ntuple.

[grzanka]

I add "&progress=modal" to list of supported options into JSROOT documentation

Thanks ! BTW, is there a typo in paltte ?

[linev]

I am also curious what is the loading time of this case for you

From Darmstadt/Germany it takes about 7 seconds

[linev]

Feature with modal progress bar released with https://github.com/root-project/jsroot/releases/tag/7.6.0