On really big files (60MB) the browser stalls

[breznak]

As @jefffohl mentioned in #27 , the functionality for parsing large files is much improved by that PR, but for a really big file (60MB+) the browser gets on the knees.

To improve this, we could

• use subsampling, similar to Preprocess/Compress when too much data, so the presentation is clear #43
• have a MAX_SIZE field and if the graph is larger than that
  • switch to the "sliding window" mode

[jefffohl]

I am not sure which option is the best. I am leaning towards the windowing option. We can post an alert stating that the data is being windowed, and allow the user to turn that off, and load the entire file if they really want to try that - knowing that it might crash the browser.

What option do you think users would prefer?

[breznak]

@jefffohl I don't know either. The widowing option will be used anyway for the streaming data, so we can start with that and enable #43 later?

[jefffohl]

@breznak - we can automatically use windowing on files over a certain size. I am going to arbitrarily set this to 5MB.

[breznak]

Fixed with #64 , large files use 5MB size window