Media slow to load in public view (or doesn't load at all...)

[AJLinn]

I shared a saved search with a researcher: https://arctos.database.museum/saved/Loan202107EH
Normally I also share instructions on how to show the media thumbnail in the search results by adding the media column, saving and refreshing.
From the public interface (not logged in) I'm getting either a super slow load rate or a total hang, depending on what browser I use. Is this a local problem or something bigger? There are only 47 items in this saved search and while some items do have several images per record, it doesn't seem like it should create a log-jam like I'm experiencing.

I've tried multiple browsers and computers with varying amounts of RAM, processor speed, etc.
I'm using Brave as my main browser these days (love it) and have had zero issues thus far. The hang has occurred using Safari, Firefox, and Brave. It has also occurred using an old iMac (2012 with 8 GB RAM and slow processor) and my new MBP (2019, 2.3 GHz 8-core i9 with 16 GB ram). It loads pretty much instantly (~1 sec) without media selected (public or logged in).

Here's the process I've followed:

1. Log in, enter https://arctos.database.museum/saved/Loan202107EH
2. See media (my login always has media showing at search results).
3. Log out
4. Load saved search: https://arctos.database.museum/saved/Loan202107EH
5. See 47 records
6. Click Tools / Add column / Media / save & refresh
7. Hang...

8. Pop up window - Page Unresponsive

9. Click wait & hang... 10. Continue to wait... indefinitely 11. Become frustrated and close tab. Realize there's no way out because now that i've added the media column my browser cookies are telling it to keep using the media column. 12. Try a different browser (Firefox), which then spends lots of time loading the images.

Not sure what to tell my users who might not know how to get out of the loop or have the fast internet or newish computer or multiple browsers, etc.
Recommend not showing media at search results? That is a major sad face for cultural collection user who are very visually-oriented.
Recommending show fewer results at one time (I have 100 rows showing... maybe only do 25?) BEFORE adding the media column? (Still loads images slowly)
Why does it load faster for an operator vs a public interface? Something to do with the cache?

Sorry for the long question/many questions. Just hoping to make my Indigenous and remote community users understand how to use a system they might not be familiar with.
Thanks for any tips for figuring out this problema.

Priority: Medium (inconvenient mostly)

[dustymc]

There might be something interesting in your javascript console, but my first guess is your network. Here's what I get:

The page loads for me (with JSON/text in the Media column) in 1.63 seconds, then it takes about 12 more seconds to grab the thumbs and patch them in (which is faster than I scroll, so it more or less looks like the thumbs have always been there).

Several of your thumbs are pretty huge, which isn't helping anything.

#2863 would be handy about now....

[dustymc]

I think this is also related to #3155 (comment) - the API call that gets thumbs is slow (on a VPN with certain browsers, if I hold my tongue JUST right, sometimes). IIIF would help with that too - I currently have to check thumbs to avoid other flavors of evil, if I had real media tools I'd KNOW I had good thumbs and could just serve them.

Arctos is also getting hammered on by some jerk right now, but I can't see evidence that they're successfully slowing anything down.

[AJLinn]

There might be something interesting in your javascript console,

Is there something I can share so you can see it?

but my first guess is your network.

This happened both at the museum and at home (non-VPN connection). Tested again this morning and am getting the same results - total hang in Brave, very slow in Firefox.

Strangely, I also noticed in Brave when I add the column, save and refresh, it doesn't add the column when the results re-load. I have to do it a second time in order for the new column to get added. Same thing with removing... must be something to do with the Brave browser settings?

[dustymc]

something

Google "{my browser} javascript console" or something of the sort, but see below.

IDK much about Brave, but that sounds like overly-aggressive caching.

I've been watching the logs and running around with my dev tools on, and I now think this is mostly #3155-like behavior - it sometimes (there probably is some related SOMETHING, SOMEWHERE, but I can't find it) takes a ridiculously long time to establish a session, and API calls (in this case to turn the JSON into thumbnails and links and such) each require a session. I can't replicate on test, and I can't see any evidence of this ever happening on test - I'm not confident I'm going to find an actual solution without https://github.com/ArctosDB/internal/issues/64 (or perhaps something like #2863, which might let me skip those calls altogether).

I'm not sure where to go from here. I might find some workaround, but I hate to introduce more complexity in an attempt to mitigate limited resources - that can too easily backfire. Lacking better ideas, I'll keep staring at it and hoping something comes to me....

[AJLinn]

Are these helpful? If not, let's just close it and I'll warn folks they might have issues. This is a major bummer, connected to what I see as an essential delivery service for users of cultural collections (i.e., seeing thumbnails in the search results view). And just having access to the Media hyperlink in that screen is not the same for our people.

[dustymc]

It's probably the network tab that has what I need.

I agree with major bummer, hopefully we've got a fix in the works via @mkoo

Whatever the immediate solution (or lack thereof), please leave this open - maybe I'll find a way around what I think is happening if I stare at it long enough.

[AJLinn]

here's a few shots as its been attempting to load (I'm at the museum today):

And after it finally loaded all the images (sorry if it's too many or the wrong info!):

[dustymc]

Idea: set up a cfschedule to process thumbs in media_flat, use that in UIs

[dustymc]

Added calculated/verified thumb to media flat. Update is running in production, should take about a week to complete if the current rate turns out to be sustainable.

Need to rebuild all code to use the cache instead of getMediaPreview()

Check back in a few days, make sure this is going to work before proceeding.

For the list:

1. This behavior does not occur in test, which makes everything more difficult. https://github.com/ArctosDB/internal/issues/64 should fix that.
2. The lack of a synchronized test environment means running scheduled tasks is difficult, which also complicates everything (and provides a great opportunity to melt production - so far so good...). https://github.com/ArctosDB/internal/issues/65 should fix.

[dustymc]

Production is now running precalculated thumbnails and no service calls.

A few good thumbs missed, probably for the same reasons the original problem exists, need a way to flag those for reprocessing.

[dustymc]

In next release:

I'm reprocessing all misses, that should finish in a day or two.