Filtering by mod requirements does not work for beat maps that have never been selected or played.

[Jake-Thomas-Hall]

Latest Log File:
_latest.log

Description:
Having just updated to 1.11.0 (which required a re-install in my case), I've noticed some odd behaviour from this mod. That is, when playing before the update I could filter by "Mod Requirements" under other filters and it would display either maps that require map extensions, noodle extensions etc.

However, now that I've copied all the songs back into the CustomLevels folder after the reinstall, when performing these filter options, it does not display any of these maps that have not yet been selected/played. Once a song as been selected in the list (and/or played), it shows up in the search results as expected.

This was not occurring before, because (I'm assuming) all of the maps had been played at some point, meaning they would therefore show up in the results as expected. Looking at the logs, this line in particular seems to potentially be an indicator of what is going wrong?

[DEBUG @ 21:51:20 | EnhancedSearchAndFilters] Applying filters without a provided list of BeatmapDetails objects. Filtered levels may not be correct if some custom levels are uncached

Steps To Reproduce

1. Install fresh copy of game, and install basic set of mods including this one and Song browser (although I doubt these other mods are needed to reproduce?).
2. Copy a collection of maps into the CustomLevels folder, including some that require map extensions, noodle extensions etc.
3. Start the game, but do not select or play the maps, instead try to filter for them using Other filters > Mod Requirements
4. If bug is being reproduced, the search results should be empty.
5. Select one or more of the maps that has a mod requirement to preview them, or play them. Then perform the search.
6. Now when performing the same filter as before, the songs will show in the results.

Expected Behaviour:
Even if the songs have not been selected or played before, they should still be able to be filtered - work around at the moment is to go down the list and select every single song at least once so that they are cached... which is kind of an inconvenience when you have hundreds of songs.

[chrislee0419]

Hey Diedinium,

Thanks for sending in a bug report!

I haven't been able to reproduce the issue on my end (it appears to be working fine on my machine).
Could you send me the _latest.log file from the root Logs folder, rather than the one specific to this mod? I want to check out if there are any errors in other mods (most notably, SongCore) that might affect the filter.

Also, about that specific line from the log, that shouldn't be an issue in this case. ESAF does its own caching of song data whenever the game is started and stores those in a file for much faster access. You can actually see it retrieved the premade cache file near the beginning of the logs:

[INFO @ 21:48:41 | EnhancedSearchAndFilters] Retrieved 805 cached beatmap details from file
[INFO @ 21:48:41 | EnhancedSearchAndFilters] Finished caching the details of 808 beatmaps (took 0.047 seconds)

ESAF's cache actually shouldn't even be relevant to this issue. Rather, I think SongCore's ExtraSongData cache can be relevant. Which reminds me, maybe it would also be helpful if you sent that file as well. It's located at:

C:\Users\you\AppData\LocalLow\Hyperbolic Magnetism\Beat Saber\SongCoreExtraData.dat

Also, I wanted to add that I don't actually expect the issue to be with SongCore. Typically, the files in AppData aren't even deleted in a fresh install. so it should still be populated. But, it would at least help confirm to me that there's some issue with the retrieval of that data on my end.

[Jake-Thomas-Hall]

Hey Chris, thank you for the prompt response - I manually deleted the AppData folder for beat saber before reinstalling the game as that was part of what was causing me issues (RIP my scores lol). Could that in some way be causing this issue? Maybe to try and reproduce the issue, delete the AppData folder as well before reinstalling and see what happens.

Here are the two files you requested.

_latest.log
SongCoreExtraData.zip

Maybe I should just try another reinstall of the game (this time without deleting the appdata folder), or just reinstall the mods and see if that makes any difference?

[chrislee0419]

Seeing as how ESAF detects that you have a library of over 800 songs and the SongCoreExtraData.dat cache file only has about ~100 entries, it looks like the reason why there is a problem is that ESAF had no data to access. So yeah, it looks like the SongCoreExtraData.dat cache file is only added to once the song has been selected once.

A possible solution could be to force SongCore to load and store the ExtraSongData when ESAF is caching. It would only need to occur once, but would also make the first ESAF load extremely slow (and it was already pretty slow before as well), especially since the ESAF loading is done on another thread and the SongCore stuff probably isn't thread-safe so I would need to send tasks back to the main thread.

Another potential solution is to load and store it in my own cache, but I'd prefer not having to redo work that another mod (and in this case, a prerequisite mod) already does.

I'll try and think up some better solutions, but for now, you would have to select each song manually to have SongCore store that extra info (big yikes, but there's no other way around it).

[Jake-Thomas-Hall]

Well, sounds like you might have your work cut out to find a solution to this odd edge case... maybe it should be SongCore itself that gets changed, rather than your mod - it would make sense that it should add all current existing songs to the SongCoreExtraData.dat cache file when the game loads, without a mod forcing it to do so?

In the meantime though, I can live with the workaround... looks like I have some find times ahead, selecting all of my 800 songs 😅

Anyway, Thank you for your prompt replies, and being willing to look into this.

[Jake-Thomas-Hall]

I'll try and think up some better solutions, but for now, you would have to select each song manually to have SongCore store that extra info (big yikes, but there's no other way around it).

By the way, have talked to some of the modders on the BSMG discord, and this is what they had to say about it:

[chrislee0419]

I talked to Kyle about it in the discord channel, and will probably just go ahead with the "force SongCore to cache when ESAF is caching" route, since it's a simple (and fairly easy) option.