Replays regularly show wrong accuracy & judgement counts

[120-cell]

Type

Game behaviour

Bug description

The accuracy and other stats shown at the end of replays do not match the ones I got when setting the score (which are correctly displayed online). The replay acc is usually too high and stays constant upon rewatching the replay. This is not a rare occurrence, it happens with almost every replay that I look at. See my top plays below for example.

Screenshots or videos

should be 97.57%, 121x 100, 3x 50
should be 94.82%, 68x 100, 1x 50
should be 97.08%, 26x 100, 4x 50
this one is correct
should be 96.65%, 27x 100

Version

2024.625.2

Logs

compressed-logs.zip

[smoogipoo]

Sticking to just the first one, do you have the replay for that one in your local leaderboard with the correct stats? If so can you provide that?

[120-cell]

The screenshots are from my local replays, meaning the local replay is incorrect. Here's the file
120-cell playing -45 - Crimsonic dimension (Sanch-KK) [Phantasm] (2024-05-20_22-29).zip
I also checked the online replay and it seems to be the same as the local one.

[ssz7-ch2]

duplicate of #11311

[smoogipoo]

@ssz7-ch2 Have you actually tested to make sure that this is that issue, or are you just making a similarity inference?

[ssz7-ch2]

@ssz7-ch2 Have you actually tested to make sure that this is that issue, or are you just making a similarity inference?

I should've added "most likely" before it as I am making a similarity inference. But I'm not sure how it would be possible to test if these specific replay discrepancies are caused by that issue. Since won't you need the unrounded timestamps for the clicks? which I don't think is possible to retrieve anymore.

Afaik that is the only issue that causes differences in 300s/100s/50s for non-classic std plays, so I think it's very likely that that's the cause of the discrepancies seen here.

Edit: looked through the Ki ga Kurutta Noise score and the missing 50 is very likely caused by rounded replay frames
There is a click at 92195 which is 68ms late for the circle at 92127. The hitwindow100 for this map also happens to be ±68ms.

[Wieku]

I was doing lazer replay playback tests in danser and I got wrong results. I've decided to double check in lazer and noticed a discrepancy here as well:
local score:

1269x great, 119x ok

after replay playback:

1261x great, 127x ok

Replay Wiek playing KASAI HARCORES - Cycle Hit (Worminators) [Home Run] (2024-09-22_01-16).zip

I've added an artificial 1ms to great hit window in danser and I got 1276x great 112x ok so it seems to be replay frame rounding.

[smoogipoo]

As a bit of an update, peppy and I briefly discussed tending to this issue as a somewhat priority-1 urgency.

The gist of it is that we could break the replay format and store frame times as doubles. The idea being that lazer replays don't need to be compatible with stable - that's something we did while assuming stable would eventually show lazer scores in the leaderboard but that hasn't panned out in practice.

However, there are also other paths that we could go down. For example, we could keep the existing frame signature and instead append the HitResult that occurred to the file, and fake it during gameplay. This could improve more than just replay accuracy - we would also be able to have instant accurate seeks and likely simplify the server-side structures that currently serialise internal scoring values to achieve the same. (In fact I wonder why we need the rest of the data in FrameHeader and can't just use the Statistics...).

We'd need to consider that it probably won't work well if we change gameplay and a result is no longer valid for the hit object type, but I'm not sure what the plans are if that changes anyway - invalidate scores?

At the end of the day, this needs some experimentation/an MVP implementation, but tl;dr we're not against breaking backwards compatibility if required.

[felipemarins]

Thank you for looking into this issue. I only have a few questions about your comment. I hope I'm not misunderstanding it, but wouldn't "instant accurate seek" break the current method of opening a replay and skipping it to the end to see the extended score statistics at the result screen? As I understand it, the replay player wouldn't process the actual hits between the current frame and the final one, so there wouldn't be a way to get the stats. I get that this isn't an optimal way either, but I guess it's something to consider.

Also, as much as I think it would be nice if lazer replays could work on stable, wouldn't it be incompatible with lazer-only mods or mod settings (such as rate adjust) anyway?

[smoogipoo]

I consider that process a hack that should be a background operation. Related: the PP community has asked about storing more info in scores that could be used in PP calc, and that could be fulfilled by replays...

[tsunyoku]

I think I'd want to be careful with involving PP in this. I think the general direction of including the timed statistics in replays is good, I'm not commenting on that.

However, I have concerns that if we start encouraging changes to PP that use those timed statistics we may result in an extremely unbalanced system as we'd have to consider and separately handle scores that don't have access to this data (for example, there are many legacy scores with no replays so there is absolutely zero chance that we could have this data for those) and I don't imagine this ever working well in practice - it'd be unbalanced, and also a very confusing experience.

This is before even thinking about any possible concerns with the individual ideas that people may have that would involve these extra statistics... I'm not saying that the committee have a hard "no" stance to using these in PP but at a glance I don't see it working so I'm wary of encouraging it. This is just my personal opinion, I'm not sure how the other committee members feel about it and it's probably also not very relevant at this stage - I just see PP mentioned and feel the need to leave my opinion.

[stanriders]

I'd rather have more timed info in the scores themselves so that it can be clearly separated into the "has new data" and "has not" categories that would be more obvious for everyone. Using replays would muddy everything very hard very quickly to the point that people might have different pp on identical scores - for example if one player set it when the map got ranked, and other set it 5 years later.
I think the only way replays could be used in PP if all replays were preserved, not necessarily from the beginning of the game. I'm definitely in favor of extending score data by any means though, even without populating all legacy scores. My only concern is consistency

[ssz7-ch2]

I think storing frame times as doubles should be done first. Then other methods of storing replay data can be considered later on.

Since currently that info is permanently lost, so replays that are inaccurate today will be inaccurate forever.

[Givikap120]

I think storing frame times as doubles should be done first. Then other methods of storing replay data can be considered later on.

Since currently that info is permanently lost, so replays that are inaccurate today will be inaccurate forever.

It's already stored as doubles

/// <summary>
/// The time at which this <see cref="ReplayFrame"/> takes place.
/// </summary>
[Key(0)]
public double Time;

EDIT: even tho Time is stored as doubles - it's truncated immediately after setting the replay. Meaning that all replays are permanently damaged.
This issue should be resolved ASAP.