When import beatmaps downloaded from osu! website, the game marked the beatmap is not ranked, but beatmap downloaded from osu!direct is ranked #31496

NammIsADev · 2025-01-13T11:08:28Z

Type

Game behaviour

Bug description

When I open osu!lazer on my Debian 12 and try to import a beatmap from the osu! website I downloaded, the beatmap is marked as not ranked. However, when I download the SAME beatmap from osu!direct, it is marked as ranked.

Screenshots or videos

Version

2025.101.0

Logs

logs.tar.gz

bdach · 2025-01-13T12:46:00Z

As far as I can see, you tried to import these maps either from stable or in a large batch:

2025-01-07 06:38:58 [verbose]: 📺 ScreenStack#378(depth:4) loading ScreenImportFromStable#318
2025-01-07 06:38:58 [verbose]: 📺 ScreenStack#378(depth:4) entered ScreenImportFromStable#318

Additionally, your local online metadata cache is corrupted:

2025-01-07 06:39:07 [verbose]: [1580c] [LocalCachedBeatmapMetadataSource] Cached local retrieval for Kuba Oms - My Love (W h i t e) [Hard] failed with Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 11: 'database disk image is malformed'.
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements()+MoveNext()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.GetStatements()+MoveNext()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
2025-01-07 06:39:07 [verbose]: at osu.Game.Beatmaps.LocalCachedBeatmapMetadataSource.getCacheVersion(SqliteConnection connection)

Normally this would lead to falling back to an API request, but it looks like you were also not logged in at the time:

2025-01-07 06:40:17 [verbose]: Request to https://osu.ppy.sh/api/v2/users/helinatri/?key=username failed with System.Net.WebException: Unauthorized.
2025-01-07 06:40:17 [verbose]: Response was: {"authentication":"basic"}

so the online fallback could not be used and ultimately the status of the beatmap could not be determined.

As a short term fix please delete online.db from your game files and try again.

On our side there are probably two things that can be done:

sqlite corruptions should probably lead to the offline metadata cache getting purged
metadata lookup API requests should maybe be opened to anonymous users

@peppy opinions?

NammIsADev · 2025-01-13T13:42:20Z

As far as I can see, you tried to import these maps either from stable or in a large batch:
2025-01-07 06:38:58 [verbose]: 📺 ScreenStack#378(depth:4) loading ScreenImportFromStable#318
2025-01-07 06:38:58 [verbose]: 📺 ScreenStack#378(depth:4) entered ScreenImportFromStable#318
Additionally, your local online metadata cache is corrupted:
2025-01-07 06:39:07 [verbose]: [1580c] [LocalCachedBeatmapMetadataSource] Cached local retrieval for Kuba Oms - My Love (W h i t e) [Hard] failed with Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 11: 'database disk image is malformed'.
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements()+MoveNext()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.GetStatements()+MoveNext()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
2025-01-07 06:39:07 [verbose]: at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
2025-01-07 06:39:07 [verbose]: at osu.Game.Beatmaps.LocalCachedBeatmapMetadataSource.getCacheVersion(SqliteConnection connection)
Normally this would lead to falling back to an API request, but it looks like you were also not logged in at the time:
2025-01-07 06:40:17 [verbose]: Request to https://osu.ppy.sh/api/v2/users/helinatri/?key=username failed with System.Net.WebException: Unauthorized.
2025-01-07 06:40:17 [verbose]: Response was: {"authentication":"basic"}
so the online fallback could not be used and ultimately the status of the beatmap could not be determined.

As a short term fix please delete online.db from your game files and try again.

On our side there are probably two things that can be done:

sqlite corruptions should probably lead to the offline metadata cache getting purged

metadata lookup API requests should maybe be opened to anonymous users

@peppy opinions?

I'm not at home at the moment, when I get home, I'll try to delete online.db on the Stable and Lazer side, I'll also try to delete all beatmaps + user data on the Lazer side. After clean all the data in Lazer (Debian), I will start the setup wirazd to import map from Stable again.

bdach · 2025-01-13T13:44:30Z

I'll try to delete online.db on the stabilization and lazer side

There is no need to touch stable. Please do not.

NammIsADev · 2025-01-13T13:44:52Z

I'll try to delete online.db on the stabilization and lazer side

There is no need to touch stable. Please do not.

Okay okay.

NammIsADev · 2025-01-14T04:35:13Z

Normally this would lead to falling back to an API request, but it looks like you were also not logged in at the time:
2025-01-07 06:40:17 [verbose]: Request to https://osu.ppy.sh/api/v2/users/helinatri/?key=username failed with System.Net.WebException: Unauthorized.
2025-01-07 06:40:17 [verbose]: Response was: {"authentication":"basic"}
so the online fallback could not be used and ultimately the status of the beatmap could not be determined.

I understand your point. Normally, the API will continue sending data to update the beatmap status even if the user is not logged in, and the status would be shown as ranked. However, in this case, it appears that the API request failed due to an authorization error, as indicated by the "Unauthorized" response.

After deleting the online.db, I attempted to log in again, which initiated the star ratings calculation process. However, I noticed that not all beatmaps have been set to ranked status. Specifically, some beatmaps older than 2012 remain unranked.

For example, "Baby Crushing Love" (mapped by peppy) is still unranked, even though the osu! website indicates it as ranked. This issue seems to affect multiple older beatmaps. (older than 2010)

Additionally, I encountered a new error: beatmaps downloaded from the setup wirazd are still set to unranked status.

Even reimporting from Stable did not resolve this issue.

I'm unsure what might be causing these weird issues?

Edit: here logs for debugging
compressed-logs.zip

peppy · 2025-01-14T06:03:17Z

sqlite corruptions should probably lead to the offline metadata cache getting purged
metadata lookup API requests should maybe be opened to anonymous users

both of these sound fair, yeah.

bdach · 2025-01-14T09:36:59Z

For example, "Baby Crushing Love" (mapped by peppy) is still unranked, even though the osu! website indicates it as ranked. This issue seems to affect multiple older beatmaps. (older than 2010)

Additionally, I encountered a new error: beatmaps downloaded from the setup wirazd are still set to unranked status.

Even reimporting from Stable did not resolve this issue.

Please remove the beatmaps, restart the game, and only then try to reimport the beatmaps in question again. It's not working for you because beatmaps are not actually deleted from the hard drive until you restart, and for optimisation purposes reimporting undeletes beatmaps if it can.

Addresses one of the points in ppy#31496. Not going to lie, this is mostly best-effort stuff (while the refetch is happening, metadata lookups using the local source *will* fail), but I see this as a marginal scenario anyways.

bdach · 2025-01-14T10:36:36Z

@ppy/team-web how feasible would it be to make the GET /beatmaps/lookup endpoint accessible anonymously / without authentication? I can probably try doing it if you think it's possible, but I figure I'd ask for your opinions before I faceplant into it...

nanaya · 2025-01-14T10:55:35Z

it's just adding it to the token requirement exception list in RequireScopes middleware

alternatively the client can request client credential token and do away with all the exception altogether (the oauth client will need adjustment for this)

NammIsADev · 2025-01-14T11:45:13Z

For example, "Baby Crushing Love" (mapped by peppy) is still unranked, even though the osu! website indicates it as ranked. This issue seems to affect multiple older beatmaps. (older than 2010)
Additionally, I encountered a new error: beatmaps downloaded from the setup wirazd are still set to unranked status.
Even reimporting from Stable did not resolve this issue.

Please remove the beatmaps, restart the game, and only then try to reimport the beatmaps in question again. It's not working for you because beatmaps are not actually deleted from the hard drive until you restart, and for optimisation purposes reimporting undeletes beatmaps if it can.

Fixed. But beatmaps downloaded from setup wirazd still marked as not ranked.

bdach · 2025-01-14T11:51:04Z

But beatmaps downloaded from setup wirazd still marked as not ranked.

Did you delete those, close the game, and reimport again? If yes attach logs again please.

NammIsADev · 2025-01-14T11:53:49Z

But beatmaps downloaded from setup wirazd still marked as not ranked.

Did you delete those, close the game, and reimport again? If yes attach logs again please.

Yes, even I run the setup wirazd to download beatmaps (testing). Now, I'm not home so... no logs! (sorry)

NammIsADev · 2025-01-14T12:38:06Z

@bdach

compressed-logs.zip

Grab logs remotely by using VNC + FTP

bdach · 2025-01-14T12:55:10Z

The latest logs show rather clearly that the beatmaps were not deleted from the game...

2025-01-14 11:56:14 [verbose]: [?????] Beginning import from tmpqnxIAP.osz...
2025-01-14 11:56:15 [verbose]: [1580c] Found existing beatmap for Kuba Oms - My Love (W h i t e) (ID de15a1c5-a2cf-4c8e-b152-316b6fb06415) – skipping import.
2025-01-14 11:56:15 [verbose]: [?????] Beginning import from tmpQgsuMx.osz...
2025-01-14 11:56:16 [verbose]: [672a6] Found existing beatmap for Soleily - Renatus (Deif) (ID 9ef4ddff-35fb-426c-bec7-ea3e37b0c842) – skipping import.
2025-01-14 11:56:17 [verbose]: [?????] Beginning import from tmp3UnY2F.osz...
2025-01-14 11:56:18 [verbose]: [31322] Found existing beatmap for LukHash - WHEN AN ANGEL DIES (Nelly) (ID 8538e8d2-d2d4-4c79-a67e-7629d00d6c79) – skipping import.
2025-01-14 11:56:18 [verbose]: [?????] Beginning import from tmpDV4sjt.osz...
2025-01-14 11:56:18 [verbose]: [c74a3] Found existing beatmap for Toni Leys - Dragon Valley (Toni Leys Remix feat. Esteban Bellucci) (LordRaika) (ID 91834c32-9aef-47f3-8153-1e45e74bf7ee) – skipping import.
2025-01-14 11:56:19 [verbose]: [?????] Beginning import from tmpoZfNDA.osz...
2025-01-14 11:56:20 [verbose]: [d2674] Found existing beatmap for cYsmix - triangles (yf_bmp) (ID c8932fde-6359-4c48-ae29-03d6e2997d90) – skipping import.
2025-01-14 11:56:20 [verbose]: [?????] Beginning import from tmp97cZsQ.osz...

These are the ones that don't have ranked status set, correct?

NammIsADev · 2025-01-14T13:11:00Z

The latest logs show rather clearly that the beatmaps were not deleted from the game...

2025-01-14 11:56:14 [verbose]: [?????] Beginning import from tmpqnxIAP.osz...
2025-01-14 11:56:15 [verbose]: [1580c] Found existing beatmap for Kuba Oms - My Love (W h i t e) (ID de15a1c5-a2cf-4c8e-b152-316b6fb06415) – skipping import.
2025-01-14 11:56:15 [verbose]: [?????] Beginning import from tmpQgsuMx.osz...
2025-01-14 11:56:16 [verbose]: [672a6] Found existing beatmap for Soleily - Renatus (Deif) (ID 9ef4ddff-35fb-426c-bec7-ea3e37b0c842) – skipping import.
2025-01-14 11:56:17 [verbose]: [?????] Beginning import from tmp3UnY2F.osz...
2025-01-14 11:56:18 [verbose]: [31322] Found existing beatmap for LukHash - WHEN AN ANGEL DIES (Nelly) (ID 8538e8d2-d2d4-4c79-a67e-7629d00d6c79) – skipping import.
2025-01-14 11:56:18 [verbose]: [?????] Beginning import from tmpDV4sjt.osz...
2025-01-14 11:56:18 [verbose]: [c74a3] Found existing beatmap for Toni Leys - Dragon Valley (Toni Leys Remix feat. Esteban Bellucci) (LordRaika) (ID 91834c32-9aef-47f3-8153-1e45e74bf7ee) – skipping import.
2025-01-14 11:56:19 [verbose]: [?????] Beginning import from tmpoZfNDA.osz...
2025-01-14 11:56:20 [verbose]: [d2674] Found existing beatmap for cYsmix - triangles (yf_bmp) (ID c8932fde-6359-4c48-ae29-03d6e2997d90) – skipping import.
2025-01-14 11:56:20 [verbose]: [?????] Beginning import from tmp97cZsQ.osz...

These are the ones that don't have ranked status set, correct?

Yes. But more than this.

NammIsADev · 2025-01-16T10:43:17Z

Now the issues is dead 💀 (waiting for fix)

bdach · 2025-01-16T10:56:23Z

I can't help you more than I already have. "but more than this" is not enough information. As far as I can tell you didn't actually delete the beatmaps in the way that I asked because otherwise I wouldn't see the log snippets I pasted in #31496 (comment).

NammIsADev · 2025-01-16T11:08:20Z

I can't help you more than I already have. "but more than this" is not enough information. As far as I can tell you didn't actually delete the beatmaps in the way that I asked because otherwise I wouldn't see the log snippets I pasted in #31496 (comment).

What? I deleted all the beatmaps via settings, and restart the game, open the setup wirazd, downloaded the beatmaps via setup wirazd and the beatmaps still marked as not ranked.

bdach · 2025-01-16T12:50:21Z

What? I deleted all the beatmaps via settings, and restart the game, open the setup wirazd, downloaded the beatmaps via setup wirazd and the beatmaps still marked as not ranked.

I cannot reproduce this, sorry. Not sure what else to tell you.

NammIsADev changed the title ~~Beatmaps downloaded from osu! website has not ranked, but downloaded from osu!direct is ranked~~ When import beatmaps downloaded from osu! website has not ranked, but downloaded from osu!direct is ranked Jan 13, 2025

NammIsADev changed the title ~~When import beatmaps downloaded from osu! website has not ranked, but downloaded from osu!direct is ranked~~ When import beatmaps downloaded from osu! website, the game marked the beatmap is not ranked, but beatmap downloaded from osu!direct is ranked Jan 13, 2025

bdach added type:online area:import dealing with importing of data from stable or file formats labels Jan 13, 2025

peppy added the priority:1 Very important. Feels bad without fix. Affects the majority of users. label Jan 14, 2025

bdach mentioned this issue Jan 14, 2025

Refetch local metadata cache if corruption is detected #31509

Merged

Uh oh!

When import beatmaps downloaded from osu! website, the game marked the beatmap is not ranked, but beatmap downloaded from osu!direct is ranked #31496

When import beatmaps downloaded from osu! website, the game marked the beatmap is not ranked, but beatmap downloaded from osu!direct is ranked #31496

Comments

NammIsADev commented Jan 13, 2025

Type

Bug description

Screenshots or videos

Version

Logs

bdach commented Jan 13, 2025

Uh oh!

NammIsADev commented Jan 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bdach commented Jan 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NammIsADev commented Jan 13, 2025

Uh oh!

NammIsADev commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

peppy commented Jan 14, 2025

Uh oh!

bdach commented Jan 14, 2025

Uh oh!

bdach commented Jan 14, 2025

Uh oh!

nanaya commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NammIsADev commented Jan 14, 2025

Uh oh!

bdach commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NammIsADev commented Jan 14, 2025

Uh oh!

NammIsADev commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bdach commented Jan 14, 2025

Uh oh!

NammIsADev commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NammIsADev commented Jan 16, 2025

Uh oh!

bdach commented Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NammIsADev commented Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bdach commented Jan 16, 2025

Uh oh!

NammIsADev commented Jan 13, 2025 •

edited

Loading

bdach commented Jan 13, 2025 •

edited

Loading

NammIsADev commented Jan 14, 2025 •

edited

Loading

nanaya commented Jan 14, 2025 •

edited

Loading

bdach commented Jan 14, 2025 •

edited

Loading

NammIsADev commented Jan 14, 2025 •

edited

Loading

NammIsADev commented Jan 14, 2025 •

edited

Loading

bdach commented Jan 16, 2025 •

edited

Loading

NammIsADev commented Jan 16, 2025 •

edited

Loading