OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash #2433

drjarno · 2023-06-20T12:36:59Z

Describe the bug

I just updated to onedrive v2.4.24 which incorporates the deprecation of the SHA checksum. However, now my logs are full of these lines now:

Jun 20 08:20:44 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash
Jun 20 08:20:44 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash
Jun 20 08:20:44 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash
Jun 20 08:20:45 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash
Jun 20 08:20:45 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash
Jun 20 08:20:45 goat onedrive[29778]: ERROR: OneDrive API inconsistency - the file does not have any hash

So it kind of looks like something broke in the hashing.

It's possible these errors are harmless, but I don't know how to tell which files are affected. It's also possible it will give this error just once for every file while it switches over to another checksum algorithm, but if so it should maybe mention that.

I tried a onedrive --synchronize operation too which did not have these errors. I usually use onedrive --monitor though.

Operating System Details

$ uname -a
Linux goat 6.3.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 14 Jun 2023 20:10:31 +0000 x86_64 GNU/Linux

$ lsb_release -a
LSB Version:    n/a
Distributor ID: Arch
Description:    Arch Linux
Release:        rolling
Codename:       n/a

Client Installation Method

From Distribution Package

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

onedrive v2.4.24

What is your OneDrive Application Configuration

Configuration file successfully loaded
onedrive version                             = v2.4.24
Config path                                  = /home/jvanderk/.config/onedrive
Config file found in config path             = true
Config option 'sync_dir'                     = /home/jvanderk/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     = large
Config option 'skip_dir_strict_match'        = true
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = 
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   = 
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 0
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = false
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 8.1.2 (x86_64-pc-linux-gnu) libcurl/8.1.2 OpenSSL/3.1.1 zlib/1.2.13 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) libssh2/1.10.0 nghttp2/1.54.0
Release-Date: 2023-05-30
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

/dev/nvme1n1p1 on /home type ext4 (rw,relatime)
/home/.ecryptfs/jvanderk/.Private on /home/jvanderk type ecryptfs (rw,nosuid,nodev,relatime,ecryptfs_fnek_sig=a8d56b9fef6ef695,ecryptfs_sig=0489a851517a6a34,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_unlink_sigs)

What are all your local file system partition types

nvme1n1                                                                            
└─nvme1n1p1 ext4   1.0         2f7cf7f5-70a0-4cac-9211-24a4f69137b7  202.2G    73% /home

How do you use 'onedrive'

I have the client on my laptop and on my desktop. They are never in operation at the same time.

Steps to reproduce the behaviour

Start onedrive client in a user SystemD service using onedrive --monitor
Observe errors in the logs

Complete Verbose Log Output

Relevant output of onedrive --monitor --verbose:
Processing 213 OneDrive items to ensure consistent local state due to a full scan being requested
Processing 212 OneDrive items to ensure consistent local state due to a full scan being requested
Processing 217 OneDrive items to ensure consistent local state due to a full scan being requested
Processing 214 OneDrive items to ensure consistent local state due to a full scan being requested
Processing 231 OneDrive items to ensure consistent local state due to a full scan being requested
Processing 217 OneDrive items to ensure consistent local state due to a full scan being requested
The Microsoft OneNote Notebook 'Notebooks/IT Managers Report' is not supported by this client
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
Skipping item - excluded by skip_dir config: large
Processing 217 OneDrive items to ensure consistent local state due to a full scan being requested
The Microsoft OneNote Notebook 'Notebooks/IT Managers Report' is not supported by this client
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
ERROR: OneDrive API inconsistency - the file does not have any hash
Processing 208 OneDrive items to ensure consistent local state due to a full scan being requested

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

The text was updated successfully, but these errors were encountered:

Totto16 · 2023-06-20T15:42:50Z

Same here, but it also occurs during --synchronize for me

drjarno · 2023-06-20T17:11:39Z

Just adding another observation, The client is still uploading changed files and it also downloads files I manually upload to OneDrive through the web interface. So it doesn't seem to stop the client from working, at least not for the few files I tested.

abraunegg · 2023-06-20T19:29:41Z

@Totto16 , @drjarno

This is NOT a bug with this application, but a BUG with the OneDrive API.

The content being provided by @microsoft by their OneDrive API is being inconsistent - and the files that are reporting this error message have no hash as provided by the API.

Essentially - for these files - they have no hash, and in the v2.4.23 application, the file item would have just been stored without a hash, making any file comparison for that file impossible - leading to all of unpredictable of situations.

This is NOT a bug I can solve for - if the OneDrive API data does not contain the hash values - I cant fix that.

@ificator , @microsoft
As per #2423 - sha1 has been removed, however it appears that many users are now saying that there are many files across their ecosystem that report as having no hash in the API data. This is primarily being seen in Business Accounts.

I have evidenced this in debug logs as per below:

Please urgently fix the OneDrive API

Totto16 · 2023-06-20T19:33:48Z

Thanks for clearing this up 😄
Mine is indeed a Business Account, but something that would be nice, would be, if the filename was logged too, since without it, it's really hard to know which files are affected and if an important file is affected trying to solve it, by renaming, removing and adding or similar.

At the moment only ERROR: OneDrive API inconsistency - the file does not have any hash is logged, so adding a filename there would help a lot ❤️

ificator · 2023-06-20T19:35:02Z

Those logs are for a business account, which has never has sha1Hash so that shouldn't be new. If I remember correctly there are cases where SPO can be missing the hash, and for enumeration scenarios it's too expensive to generate at query time.

If you have logs for a consumer user I can definitely drill in.

abraunegg · 2023-06-20T19:41:14Z

@ificator
Raised OneDrive/onedrive-api-docs#1704 as well

It appears to be only OneDrive Business Accounts at the moment ... it is a valid issue as the file hash is needed to accurately compare above timestamp as to if the file has indeed been modified.

It does not stand for reason why, on the same OneDrive Business driveId that some files are returned with a hash - and others not .. as per the above log extract where this has been evidenced.

If I remember correctly there are cases where SPO can be missing the hash, and for enumeration scenarios it's too expensive to generate at query time.

Computationally expensive or not - all files should have at least 1 hash available ... not 0 ! I can understand not wanting to compute quickXorHash & sha256Hash .. but to have nothing at all ... and to have this randomly in the /delta data response - that feels like a bug in the API platform.

ificator · 2023-06-20T19:42:19Z

I completely agree - there's no reason we shouldn't have a hash to return.

Do you have any idea why your change would have started manifesting this for OneDrive Business?

abraunegg · 2023-06-20T19:42:24Z

@Totto16

At the moment only ERROR: OneDrive API inconsistency - the file does not have any hash is logged, so adding a filename there would help a lot

Noted .. will update this in the v2.5.x codebase

abraunegg · 2023-06-20T19:44:08Z

@ificator

Do you have any idea why your change would have started manifesting this for OneDrive Business?

Absolutely - in prior releases, there was no warning message printed if there was a hash missing - now it does .. to catch this sort of situation .... so I am guessing that having the hash data missing has been a long standing API bug or problem that has only now just been surfaced.

ificator · 2023-06-20T20:01:18Z

Has anyone noticed a pattern with the missing hashes? E.g. specific file types etc?

drjarno · 2023-06-20T20:04:45Z

Has anyone noticed a pattern with the missing hashes? E.g. specific file types etc?

I'm not sure how to tell since there's no way to see what file has the missing hash currently. Well, that I know of anyway.

ificator · 2023-06-20T20:07:21Z

The snippet of response with the missing hash is for a 0-byte file.. I wonder if that's an edge case where the hash calculation is skipped.

abraunegg · 2023-06-20T20:24:36Z

@ificator

The snippet of response with the missing hash is for a 0-byte file.. I wonder if that's an edge case where the hash calculation is skipped.

Potentially .. looking at the log file provided - all of those items are indeed zero size from what I can tell

@Totto16 , @drjarno

Can you generate a debug log please as per: https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support and email to the support address.

Totto16 · 2023-06-20T20:27:33Z

I think using --force did something strange, I don't receive any Error anymore, it seems as if something changed, but since I'm clueless which files where exactly affected, I can't check, if they have a hash now or if they are gone 🤷

Edit: I can confirm, that they were not deleted, as per The Recycle Bin of the onedrive web-interface

Rmano · 2023-06-20T20:31:01Z

Has anyone noticed a pattern with the missing hashes? E.g., specific file types etc?

I reported this also; you can see that at #2430 (comment).

I generated a verbose log as explained in https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support, with onedrive --monitor --verbose --verbose > debug_output.log 2>&1. Then I isolated the name of the erroring files like this:

 cat debug_output.log | egrep "ERROR|thisItemFullPath" | grep -B1 ERROR

I can't say for sure, but all the files that failed for me that I could track in the file (I had to stop it when it passed 50 Mbyte) are either

from the Notebooks / Bloc de Notas folder
from a (huge) directory that I call Backups, and that I update every weekend using rclone.

Both subtrees are ignored in my configuration, but they still send the delta data (which I think they shouldn't), even if there have been no changes at all.

Given that the files I can easily check on my PC are under the Backup/Live directory, I then did the following:

cat debug_output.log | egrep "ERROR|thisItemFullPath" | grep -B1 ERROR | grep -v ERROR | grep -v -- -- | grep Backups/ | sed 's/^\[DEBUG\].*Backups\/Live\///' > listfiles.txt

and the used ls -l $(cat listfiles.txt).

I can confirm that in my case (I repeat, this is only a subset of the files) those are all zero-length files.

Totto16 · 2023-06-20T20:36:40Z

After creating a bunch of empty files, the error message reappears, so I can confirm, that the empty files are the problem on my site

ificator · 2023-06-20T20:36:52Z

Well that's an easy fix - I have a very efficient algorithm for generating the hash of a 0-byte file ;)

Totto16 · 2023-06-20T20:37:33Z

Well that's an easy fix - I have a very efficient algorithm for generating the hash of a 0-byte file ;)

And that would be 👀 🕋 (imagine a kappa there)

Rmano · 2023-06-20T20:45:04Z

But in the example (from my log) that @abraunegg posted (thanks!) in OneDrive/onedrive-api-docs#1704 (comment), it seems that one the file has non-zero length... let me check.

Ah, ok, it seems a cut-and-paste problem --- there is a menudep.h which is indeed zero-sized and it generates the error. The menudep.c file is working ok.

abraunegg · 2023-06-20T21:09:16Z

For those that are impacted by this issue, please can you test the following PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/2436/head:pr2436
git checkout pr2436

To run the PR, you need to run the client from the PR build directory:

./onedrive <any options needed>

To install the PR, you will need to perform sudo make install to install the PR version.

When running the PR, your version should be: onedrive v2.4.24-2-g1d406e0 or greater.

abraunegg · 2023-06-20T23:19:03Z

@Totto16

Were you able to test the PR ?

With a slight tweak locally to actually print a message on a known zero byte file .. this PR is known to be working as intended:

Using 'user' Config Dir: /home/alex/.config/onedrive-business/
Configuration file successfully loaded
Deleting the saved application sync status ...
Checking Application Version ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Using Curl defaults for all HTTP operations
Opening the item database ...
All operations will be performed in: /home/alex/OneDriveBusiness
Application version: v2.4.24-2-g1d406e0
Account Type: business
Default Drive ID: b!bO8V7s9SSk6r7mWHpIjURotN33W1W2tEv3OXV_oFIdQimEdOHR-1So7CqeT1MfHA
Default Root ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ
Remaining Free Space: 1099499773050
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Initializing the Synchronization Engine ...
Syncing changes and items from OneDrive ...
Applying changes of Path ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ
Updated Remaining Free Space: 1099499773050
Processing 12 OneDrive items to ensure consistent local state
Skipping item - excluded by skip_dir config: target_excluded
Creating local directory: ./source
Creating local directory: source/folder_a
Downloading file source/folder_a/solaris crash.docx ... done.
Downloading file source/folder_a/Remove Ubuntu Package.docx ... 
Downloading 100% |oooooooooooooooooooooooooooooooooooooooo| DONE IN 00:00:01                                                                                                         
done.
Creating local directory: ./Music
Creating local directory: ./Double Folder
WARNING: OneDrive API inconsistency - this zero byte file does not have any hash: ./zero-byte-file
Downloading file ./zero-byte-file ... done.
Performing a database consistency and integrity check on locally stored data ... 
Uploading differences of ~/OneDriveBusiness
Processing .
The directory has not changed
Processing source
The directory has not changed
Processing source/folder_a
The directory has not changed
Processing source/folder_a/solaris crash.docx
The file has not changed
Processing source/folder_a/Remove Ubuntu Package.docx
The file has not changed
Processing Music
The directory has not changed
Processing Double Folder
The directory has not changed
Processing zero-byte-file
The file has not changed
Uploading new items of ~/OneDriveBusiness
Applying changes of Path ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ
Updated Remaining Free Space: 1099499773050
Processing 12 OneDrive items to ensure consistent local state due to a full scan being triggered by actions on OneDrive
Skipping item - excluded by skip_dir config: target_excluded
WARNING: OneDrive API inconsistency - this zero byte file does not have any hash: ./zero-byte-file
Sync with OneDrive is complete

The PR version does not output this line: WARNING: OneDrive API inconsistency - this zero byte file does not have any hash:

Totto16 · 2023-06-20T23:29:16Z

I tested it, and it doesn't log any error, it seems to fix it. I also tested downloading on another device, and it worked too.
I managed to replace the original ondrive executable, so it took me longer 🤦 but it works 👍

Rmano · 2023-06-21T07:27:20Z

I still have some warnings, although much less than before. My problem is:

Jun 21 09:22:28 pern onedrive[594453]: WARNING: OneDrive API inconsistency - this file does not have any hash: 01CRET2R2N43PLCNJNN5BIJRSMB3AWN4LW

How do I get from here to the file/directory name?

abraunegg · 2023-06-21T07:47:31Z

I still have some warnings, although much less than before. My problem is:
Jun 21 09:22:28 pern onedrive[594453]: WARNING: OneDrive API inconsistency - this file does not have any hash: 01CRET2R2N43PLCNJNN5BIJRSMB3AWN4LW 
How do I get from here to the file/directory name?

So that will take some tracking down. You are getting that message as the file is > 0 in size and has no hash ... so it is a legitimate problem from the OneDrive API

The item id is being printed - so this means that the parents of this item are not in the database - thus the path cannot be calculated.

If the parents were in the database - the actual path would have been calculated and presented instead of the item id

This tells me then that this particular item is potentially being skipped by your configuration.

Rmano · 2023-06-21T07:50:42Z

Yes, I suppose it's being skipped. The previous warning was for

Jun 21 09:47:35 pern onedrive[594453]: WARNING: OneDrive API inconsistency - this file does not have any hash: ./Romano @ Universidad Pontificia Comillas

which are the strange things created by OneNote, I suppose, and that I do not know how to get rid of ;-)

abraunegg · 2023-06-21T08:04:26Z

Yes, I suppose it's being skipped. The previous warning was for
Jun 21 09:47:35 pern onedrive[594453]: WARNING: OneDrive API inconsistency - this file does not have any hash: ./Romano @ Universidad Pontificia Comillas 
which are the strange things created by OneNote, I suppose, and that I do not know how to get rid of ;-)

That warning is for a legitimate file + path - again, has zero hash.

It should not be a OneNote item - as they are filtered out - this item has the item type = file and is denoted by the API as such.

Rmano · 2023-06-21T08:11:25Z

This is at my "top" OneDrive directory:

and if I click on it in the browser, it will open OneNote with my main content there (lot of things). As I said, I got lost in these MS applications...

abraunegg · 2023-06-21T08:15:28Z

This is at my "top" OneDrive directory:

and if I click on it in the browser, it will open OneNote with my main content there (lot of things). As I said, I got lost in these MS applications...

Hrmm .. OK .. will have to look into this as part of v2.5.x work ... they should be filtered out from the get-go

abraunegg · 2023-06-29T02:02:45Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

drjarno added the Bug Something isn't working label Jun 20, 2023

abraunegg mentioned this issue Jun 20, 2023

ODB: The OneDrive API is inconsistent with providing hash data in /delta JSON response (URGENT BUG) OneDrive/onedrive-api-docs#1704

Closed

3 tasks

abraunegg added OneDrive API Inconsistency Not a bug and removed Bug Something isn't working labels Jun 20, 2023

abraunegg changed the title ~~Bug: OneDrive API inconsistency - the file does not have any hash~~ API Bug: OneDrive API inconsistency - the file does not have any hash Jun 20, 2023

abraunegg changed the title ~~API Bug: OneDrive API inconsistency - the file does not have any hash~~ OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash Jun 20, 2023

abraunegg removed the Not a bug label Jun 20, 2023

abraunegg added the PR Provided - Needs Testing label Jun 20, 2023

abraunegg linked a pull request Jun 20, 2023 that will close this issue

Fix that zero-byte files do not have a hash as reported by the OneDrive API thus should not generate an error message #2436

Merged

abraunegg closed this as completed in #2436 Jun 21, 2023

abraunegg added Fixed and removed PR Provided - Needs Testing labels Jun 21, 2023

This was referenced Jun 21, 2023

Bug: OneDrive API inconsistency #2442

Closed

BUG: Formerly working config. Updated recently to latest #2444

Closed

Repository owner locked as resolved and limited conversation to collaborators Jun 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash #2433

OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash #2433

drjarno commented Jun 20, 2023

Totto16 commented Jun 20, 2023

drjarno commented Jun 20, 2023

abraunegg commented Jun 20, 2023

Totto16 commented Jun 20, 2023

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023 •

edited

Loading

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023

abraunegg commented Jun 20, 2023 •

edited

Loading

ificator commented Jun 20, 2023

drjarno commented Jun 20, 2023

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023

Totto16 commented Jun 20, 2023 •

edited

Loading

Rmano commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading

ificator commented Jun 20, 2023

Totto16 commented Jun 20, 2023 •

edited

Loading

Rmano commented Jun 20, 2023

abraunegg commented Jun 20, 2023

abraunegg commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

abraunegg commented Jun 29, 2023

OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash #2433

OneDrive API Bug: OneDrive API inconsistency - the file does not have any hash #2433

Comments

drjarno commented Jun 20, 2023

Describe the bug

Operating System Details

Client Installation Method

OneDrive Account Type

What is your OneDrive Application Version

What is your OneDrive Application Configuration

What is your 'curl' version

Where is your 'sync_dir' located

What are all your system 'mount points'

What are all your local file system partition types

How do you use 'onedrive'

Steps to reproduce the behaviour

Complete Verbose Log Output

Screenshots

Other Log Information or Details

Additional context

Totto16 commented Jun 20, 2023

drjarno commented Jun 20, 2023

abraunegg commented Jun 20, 2023

Totto16 commented Jun 20, 2023

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023 • edited Loading

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023

abraunegg commented Jun 20, 2023 • edited Loading

ificator commented Jun 20, 2023

drjarno commented Jun 20, 2023

ificator commented Jun 20, 2023

abraunegg commented Jun 20, 2023

Totto16 commented Jun 20, 2023 • edited Loading

Rmano commented Jun 20, 2023 • edited Loading

Totto16 commented Jun 20, 2023 • edited Loading

ificator commented Jun 20, 2023

Totto16 commented Jun 20, 2023 • edited Loading

Rmano commented Jun 20, 2023

abraunegg commented Jun 20, 2023

abraunegg commented Jun 20, 2023 • edited Loading

Totto16 commented Jun 20, 2023 • edited Loading

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

Rmano commented Jun 21, 2023

abraunegg commented Jun 21, 2023

abraunegg commented Jun 29, 2023

abraunegg commented Jun 20, 2023 •

edited

Loading

abraunegg commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading

Rmano commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading

abraunegg commented Jun 20, 2023 •

edited

Loading

Totto16 commented Jun 20, 2023 •

edited

Loading