Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: files with no mime type cause sync to crash #2963

Merged

Conversation

celsopalmeiraneto
Copy link
Contributor

Upon syncing the OneDrive files the following error has happened:

(...)
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 33 of 232 from API Response Bundle 107
DEBUG: Raw JSON OneDrive Item: {"cTag":"aYzpEOTlFNjNDQzJDRTdBRTJEITE5MzIzOC4yNTc","eTag":"aRDk5RTYzQ0MyQ0U3QUUyRCExOTMyMzguMA","file":{"hashes":{"quickXorHash":"ek3Yb91dB9lsH3HC142sW\/wLMGg=","sha1Hash":"427ACCF3D80F00897CDF4362C56EB9F0F9CFADC1","sha256Hash":"5313530468673A49B0553890C03F5C91DDA711DDB319DF75217169B588DFE340"}},"fileSystemInfo":{"createdDateTime":"2016-10-06T17:51:18.316Z","lastModifiedDateTime":"2016-10-06T17:51:18.316Z"},"id":"D99E63CC2CE7AE2D!193238","name":"All mail Including Spam and Trash-3.mbox","parentReference":{"driveId":"d99e63cc2ce7ae2d","driveType":"personal","id":"D99E63CC2CE7AE2D!193227","name":"Google⁠ Download Your Data","path":"\/drive\/root:\/Aplicativos\/Google⁠ Download Your Data"},"size":2493329731}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = D99E63CC2CE7AE2D!133
DEBUG: 'search id'                                          = D99E63CC2CE7AE2D!193238
DEBUG: id == defaultRootId                                  = false
DEBUG: isItemRoot(onedriveJSONItem)                         = false
DEBUG: onedriveJSONItem['name'].str == 'root'               = false
DEBUG: itemHasParentReferenceId                             = true
DEBUG: Failure scope was called
DEBUG: performSynchronisedExitProcess called by: failureScope
DEBUG: Shutting down Filesystem Monitoring instance
DEBUG: Shutdown of Filesystem Monitoring instance is complete
DEBUG: Shutting down Sync Engine instance
DEBUG: SyncEngine: Waiting for all internal threads to complete
DEBUG: Shutting down processPool in a thread blocking manner
DEBUG: Shutdown Sync Engine instance is complete
DEBUG: CurlEngine releaseAllCurlInstances() called
DEBUG: CurlEngine curlEnginePool size to release: 2
DEBUG: CurlEngine cleanup() called on instance id: 1LH9KfapCV3obwky
DEBUG: CurlEngine shutdownCurlHTTPInstance() called on instance id: 1LH9KfapCV3obwky
DEBUG: HTTP instance still active: 1LH9KfapCV3obwky
DEBUG: HTTP instance isStopped state before http.shutdown(): false
DEBUG: HTTP instance isStopped state post http.shutdown(): true
DEBUG: HTTP instance shutdown and destroyed: 1LH9KfapCV3obwky
DEBUG: CurlEngine destroyed
DEBUG: CurlEngine cleanup() called on instance id: P1iyDLV5hYXwF6Mb
DEBUG: CurlEngine shutdownCurlHTTPInstance() called on instance id: P1iyDLV5hYXwF6Mb
DEBUG: HTTP instance still active: P1iyDLV5hYXwF6Mb
DEBUG: HTTP instance isStopped state before http.shutdown(): false
DEBUG: HTTP instance isStopped state post http.shutdown(): true
DEBUG: HTTP instance shutdown and destroyed: P1iyDLV5hYXwF6Mb
DEBUG: CurlEngine destroyed
DEBUG: CurlEngine releaseAllCurlInstances() completed
DEBUG: Shutting down Client Side Filtering instance
DEBUG: Shutdown of Client Side Filtering instance is complete
DEBUG: Shutting down Database instance
DEBUG: Merge contents of WAL and SHM files into main database file before shutting down database
DEBUG: Attempting to perform a database checkpoint to merge temporary data
DEBUG: Counting open SQL statements
DEBUG: Database checkpoint is complete
DEBUG: Attempting to perform a database vacuum to optimise database
DEBUG: Counting open SQL statements
DEBUG: Database vacuum is complete
DEBUG: Shutdown of Database instance is complete
DEBUG: Shutting down Application Configuration instance
DEBUG: Shutdown of Application Configuration instance is complete
DEBUG: Application is exiting
DEBUG: Shutting down Application Logging instance
std.json.JSONException@std/json.d(641): Key not found: mimeType
----------------
/usr/lib/ldc/x86_64-linux-gnu/include/d/std/exception.d:519 pure @safe noreturn std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x55d2c871bfe3]
??:? inout pure ref @safe inout(std.json.JSONValue) std.json.JSONValue.opIndex(return scope immutable(char)[]) [0x7f8253acfdc9]
src/util.d:1065 [0x55d2c8533351]
src/sync.d:1350 [0x55d2c852bfb6]
src/sync.d:1028 [0x55d2c8522fd3]
src/sync.d:568 [0x55d2c852120b]
src/main.d:1274 [0x55d2c85e6916]
src/main.d:1059 [0x55d2c85e4193]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x7f82537d5b7b]
??:? _d_run_main2 [0x7f82537d5995]
??:? _d_run_main [0x7f82537d57ed]
/usr/lib/ldc/x86_64-linux-gnu/include/d/core/internal/entrypoint.d:42 [0x55d2c8616f91]
??:? [0x7f8253432249]
??:? __libc_start_main [0x7f8253432304]
??:? [0x55d2c84e6120]

The error was caused because, for the file in question, OneDrive has not returned the MIME type:

  {
    "cTag": "aYzpEOTlFNjNDQzJDRTdBRTJEITE5MzIzOC4yNTc",
    "eTag": "aRDk5RTYzQ0MyQ0U3QUUyRCExOTMyMzguMA",
    "file": {
      "hashes": {
        "quickXorHash": "ek3Yb91dB9lsH3HC142sW/wLMGg=",
        "sha1Hash": "427ACCF3D80F00897CDF4362C56EB9F0F9CFADC1",
        "sha256Hash": "5313530468673A49B0553890C03F5C91DDA711DDB319DF75217169B588DFE340"
      }
    },
    "fileSystemInfo": {
      "createdDateTime": "2016-10-06T17:51:18.316Z",
      "lastModifiedDateTime": "2016-10-06T17:51:18.316Z"
    },
    "id": "D99E63CC2CE7AE2D!193238",
    "name": "All mail Including Spam and Trash-3.mbox",
    "parentReference": {
      "driveId": "d99e63cc2ce7ae2d",
      "driveType": "personal",
      "id": "D99E63CC2CE7AE2D!193227",
      "name": "Google⁠ Download Your Data",
      "path": "/drive/root:/Aplicativos/Google⁠ Download Your Data"
    },
    "size": 2493329731
  }

Upon fixing the issue this file was successfully synced:

(...)
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 33 of 232 from API Response Bundle 107
DEBUG: Raw JSON OneDrive Item: {"cTag":"aYzpEOTlFNjNDQzJDRTdBRTJEITE5MzIzOC4yNTc","eTag":"aRDk5RTYzQ0MyQ0U3QUUyRCExOTMyMzguMA","file":{"hashes":{"quickXorHash":"ek3Yb91dB9lsH3HC142sW\/wLMGg=","sha1Hash":"427ACCF3D80F00897CDF4362C56EB9F0F9CFADC1","sha256Hash":"5313530468673A49B0553890C03F5C91DDA711DDB319DF75217169B588DFE340"}},"fileSystemInfo":{"createdDateTime":"2016-10-06T17:51:18.316Z","lastModifiedDateTime":"2016-10-06T17:51:18.316Z"},"id":"D99E63CC2CE7AE2D!193238","name":"All mail Including Spam and Trash-3.mbox","parentReference":{"driveId":"d99e63cc2ce7ae2d","driveType":"personal","id":"D99E63CC2CE7AE2D!193227","name":"Google⁠ Download Your Data","path":"\/drive\/root:\/Aplicativos\/Google⁠ Download Your Data"},"size":2493329731}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = D99E63CC2CE7AE2D!133
DEBUG: 'search id'                                          = D99E63CC2CE7AE2D!193238
DEBUG: id == defaultRootId                                  = false
DEBUG: isItemRoot(onedriveJSONItem)                         = false
DEBUG: onedriveJSONItem['name'].str == 'root'               = false
DEBUG: itemHasParentReferenceId                             = true
DEBUG: Attempting to calculate local filesystem path for d99e63cc2ce7ae2d and D99E63CC2CE7AE2D!193227
DEBUG: New Item calculated full path is: Aplicativos/Google⁠ Download Your Data/All mail Including Spam and Trash-3.mbox
DEBUG: skip_file item to check (full calculated path): /Aplicativos/Google⁠ Download Your Data/All mail Including Spam and Trash-3.mbox
DEBUG: skip_file evaluation for: /Aplicativos/Google⁠ Download Your Data/All mail Including Spam and Trash-3.mbox
DEBUG: Result: false
DEBUG: Adding this Raw JSON OneDrive Item to jsonItemsToProcess array for further processing
DEBUG: Initial JSON item processing time: 376 μs and 2 hnsecs
DEBUG: ------------------------------------------------------------------
DEBUG: Processing OneDrive Item 34 of 232 from API Response Bundle 107
DEBUG: Raw JSON OneDrive Item: {"cTag":"aYzpEOTlFNjNDQzJDRTdBRTJEITE5MzIzOS4yNTc","eTag":"aRDk5RTYzQ0MyQ0U3QUUyRCExOTMyMzkuMA","file":{"hashes":{"quickXorHash":"hxFcerukdrdGGIttaH2kBj44CHw=","sha1Hash":"0866859B2BC07E6E5C6087CC766FA787A495157D","sha256Hash":"26BD0E61C87C8ED45D80DC95D82637C95993A3BA6EA133F0735B347419E6EF20"},"mimeType":"application\/x-gzip"},"fileSystemInfo":{"createdDateTime":"2016-10-06T17:51:21.456Z","lastModifiedDateTime":"2016-10-06T17:51:21.456Z"},"id":"D99E63CC2CE7AE2D!193239","name":"takeout-20161005T212339Z-4.tgz","parentReference":{"driveId":"d99e63cc2ce7ae2d","driveType":"personal","id":"D99E63CC2CE7AE2D!193227","name":"Google⁠ Download Your Data","path":"\/drive\/root:\/Aplicativos\/Google⁠ Download Your Data"},"size":208607}
DEBUG: This item is not a OneDrive deletion change
DEBUG: defaultRootId                                        = D99E63CC2CE7AE2D!133
DEBUG: 'search id'                                          = D99E63CC2CE7AE2D!193239
(...)

This comment has been minimized.

This comment has been minimized.

* Update PR
Copy link

@check-spelling-bot Report

🔴 Please review

See the 📂 files view, the 📜action log, or 📝 job summary for details.

Unrecognized words (40)
bashisms
capitalisation
containerised
customisable
customisation
cuz
finalise
finalised
fullstop
gzipping
initialising
Ioctl
ISPs
kinda
Kubuntu
logfiles
Lubuntu
malware
meh
optimisations
preinstalled
prioritise
prolly
reparse
resynchronisation
sandboxing
segfault
shitload
Slackware
sourced
Sourcing
spamming
thingies
Thu
uninitialised
uninstallation
upgradable
utilisation
websites
Xubuntu
To accept these unrecognized words as correct, you could run the following commands

... in a clone of the git@github.com:celsopalmeiraneto/onedrive.git repository
on the fix-mime-type-validation branch (ℹ️ how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' |
perl - 'https://github.com/abraunegg/onedrive/actions/runs/11804225805/attempts/1'

@abraunegg abraunegg merged commit 6024d30 into abraunegg:master Nov 12, 2024
4 of 5 checks passed
@celsopalmeiraneto celsopalmeiraneto deleted the fix-mime-type-validation branch November 12, 2024 21:46
@abraunegg abraunegg added this to the v2.5.3 milestone Nov 15, 2024
@abraunegg
Copy link
Owner

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Repository owner locked and limited conversation to collaborators Nov 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants