Bug: HTML encoding requires to add filter entries twice #2740
Comments
This is not a bug. For whatever reason - you are the only person who seems to have paths stored online that use HTML entities - I am yet to even find a way locally or remotely that creates folders that contain things like '%20'. How are these being even created in the first place or is this some sort of 'legacy' naming issue ? The folder entity that is returned from Microsoft OneDrive includes the HTML entity itself in the JSON response - thus, to filter against that specific folder you need to add the folder name that contains that entity. Your solution here is to really rename your folders .. failing that it is double entries .. sorry. |
|
Very strange, I don't think I did anything special to create these. Maybe the HTML entities are an artifact of me "mounting" other OneDrives within my main OneDrive? |
No, that does not create that effect, neither can you create a folder or link with |
|
You are probably right: even if I create a folder "BücherTest" within my main OneDrive (not a linked one) in the main root dir the special letter is HTML encoded even in the browser: |
|
For reference I also tested on a corporate SharePoint and also here folders with special characters are HTML encoded: |
|
@robertschulze These are stored online correctly as UTF-16, for example:
You also have some sort of DecodePath item also being 'included' .. which is not occurring for any item I am using:
So I created a new folder online called
The 'name' as provided by Microsoft OneDrive API includes the umlaut ... not some HTML entity version of the text. The JSON is processed 100% normally: So I think that there is 100% something odd with your OneDrive account that I literally cannot assist with. At this point I would be raising a case with Microsoft to understand why this is being used and how this is being triggered:
|
|
In the Elements view in the browser it is indeed correctly encoded also for my OneDrives: |
|
@robertschulze
Subfolders within
Everything is being presented correctly for the 'name' JSON element, which the 'onedrive' client uses for all operations. So right now, I suspect that what you are seeing is the result one of the following:
What I have in 'My Profile' for this test account is the following: What is yours set to? If I change my account to match yours - do I get the same experience? If you change yours to match mine does this change your experience? In my view this is potentially a OneDrive API Bug as the 'name' element in the API responses should never be URL encoded. The OneDrive API does not specify at all that the 'name' element may contain URL encoded characters - thus this is why I think this is an API bug. That no-one else however has reported this (only yourself and this started off with |
|
@abraunegg I did a number of tests. Disabling all plugins in Chrome did not help, changing my country to Australia also did not help (language was already English (United States)). |
|
@robertschulze
|
They both use the same Microsoft Graph API |
|
@ificator Any input you have would be greatly appreciated. |
|
@abraunegg You are right, i was mistaking B\u00fccher for URL encoding of Bücher, but it is just UTF. But then I think the response from the API is actually fine: With sync_list: actually nothing is synced because items within "OneDrive - Elias" are considered root items (which I think is due to the fact that they are linked even though they are linked from folder subfolder Documents within the other OneDrive): With sync_list: the folder Bücher is matched per se: Then the list of files within the folder is retrieved: where the name of the parentReference is fine (B\u00fccher). This is why the file is not being synced (except if I would add B%C3%BCcher to the sync_list) At the same time, your screenshot above also shows the path of the parentReference containing URL encoded letters: What I meant w.r.t. to Firefox: it appears to convert UTF, i.e. if you look at the RAW json response you see B\u00fccher while in the "formatted" JSON view you see Bücher. Chrome shows B\u00fccher also in the formatted JSON view. In any case, as you correctly state, no mangling of URL encodes, this is just a question of how they handle UTF. |
|
@robertschulze |
|
The whole JSON response item you need to reference is 'name' - this is the only item which is being HTML encoded in your responses. This is why:
|
Not correct ... event the UTF encoding is wrong .. in ALL of my examples the 'name' JSON element contains the correct text - unmangled by either UTF encoding or HTML encoding. The only thing I can think of is you have configured something somewhere for Microsoft to auto-translate or auto do something. |
|
For my reference, could you please share the full RAW json of your "Bücher" example (from the onedrive client with --debug-https or in Firefox, selecting children on the left and on the right side scrolling all the way down to the actual json string)? |
|
Details have been sent offline. |
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
|
@robertschulze |
|
@robertschulze |
abraunegg
removed
Unable to reproduce
OneDrive API Inconsistency
OneDrive API Bug
* If we have to 'build' the path use for checking against Client Side Filtering rules - make sure this path is free from HTML encoded characters
|
@robertschulze Using just a sync_list entry of the following:
Generates the following: Can you please verify / rebuild your v2.5.x client using PR #2728 ? When you rebuild , your client version should be: |
|
Marking this as fixed as testing evidence shows that this issue is resolved. |
abraunegg
added
Fixed
and removed
PR Provided - Needs Testing
Waiting on Response
labels
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Describe the bug
In my sync_list I have (among others):
To sync subdirs "Backup" and "Bücher" of folder "OneDrive - Elias".
However, directory "Bücher" is not entered, rather the seek continues in "Dokumente"
while for the parallel directory "Backup", the subdirectory "Backup/20221026 - Kindle" is considered
Further down in the log file it turns out the subdir "20181006 - Fredo 1 Mathematik Arbeitsheft" was excluded
Only when now adding "B%C3%BCcher" to the sync_list as well, the files and folders within "Bücher" are properly synchronized:
Operating System Details
Linux atom 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.4 LTS Release: 22.04 Codename: jammyClient Installation Method
From Source
OneDrive Account Type
Personal
What is your OneDrive Application Version
onedrive v2.5.0-rc2-37-g3f7fb5a
What is your OneDrive Application Configuration
What is your 'curl' version
Where is your 'sync_dir' located
Network
What are all your system 'mount points'
//hyperv/3 on /media/3 type cifs (rw,relatime,vers=3.1.1,cache=strict,username=Administrator,uid=1000,noforceuid,gid=100,noforcegid,addr=192.168.3.11,file_mode=0777,dir_mode=0777,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1)What are all your local file system partition types
How do you use 'onedrive'
./onedrive --confdir='/home/robert/.config/onedrive/accounts/robert@guitaronline.de' --sync --verbose --verbose --resync --resync-auth > debug_output.log 2>&1Steps to reproduce the behaviour
see description above
Complete Verbose Log Output
see description above, can send via mail if requiredScreenshots
No response
Other Log Information or Details
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: