ODP Regression: OneDrive Graph API /delta deltaLink response is malformed

[abraunegg]

Category

• Question
• Documentation issue
• Bug

Expected or Desired Behavior

The deltaLink should return the correct data

Observed Behavior

For the last 24hr's ODP deltaLink response from the OneDrive API is malformed.

When calling /delta, the deltaLink is responding as per the following example:

[DEBUG] deltaLink: https://graph.microsoft.com/v1.0/drives('66d53be8a5056eca')/items('66D53BE8A5056ECA!101')/microsoft.graph.delta(token='aTE09NjM3NDI5NzI5OTI.....

Previously, this would respond as:

[DEBUG] deltaLink: https://graph.microsoft.com/v1.0/drives/66d53be8a5056eca/items/66D53BE8A5056ECA!101/delta?select=id%2cname%2ceTag%2ccTag%2cdeleted%2cfile%2

This means that the OneDrive Personal /delta and associated deltaLink response cannot be consumed or used. When attempting to use the bad deltaLink URL as provided by the API, it generates the following:

ERROR: OneDrive returned an error with the following message:
  Error Message:    HTTP request returned status code 400 (Bad Request)
  Error Reason:     Invalid API or resource
  Error Date:       2020-12-08T04:17:18
  Error Request ID: d704a5a7-35ff-472b-97d9-208f4f3286e2

Full Example:

[DEBUG] Attempting query 'changes = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLink)'
[DEBUG] driveId: 66d53be8a5056eca
[DEBUG] idToQuery: 66D53BE8A5056ECA!101
[DEBUG] Previous deltaLink: https://graph.microsoft.com/v1.0/drives/66d53be8a5056eca/items/66D53BE8A5056ECA!101/delta?select=id%2cname%2ceTag%2ccTag%2cdeleted%2cfile%2cfolder%2croot%2cfileSystemInfo%2cremoteItem%2cparentReference%2csize&$skiptoken=aTE09...redacted
[DEBUG] Request URL = https://graph.microsoft.com/v1.0/drives/66d53be8a5056eca/items/66D53BE8A5056ECA!101/delta?select=id%2cname%2ceTag%2ccTag%2cdeleted%2cfile%2cfolder%2croot%2cfileSystemInfo%2cremoteItem%2cparentReference%2csize&$skiptoken=aTE09...redacted
[DEBUG] Query 'changes = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLink)' performed successfully
[DEBUG] OneDrive API /delta response: {"@odata.context":"https:\/\/graph.microsoft.com\/v1.0\/$metadata#Collection(driveItem)","@odata.deltaLink":"https:\/\/graph.microsoft.com\/v1.0\/drives('66d53be8a5056eca')\/items('66D53BE8A5056ECA!101')\/microsoft.graph.delta(token='aTE09...redacted')?select=id%2cname%2ceTag%2ccTag%2cdeleted%2cfile%2cfolder%2croot%2cfileSystemInfo%2cremoteItem%2cparentReference%2csize&$skiptoken=aTE09...redacted","value":[{"@odata.type":"#microsoft.graph.driveItem","cTag":"adDo2NkQ1M0JFOEE1MDU2RUNBITEwMS42Mzc0Mjk3Mjk5MjE4MDAwMDA","eTag":"aNjZENTNCRThBNTA1NkVDQSExMDEuMQ","fileSystemInfo":{"createdDateTime":"2018-06-06T20:45:24.436Z","lastModifiedDateTime":"2019-04-24T07:09:31.29Z"},"folder":{"childCount":0,"view":{"sortBy":"takenOrCreatedDateTime","sortOrder":"ascending","viewType":"thumbnails"}},"id":"66D53BE8A5056ECA!101","name":"root","parentReference":{"driveId":"66d53be8a5056eca","driveType":"personal","id":"66D53BE8A5056ECA!0","path":"\/drive\/root:"},"root":{},"size":421686}]}

Steps to Reproduce

1. Perform a /delta query for OneDrive Personal Accounts
2. Analyse the /delta response

Currently this only impacts OneDrive Personal Accounts. OneDrive Business Accounts are not impacted

[ ]: http://aka.ms/onedrive-api-issues
[x]: http://aka.ms/onedrive-api-issues

[ghost]

Thank you for your contribution to OneDrive API Docs. We will be triaging your incoming issue as soon as possible.

[danpeig]

Same problem here...

[pcmagic]

Same problem. should we upgrade the client or just wait msftbot?

[matekolonics]

Business user here, I have the same problem.

[lahtis]

Normal user here and same problem.

[JeremyKelley]

Hi everyone, we're investigating now to see what's happening.

Note: We do expect that the format of the URL can change over time, however whatever URL is returned in the property should be valid. Just want to make sure that people don't take a dependency on the format of the URL, just that it is returned and works correctly. (Which in this case it isn't...)

[JeremyKelley]

Please let me know if this issue persists, it should return to working very shortly if it's not already.

[srm39]

Still happening -Joplin app syncing via OneDrive. Originally found here: https://discourse.joplinapp.org/t/error-invalid-api-or-resource/12966/2

[abraunegg]

@JeremyKelley
Confirm that it is working, however any client side 'cache' of the badly formatted deltaLink response needs to be removed as otherwise OneDrive clients still use the bad formatted URL and does not get told expire the old deltaLink URL as there is not date expiry associated with it.

[JeremyKelley]

We apologize for the interruption this caused, we're working on improvements for Delta and unfortunately some parts got ahead of other parts in the release train. Our intent was that the "new" URLs would be just working for you alongside the old ones and you wouldn't actually have noticed that there was even a change.

[abraunegg]

@JeremyKelley
Is there any way for the API team to engage with the Linux OneDrive client developers more directly, so when stuff like this is going on, there can be some sort of engagement and validation before things go more 'public' ?

[krkhan]

Chiming in, I use the open-source OneDrive client on roughly 8 of my machines -- on various distros e.g., Gentoo/Arch/Fedora. Thanks for the quick resolution! However, to @abraunegg 's point please find a way to coordinate breaking changes with client developers. For a while there I panicked :)

[JeremyKelley]

We will certainly make an effort for planned breaking changes, which should be rare, if ever, and generally accompanied by a major version bump and announcement beforehand with plenty of time to adapt any clients.

As mentioned, this should have been a silent change but a portion of the feature was deployed in an "on" state before supporting code was deployed. The feature change will likely be re-enabled after all necessary parts of the code are verified to have rolled out to production later in the week or possibly next.

The feature this is intended to support is adding an OData annotation for deleted items in addition to our "Deleted" facet that currently shows up. We aren't planning to remove the existing deleted indicator but we want to align with the rest of Microsoft Graph's handling of Delta to be consistent. To do that requires a few other changes on our end, one of which is supporting the URL pattern that got enabled today.

Again, apologies for the disruption, we'll continue to try to do better in the future.

[lahtis]

I have a few files on OneDrive cloud. The all have been synced to my computer. I went to copy one file from Onedrive’s web interface and edited it libreoffice and then uploaded it to the cloud via a browser in second computer. When I synced with the app then an error occurred. Im a normal user. Onedrive version is 2.4.8, Ubuntu 18.04.5 LTS

my logs are:
onedrive --synchronize
Configuring Global Azure AD Endpoints
Initializing the Synchronization Engine ...
Syncing changes from OneDrive ...

ERROR: OneDrive returned an error with the following message:
Error Message: HTTP request returned status code 400 (Bad Request)
Error Reason: Invalid API or resource
Uploading differences of ~/OneDrive
Uploading modified file Carpenter/sold.ods ... done.
Uploading new items of ~/OneDrive

ERROR: OneDrive returned an error with the following message:
Error Message: HTTP request returned status code 400 (Bad Request)
Error Reason: Invalid API or resource

If you've modified the Linux machine files from any defect has not been previously entered. Sync has worked seamlessly in the cloud. Otherwise, the files will be synced to the correct cloud service. Even if an error message appears.

[abraunegg]

@JeremyKelley
Happy to have you close this issue ticket given the resolution & fix confirmation

[laurent22]

Could someone explain what the issue is and how to fix it? I guess clients also need to do something since all of our users are now unable to use OneDrive for synchronisation (forum link).

Someone mentioned there that the delta link we cache should be cleared, but if we do that doesn't it mean clients will have to re-download all the data?

[laurent22]

In fact one users tried this:

As I'm a new Joplin user, I deleted everything and started again and also revoked/replaced the OneDrive auth but I still get the same error....

So it seems the bug is not fixed on OneDrive side yet?

[abraunegg]

@laurent22
The API issue is certainly fixed. The badly formatted deltaLink value is now correctly formatted.

How Joplin uses / saves / error checks this value, is an issue for Joplin to now fix.

For the Linux OneDrive Client which I am the developer / maintainer of, this required the following:

• Implement a check if the deltaLink URL causes an API error
  • If yes, remove it from any cached / stored value & retry the actual /delta call to get a new deltaLink value

It sounds like Joplin needs to implement something similar.

[laurent22]

This bug is not fixed - on a fresh sync, with no previous token, I'm still getting a malformed delta url:

https://graph.microsoft.com/v1.0/drives('9ada0xxxx073d0a')/root/microsoft.graph.delta(token='aTE09NjM3NDY1MDA1OTEzOTM7SUQ9OUFEQTBFQURGQTA3M0QwQSExxxxxxx1OTQyMzA7RVA9MjA7U0k9MTI7RExFUDxxxxxxxU089MjtQST0z')?select=name%2cfile%2cfolder%2cfileSystemInfo%2cparentReference%2cdeleted&$skiptoken=aTE09NjM3NDY1MDEwNzY1ODM7SUQ9Oxxxx0QwQSExMDY7TFI9NjM3NDY1MjMyNjMwMzA7RVAxxxxxxExFUD0wO1NHPTE7U089MjtQST0z

Maybe related to this - sync in some cases never finishes because the delta link provided by the last API call (@odata.nextLink) keeps returning changes, even though nothing has actually changed.

This is not a problem in our app as far as we can tell, and sync has been working properly for several years. It's just since this bug happened OneDrive became terribly unreliable for all our users. Is it a known issue and are there any plans to fix it?

[abraunegg]

@JeremyKelley
This issue has re-occurred:

When calling /delta, the deltaLink is responding as per the following example:

[DEBUG] deltaLink: https://graph.microsoft.com/v1.0/drives('66d53be8a5056eca')/items('66D53BE8A5056ECA!101')/microsoft.graph.delta(token='aTE09NjM3NDI5NzI5OTI.....

Previously, this would respond as:

[DEBUG] deltaLink: https://graph.microsoft.com/v1.0/drives/66d53be8a5056eca/items/66D53BE8A5056ECA!101/delta?select=id%2cname%2ceTag%2ccTag%2cdeleted%2cfile%2

[v-pajorg]

@JeremyKelley I have a customer with this issue it started to happen on the 20th of January, can you check this please?

[JeremyKelley]

Replying here too, we're investigating the necessary fixes here and will let people know when we have a final fix in place.

[snarkywolverine]

@JeremyKelley Any update here? Has this functionality really been regressed for over 2 months?

[ddkserv]

@JeremyKelley any update to the problem at hand?

[patrick-rodgers]

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.