Partially updating an object sends the full object to the server

[dblythy]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest versions of Parse Server and the Parse Swift SDK.

Issue Description

When fetching a saved object and then calling .save, all keys are marked as dirty and saved to the backend, causing unnecessary traffic and database ops.

Steps to reproduce

1. Fetch an existing object
2. Have a Parse Cloud beforeSave trigger that logs object.dirtyKeys().

Actual Outcome

All fields are dirty.

Expected Outcome

No fields should be dirty unless mutated.

Environment

Client

• Parse Swift SDK version: 1.9.10
• Xcode version: 10.15.7
• Operating system (iOS, macOS, watchOS, etc.): iOS
• Operating system version: 14.7

Server

• Parse Server version: 4.5.0
• Operating system: macOS
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): localhost

Database

• System (MongoDB or Postgres): mongodb
• Database version: 4.2
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Atlas

Logs

[parse-github-assistant]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[cbaker6]

Posting my responses to your comments in #241:

So there's no way to only send mutated keys to the server using Parse Swift?

There are a few ways, but the developer has to take some action here. For example, developers can add the following method (or a similar one) to their ParseObject's:

struct MyUser: ParseUser {
    var authData: [String: [String: String]?]?
    var username: String?
    var email: String?
    var emailVerified: Bool?
    var password: String?
    var objectId: String?
    var createdAt: Date?
    var updatedAt: Date?
    var ACL: ParseACL?

    // method that creates empty versions of self
    func emptyObject() -> Self {
        var object = Self()
        object.objectId = objectId
        return object
    }
}

Then add properties what will be mutated and saved. Only these get sent to the server:

var myObject = MyUser()
myObject.email = "myNewEmail@parse.com"
myObject.save { _ in }

This also can be easily achieved by using ParseOperation:

Parse-Swift/ParseSwift.playground/Pages/13 - Operations.xcplaygroundpage/Contents.swift

Lines 36 to 58 in 087227e

	//: You can have the server do operations on your `ParseObject`'s for you.
	//: First lets create another GameScore.
	let savedScore: GameScore!
	do {
	savedScore = try GameScore(score: 102).save()
	} catch {
	savedScore = nil
	fatalError("Error saving: \(error)")
	}
	//: Then we will increment the score.
	let incrementOperation = savedScore
	.operation.increment("score", by: 1)
	incrementOperation.save { result in
	switch result {
	case .success:
	print("Original score: \(String(describing: savedScore)). Check the new score on Parse Dashboard.")
	case .failure(let error):
	assertionFailure("Error saving: \(error)")
	}
	}

These examples should address most of your comments below:

Just a quick comment - whilst this will likely fix the issue, the core problem of dirtyKeys still remains. This is problematic because:

• unnecessary data is sent to parse server, which in developing countries may result in unbearable latency, or, on a high scaled system, result in additional data expenses
• unnecessary database ops will happen when no data is actually changing
• Cloud functions will tell developers to assume keys are dirty when they are not
• email being set to dirty is just an example of an unpredictable consequences of all the keys being sent, there could be other cases which we don’t know about.

When it comes to:

@cbaker6 is there a way we could perhaps store the serverData from the initial fetch of the object, and then compare changes to keys onSave if we can’t directly listen to property changes on a struct?

"Listening to property changes" is typically associated to reference types or "instances of Classes" in many languages, not just Swift. Most/All of the Parse SDK's use "classes" with the exception of the Swift SDK. value types or "structs" don't have this ability and are one of the distinct difference between a struct and a class. The Swift SDK doesn't need to listen property changes and instead should use a method similar to what I mentioned at the beginning of this comment. It's important not to think or compare of the Swift SDK to the other SDKs as it has a completely different design philosophy. Flo's comments and thread provide good insight into the design philosophy and the reasoning. For these reasons, the Swift SDK can exist with zero dependencies, not run into the threading issues the other client SDKs encounter, along with having ParseObject's play nicely with MVVM and SwiftUI (from what I see reference types is what's causing a major barrier when using MVVM in the issues you and others mention with the JS SDK).

[dblythy]

Thank you for your reply here @cbaker6.

In my view, it shouldn't be up to the developer to custom code a solution for the SDK to work as expected with the core Parse Server. One of our attractive features is simple SDKs which offer a broad range of consistent features. I understand that there are tradeoffs with the current implementation, but I still think it's problematic that at the end of the day, Parse Server has no distinction for dirtyKeys in the Swift SDK, which is different from any other SDK.

Although you've said in terms of philosophy it's not valuable to compare across SDKs, I think from both a developer and Server team perspective, it would be expected that beforeSave functions would function the same, regardless of the SDK. It just seems a little odd to me to say "if you use .dirtyKeys in Parse Cloud, make sure you customise ParseSwift so it doesn't send all the data".

And on the race condition, let's say you have an object that is being managed by two Swift SDK users at the same time. They both fetch the object at the same time.

The object looks like:

{
   foo: "bar",
   yolo: "xyz"
}

Both users query this object at the same time.

User one calls:

obj.foo = "aaa", and then saves.

A few minutes later, user two calls:

obj.yolo = "aaa", and then saves.

The result that will be saved in Parse Server would be:

{
   foo: "bar",
   yolo: "aaa"
}

As all keys saved by user two will be overriden. If this was any other SDK, the result would be:

{
   foo: "aaa",
   yolo: "aaa"
}

Meaning again, if you have a multi tenant solution (iOS App, Web App, Android App), using the Swift SDK in its purest form could be problematic and deliver unexpected results.

Could we perhaps store a clone of the fetched object somewhere, and then compare keys as part of the save operation? So when building the save operation, Parse Swift runs something like:

const fetchedObj // this has the fetched state
const updatedObj // this has all the user mutations
const bodyToSend = {};
for (const key in updatedObject) {
  if (fetchedObj[key] !== updatedObj[key]) {
    bodyToSend[key] = updateObj[key]
  }
}

Forgive the JS as said I am not a Swift dev 😊

[dblythy]

If this technically can't be solved on the SDK side because the use of struct, maybe the solution is to override dirtyKeys in RestWrite if object.key === original.key. I've discussed this with @mtrezza, however he has stated that believes:

Adding this to the server may be a mitigating step, but that doesn’t fully solve the cost and efficiency issue

[cbaker6]

In my view, it shouldn't be up to the developer to custom code a solution for the SDK to work as expected with the core Parse Server. One of our attractive features is simple SDKs which offer a broad range of consistent features. I understand that there are tradeoffs with the current implementation, but I still think it's problematic that at the end of the day, Parse Server has no distinction for dirtyKeys in the Swift SDK, which is different from any other SDK.

Although you've said in terms of philosophy it's not valuable to compare across SDKs, I think from both a developer and Server team perspective, it would be expected that beforeSave functions would function the same, regardless of the SDK. It just seems a little odd to me to say "if you use .dirtyKeys in Parse Cloud, make sure you customise ParseSwift so it doesn't send all the data".

It seems you are thinking in terms of the other SDKs, OOP, and Classes. I pointed to Flo's comment because the philosophy eludes to a number of things traditional users of the other SDKs may be unfamiliar with and the Swift SDK wasn't designed with the restrictions of OOP and Classes. I recommend looking into protocol oriented programing (POP) as it already forces the developer to customize and add their own keys for their ParseObjects, even for the Parse provided keys. The addition of the method I mentioned is simple and straightforward. In POP, developers should feel comfortable using the provided methods, replacing with their own, and adding extensions. As I mentioned in my previous comment, the design patterns of the other SDKs in terms of using OOP and Classes lead to other issues that are not present in the Swift SDK.

Could we perhaps store a clone of the fetched object somewhere, and then compare keys as part of the save operation? So when building the save operation, Parse Swift runs something like:

const fetchedObj // this has the fetched state
const updatedObj // this has all the user mutations
const bodyToSend = {};
for (const key in updatedObject) {
if (fetchedObj[key] !== updatedObj[key]) {
bodyToSend[key] = updateObj[key]
}
}

ParseSwift doesn't have local storage (besides using the Keychain for current) and if it did have storage, it would be protocol based, meaning the developer would implement it or use some 3rd party storage implementation that conformed to the "parse storage protocol". The 3rd party storage can have some notion of "dirty" if it felt it was needed, but depending on the implementation, it may not be necessary. The JS code you mentioned is in the style of the other SDKs which all use runtime encoders/decoders to dynamically access key/value pairs. Those encoders/decoders are also slower and prone to runtime bugs. ParseSwift uses compile time encoders/decoders and doesn't/shouldn't be doing anything like toJSON like the other SDKs to access/modify properties of an object.

Of course, a developer doesn't have to use the Swift SDK and can use the iOS SDK if they want OOP and Class based parse objects.

[cbaker6]

As all keys saved by user two will be overriden. If this was any other SDK, the result would be:

{
foo: "aaa",
yolo: "aaa"
}
Meaning again, if you have a multi tenant solution (iOS App, Web App, Android App), using the Swift SDK in its purest form could be problematic and deliver unexpected results.

This looks like a synchronization issue as a developer can run into the same issue with any of the SDKs if 2 clients attempted to modify/save the same property of an object at the same time. The race condition you mentioned is because Parse doesn't have a real synchronization mechanism and out-of-the-box asks developers to depend on a wall-clock (createdAt and updatedAt). If developers implement their storage in a decentralized way as I suggested, they will never run into the issue you mentioned with the Swift SDK or any of the other SDKs.

[dblythy]

Right, you're much more familiar with Swift and POP than I am, so I can't confidently argue either way. It just seems strange to me that updating an object with save fundamentally interacts differently with Parse Server than any other SDK. Creating, deleting objects is great with Parse Swift, but I'm having to create a custom method to update an existing object - a core feature of Parse. As I've said previously, I think this should be a consideration developers should be aware of, especially if they use cloud functions.

The synchronization issue will happen for the clients in the other SDKs, but it won't happen on the Parse Server as with the other sdks, only mutated keys are dirty. When a user calls .fetch in the JS SDK, they will get an object with both keys set by each respective user, whereas in the swift sdk they will only get the second users' save.

[cbaker6]

As I've said previously, I think this should be a consideration developers should be aware of, especially if they use cloud functions.

Agreed, they should definitely be made aware and see the workarounds I mentioned earlier.

[dblythy]

Right, so the consideration here to be aware of, is that if you use Parse Swift, calling .save on an existing object, will always send all keys to be updated, regardless of whether they are mutated? And this can't be avoided internally on the SDK side because of the way that POP works? If this is the case, I think we should mitigate the dirtyKeys on the server to minimise the potential confusion with cloud code, or unpredictable consequences of internal read-only keys becoming modified.

[cbaker6]

If this is the case, I think we should mitigate the dirtyKeys on the server to minimise the potential confusion with cloud code, or unpredictable consequences of internal read-only keys becoming modified.

Im not sure what you mean here. Internal parse keys such as objectId, createdAt, etc. are always skipped when using the Swift SDK (unless when using customObjectId), if they get modified, it’s not because of the Swift SDK. Some of the other Client SDKs send the internal keys and depends on the server to strip or ignore them.

[cbaker6]

I see your server PR, makes sense and my guess is the trade off will be negligible when compared to the DB writes. The server does a lot of key checking already, the only possible improvement I can think of there is if you added your check to a place that was already checking keys to avoid another for loop, but I’m not sure if/where that place exists