Consider usage of msgpack.object feature for rebalansing #328
Labels
Comments
Gerold103
added a commit
that referenced
this issue
Jun 2, 2022
bucket_recv() used to turn received tuples into Lua tables like all Lua functions do by default when called remotely. But that changed types of some values. The confirmed bug was with the 'varbinary' and 'double' space field types. Varbinary values turned into Lua strings when were received by bucket_recv(). Double values turned into integers during serialization into a tuple if they didn't have a fraction. Some other msgpack types like MP_EXT uuid, decimal, datetime could potentially be a problem too, although I didn't really try to find a reproducer for them. The patch makes bucket_recv() take raw arguments as an msgpack object. It decodes all arguments back to Lua except the tuples which stay msgpack objects and are inserted into spaces exactly like they were stored on the original node. There was an alternative - introduce a new function bucket_recv_raw(), but that would mean bucket_send() needs to handle the case when bucket_recv_raw() is not found and fallback to bucket_recv(). It seemed more complicated that supporting raw args in the regular bucket_recv(). Also there is a problem which appears with any solution which wants to use takes_raw_args. It can not be introduced just on a regular vshard schema version bump. It depends on certain Tarantool core version. It means a user might get, say, vshard 0.1.21 schema and yet stay on Tarantool 1.10 without raw args support. Tarantool core feature set version and basic vshard schema version can progress in parallel. There is no one single version but rather 2 schema versions now. The feature set version is not stored explicitly yet while the feature is just one (takes_raw_args). Closes #327 Closes #328
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Probably it's possible to declare function that receives tuples during rebalancing as
takes_raw_args(tarantool/tarantool@c03f9c7). And after it put directly to space (tarantool/tarantool@ae48d9e).Probably it could speedup rebalancing procedure.
The text was updated successfully, but these errors were encountered: