While reconstructing my app (WordsBaking), a big challenge is data synchronizing. I’ve spent a lot of time thinking about possible ways, and still not sure if I got the best one.
There are two main problems I am facing.
Physical timestamps can’t be trusted
When the user is online, well we may actually calculate the timestamp based on the difference between client and server that can be somehow trusted at some degree. However the process could be really complex if we want to get every detail right. And if the user is offline, you can never tell the timestamp that can be trusted.
So I have to get rid of it, and choose only to trust the physical timestamps with limited correction (lower and upper bounds).
There are too many records to compare when syncing
For my app, there could be thousands of records. And I can’t just upload all of them every time to check whether their timestamps are smaller than the copies on the server. Another independent timestamp for syncing has to be set for quickly picking up records that might have newer version.
The solution seems to be really obvious, just add another timestamp for sync seeking. The question keeps around in my mind is, do both of the timestamp for seeking and for version comparing necessary?
If they are, the complete synchronizing logic would be:
Client
clientStamp: updated every time after syncing, and the value equals the server time when that syncing happens.
itemUpdateStamp: updated every time when an item gets created/updated/removed, and the value should be greater than the latest clientStamp.
When syncing, client would upload the clientStamp and changed items along with their own itemUpdateStamps. And then it would receive changed items from the server along with the new clientStamp, it won’t receive new itemUpdateStamps because it’s not the device that would compare the records version.
Server
For every record in the database, it will have both the clientStamp and the itemUpdateStamp. When receiving a syncing request, it will first find all the records that have the clientStamp greater than the value it receives or have the same ids, and then comparing itemUpdateStamp with the items it receives respectively if they match the items it receives. If a change from the client has a newer itemUpdateStamp, then update the record in database (with the upper bound of itemUpdateStamp being the current time of the server). Otherwise, send the change to the client.
Seems to be quick clear after writing the logic down. 😉
—
Later I added another feature called passive syncing, thus the client is now able to sync down some related data on demand.