I was working on the offline feature of WordsBaking last month. Considering the data needs to be stored offline may exceed the limit of DOM storage (usually 5 MB as I can find on other websites), I have to pick up a database to handle my needs.
Luckily, there are solutions for different platforms:
- IndexedDB for Internet Explorer (and other browsers built in Android)
- WebSQL for Safari (and other browsers built in Android)
But as WordsBaking runs on both Windows Phone and iOS, I need a wrap-up that adapts to those databases. Here’s what I found:
Lawnchair is an extremely lite wrapper with varieties of database adapters built-in, and basically it’s just switching adapters for us. It seems that several people have contributed to this project because the source file, though not big, is written in different code styles… And this is one of the reasons that why I didn’t choose it.
The second reason that made me give up Lawnchair is that it has only a global handler that handles failure events, which will make fallbacks much more tricky. However, it turned out that I did little fallbacks on database errors in the current version as the errors simply won’t occur in normal situations.
PouchDB as explained on its official site, is a project inspired by CouchDB, and . As I am new to NoSQL (maybe new to SQL too), I didn’t know CouchDB much before. But as I was considering using PouchDB as my local database solution, I dug a little deeper and found that it’s something (as sharing the same API of CouchDB) based on revision, and every update needs a revision as parameter and it’s the core of CouchDB.
Actually I even considered replacing current MongoDB with CouchDB as PouchDB is created for offline apps and syncs with CouchDB automatically, but later I realized that CouchDB doesn’t fit the needs of WordsBaking as it updates the records data all the time, which is not the strength of CouchDB.
So while the syncing doesn’t work for me because I have MongoDB on the server, and there’s a surplus “revision” parameter that needs me to handle, PouchDB is out of my little game.
And here comes the hero, YDN-DB. It seems to be a personal project but it’s full-featured and well documented, and it also provides customized code downloading as you may need only some features of this lib. BTW, I noticed it has syncing APIs, but I didn’t get much time to give it a try.
After integrated it with WordsBaking, everything seems to work fine. But as the records growing, initializing of WordsBaking became really slow. The main reason is that I need to read all the study records (maybe thousands) at the beginning. So I wrote a little class to handle this specific situation, to merge these records into a single one (TypeScript), and it works like a charm.
// Edit:
// The code below is written when I had just a little TypeScript and Promise experience.
// And it doesn't look very well right now.
class DBStore {
name: string;
keyPath: string;
mergeLimit: number;
private merging = false;
private mergeChanged;
private merges: { (): void; }[] = [];
constructor(name: string, keyPath: string, mergeLimit: number) {
this.name = name;
this.keyPath = keyPath;
this.mergeLimit = mergeLimit;
}
values = (callback: { (err, values?: any[], mergePendings?: string[]): void; }) => {
db.values(this.name, null, 1000000)
.then((values: any[]) => {
db.get(this.name + '-single', 'values')
.then((item) => {
var mergePendings: string[] = [];
var hash: { [term: string]: boolean; } = {};
values.forEach((value) => {
var key = value[this.keyPath];
mergePendings.push(key);
hash[key] = true;
});
if (item) {
var svalues: any[] = item.data;
svalues.forEach((value) => {
var key = value[this.keyPath];
if (!hop.call(hash, key)) {
values.push(value);
}
});
}
callback(null, values, mergePendings);
})
.fail((e) => {
callback(e);
});
})
.fail((e) => {
callback(e);
});
};
putOne = (value, callback: { (err): void; }) => {
this.put([value], callback);
};
put = (values: any[], callback: { (err): void; }) => {
if (values.length >= this.mergeLimit) {
this.merge(values, callback);
return;
}
if (this.merging) {
values.forEach((value) => {
var key = value[this.keyPath];
this.mergeChanged[key] = true;
});
}
db.put(this.name, values)
.then(() => {
callback(null);
db.count(this.name)
.then((count) => {
if (count >= this.mergeLimit) {
this.merge();
}
})
.fail((e) => {
alert(e.message);
});
}, (e) => {
callback(e);
})
.fail((e) => {
alert(e.message);
});
};
merge = (newValues?: any[], callback: { (err): void; } = () => { }) => {
var hop = Object.prototype.hasOwnProperty;
var invokeCallback = (err) => {
callback(err);
this.merging = false;
this.mergeChanged = undefined;
var next = this.merges.shift();
if (next) {
next();
}
};
var process = () => {
this.merging = true;
this.mergeChanged = {};
this.values((err, values, mergePendings) => {
if (err) {
callback(err);
return;
}
if (newValues) {
var hash: { [key: string]: boolean; } = {};
newValues.forEach((value) => {
var key = value[this.keyPath];
hash[key] = true;
});
values.forEach((value) => {
var key = value[this.keyPath];
if (!hop.call(hash, key)) {
newValues.push(value);
}
});
values = newValues;
}
db.put(this.name + '-single', {
id: 'values',
data: values
})
.then(() => {
var promise;
var changed = Object.keys(this.mergeChanged);
if (changed.length) {
mergePendings = lodash.difference(mergePendings, changed);
if (mergePendings.length) {
var keys = [];
mergePendings.forEach((id) => {
keys.push(new ydn.db.Key(this.name, id));
});
promise = db.remove(keys);
}
}
else {
promise = db.clear(this.name);
}
promise
.then(() => {
invokeCallback(null);
}, (e) => {
invokeCallback(e);
})
.fail((e) => {
alert(e.message);
});
})
.fail((e) => {
alert(e.message);
});
});
};
if (this.merging) {
this.merges.push(process);
}
else {
process();
}
};
clear = () => {
db.clear([this.name, this.name + '-single']);
};
}