Database Solutions for Web Apps

I was working on the offline feature of WordsBaking last month. Considering the data needs to be stored offline may exceed the limit of DOM storage (usually 5 MB as I can find on other websites), I have to pick up a database to handle my needs.

Luckily, there are solutions for different platforms:

  • IndexedDB for Internet Explorer (and other browsers built in Android)
  • WebSQL for Safari (and other browsers built in Android)

But as WordsBaking runs on both Windows Phone and iOS, I need a wrap-up that adapts to those databases. Here’s what I found:

Lawnchair is an extremely lite wrapper with varieties of database adapters built-in, and basically it’s just switching adapters for us. It seems that several people have contributed to this project because the source file, though not big, is written in different code styles… And this is one of the reasons that why I didn’t choose it.

The second reason that made me give up Lawnchair is that it has only a global handler that handles failure events, which will make fallbacks much more tricky. However, it turned out that I did little fallbacks on database errors in the current version as the errors simply won’t occur in normal situations.

PouchDB as explained on its official site, is a project inspired by CouchDB, and . As I am new to NoSQL (maybe new to SQL too), I didn’t know CouchDB much before. But as I was considering using PouchDB as my local database solution, I dug a little deeper and found that it’s something (as sharing the same API of CouchDB) based on revision, and every update needs a revision as parameter and it’s the core of CouchDB.

Actually I even considered replacing current MongoDB with CouchDB as PouchDB is created for offline apps and syncs with CouchDB automatically, but later I realized that CouchDB doesn’t fit the needs of WordsBaking as it updates the records data all the time, which is not the strength of CouchDB.

So while the syncing doesn’t work for me because I have MongoDB on the server, and there’s a surplus “revision” parameter that needs me to handle, PouchDB is out of my little game.

And here comes the hero, YDN-DB. It seems to be a personal project but it’s full-featured and well documented, and it also provides customized code downloading as you may need only some features of this lib. BTW, I noticed it has syncing APIs, but I didn’t get much time to give it a try.

After integrated it with WordsBaking, everything seems to work fine. But as the records growing, initializing of WordsBaking became really slow. The main reason is that I need to read all the study records (maybe thousands) at the beginning. So I wrote a little class to handle this specific situation, to merge these records into a single one (TypeScript), and it works like a charm.

// Edit:
// The code below is written when I had just a little TypeScript and Promise experience.
// And it doesn't look very well right now.

class DBStore {
    name: string;
    keyPath: string;
    mergeLimit: number;

    private merging = false;
    private mergeChanged;

    private merges: { (): void; }[] = [];

    constructor(name: string, keyPath: string, mergeLimit: number) {
        this.name = name;
        this.keyPath = keyPath;
        this.mergeLimit = mergeLimit;
    }

    values = (callback: { (err, values?: any[], mergePendings?: string[]): void; }) => {
        db.values(this.name, null, 1000000)
            .then((values: any[]) => {
                db.get(this.name + '-single', 'values')
                    .then((item) => {
                        var mergePendings: string[] = [];
                        var hash: { [term: string]: boolean; } = {};

                        values.forEach((value) => {
                            var key = value[this.keyPath];
                            mergePendings.push(key);
                            hash[key] = true;
                        });

                        if (item) {
                            var svalues: any[] = item.data;

                            svalues.forEach((value) => {
                                var key = value[this.keyPath];
                                if (!hop.call(hash, key)) {
                                    values.push(value);
                                }
                            });
                        }

                        callback(null, values, mergePendings);
                    })
                    .fail((e) => {
                        callback(e);
                    });
            })
            .fail((e) => {
                callback(e);
            });
    };

    putOne = (value, callback: { (err): void; }) => {
        this.put([value], callback);
    };

    put = (values: any[], callback: { (err): void; }) => {
        if (values.length >= this.mergeLimit) {
            this.merge(values, callback);
            return;
        }

        if (this.merging) {
            values.forEach((value) => {
                var key = value[this.keyPath];
                this.mergeChanged[key] = true;
            });
        }

        db.put(this.name, values)
            .then(() => {
                callback(null);

                db.count(this.name)
                    .then((count) => {
                        if (count >= this.mergeLimit) {
                            this.merge();
                        }
                    })
                    .fail((e) => {
                        alert(e.message);
                    });
            }, (e) => {
                callback(e);
            })
            .fail((e) => {
                alert(e.message);
            });
    };

    merge = (newValues?: any[], callback: { (err): void; } = () => { }) => {

        var hop = Object.prototype.hasOwnProperty;

        var invokeCallback = (err) => {
            callback(err);

            this.merging = false;
            this.mergeChanged = undefined;

            var next = this.merges.shift();
            if (next) {
                next();
            }
        };

        var process = () => {
            this.merging = true;
            this.mergeChanged = {};

            this.values((err, values, mergePendings) => {
                if (err) {
                    callback(err);
                    return;
                }

                if (newValues) {
                    var hash: { [key: string]: boolean; } = {};

                    newValues.forEach((value) => {
                        var key = value[this.keyPath];
                        hash[key] = true;
                    });

                    values.forEach((value) => {
                        var key = value[this.keyPath];
                        if (!hop.call(hash, key)) {
                            newValues.push(value);
                        }
                    });

                    values = newValues;
                }

                db.put(this.name + '-single', {
                    id: 'values',
                    data: values
                })
                    .then(() => {
                        var promise;

                        var changed = Object.keys(this.mergeChanged);

                        if (changed.length) {
                            mergePendings = lodash.difference(mergePendings, changed);

                            if (mergePendings.length) {
                                var keys = [];
                                mergePendings.forEach((id) => {
                                    keys.push(new ydn.db.Key(this.name, id));
                                });

                                promise = db.remove(keys);
                            }
                        }
                        else {
                            promise = db.clear(this.name);
                        }

                        promise
                            .then(() => {
                                invokeCallback(null);
                            }, (e) => {
                                invokeCallback(e);
                            })
                            .fail((e) => {
                                alert(e.message);
                            });
                    })
                    .fail((e) => {
                        alert(e.message);
                    });
            });
        };

        if (this.merging) {
            this.merges.push(process);
        }
        else {
            process();
        }

    };

    clear = () => {
        db.clear([this.name, this.name + '-single']);
    };
}

iOS Web Application Directly Using Online URI

Will Apple App Store approve web apps that directly using online URL and resources?

Yes.

When I submitted my app to Apple App Store, I was worrying about a rule that forbidding downloading code in any way or form. And as a web app that downloads its main code only after installation, it does seem to violate this rule. But luckily, it’s finally approved.

Apple App Store http://itunes.apple.com/app/id843170841
See also the Windows Phone version http://www.windowsphone.com/s?appid=a3e70b25-b048-4bef-996f-cc2b30a6eda2

And congrats to myself that WordsBaking is scheduled to be FEATURED in Windows Phone Store (China) tomorrow. 😉

An Underestimated Tool in Windows: HTA (HTML Application)

I can’t find a convinced source telling in which specific date did HTA become available, but it seems to be earlier than the year 2000.

On the official introduction page of HTA, there’s a highlight saying “The Power of Trust”. Unlike normal HTML pages running in a normal browser, HTA can have almost the same permissions that an exe file could have. Which gives it access to varieties of ActiveX that are not available in normal browsers due to security reasons.

I am always wondering why only a few people know and use HTA? Hmm, maybe one of the reasons is its “Microsoft” tag. I have given up the thoughts hoping HTA could be relatively mainstream, however, for developers who use JavaScript and HTML, HTA could still be a good choice to write some lite productive tools for your colleagues and yourself.

NodeJS would certainly have much better ecosystem now but for some specific tasks I still prefer tools with GUI… T-T (Now I also use NodeJS to do some batch work.)

But it was a shame that Microsoft stopped making it better. Wishes.

Disable Scroll Bouncing Effect of WebBrowser Control on Windows Phone 8

Just another story happens when developing WordsBaking.

First, the basis. If you don’t have any element in your web page that requires overflow style being auto or scroll, “-ms-touch-action: none;” under “html” selector should work well . Actually it works all the time in Internet Explorer, but in a WebBrowser control, if there’s something like a list for which you may need that style, the solution becomes tricky.

I spent tons of hours and tried tons of ways hoping figure out a solution, and luckily, found one.

That is the good news, but the bad news is, this solution doesn’t seem to be elegant (though it works perfectly so far).

The first thing that I found might be useful is that this bouncing effect happens only when your finger touches the elements that already have their contents at the top/bottom or both. So, the first solution I thought might work was to add a pointerdown (MSPointerDown) event listener on the element, and determine whether its content has reached the top or bottom. Unfortunately, it doesn’t work well.

Later I read about an article shows a solution on suppressing scrolling and zooming in WP7 WebBrowser control, I can’t say that it works (on WP8), but it helps.

Combining these two parts (and this is why I think it’s not elegant enough), here’s the solution:

C# Part

private void mainBrowser_Loaded(object sender, RoutedEventArgs e) {
    // here we are using a library named Linq to Visual Tree.
    // http://www.scottlogic.com/blog/2010/03/04/linq-to-visual-tree.html
    var border = mainBrowser.Descendants().Last() as Border;
    border.ManipulationDelta += border_ManipulationDelta;
    border.ManipulationCompleted += border_ManipulationCompleted;
}

void border_ManipulationCompleted(object sender, ManipulationCompletedEventArgs e) {
    mainBrowser.InvokeScript("onmanipulationcompleted");
}

void border_ManipulationDelta(object sender, ManipulationDeltaEventArgs e) {
    var status = mainBrowser.InvokeScript("onmanipulationdelta") as string;
    if (status == "top" || status == "both") {
        if (e.DeltaManipulation.Translation.Y > 0) {
            e.Handled = true;
        }
    }
    if (status == "bottom" || status == "both") {
        if (e.DeltaManipulation.Translation.Y < 0) {
            e.Handled = true;
        }
    }
}

JavaScript Part

window.manipulationTarget = null;

window.onmanipulationdelta = function () {
    if (!window.manipulationTarget) {
        return '';
    }

    var target = window.manipulationTarget;

    var top = target.scrollTop == 0;
    var bottom = target.scrollTop + target.clientHeight == target.scrollHeight;

    return top ? bottom ? 'both' : 'top': bottom ? 'bottom' : '';
};

window.onmanipulationcompleted = function () {
    window.manipulationTarget = null;
};

// and you'll need to make calls to every elements
// with overflow auto or scroll with this:
function preventBouncing(ele) {
    // using jQuery.
    ele.on('MSPointerDown pointerdown', function (e) {
        window.manipulationTarget = this;
    });
}

And good luck fellows~

Let's Again Talk About IE (Internet Explorer)

As an amateur but the same time professional front-end engineer, one of the most common things in my life is to press F5 in different browsers.

When this word was not “standardized” as it is now, I was used to pressing that button in several versions of IE, Chrome, Firefox, Safari, and, if I was in a really great mood, Opera. So I hadn’t suffer less than anyone. Especially from IE 6/7.

Now I basically test only in IE 8+ and Chrome. If the project is platform targeted, even better. There has been a lot of scoffs about IE  on compatibility and performance these years, which make me sort of sorry. People would always be fond of the new and tired of the old, even though it is the old one which brings the bright start to all these wonderful things.

IE 6 in my opinion is a great browser. It is out of date but It was well-designed and ignited many features in modern browsers. Actually, tons of newly added features in HTML5 or CSS3 can be imitated in IE 6, a browser which was released 12 years ago. And many earlier IE-only features have become the standards.

The earlier versions of IE is buggy and perform slow, but they perfectly served the days when it was released.

Well I am not trying to ask people not to give it up, but just hoping we can keep the respect, which this browser deserves, to it in its last days.

Microsoft seems to have realized their mistakes on detaining the developing of IE these years and brings us the new IE 9/10/11. Compatibility issues have become minor since version 9 (also it brings us fully GPU acceleration, which is a really important event for web apps), and since version 10, IE finally caught up the steps of modern browsers.

IE now is almost ready for web apps which could be as great as native apps, and I am looking forward to the booming of mobile web apps in the next several years. 🙂

Thoughts after Developing WordsBaking

Several months ago I started my own project called WordsBaking, which is a tool developed for helping students or learners like me to remember new English words. Considering the time and resources I have, I finally decided to use HTML/CCS/JS technologies so that it could be easier to have the versions for the three major mobile platforms.

But there would certainly be some subsequent issues. In last post, I wrote something about performance, gestures and preventing tap highlight. And in this post, I am gonna write some notes about browser bugs.

First of all, I like IE 10 Mobile quite much, comparing to original Android browser. But unfortunately, it still gets some weird bugs.

1. Scrolling issue with CSS3 transform.

I spent a lot of time locating this issue, as scrolling should be one of the most basic features, which in other word should be stable enough. It is a list of which the parent node has the style “overflow” with value “auto”. And the pseudo element “:active” of the list items has a transform with transition: translateX, so that when finger is on the item, it moves a little bit rightwards. Then the problem came, you can’t scroll the list down directly. If you want to do so, you’ll have to scroll up a little first. And, after you scroll to the bottom, you can even continue scrolling up (though the scrolling becomes no longer smooth).

So I have to change the transform to some older ways, like margin, padding, etc.

2. Scrolling issue with phone:WebBrowser control.

Ohh, again scrolling. To use pointer series of events built in IE, I have set -ms-touch-action to none under html selector. So that the page won’t move around or dance with your fingers. It works well until you get some elements with style “overflow: auto;” (or “scroll” I guess). After a list is scrolled to the top or bottom, if you put away and back your finger and continue scrolling, the whole page starts a bouncing effect. Luckily, this only happens in a phone:WebBrowser control. But unfortunately, I am using that control to wrap up my app.

I guess there should be a solution but I really haven’t found it.

Okay, so these are issues with the front end. On back end, I chose NodeJS as I am more confident with my JS skill. But it really took me quite a lot time to start due to the poor documentation. I am not familiar with back end and everyone knows I write front end all the time. I can’t say it would be a challenge to write the back-end stuffs I’ve done these days, but it was a great chance to learn new things.

The main problem I have is lack of database related experience, I don’t know how the db engine works in the box, so I don’t know how would my structures and logic perform. T-T

But back to NodeJS, I am quite missing my VEJIS, which helps JSers coding without looking for online documentations. Oh dear you know how it feels when you press the “.” but there is no accurate auto completion and hints? It might not be that helpful after one is extremely familiar with a framework, but before that, it helps a lot. And one more thing, we are never be that familiar with every new code line we write…

A thousand words are reserved here…

wb

So, I am very excited as WordsBaking for WP is almost ready for beta, after months. I always believe the power of a team, but when a team is not available, I also believe my own skills. Thanks my CTO (Chief Track Officer) for Tracking my developing progress btw, or I wouldn’t be able to finish the beta version. 🙂

Some Thoughts on Touch-friendly Mobile Web Apps

These days I am working on a project related to English study as my startup (though part-time), due to the limited resources, I chose a native browser based solution, So that we’ll save some time for building only the single main part of the app which will run on multiple devices.

However, the app I want to build has to be user-friendly, and for an HTML based app on a mobile device, which means, relatively poorer performance, to make the user experience acceptable forms a challenge. Also, as the UX we designed, there would be some touch gestures, how to manage these gestures is also a core issue to figure out.

Luckily, both of these two issues are not tricky.

There are some transition animations during page switching, elements removing, etc. I tried to use something like margin (CSS), but it doesn’t work that well, the animation doesn’t look smooth. So I made some searches, an article says CSS 3 transform has even slower performance comparing to CSS properties like margin and the position stuffs. I believed it… But it turned to be wrong. After I randomly changed the page switching from margin to CSS 3 transform + transition, the animation runs like a charm! I can’t say it’s as good as native apps, but it’s far from acceptable.

I am using Lumia 1020 with IE 10 Mobile myself, and get a Xiaomi 2S with its built-in browser and Chrome Mobile. These browsers all perform well, especially IE 10 and Chrome Mobile. Also, I tested it on an iPhone 5 of my friend, Safari Mobile does a great job, too. That’s actually not a surprise, I wrote some web games in 2010 runs on my iPod touch 3 32G using CSS 3 animations, even with some 3D stuffs, and the game performed well.

As for gestures management, I wrote two classes respectively called GestureIdentifier and GestureDelegate and so far they works well. The idea is really simple but it really makes things much easier, hoping telling this will help some starters learn about the power of ideas which is far above code itself.

And also some quick notes:

About “meta viewport”, seems that some phones have a minimum valid width of 360 instead of 320, so any value below 360 will not work… T-T gotta make some changes to fit this new value…

About click and “tap”. Seems that if these is a click event (especially on an “a” tag), these will be a semi-transparent rectangle when you “tap”. But sometimes, it doesn’t look well… Luckily “tap” based on touch event would be a solution… As I got the two gesture classes, I can simply define a new GestureIdentifier called “tap” and write a simple rule like the path of touch has only one point. Solved.

Wishing this tool will soon be available to everyone!

[Update]

The touch event trick for preventing tap highlight works most of the time, but on IE Mobile there are still some conditions on which the semi-transparent rectangle (tap highlight) will show up, so I continued searching for solutions. Luckily, there are way better and more official ones:

For IE, there’s a meta item named “msapplication-tap-highlight”, and simply setting the content to “no” solves the problem.

<meta name="msapplication-tap-highlight" content="no" />

For Webkit (Chrome and Safari), there’s a style named “-webkit-tap-highlight-color”, and “transparent” is the answer.

html { -webkit-tap-highlight-color: transparent; }

C# Refresh System Tray Icons (Remove the dead ones)

I was writing a small tool to restart a related application when it seems to stop working. As I don’t want to spend much time on this thing, I use Process.Kill to do that part of job. But after that, I will need to clean the notification area and remove the dead system tray icons.

I made searches and found one working great on my computer running English Windows 7.

http://maruf-dotnetdeveloper.blogspot.com/2012/08/c-refreshing-system-tray-icon.html

However, when I copied it to the server, the code was just doing nothing. As I am not familiar with Windows API and its programming, it took me a long time to realize the problem. Though the window titles such as “User Promoted Notification Area” are not visible, it’s been translated on the non-English Windows OS.

So I tried Spy++ (which I am not familiar with, either), and found out the Chinese ones for those strings.

“User Promoted Notification Area” => “用户升级的通知区域”
“Overflow Notification Area” => “溢出通知区域”

I didn’t find “Notification Area” one, and guess it’s on Windows XP. But I think it should be translated into “通知区域”.

And, it now works like a charm. 😉

Windows 8 Earlier and Recent Start Screen Color Comparison

Every time when comes to Windows 8, I feel sad… The reason is simple, I want the latest version, but I feel better with the earlier one. So even though I was the first group of people out there who started trying Windows 8 since Developer Preview, then Customer Preview, Release Preview, and RTM. But finally I degraded (several times) to Windows 7, which is my favorite so far.

Some people might think I am not a fan of Metro UI, but they are wrong. I am a big fan of that, and I started using Windows Phone since Mango (7.5) is still in test version. I love Metro, but as well as Aero. I have posted about this and wrote Windows 7 along with VS 2010, Office 2010 etc, as well as Windows Phone 7.5 are the highest level Microsoft had reached on UI design (of course in my personal opinion).

It was not my plan to write this post actually. I made a comparison image of the start screens on early Windows 8 concept image and Windows 8 released to public a long time ago, and I have never thought that I might be wrong. Here’s the image:

win8

I like the earlier one (which in my opinion uses more reasonable colors) much more than the released version. No matter the color gradient or hue. But I was so confident and thought everyone will agree with me, until I sent this image in a group chat and get punched just now.

I always think I am person gifted on many, including some aesthetic feelings, and I don’t think this is negotiable. So I begin to wondering, how do most of the people judge the beauty of everything? One word I heard today was bright. So I started to thinking about all the facts that might bring visual impact just like bright colors, but may need more time to figure this out.

As far as I know, most of people just don’t care about the arrangement of colors. For Windows Phone users, a messy start screen is a proof. Also, most of them don’t care about the proportions of the elements (including text elements and images, no need to mention colors), and a proper margin between them just doesn’t matter to most of the people either, if it doesn’t make the features harder to use.

It is good to find that many IT industry leaders in China still have the first class designers so far, like Tencent, Alibaba. Actually even many start-up companies do. But what happened to companies like Microsoft and Apple? I still remember one of the reasons why I kind of liked Microsoft was that they had the best designers…

One of the reasons comes to me is, they might have valued the marketing researchers too much, and letting them manipulate designers and developers. Or maybe a much simpler answer, they just have lost the best people they had and that’s all.

HowTo: Add Facebook Account on a China Country Variant Lumia

Before we start, there’s something you may need to know:
1. It has to be Windows Phone 8.
2. Twitter account can’t be added this way.
3. When come to flashing ROM, I refer the WCDMA model (for China Unicom).

And the technique here is simple, though not very easy to be done. We’ll need to add the Facebook account somewhere else first, backup the settings and then restore it on the China Country Variant Windows Phone.

I was lucky because I got a Hong Kong CV Lumia 920, so I simply restored the backup data on my new Lumia 1020 and the Facebook account showed up. However for other people who don’t have an older Windows Phone runing WP8, they won’t be able to add that account at the first place. Luckily, we get Vietnam CV Lumias sharing the same models (you can check the RM-xxx code to see if that really matches) as Lumias for Mainland of China. So we can flash the Vietnam ROM and then get Facebook account added.

Actually if you are cool with that Vietnam ROM, you can just get off the bus here and enjoy the social network, or you may want to flash back ROM for China as what I would do.

Some photos here by Lumia 1020.

NOKIA Pro Camera – Auto mode:

WP_20130820_18_37_05_Pro WP_20130820_19_26_27_Pro WP_20130817_18_30_22_Pro

NOKIA Pro Camera – ISO 100 / 0.8 seconds:

WP_20130819_21_02_20_Pro

And a bonus by Lumia 920 (Edited by Fireworks CS6).

1020-small