Makeflow 标准化协作工具箱

这两年做了一个团队协作产品, 经历了三个大版本, 终于是公测了. https://www.makeflow.com

第一版主要开发于 2017 年 5~11 月, 使用 Angular 2 开发. 第二版迁移至 React, 并重新涉及了 UI. 第三版重写了前后端全部代码, 从 2018 年 6 月左右开发至今 (2019 年 11 月), 团队内部使用也有一年左右了.

不过作为 B 端定位协作标准化的产品, 自身复杂度还是有的, 前些天使用 scc 统计了一下项目代码量 (不含抽离的类库), 还是令人感动.

───────────────────────────────────────────────────────────────────────────────
Language                 Files     Lines   Blanks  Comments     Code Complexity
───────────────────────────────────────────────────────────────────────────────
TypeScript                1319    136801    21118      1961   113722       9493
TypeScript Typings          75      5720      615       735     4370        869
JSON                        54      1307       39         0     1268          0
JavaScript                  36      2293      378        28     1887        106
SVG                         17       973        0         1      972          0
Shell                       12       260       45        13      202         14
Markdown                    10       246       86         0      160          0
HTML                         6       246       15         8      223          0
YAML                         5       453       28         1      424          0
CSS                          4       266       35         0      231          0
Dockerfile                   2        70       23         6       41          2
Docker ignore                1         2        0         1        1          0
gitignore                    1        44       16         9       19          0
───────────────────────────────────────────────────────────────────────────────
Total                     1542    148681    22398      2763   123520      10484
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop $4,245,408
Estimated Schedule Effort 26.576646 months
Estimated People Required 18.922301
───────────────────────────────────────────────────────────────────────────────

不过资本主义的成本统计还是比较夸张.

目前只能说是最基本的部分完成了, 离完全体还差很多, 但作为互联网产品团队协作产品应该是绰绰有余. 如果团队组织者认可标准化的价值, 并且有推动团队标准化的意愿和执行力, 那么 Makeflow 应该会是一个不错的选择.

应用 (app) 的技术周期 – 词焙的重构实践

作为造轮子专业户, 自己的一些小东西不可避免地重写了一次又一次, 不过在应用层面, 我接触到的架构上的重新设计案例并不多. 几个月之前, 我结束了词焙 0.x 版本的开发, 开始重新设计并实现 1.0 版本, 是一次有意思的经历.

应用的技术周期

对于词焙 (web app) 来讲, 应用从最初概念的提出, 需求设计, 代码实现, 到未来逐渐成熟, 我想会经过如下几个阶段.

  1. 最初的需求实现.
    这个过程中, 会有一个针对的架构设计, 这个设计或许满足最初的需求实现, 也考虑了扩展能力, 但由于思维的局限性, 通常会遗留一部分问题.
  2. 不断补充的需求与对应的实现.
    此时, 会暴露一些原有细节实现的问题, 但更棘手的是, 可能会发现原有架构难以满足进一步的改变了.
  3. 架构的重新设计与实现.
    在应用上线一段时间后, 将暴露的问题和新增的需求纳入考量, 重新设计应用架构.
  4. 新架构实现的重构.
    在实现新架构某一部分的过程中, 分析现有技术实现遇到的问题, 并进行这部分实现的重构.
  5. 接受新需求与对应的实现的检验.

目前词焙 1.0 处在第三个阶段, 并且已经发现了一些组件设计中不完善或者不合理的地方, 会在未来进行重构.

新版词焙的架构考虑

老版本的词焙设计之初, 是需要联网使用的, 但由于用户的强烈要求, 不得不将本来为联网使用设计的简单缓存结构硬改 (其实是重写) 为支持离线和同步的结构, 很多实现非常生硬, 难以维护. 另外在 UI 这一块儿, 一开始因为进度需要, 没有做好规划, 触摸手势及相关的事件管理也没有做好统一, 这些都成了新版词焙设计时重点考虑的地方.

使用 TypeScript

前后端全面改用 TypeScript 开发, 静态类型检查与对应的自动完成很能提高开发效率.

基于 Promise (ThenFail)

刚开始接触 Promise 的时候其实有点抵触, 但后来发现上手了也挺好用的. 但本着造轮子的精神, 写了自己的 ThenFail, 使用 TypeScript 开发, 也可以更方便地添加需要的 helper.

离线数据与同步 (Syncable, 面临重写, 未来将整理开源)

除了简单的数据同步外, 词焙中需要一个可以累积的东西以储存类似每天学习时间的数据. 但考虑到未来或许还会处理更复杂的数据合并, Syncable 为这一块儿预留了相关接口. 为了进一步增加该组件的一般性, 还添加了诸如只读数据, 被动 (按需) 同步等特性.

触摸与手势 (Touch Delegate)

上一版词焙实际上也使用了类似 Touch Delegate 的做法, 但是不支持多触点, 并且缺乏终止事件传播的机制 (因为没有统一的管理), 这些是重写这部分的原因. 与此同时, 保留了之前 Gesture Identifier 的模式, 方便手势扩展.

MVC 框架 (Drop, 面临重写)

相对 ThenFail 这么个 Promise 实现来讲,,, 自己搞一套 MVC 框架确实需要点超越时代的轮子精神在里面, 但也算实现了个愿望, 因为还在 2011 年的时候就一直想写这么个框架. 考虑到开发词焙的时间不是特别充裕, 但又耐不住想要写这么个东西, 这次实现的时候调整了原有的模板语法, 力求实现简单, 同时也尽量保留原有设计的精髓 (一般性). 但感觉歪打正着, 现在反倒更喜欢这个实现简单的语法, 而一般性也得到保留 (具体可以参看 Drop 中各种 Decorator 的实现).

不过将 Drop 应用到词焙开发上之后, 还是遇到了不少问题. 其中一些问题通过添加 Decorator (修饰器) 的形式巧妙地解决了, 另一些则还有待框架本身的改进, 比如对象字面量的支持 (实现时为了解析简便, 不允许为嵌套未转义的 { }), 对表达式的解析.

服务器角色的转变和重构

目前来讲, 词焙的服务器基本就是作为数据同步中心存在的 (其实之前改离线之后就已经是这样的功能了). 同时, 新的服务器端脚本也使用了 TypeScript 和 Promise, 结构更加清晰, 实现也更加优雅.

总结

总体而言, 这次架构的重新设计自己还是比较满意的, 也产出了几个实用并且通用的组件. 我想之后会把这些经验和组件打包一下, 做成一套 web app 的解决方案. 当然, 在此之前, 还得完成具体实现的重构, 并且经得住需求的考验.

Understand Visual Studio Tools for Apache Cordova

Recently I turned to Cordova for building a new version of my app, as I am a Visual Studio user, it’s great to get an official tools from Microsoft for Cordova.

Cordova is a tool for building web applications running in native shell. Due to the nature of web apps, and the effort make by Cordova team and its ecosystem, Cordova is also a great tool to build cross-platform apps.

Actually, Cordova also provide a way to develop web apps dedicated to a specific platform. Though that won’t be discussed in this post.

For building cross-platform web apps, refer to http://cordova.apache.org/docs/en/edge/guide_cli_index.md.html

After creating a Cordova project, a “www” folder will be created with some initial files. The major project usually goes here. And there’s also a “platforms” folder that holds the project files for each platforms.

However, once a specific platform is added to your Cordova project, Cordova will leave most of the files as they are. This means that you may actually open a specific platform folder as a project in your IDE, edit and debug them.

Though these “most of the files” does not include cross-platform contents, e.g. contents in folder “www”.

The situation for plugins is quite the same, after the installation of plugins, Cordova will not touch the files that have already been generated, even if they’ve been changed by you.

This is what’s going on with Cordova CLI, however, not Visual Studio Tools for Apache Cordova.

Let’s put the directory structures of original Cordova and VS Tools together and we’ll be able to understand the differences.

Cordova CLI
- project
  - platforms
    - wp8
      - www
  - plugins
  - www
  - config.xml

VS Tools
- project
  - bld
    - Debug
      - platforms
        - wp8
          - www
      - plugins
      - www
      - config.xml
  - plugins
  - config.xml

VS Tools treat the entire project folder as “www” except for some special ones for Cordova. But after all it’s a tool box for Cordova, what’s their connection?

Basically a project defined by Visual Studio Tools for Cordova is not a real Cordova project, but a project that this tool will manage to convert to Cordova projects based on your configuration.

So what you see in bld\Debug\ (or Release) folder is the actual Cordova project.

I mentioned that Cordova won’t touch most of the files once they are generated. This good, but sometimes it will bring problems (e.g. install then uninstall plugins). However, the way Visual Studio Tools use brings also problems, and maybe even more…?

For example the installation of plugins does not support variables, and editing native projects is really painful.

But hoping they will solve these problems soon in the future when it goes to RTM.

Checkout Visual Studio Tools for Apache Cordova (CTP3).

Cordova 微信分享插件 (iOS, WP, Android)

先上 GitHub 链接 https://github.com/vilic/cordova-plugin-wechat

因为词焙需要, 然后现成的没有好用并且平台覆盖全面的, 于是就参考已有的自己写了个.

没有具体去看最低支持到 Cordova 多少, 但 3.x 应该问题不大. PhoneGap 估计也差不多.

支持三大系统, 除了安卓外, 基本不用额外配置, cordova plugin add com.wordsbaking.cordova.wechat –variable APP_ID=[你的AppID] 之后相关配置 (包括 URL Scheme) 都弄好了.

分享的内容只支持文本, 图片和链接. 以后空了再增加其他支持.

安卓需要额外修改一下 plugin.xml, 具体的参见 GitHub 上的说明.

Multi-device Syncing Strategy

While reconstructing my app (WordsBaking), a big challenge is data synchronizing. I’ve spent a lot of time thinking about possible ways, and still not sure if I got the best one.

There are two main problems I am facing.

Physical timestamps can’t be trusted

When the user is online, well we may actually calculate the timestamp based on the difference between client and server that can be somehow trusted at some degree. However the process could be really complex if we want to get every detail right. And if the user is offline, you can never tell the timestamp that can be trusted.

So I have to get rid of it, and choose only to trust the physical timestamps with limited correction (lower and upper bounds).

There are too many records to compare when syncing

For my app, there could be thousands of records. And I can’t just upload all of them every time to check whether their timestamps are smaller than the copies on the server. Another independent timestamp for syncing has to be set for quickly picking up records that might have newer version.

The solution seems to be really obvious, just add another timestamp for sync seeking. The question keeps around in my mind is, do both of the timestamp for seeking and for version comparing necessary?

If they are, the complete synchronizing logic would be:

Client

clientStamp: updated every time after syncing, and the value equals the server time when that syncing happens.
itemUpdateStamp: updated every time when an item gets created/updated/removed, and the value should be greater than the latest clientStamp.

When syncing, client would upload the clientStamp and changed items along with their own itemUpdateStamps. And then it would receive changed items from the server along with the new clientStamp, it won’t receive new itemUpdateStamps because it’s not the device that would compare the records version.

Server

For every record in the database, it will have both the clientStamp and the itemUpdateStamp. When receiving a syncing request, it will first find all the records that have the clientStamp greater than the value it receives or have the same ids, and then comparing itemUpdateStamp with the items it receives respectively if they match the items it receives. If a change from the client has a newer itemUpdateStamp, then update the record in database (with the upper bound of itemUpdateStamp being the current time of the server). Otherwise, send the change to the client.

Seems to be quick clear after writing the logic down. 😉

Later I added another feature called passive syncing, thus the client is now able to sync down some related data on demand.

{{Angular}}, {{Ember}} or {Drop}?

Okay, this post is actually not about comparing these three frameworks. But to introduce my DropJS.

Though actually I can’t compare DropJS to the others, even if I want to… Because it just can’t. But it exists for reasons. For example: why not {single braces} instead of {{double braces}}; why lacking of a gentle escaping notation; why conventions, especially why conventions on strings and letter capitalization…

I had some thoughts on MVVM framework years ago and actually tried to implement DropJS years ago. Sadly I didn’t make it. One of the reasons is that the template style I used was not very friendly to parse (though I do remember I had written a parser successfully, then maybe it was because other reasons). As for Drop, it’s clear and simple. To make everything easier, I come up with an idea called “decorators”. It’s somewhat like the concept of “directives” in Angular based on my understanding (though not very good understanding), but written in a way of {handlebars}, single-brace {handlebars}. 😉

Some code:

<!-- AngularJS -->
<input type="text" ng-model="name" />
<h1>Hello {{name}}!</h1>

<!-- EmberJS -->
{{input type="text" value=name}}
<h1>Hello {{name}}!</h1>

<!-- DropJS -->
{%var name}
{%bind-value name}
<input type="text" />
<h1>Hello {name}!</h1>

I have to say that the code written in Drop is a little bit more than Angular and Ember. But actually {%var name} can be removed if {name} exists in the model. Or you can just write your own {%bind-value} decorator with the ability to define variables under the scope. Because even {%var} is just a normal decorator.

You may want to check out the source code of these decorators here lib/decorators.ts. Yup, Drop is written in TypeScript.

GitHub https://github.com/vilic/drop
Demo https://rawgit.com/vilic/drop/master/demo/index.html

Windows Phone C++ Load System Dll Using LoadLibraryExW

I was trying to write a Windows Phone 8 app using undocumented API. @imbushuo told me that there was an example on xda-developer using complete Windows API on a store app, and it works great.

And then I found a better implementation from this post in Chinese (The approach is actually mentioned in a comment of the post in xda-developer). Though it’s missing some part.

Here is the core code of the implementation mentioned above:

#include "pch.h"

typedef struct _CLIENT_ID {
	PVOID UniqueProcess;
	PVOID UniqueThread;

} CLIENT_ID;

typedef struct _MODULE_LIST_ENTRY {
	struct  _MODULE_LIST_ENTRY* Flink;
	struct  _MODULE_LIST_ENTRY* Blink;
	DWORD* baseAddress;

} MODULE_LIST_ENTRY;

typedef struct _PEB_LDR_DATA {
	//    BYTE fill[0x1c]; x86
	ULONG Length;
	BOOLEAN Initialized;
	PVOID SsHandle;
	LIST_ENTRY InLoadOrderModuleList;
	LIST_ENTRY InMemoryOrderModuleList;
	MODULE_LIST_ENTRY* initModuleList;

} PEB_LDR_DATA;

typedef struct _PEB {
	//    BYTE fill[0x0c]; x86
	BYTE Reserved1[2];
	BYTE BeingDebugged;
	BYTE Reserved2[1];
	PVOID Reserved3[2];
	PEB_LDR_DATA* ldr;

} PEB;

typedef struct _TEB {
	//BYTE fill[0x30]; x86
	NT_TIB nt_tib;
	PVOID EnvironmentPointer;
	CLIENT_ID id;
	PVOID ActiveRpcHandle;
	PVOID ThreadLocalStoragePointer;
	PEB* currentPEB;
} TEB;

typedef HMODULE(*LoadLibraryEx)(
	LPCTSTR lpLibFileName,
	HANDLE hFile,
	DWORD dwFlags
	);

int __cdecl main(::Platform::Array<::Platform::String^>^ args) {
	TEB* teb = NtCurrentTeb();
	HMODULE kernel = (HMODULE) teb->currentPEB->ldr->initModuleList->Flink->baseAddress;
	LoadLibraryEx LoadLibraryExW = (LoadLibraryEx) GetProcAddress(kernel, "LoadLibraryExW");

	// do your jobs...
}

TypeScript or Pure JavaScript?

I was really excited when I got to know that Microsoft had released a new language called TypeScript.

Type, which means better intellisense and type safety that serve the purpose of its existence — scalable.

Why is this important?

  1. Type safety.
    Hate bugs? There’s a joke we talk a lot, which is also a truth: “The weirder the bug is, the stupider mistake we might have made.” And the compiler would now worry about some of these things for us, like typo or forgetting to change a property after copying some code.
  2. Tooling.
    When a project grows, the API might become too complex for the developer himself to remember. Visual Studio provides great experience on JavaScript intellisense by actually running the code in background. But on the one hand, it would usually requires a special rewritten-version of intellisense file. On the other hand, it would become very slow when code lines add up. (I wrote a framework called VEJIS which picks up the powerful intellisense feature of Visual Studio, provides support for classes, interfaces, delegates, etc. But it’s a runtime framework and there’s no type safety. Checkout http://vilic.github.io/vejis/)
    TypeScript makes code navigation much easier and most of the time, we won’t even need to navigate because the information we need is there with intellisense.

Searching articles related to TypeScript would result in many that are comparing it to CoffeeScript and Dart. It’s really weird because some (maybe most) of these articles treats TypeScript the same sort of thing as CoffeeScript simply because both of them compile to JavaScript. And I have even seen an opinion saying: “If you want to build a big application I’d recommend going with CoffeeScript as you end up writing less code.”

WHAT? Maybe that buddy haven’t really met something big enough.

CoffeeScript won’t do things better for people like me, who have been writing JavaScript since day one. People like to talk about the “good part” when it comes to CoffeeScript, but… does it really take more effort for one person who starts with JavaScript to remember how to avoid the bad part than learning a syntactically brand new language?

And actually I have never given the “good or bad part” thing much credit.

So back to the topic, TypeScript or pure JavaScript? Ignoring things other than the languages themselves, I go with TypeScript without hesitate. Because it’s a superset of JavaScript, and it provides much more useful features for larger-scale applications. And, the key point, I write relatively large-scale web applications. However, to use TypeScript in productive environment, there would be more to think about.

  1. Poor IDE and editor plugin support.
    There have been several IDEs with TypeScript support integrated. Visual Studio, of course, would be one of them. But the experience coding using Visual Studio can’t even beat the experience on TypeScript Playground. This really bothers me a lot. No automatic-quote/bracket completion, no snippets, strange indent behavior, etc.
    I haven’t tried TypeScript in WebStorm, maybe it would have done better job. Also ReSharper for Visual Studio may improve the experience according to some comments I’ve seen.
    There are also some plugins for Sublime Text, but… you know.
  2. Poor NPM package and cross-project referencing support.
    Actually TypeScript is capable generating definition files so that it should have been friendly to these things. But the reality is not that awesome…
    I have proposed a convention on distributing TypeScript-written NPM package, and hoping it would make things better.

If finally you choose TypeScript, here’s some information and techniques that might help.

  1. Definitions of popular JavaScript libraries.
    I don’t have any crush on open source, but here open source does it right. So far most of the declarations I need for specific JavaScript libraries can be found in DefinitelyTyped, and it’s becoming of better quality and of larger covering.
    You may install the declarations though NuGet on Visual Studio, to make things work after installation, try to refresh the solution tree.
  2. Temporary “best” practice for cross-project referencing in Visual Studio.
    In project properties page TypeScript Build tab, check “Combine JavaScript output into file” (and specify a file name) and “Generate declaration files”.
    Create a “.d.ts” file in parent project and add the declaration file generated by sub project to it as a reference.
    Visual Studio would compile all “.ts” files included in the project into the JavaScript file specified, so make sure other test files are excluded from the project or you may want to create another project for samples and tests.
    When it comes to NPM packages, I haven’t figure out an acceptable way. But following that convention I mentioned earlier in this post, I may try to write an extension for Visual Studio which would pick up declaration files automatically, wrap it up with ambient module name and add the modified declaration file into the project that’s using that module. It would also be useful even if we are using these packages ourselves without publishing it to npmjs.org only. (BTW, symlink would save you hours if you didn’t know.)

I have now a symptom trying to make everything typed, it slows me down starting up coding a project, but speeds my lines up once the skeleton completes. Hope you enjoy writing in TypeScript if you need it.

(BTW it would be great if TypeScript add support for await/async based on Promise.)