Why we need a NodeJS application server or: writing boilerplate code considered harmful

[Update] A lot of commentators have taken issue with my use of the term “application server”, which made them think of a big monolithic architecture. Even though I do refer to Java, that’s not what I mean. It was the wrong choice of terminology – I am  thinking of a modular, lightweight solution that saves you from implementing the most generic parts of the typical web application. For a better writeup with the same cause, see this post on the hood.ie blog.[/Update]

I love creating web applications, but I don’t do it for a living. I code because I like the challenge of it, but first of all: I want to solve specific problems. When I was writing my dissertation in Political Science, I collected thousands of bibliographic references, some of which I wanted to share with others. This was a time long before Zotero came into existence and there was no decent software that let me do that the way I wanted it. So I started to dabble in PHP, later in JavaScript libraries (in particular, qooxdoo), and out of this came, in several iterations, my pet project Bibliograph. Later, when I was coordinating the organization of a big conference, I had to manage a staff of 50 people working in shifts. The wasn’t any freely available software that was up to the task, so I wrote a qooxdoo-based, interactive staff scheduling web app. I won’t publish the code because it is a mess, but it worked well for the task at hand. Besides my ongoing urge to rewrite Bibliograph to make use of the latest web technologies, I have lots of other ideas for web applications to deal with that odd challenge that I come across occasionally.

There is, however, one big issue that holds me back from starting to work on any of those ideas: the lack of the right end-to-end stack including a decent application server for NodeJS, which will free me from having to worry about boilerplate code for things like authentication, user management, access control, etc.

Professional developers might argue that one should write this kind of code oneself, so that one knows the code inside out and has full control, and can then adapt it for each specific job. If you are not ready to write the boilerplate, they might argue, you should maybe not be in the business of writing applications in the first place.

I respectfully disagree. On the contrary, I think that it is wasteful, if not outright dangerous, for specialized, part-time application developers to implement their own backends from scratch. For Bibliograph, I developed a complete PHP solution. It was an interesting experience, from which I learned a lot. However, it proved impossible for me to maintain it beyond the time that I was actively working on the main application. Later, busy with the things I had to do for my real job, I couldn’t react fast when bugs were discovered in the PHP version I used. Having published the backend code, with people actually using it, I couldn’t respond to questions they had or bugs they discovered in the code. In the first place, I had invested countless of hours in reinventing the wheel, with no real benefit.

I should be using the resources in my disposal efficiently, i.e. the very specific ideas and knowledge of particular technologies that solve specialized problems (such as bibliographic data management for the humanities). I shouldn’t spend my time worrying about how to store passwords securely, or how to synchronize data between the client and the server, or how exactly model data is persisted in what database. There are quite a few tasks that can be solved in very generic ways, in order to provide a platform on top of which specialized applications can be built. This issue also pertains to testing: I should be writing tests for the main business logic, and not spend valuable time on tests for boilerplate code.

I think such a platform would not only save thousands (or more) of hours of developers’ time. This time could then be invested instead in solving the application-specific problems in thorough and creative ways. Such a collaboratively developed platform would make the applications built on top of it more secure and stable (because of collective wisdom, test coverage, and quick fixing of bugs). It would generate enough interest in its long-term maintenance.

There is a wealth of JavaScript frameworks on the client and on the server side, but, as far as I can see, no such thing as an “application server” known from the Java world yet exists. Plenty of libraries exist that can be wired together, but this wiring itself is not trivial, as I learned while writing a tutorial on exactly this topic. The large number of libraries to choose from is both a blessing and a curse – the freedom to choose comes with a price – we need more and more time to research which choices should be made, and the multiplication of APIs and protocols also adds to the expense. How neat would it be if we could agree on a meta-API for the most commonly needed features of a web application and then shims could be used to plug in the different solutions that currently exist.

I see a number of features and characteristics of such an application server (most, if not all of them provided by already existing modules):

• It should not assume any particular rendering model on the client (DOM-templating vs. widget objects)
• It should also not force a different programming model on developers (like Opa or Meteor), but let you code in plain old asynchronous javascript. No to-javascript-compilation (except optionally). No magic. Just the right tools.
  
• It should have an integrated API for client and server.
• It should provide a static HTTP server, REST routing, and bidirectional, realtime messaging and broadcasting (such as Socket.io).
• It should offer async startup/plugin/configuration system like Cloud9’s Architect.
• It should provide an out-of-the box system and API for user & group management, registration, access control, Password storage/retrieval/update etc, preferrably with a set of built-in templates that can be used for managing the most generic configuration tasks. With this, a pluggable system to use third-party authentication providers.
• It should also provide an integrated system of data modeling and persistence. I really do not care about database technology. I simply want to store, edit and retrieve my model data.
• It could also have a toolset that would allow you to deploy your application instantly to a cloud provider such as Heroku or Nodejitsu.

The existence of such an application server, I would argue, would unleash a lot of currently unused creative energy and lead to the development of a lot of interesting special-purpose applications by people who like to code to solve specific problems, but who cannot afford to invest the time to develop full-fledged application backends. It would be great to see such a project take shape, and I am sure others like me would gladly contribute small bits and pieces. Or does it exist already and I have overlooked something?