The proliferation of lightweight web frameworks combined with a multitude of talented web designers and developers has led to an unprecedented number of web-based applications. From balancing our checkbooks to managing large projects at a small business or creative agency, one can find web-based software that does just about everything.
When I’ve been in conversations with other web designers and developers the same question almost always comes up in regards to their software and the framework underneath it… does it scale?
It’s a valid question and one that every developer should be considering even before entering the fray of online software. The framework will only get you so far. After that an intimate knowledge is required of the code base, the database, and how the two relate to your online app. You never know when growth is going to happen and increased usage will almost always reveal cracks and fissures in the stability of your app.
We’ve been developing Intervals, our web-based time, task and project management software, for several years now. And during that time we’ve encountered many scenarios where scalability of our online application was critical. While we’ve depended on the shared knowledge of our development team and many veterans in our field, we’ve also learned a lot by watching how the big boys do it. Facebook, in particular, has been a poster child for learning how to address scalability.
For example, take a look at this presentation summary from “High Performance at a Massic Scale: Lessons Learned at Facebook” from Amin Vahdat, a Professor in Computer Science and Engineering at UC San Diego.
The Intervals development team has certainly benefited from the contributions made by Facebook. Here are some highlights from the article on what we believe web designers and developers should focus on most when scaling their web-based software.
- Site Architecture
Plan out how many servers you will need and leave yourself room to scale. For example, a single dedicated server may be fine for smaller applications, but if you see your app growing beyond the capabilities of one server, start out with an additional dedicated database server. Designing your online software to run on multiple hardware makes scalability less difficult because you can easily add additional servers and a load balancer to the mix. - Memcache
Memcache is perhaps one of the easiest ways to scale an application. Facebook has open sourced their additions to the memcache library which make it versatile and easy to implement. Memcache allows a web-based app to store data in RAM across multiple physical servers, saving trips to the database. Commonly accessed data can be stored in memcache which will decrease load on the database and increase the applications response times. - Faster web server
In addition to new lightweight frameworks there are a number of new web servers with a much smaller footprint than Apache. Faster web servers have been recently built from the ground up to do the minimal amount of lifting required to serve up web pages. Web servers such as Nginx and Lighttpd are two in particular that are gaining traction. Facebook has written a web server called Tornado as well. Consider serving up all or part of your application using these lighter and faster web servers.
That should be enough to get anyone started. There are a lot of other tips in the article, some of which may challenge conventional thinking (database denormalization, for example) on how we design and develop web-based applications. However, when it comes to serving up online software, speed and response times are more important than code prettiness and database neatness.
Most importantly, remember that the web is in a constant state of flux. You are going to make some mistakes and most likely your customers will forgive you for it. Amin paraphrases this well:
…an interesting philosophy from Mark Zuckerberg: “Work fast and don’t be afraid to break things.” Overall, the idea to avoid working cautiously the entire year, delivering rock-solid code, but not much of it. A corollary: if you take the entire site down, it’s not the end of your career.
