Scalability & Extensibility
Posted by alexander on Oct. 23, 2009, 6 p.m.
So you've built a web site, and eureka! -- you've struck gold. Or maybe you're just growing steadily until one day your server doesn't feel comfortable serving all those people at once. Maybe it's as simple as making the decision internally to go from a basic database system to one with more bells and whistles. In any of these scenarios, you do not want to be stuck with a web site that can't be extended or scaled.
Scalability and Extensibility refer to an application's ability to inherently support changes to the hardware and software on which it depends. I have found over the years that the majority of programmers do not keep these important concepts in mind when designing their systems.
A system that is not extensible will work against you from making even simple upgrades and changes to the way your site works, and one that is not scalable could force you back to the drawing board at a time when you need most to grow.
There are some concerns, like caching or using accelerators, for instance, which will help delay the need to scale. As important as these methods are to a high-performance web site, they will do nothing to help you when your server is about to give up the ghost. In fact, they often make things more complicated for you.
Here are some things to keep in mind when building an application that can stand up to your success:
Use an ORM. An Object Relational Mapper, like WORM, will allow you to change databases without having to rewrite your SQL. In fact, in the case of WORM, it is as easy to change a database as changing a single line of code.
If you use caching, and you should, make certain that you can push that mechanism out to its own server or cluster of servers. Having a web server, database, and cache all on the same machine is fine at first, but be sure that your application will support a cache that resides elsewhere so that when you have 3 web servers instead of just one, you're still able to take advantage of caching no matter which server the request goes to.
Having a built-in timer mechanism will help down the road when you need to begin planning for scaling. A timer will let you know how long each part of the request process takes, so you can easily identify bottlenecks and weak points in your code. Similarly, the Fusionbox framework has a “development” mode that will print out the arc of execution, all SQL queries, and the memory usage for each request at the bottom of the page. This has proven invaluable to fine-tuning and finding redundancies.
Configuration options should be separated out so that you can make changes easily months or years after the system was built.
Generally provide hooks for future growth and planned feature sets. Think ahead.
Some obvious thoughts: make sure your code is object oriented. Always practice Separation of Concerns. Having an application that supports Modules can make adding features later much easier.
Building an application that isn't extensible or doesn't scale well could unexpectedly costing you a great deal of money. Making sure that your programmers keep these principal thoughts in mind will preclude unpleasant surprises down the road.