Synthetic Monitoring
In my last post, I covered why the cloud represents a golden opportunity to perform capacity planning. Although the cloud provides an ideal environment for capacity planning, don’t make the mistake of viewing it through rose-colored glasses. You still have to make sure you have a proper test harness set up and the right testing methodology enabled. In other words, you have to test the right things, in the right way, with the right tools.
Let me explain where I got my reasoning for this. In his first book, “The Macintosh Way,” Apple evangelist, venture capitalist, and now Internet impresario Guy Kawasaki introduced a grid that defines the four ways that anything can be done in business. You can do the right thing the wrong way, the wrong thing the right way, the wrong thing the wrong way, and the right thing the right way. When it comes to testing in the cloud, Kawasaki’s decision grid applies perfectly. This is because in cloud testing, as in marketing, there are only four ways to do things.
What I’d like to talk to you about today is how to perform the right test, the right way, and it starts with the right tools.
The right tools
Obviously, this means choosing an appropriate testing tool (if you plan on executing the test yourself), or employing the right testing service that’s the right fit for you. One key feature you need in your load testing tool is the ability to mimic the exact same load over and over again, so it’s easy to do regression or A and B testing. With the right tool, you can test with two web servers to find how many page views can be handled per minute. Then you can continue to test with three, four, five, or more web servers until you can see how many page views per minute the system can handle.
Your load testing tool should also show you the whole picture, both the inside and the outside of the firewall as you stress the servers. The test on the outside needs to be kept exactly the same in terms of the geographic location of the traffic, and show the exact same screen that any user would see. From the start, the tool should simultaneously be monitoring the servers on the inside of the firewall to see where you’re utilizing the most resources performance-wise.
The right things
One of the most important things to be testing is the server. You should be testing it until you can’t get any more out of it. Hit the limit on a server, whether that is transactions per second, page views per minute, or megabits per second. When you’ve reached that maximum level on the server, take it even further to see how it degrades. As you add load to a maxed-out server, you want to see a linear increase in response time. This means the system is healthy, even though it is beyond its maximum capacity.
The right way
The length of the test is one of the most important factors in capacity planning. At Apica, we perform bell tests over a 24-hour period to identify any stability issues (such as a memory leak) within a system. After 24 hours, we look at the memory consumption. If all goes well when we close the test, everything should go back to normal and the application shouldn’t have eaten up all the server memory.
Thanks to the cloud, you have myriad tests at your disposal to help you determine what sort of usage your system can handle. If you use the right tests to determine what your limits are, you can create a road map to abide by for future usage.