For some time now I've been playing with Grinder as a load-testing tool. There was much to like about Grinder - it seemed like one of the more advanced open-source load/capacity testers. The scripting language is Jython, which I love as a result of long being a fan of Python. Still, it is clearly underpowered compared to commercial tools. Then, one day when I was visiting Open Source Testing, I ran across WebLOAD and I knew that I was on to something special.
WebLOAD claims to contain over 250 man-years of development. It was converted to open source in April 2007. It was created by Ilan Kinreich, a co-founder of Mercury Interactive. In my opinion, this is a commercial quality application. One area that really shows is the documentation. After trying to piece together how some open-source tools work by reading forum entries, having real documentation is a godsend.
Getting Started
WebLOAD has an excellent Quick Start guide. I highly recommend starting there. The basic steps for running a load test are:
- Create an Agenda
- Configure a load template
- Configure session options
- Run the test
- Analyze the results
Creating agendas are easy. WebLOAD comes with a proxy recorder. You simply start the recorder and execute a user scenario in your browser. The proxy is set up automatically - no manual steps in your browser options. The commands are stored as Javascript. I really appreciate not having to learn yet another proprietary language.
The recorder also has playback and debug capabilities. It is really nice to get your agenda working cleanly before running 1,000 concurrent versions of it!
Configure a Load Template
A load template is when you actually configure the test to be run. What agenda(s) will be used? What statistics will be collected? How many simultaneous users will you simulate? Will they hit the site all at once or ramp up linearly? What host or hosts will provide the virtual clients?
All of this is done from the main WebLOAD console, not the IDE that you used to record the Agenda.
I highly recommend using the wizard that runs through the configuration options to set up the template.
Setting Session Options
Under Tools > Current Session Options, there are a large number of controls you can set, like the maximum time it can take for a page to return before it is considered a failure. In the Quick Start guide, they set a 20-second maximum time and verify 5% of the sessions.
Run the Test
From the Console, you just click the the Start Session icon and you are under way.
I love watching the various stats displaying in real time. Very cool! When running a stress test, monitoring the log view gives a good indication of when things start to break down.
Analyze the Results
WebLOAD has the ability to graph any number of parameters and statistics, including things like CPU and disk utilization on the server(s). It also supports export of data to popular formats such as Excel.
Hurdles
I did have to overcome several problems before WebLOAD would work with the main application I test. The first is that my application uses URL rewriting to track user sessions. So any URL recorded with the IDE will be incorrect at testing time, since each session is assigned a new sessionid which is part of the URL. This is where WebLOAD's professional documentation comes in again. On their forum, a white paper called "Session Management in Performance Testing [How-to]" describes handling URL re-writing along with cookies and hidden-form fields. You've got to love professional documentation!
The second issue I haven't figured out yet is why I was not able to record from our test site. I had to build sessions from our production site, then went in and modified the URLs to reference our test server. I'll let you know what I figure out with this one.
Gotchas
I learned a lesson the hard way. Do not run a stress test on your production system during business hours! (Notice issue number two in "Hurdles.") I managed to run an Agenda that hadn't been converted to the testing server's URLs. Ouch. It brought down the server at one point. Not a good thing to be doing at 10 AM! There is one good thing that came from this - it convinced me that WebLOAD was actually doing something. It's not just a program that draws pretty graphs. ;-)
Summary
Indeed, I am impressed with WebLOAD. It comes closest to an "ideal" load-testing solution than anything I've seen in the open source world. I especially like the professional quality documentation. With most open-source load testers, I give up in frustration before I ever get anything working. With WebLOAD, I had things working in a morning.
Perhaps others who have experience with commercial tools such as HP's LoadRunner will chime in on how it compares to the "big boys".
No comments:
Post a Comment