Notes:
h1. Stella - Your Performance Testing Friend
Release: 0.5.5 (2009-01-14)
This is a BETA release. Proceed in double verify mode!
h2. Overview
Stella is a command-line tool for running, reporting, and analyzing performance tests.
h2. Prerequisites
* Linux, BSD, Solaris
* Ruby 1.8.x, 1.9.x, or JRuby 1.1
* Native Libraries (c or c++)
** Pcap (optional, for packet sniffing)
* Ruby Libraries
** rspec
** net-dns
** Ruby-Pcap (optional, for packet sniffing)
* One of:
** "Apache Bench":http://httpd.apache.org/docs/2.2/programs/ab.html
** "Siege":http://www.joedog.org/index/siege-home
** "Httperf":http://www.hpl.hp.com/research/linux/httperf/
*** _Note: Argument handling for Httperf is incomplete_
h2. Installation
Get it in one of the following ways:
* RubyForge: http://stella.rubyforge.org/
** gem install stella
* git clone git://github.com/solutious/stella.git
** git submodule init
** git submodule update
Use ab, siege, and httperf like you normally would with the addition of stella at the beginning (examples are below).
h3. Debian (and derivatives)
Debian and its derivative (Ubunutu) handling packing a bit differently (see: http://pkg-ruby-extras.alioth.debian.org/rubygems.html). There are a couple errors to watch out for during the installation. The solutions are below:
"no such file to load -- mkmf (LoadError)"
$ apt-get install ruby1.8-dev
"ERROR: RDoc documentation generator not installed!"
$ apt-get install rdoc
h3. Installing libpcap and Ruby-Pcap
*Stella currently supports packet filtering with Native Ruby (no JRuby) on Linux/Unix/OS X only.*
*Ruby-Pcap does not compile with Ruby 1.9.1-RC1*
libpcap and Ruby-Pcap are used to by Stella's watch command to filter packets off the network. This allows you to view HTTP requests and response (including headers and content) without having to configure a proxy. Both need to be compiled and installed separately:
* "libpcap":http://www.tcpdump.org/release/
* "Ruby-Pcap":http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html
On Ruby, 1.8.6, Ruby-Pcap produces the following warnings at runtime: "pcap.bundle: warning: do not use Fixnums as Symbols". See: support/ruby-pcap-takuma-notes.txt.
h2. Usage Examples
Run Apache Bench with a warmup and rampup from 10 to 20 virtual users in increments of 5
$ stella --warmup=0.5 --rampup=5,20 ab -c 10 -n 50 http://stellaaahhhh.com/search?term=trooper
Run Siege, repeat the test 5 times. Automatically creates a summary averages and standard deviations.
$ stella --agent=ff-3-osx --testruns=5 siege -c 10 -r 5 -b http://stellaaahhhh.com/search?term=flock+of+seagulls
Run Httperf like you normally would (but all the test data will be collected for you)
$ stella httperf --hog --client=0/1 --server=127.0.0.1 --port=5600 --uri=/ --rate=50 --num-conns=100 --timeout=5
Monitor DNS traffic using Pcap
$ stella watch -C dns
Monitor HTTP traffic using Pcap
$ stella watch -C http
Monitor HTTP traffic using a proxy (don't forget to configure your browser)
$ stella watch -P
h3. Sample Output
<pre><code>
$ stella -f csv -x 5 -w 0.75 -r 25,125 -m "httpd 2.2.9-prefork" siege -c 75 -r 10 -b http://testapp:5600/
Writing test data to: stella/testruns/2008-12-23/test-054
Warmup: 3750@37/1 100% 264.29/s 0.140s 0.024MB/s 0.340MB 14.000s .
-------------------------------------------------------------------
REQ@VU/s AVAIL REQ/s RTIME DATA/s DATA TIME
Run 01: 7500@75/1 100% 345.30/s 0.210s 0.032MB/s 0.690MB 21.720s
Run 02: 7500@75/1 100% 360.58/s 0.200s 0.033MB/s 0.690MB 20.800s
Run 03: 7500@75/1 100% 359.02/s 0.210s 0.033MB/s 0.690MB 20.890s
-------------------------------------------------------------------
Total: 22500@73 100% 354.97/s 0.207s 0.033MB/s 2.070MB 63.410s
Std Dev: 6.86/s 0.005s 0.001MB/s 0.414s
Run 04: 10000@100/1 100% 384.47/s 0.260s 0.035MB/s 0.920MB 26.010s
Run 05: 10000@100/1 100% 385.06/s 0.260s 0.035MB/s 0.920MB 25.970s
Run 06: 10000@100/1 100% 380.95/s 0.260s 0.035MB/s 0.920MB 26.250s
-------------------------------------------------------------------
Total: 30000@98 100% 383.49/s 0.260s 0.035MB/s 2.760MB 78.230s
Std Dev: 1.81/s 0.000s 0.000MB/s 0.124s
Run 07: 12500@125/1 100% 397.20/s 0.310s 0.036MB/s 1.140MB 31.470s
Run 08: 12500@125/1 100% 397.08/s 0.310s 0.036MB/s 1.140MB 31.480s
Run 09: 12500@125/1 100% 397.58/s 0.310s 0.036MB/s 1.140MB 31.440s
-------------------------------------------------------------------
Total: 37500@123 100% 397.29/s 0.310s 0.036MB/s 3.420MB 94.390s
Std Dev: 0.21/s 0.000s 0.000MB/s 0.017s
-------------------------------------------------------------------
Total: 90000@98 100% 378.58/s 0.259s 0.035MB/s 8.250MB 236.030s
Std Dev: 18.09/s 0.042s 0.002MB/s 4.225s
</pre></code>
All test data is collected under ./stella (this can be changed with the parameter --datapath):
$ ls -l ./stella/testruns/2008-12-23/
test-001 test-002 test-003 test-004 test-005 test-006 ... test-054
A symbolic link points to the most recent test:
$ ls -l ./stella/latest/
ID.txt MESSAGE.txt SUMMARY.csv run01 run02 run03 run04 run05 warmup
Each run directory contains all associated data, including the command and configuration
$ ls -l ./stella/latest/run01/
COMMAND.txt STDOUT.txt siege.log STDERR.txt SUMMARY.csv siegerc
h2. Known Issues
* The output for the REQ@VU/s columns is a work in progress. It's not aligned across tools and it will likely change in the next release.
* The summary data has not been audited. Don't trust and double verify!
* httperf is functional but not all parameters are supported.
* The Ruby API has not been finalized. It's functional but there's no example because it is subject to change.
* There is an issue with parsing ipv6 ip addresses with Ruby-Pcap or Net-DNS. This affects only dns sniffing with pcap. Reproduce by running "stella watch -C dns" and requesting "apple.com"
h2. Report an issue
Email issues and bugs to stella@solutious.com
h2. More Information
"Stellaaahhhh":http://www.youtube.com/watch?v=wmq-JDonTpc
h2. License
See LICENSE.txt
Changes:
STELLA, CHANGES
#### 0.5.5 (2009-01-14) ###############################
http://github.com/solutious/stella/tree/0.5.5
* Now has specs (33 specifications, 303 requirements)
* New dependency: test-spec (development copy only)
* Added delano/drydock to vendor (was delano/frylock).
* Improved argument handling for load tool adapters (httperf not complete)
* Re-built Stella::Storable and subclasses (now using "field" method)
* Test::Stats, Test::Run::Summary cleanup
* Fixed RubyGems require error
* Fixed stddev bug by removing weak math, adding stats class from Mongrel
* Fixed --agent error
#### 0.5.4-alpha (2009-01-07) #########################
http://github.com/solutious/stella/tree/0.5.4-alpha
* New command: watch. Monitor HTTP traffic with Ruby-Pcap (unix only) or proxy. Pcap also supports DNS sniffing.
* Improved error handling for stats output
* Now runs on Ruby 1.9.1, JRuby
* Added Windows support for most features
* Added patch for Ruby-Pcap
* Added friendly error handling for Apache Bench, Siege, and Httperf
#### 0.5.3-preview (2008-12-23) #######################
http://github.com/solutious/stella/tree/e4d2bd8db3d1323fbc3b8bbb7f703ce7b8c88551
* Initial public release
|
