[rspec-users] How much details is enough?

Pat Maddox pergesu at gmail.com
Fri Apr 18 05:47:58 EDT 2008


It seems like I'm constantly making long-winded replies that would be
better off in a blog post or in a book.

Pat


On Fri, Apr 18, 2008 at 2:44 AM, Chris Parsons
<chris at edendevelopment.co.uk> wrote:
> Very nice reply Pat. This would make a great blog post if you get a chance.
>
> Thanks
> Chris
>
>
>
> On 18 Apr 2008, at 10:15, Pat Maddox wrote:
>
>
> Hey Matt,
>
> The ultimate test would be one that is focused on one thing such that
> the test would
>   - break every time that thing broke
>   - break only when that thing broke
>   - give detailed feedback enabling you to focus on the thing's
> subpart necessary to identify and fix the problem
>
>   The ultimate test suite would be the set of such tests that covered
> every single concern that exists in a project.
>
>   Some of these are concerns are easier to test than others.  Some are
> important and lend themselves to automation, thus enjoy great tool
> support.  Others take something far more abstract like a person's
> aesthetic appeal.
>
>   When considering a new test, I should ask myself what problem the
> test solves, and what problem I really want the test to solve.  I try
> to write the test in terms of the second.  For example, if I were to
> write the following test:
>
>   it "should allow deposits and withdrawals" do
>     @account.deposit 80
>     @account.withdraw 25
>     @account.balance.should == 55
>   end
>
> The test would be valuable when the problem is that you need to track
> how much money people have in their accounts.
>
> If you faced a problem such as "Make sure the user sees an error when
> they withdraw more than their balance," you would not want a test
> like:
>
>   it "should have an overdraft error" do
>     post :withdrawals, :amount => 1_000_000_000_000 # even bill can't do
> this!
>     assigns[:withdrawal].should have_at_least(1).error_on(:amount)
>   end
>
> If this test breaks at some point, we would make a change to the test
> or production code in order to make it pass. We wouldn't think of the
> problem it was intended to solve though, because we are absorbed in
> the problem that initiated the break-inducing change.  The danger in
> this is that the test appears to be robustly covering something
> useful, but in effect can let problems leak through.  Imagine that we
> remove an important partial from a view.  This test, whose ultimate
> goal was to ensure that users see an error message, failed to catch
> the problem where the message wasn't displayed at all.
>
> So you should write tests expressing the same level abstraction as the
> problem you want them to solve.  Somewhere you would need a test like:
>
>   it "should have an overdraft error" do
>     @page.should include("Can't withdraw more than your balance")
>   end
>
> Now you would need some way to get @page.  Perhaps you make a request
> to a controller, it hits a database, renders a response, and assigns
> the response body to @page.
>
> Or maybe you rendered a view using a fake @withdrawal, and you've got
> another test that verifies that you assign @withdrawal in the
> controller, and another test verifies that Withdrawal objects get an
> error when you try to create them for an amount greater than the
> target account balance.  And you've got a test that you label an
> Integration test to signal the fact that it integrates all of these
> pieces.
>
> You should only write integration tests that check across valuable
> boundaries.  This does not restrict it to stuff like company-specific
> code using an ORM framework, though.  Because you should only write
> tests that are valuable, sets of layered tests forms a small subsystem
> requiring integration testing.
>
> It is sometimes useful to have layers of tests that enable you to
> localize problems.  Other times the types of problems you solve will
> be trivial or obvious and won't require localization.
>
> As a simple rule, more tests == more overhead.  But if you're missing
> certain tests then you will not notice certain problems when they
> appear.  The art of all of this is identifying the set of tests that
> maximizes your confidence and ability to produce valuable software.
>
> With all that theory out of the way, what can we say about the tests
> you presented?
>
>
>  describe FeedsController, 'get /feeds/' do
>    before(:each) do
>      @request.env["HTTP_ACCEPT"] = "application/xml"
>      Model.should_receive(:find).with(any_args).and_return
>  mock_model(Model)
>    end
>
>    it "should return success" do
>      get '/'
>      response.should be_success
>    end
>
>    it "should return 405 (Method Not Allowed) if HTTP_ACCEPT is text/
>  html" do
>      @request.env["HTTP_ACCEPT"] = "text/html"
>      get '/'
>      response.response_code.should == 405
>    end
>  end
>
> This test would be good in a situation where we had published an API
> stating only XML requests were allowed.
>
>
>  The second one is much more detailed:
>
>  describe FeedsController, 'get /feeds/' do
>    before(:each) do
>      @model = mock_model(Model)
>      @request.env["HTTP_ACCEPT"] = "application/xml"
>      Model.should_receive(:find).with(any_args).and_return @model
>    end
>
>    it "should assign to the model" do
>      get '/'
>      assigns[:model].should == model
>    end
>
>    it "should render feed template" do
>      get '/'
>      response.should render_template('feeds/model_feed.xml.erb')
>    end
>  end
>
> This test would be valuable in a context where the XML feed output is
> complex.  In that case, testing the output directly might not
> sufficiently enable us to localize issues.
>
> If you could write tests that examine the response body, without
> reducing the clarity of the example group, you should do so.  Fewer
> tests == less overhead.
>
>
>  Obviously, both are very basic in their implementation, but still, I
>  ask... If you were writing the specs, which way would you write them?
>  Thanks for any guidance.
>
> I hope that, despite the typical "it-all-depends-on-context", I was
> able to give you some insight into identifying and analyzing possible
> contexts.
>
> Pat
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
>
> _______________________________________________
>  rspec-users mailing list
>  rspec-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/rspec-users
>


More information about the rspec-users mailing list