<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Very nice reply Pat. This would make a great blog post if you get a chance.<div><br></div><div>Thanks</div><div>Chris<br><div><br><div><div>On 18 Apr 2008, at 10:15, Pat Maddox wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hey Matt,<br><br>The ultimate test would be one that is focused on one thing such that<br>the test would<br> - break every time that thing broke<br> - break only when that thing broke<br> - give detailed feedback enabling you to focus on the thing's<br>subpart necessary to identify and fix the problem<br><br> The ultimate test suite would be the set of such tests that covered<br>every single concern that exists in a project.<br><br> Some of these are concerns are easier to test than others. Some are<br>important and lend themselves to automation, thus enjoy great tool<br>support. Others take something far more abstract like a person's<br>aesthetic appeal.<br><br> When considering a new test, I should ask myself what problem the<br>test solves, and what problem I really want the test to solve. I try<br>to write the test in terms of the second. For example, if I were to<br>write the following test:<br><br> it "should allow deposits and withdrawals" do<br> @account.deposit 80<br> @account.withdraw 25<br> @account.balance.should == 55<br> end<br><br>The test would be valuable when the problem is that you need to track<br>how much money people have in their accounts.<br><br>If you faced a problem such as "Make sure the user sees an error when<br>they withdraw more than their balance," you would not want a test<br>like:<br><br> it "should have an overdraft error" do<br> post :withdrawals, :amount => 1_000_000_000_000 # even bill can't do this!<br> assigns[:withdrawal].should have_at_least(1).error_on(:amount)<br> end<br><br>If this test breaks at some point, we would make a change to the test<br>or production code in order to make it pass. We wouldn't think of the<br>problem it was intended to solve though, because we are absorbed in<br>the problem that initiated the break-inducing change. The danger in<br>this is that the test appears to be robustly covering something<br>useful, but in effect can let problems leak through. Imagine that we<br>remove an important partial from a view. This test, whose ultimate<br>goal was to ensure that users see an error message, failed to catch<br>the problem where the message wasn't displayed at all.<br><br>So you should write tests expressing the same level abstraction as the<br>problem you want them to solve. Somewhere you would need a test like:<br><br> it "should have an overdraft error" do<br> @page.should include("Can't withdraw more than your balance")<br> end<br><br>Now you would need some way to get @page. Perhaps you make a request<br>to a controller, it hits a database, renders a response, and assigns<br>the response body to @page.<br><br>Or maybe you rendered a view using a fake @withdrawal, and you've got<br>another test that verifies that you assign @withdrawal in the<br>controller, and another test verifies that Withdrawal objects get an<br>error when you try to create them for an amount greater than the<br>target account balance. And you've got a test that you label an<br>Integration test to signal the fact that it integrates all of these<br>pieces.<br><br>You should only write integration tests that check across valuable<br>boundaries. This does not restrict it to stuff like company-specific<br>code using an ORM framework, though. Because you should only write<br>tests that are valuable, sets of layered tests forms a small subsystem<br>requiring integration testing.<br><br>It is sometimes useful to have layers of tests that enable you to<br>localize problems. Other times the types of problems you solve will<br>be trivial or obvious and won't require localization.<br><br>As a simple rule, more tests == more overhead. But if you're missing<br>certain tests then you will not notice certain problems when they<br>appear. The art of all of this is identifying the set of tests that<br>maximizes your confidence and ability to produce valuable software.<br><br>With all that theory out of the way, what can we say about the tests<br>you presented?<br><br><br><blockquote type="cite"> describe FeedsController, 'get /feeds/' do<br></blockquote><blockquote type="cite"> before(:each) do<br></blockquote><blockquote type="cite"> @request.env["HTTP_ACCEPT"] = "application/xml"<br></blockquote><blockquote type="cite"> Model.should_receive(:find).with(any_args).and_return<br></blockquote><blockquote type="cite"> mock_model(Model)<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> it "should return success" do<br></blockquote><blockquote type="cite"> get '/'<br></blockquote><blockquote type="cite"> response.should be_success<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> it "should return 405 (Method Not Allowed) if HTTP_ACCEPT is text/<br></blockquote><blockquote type="cite"> html" do<br></blockquote><blockquote type="cite"> @request.env["HTTP_ACCEPT"] = "text/html"<br></blockquote><blockquote type="cite"> get '/'<br></blockquote><blockquote type="cite"> response.response_code.should == 405<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"> end<br></blockquote><br>This test would be good in a situation where we had published an API<br>stating only XML requests were allowed.<br><br><br><blockquote type="cite"> The second one is much more detailed:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> describe FeedsController, 'get /feeds/' do<br></blockquote><blockquote type="cite"> before(:each) do<br></blockquote><blockquote type="cite"> @model = mock_model(Model)<br></blockquote><blockquote type="cite"> @request.env["HTTP_ACCEPT"] = "application/xml"<br></blockquote><blockquote type="cite"> Model.should_receive(:find).with(any_args).and_return @model<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> it "should assign to the model" do<br></blockquote><blockquote type="cite"> get '/'<br></blockquote><blockquote type="cite"> assigns[:model].should == model<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> it "should render feed template" do<br></blockquote><blockquote type="cite"> get '/'<br></blockquote><blockquote type="cite"> response.should render_template('feeds/model_feed.xml.erb')<br></blockquote><blockquote type="cite"> end<br></blockquote><blockquote type="cite"> end<br></blockquote><br>This test would be valuable in a context where the XML feed output is<br>complex. In that case, testing the output directly might not<br>sufficiently enable us to localize issues.<br><br>If you could write tests that examine the response body, without<br>reducing the clarity of the example group, you should do so. Fewer<br>tests == less overhead.<br><br><br><blockquote type="cite"> Obviously, both are very basic in their implementation, but still, I<br></blockquote><blockquote type="cite"> ask... If you were writing the specs, which way would you write them?<br></blockquote><blockquote type="cite"> Thanks for any guidance.<br></blockquote><br>I hope that, despite the typical "it-all-depends-on-context", I was<br>able to give you some insight into identifying and analyzing possible<br>contexts.<br><br>Pat<br>_______________________________________________<br>rspec-users mailing list<br><a href="mailto:rspec-users@rubyforge.org">rspec-users@rubyforge.org</a><br><a href="http://rubyforge.org/mailman/listinfo/rspec-users">http://rubyforge.org/mailman/listinfo/rspec-users</a><br></blockquote></div><br></div></div></body></html>