Category: Effect Testing with RSpec 3

Notes from Effective Testing with RSpec 3, chapter 4.

In this chapter we start by building our application guided by tests. This seemed extreme for me. Normally, I would setup the required dependencies; install libraries, setup database, and initiate the application.

Effective Testing does everything through the test file. Our first file is a test to ensure we can post an expense. It doesn’t even contain any expectations, it just assumes we have an api and we can perform a post (we do not). We then start running tests and build the dependencies based on failures.

Here’s a TDD example of vacuuming the rug.

1. start vacuum
2. FAIL – there is no vacuum
3. buy vacuum
4. FAIL – cannot find vacuum
5. put vacuum in the closet

This goes on and on until we have an api mocked and ready to go. Throughout the chapter the focus is always on the tests. In ‘Filling In the Response Body’ we go as far as to put a hard coded response in our api. No matter what we post we will always receive { “expense_id”: 42 }. We continue to build out a test suit on this hard coded response.

Top Down Testing

There are a few names for this philosophy of testing; Top-Down, Outside-In, Discovery, or London style. The idea is that we start with the purpose of the application and work down to the details. This is accomplished by mimicking the response we’re looking for as we’ve done with { “expense_id”: 42 }.

Justin Searls, vocal proponent for Top Down has written about the benefits and provides a youtube series of it in action.

The appeal to this style of testing is that it naturally flows with our train of thought. We know the purpose, what components we need to serve that purpose, and how those components act. The perceived bottle neck with TDD is that we stay in the details too long and forget how those details talk to each other.

Top down has it’s own criticisms. It’s mock heavy which can lead to false positives, create a test suite full of objects that don’t exist, and some consider mocking a code smell.

I’m expecting that Effective Testing will address these criticisms and provide solutions to avoid them.

Notes & Observations

Effective testing instructs us to use bundle exec rspec. I prefer binstubs for the tab-complete goodness. You can generate one with bundle binstubs rspec-core from your projects root directory. Then run rspec with bin/rspec.

Notes from Effective Testing with RSpec 3, chapter 3.

Benefits

Out of the half dozen listed benefits of TDD, I connected with two of them.

Enabling Refactoring/Create confidence

There have been moments where I completed a feature with 20-30 minutes left in the day. Enough time to clean up my code. This could lead to application failures surprising in scope or process. Scrambling to revert my changes I would end the day lost and confused. With tests already written, refactoring is a rewarding process. Having immediate failure in context teaches me where I am wrong about the application or technology.

Guiding Design

Writing tests get’s the abstract concepts out of our heads and into a concrete plan. It helps us recognize better design and separate ideas. Effective Testing points out that if writing specs for your current code base feels painful then it indicates the code is difficult to maintain and is an opportunity for refactoring.

Costs of a Test Suite

When a test suite is slow it translates into a monetary cost. If a developer is waiting 4 minutes for a tests suite to run there is an additional cost as that developer eases back into their workflow.

Tests that break easily or throw false positives are another time sink. It takes time and cognitive ability to understand the scope of the problem and how to resolve it.

An overdone test suite contains all of the above problems in addition to maintenance. Effective Testing quotes Kent Beck from stack overflow, I believe this is the quote I was thinking of from chapter 1,

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence…

A test creates a dependency on your codebase. If that bit of code is modified or removed you also have to update that test. Effect Testing recommends avoiding this dependency as best as you can. One way is to not tests code that frequently changes such as a user interface. Another way is to generalize your tests. Instead of looking for an ‘exact match’ check if the value is included.

Sandi Metz presents an additional view for writing tests. Separate tests from your public and private interface. We should only be concerned with the accessible parts of our classes. She keeps tests for private methods in a separate file. In a comment it instructs the reader to not fix failing tests but delete them. Private methods are meant to support the public. They are likely to change more frequently and drastically over their public siblings.

Types of Specs

RSpec supports many different types of tests but Effective Testing will focus on three.

Acceptance

End to end testing. I’ve written about acceptance testing before. It’s a great way to lay some ground coverage and start refactoring.

It’s not ideal for long term maintenance as it tends to be brittle and change frequently as an application changes.

Unit

Unit specs are tests that describe a class or methods behavior. These seem to be the most common type of test. It’s the fastest turn around time for writing a spec, failing it, making it succeed, and then refactoring.

These tests aren’t as useful for larger scopes of refactoring. They often describe your application in bite sized chucks but not a feature from end to end.

Integration

Integration specs sit in-between acceptance and unit. This is the code that interacts with external services and api’s. In Ruby on Rails projects we use integration tests to describe behavior at the controller layer.

---

The authors of Effective Testing have prepared us with the benefits and costs of testing. They have described three different types of tests with the benefits of each. With the basics of RSpec at hand we will spend Part 2 of the book diving deeper into Acceptance, Integration, and Unit testing.

Notes from Effective Testing with RSpec 3, chapter 2.

Chapter two walked us through creating a spec to guide the development of our class. This time we explored the useful ways RSpec runs tests. We learned about structuring specs with tags and context blocks. Then filtering what specs are run.

Tags

We can assign meta data to an example block using hash value syntax. Rspec can then filter these tests to only run with or without them.

In ‘Identifying Slow Examples’ we used profile to identify the last two examples as our slowest tests. We can mark these tests as ‘slow’ using the code below:

RSpec.describe 'The sleep() method' do
  it ('can sleep for 0.1 second') { sleep 0.1 }
  it ('can sleep for 0.2 second') { sleep 0.2 }
  it ('can sleep for 0.3 second') { sleep 0.3 }
  it 'can sleep for 0.4 second', slow: true do; sleep 0.4; end
  it 'can sleep for 0.5 second', slow: true do; sleep 0.5; end
end

We can then run just the ‘slow’ tests, or exclude them using command line options or configurations:

# exclude slow tests from our run
rspec --tag ~slow

# only run the slow tests
rspec --tag slow

# Configure Rspec to auto exclude tests marked as slow
RSpec.configure do |config|
  config.filter_run_excluding slow: true
end

Effective Testing walks us through configuring RSpec with `filter_run_when_matching`, but there are many filters we can use to configure RSpec.

In addition to slow, we can use meta-tags to filter

• Required: Features that have contractual obligation to be available. You may never use TDD to ensure a Partners link is available on your front page. You may want a set of tests for ease-of-mind on your business relationships.
• Components & Stories: Tests that cover a general idea or the next milestone.
• Smoke tests: quick check to ensure application is functional.

Prepare Tests with Pending

Another way of filtering specs is with pending. Pending is way of categorizing specs that are not yet ready. It allows us to sit down and completely think out the attributes of a class.

  RSpec.describe 'blog post' do
    it 'has a title'
    it 'has a subtitle'
    it 'has content'
  end

This is easy to knockout and easy to read. I don’t have to know how I am going to accomplish this behavior but get’s things out of my head. Making me less likely to forget the big picture.

Pending goes further into detail. We can mark a test with pending and provide a helpful message.

it​ ​'is light in color'​ ​do​
  pending ​'Color not implemented yet'​
  expect​(coffee.color).to be(​:light​)
end​

#Rspec output
1) A cup of coffee with milk is light in color
  # Color not implemented yet
​  Failure/Error: expect(coffee.color).to be(:light)”

#Rspec output once completed
1) A cup of coffee with milk is light in color FIXED
  Expected pending ’Color not implemented yet’ to fail. No error was raised.
​   # ./spec/coffee_spec.rb:42”

# Excerpt From: Myron Marston, Ian Dees. “Effective Testing with RSpec 3.” iBooks. 

If we forget to remove pending after completing a feature, RSpec will remind us to clean up our test. The authors point out that this is handy for bugs. An unexpected failing test can be marked as ‘pending’ with the issue tracker in the description.

Example Filter & Dry Run

In ‘Running Just What You Need’ we learn about the −−example or -e flag. This is exciting because combined with −−dry-run we can easily explore our specs as documentation. In the last chapter, the authors pointed out that let safeguards us from memoization gotchas. Curios about what let is expected to do I can quickly find out.

rspec --example let --dry-run
...
#let
  raises an error when referenced from `before(:all)`
  yields the example
  raises a useful error when called without a block
  caches a nil value
  caches the value
  raises an error when attempting to define a reserved method name
  generates an instance method
  raises an error when referenced from `after(:all)`
  does not pass the block up the ancestor chain
  when the declaration uses `return`
    can get past a conditional `return` statement
    can exit the let declaration early
  when overriding let in a nested context
    can use `super` to reference the parent context value
  when included modules have hooks that define memoized helpers
    allows memoized helpers to override methods in previously included modules
...

47 examples are returned. This is more than we wanted, but it’s much easier to scan 47 examples instead of thousands. It’s also easier to read as it is just the authors intentions. There are no stack traces, profiling, or meta-data.