Effective Testing with RSpec 3, The RSpec Way

Notes from Effective Testing with RSpec 3, chapter 3.

Benefits

Out of the half dozen listed benefits of TDD, I connected with two of them.

Enabling Refactoring/Create confidence

There have been moments where I completed a feature with 20-30 minutes left in the day. Enough time to clean up my code. This could lead to application failures surprising in scope or process. Scrambling to revert my changes I would end the day lost and confused. With tests already written, refactoring is a rewarding process. Having immediate failure in context teaches me where I am wrong about the application or technology.

Guiding Design

Writing tests get’s the abstract concepts out of our heads and into a concrete plan. It helps us recognize better design and separate ideas. Effective Testing points out that if writing specs for your current code base feels painful then it indicates the code is difficult to maintain and is an opportunity for refactoring.

Costs of a Test Suite

When a test suite is slow it translates into a monetary cost. If a developer is waiting 4 minutes for a tests suite to run there is an additional cost as that developer eases back into their workflow.

Tests that break easily or throw false positives are another time sink. It takes time and cognitive ability to understand the scope of the problem and how to resolve it.

An overdone test suite contains all of the above problems in addition to maintenance. Effective Testing quotes Kent Beck from stack overflow, I believe this is the quote I was thinking of from chapter 1,

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence…

A test creates a dependency on your codebase. If that bit of code is modified or removed you also have to update that test. Effect Testing recommends avoiding this dependency as best as you can. One way is to not tests code that frequently changes such as a user interface. Another way is to generalize your tests. Instead of looking for an ‘exact match’ check if the value is included.

Sandi Metz presents an additional view for writing tests. Separate tests from your public and private interface. We should only be concerned with the accessible parts of our classes. She keeps tests for private methods in a separate file. In a comment it instructs the reader to not fix failing tests but delete them. Private methods are meant to support the public. They are likely to change more frequently and drastically over their public siblings.

Types of Specs

RSpec supports many different types of tests but Effective Testing will focus on three.

Acceptance

End to end testing. I’ve written about acceptance testing before. It’s a great way to lay some ground coverage and start refactoring.

It’s not ideal for long term maintenance as it tends to be brittle and change frequently as an application changes.

Unit

Unit specs are tests that describe a class or methods behavior. These seem to be the most common type of test. It’s the fastest turn around time for writing a spec, failing it, making it succeed, and then refactoring.

These tests aren’t as useful for larger scopes of refactoring. They often describe your application in bite sized chucks but not a feature from end to end.

Integration

Integration specs sit in-between acceptance and unit. This is the code that interacts with external services and api’s. In Ruby on Rails projects we use integration tests to describe behavior at the controller layer.


The authors of Effective Testing have prepared us with the benefits and costs of testing. They have described three different types of tests with the benefits of each. With the basics of RSpec at hand we will spend Part 2 of the book diving deeper into Acceptance, Integration, and Unit testing.

Effective Testing with RSpec 3, From Writing Specs to Running Them

Notes from Effective Testing with RSpec 3, chapter 2.

Chapter two walked us through creating a spec to guide the development of our class. This time we explored the useful ways RSpec runs tests. We learned about structuring specs with tags and context blocks. Then filtering what specs are run.

Tags

We can assign meta data to an example block using hash value syntax. Rspec can then filter these tests to only run with or without them.

In ‘Identifying Slow Examples’ we used profile to identify the last two examples as our slowest tests. We can mark these tests as ‘slow’ using the code below:

RSpec.describe 'The sleep() method' do
  it ('can sleep for 0.1 second') { sleep 0.1 }
  it ('can sleep for 0.2 second') { sleep 0.2 }
  it ('can sleep for 0.3 second') { sleep 0.3 }
  it 'can sleep for 0.4 second', slow: true do; sleep 0.4; end
  it 'can sleep for 0.5 second', slow: true do; sleep 0.5; end
end

We can then run just the ‘slow’ tests, or exclude them using command line options or configurations:

# exclude slow tests from our run
rspec --tag ~slow

# only run the slow tests
rspec --tag slow

# Configure Rspec to auto exclude tests marked as slow
RSpec.configure do |config|
  config.filter_run_excluding slow: true
end

Effective Testing walks us through configuring RSpec with `filter_run_when_matching`, but there are many filters we can use to configure RSpec.

In addition to slow, we can use meta-tags to filter

  • Required: Features that have contractual obligation to be available. You may never use TDD to ensure a Partners link is available on your front page. You may want a set of tests for ease-of-mind on your business relationships.
  • Components & Stories: Tests that cover a general idea or the next milestone.
  • Smoke tests: quick check to ensure application is functional.

Prepare Tests with Pending

Another way of filtering specs is with pending. Pending is way of categorizing specs that are not yet ready. It allows us to sit down and completely think out the attributes of a class.

  RSpec.describe 'blog post' do
    it 'has a title'
    it 'has a subtitle'
    it 'has content'
  end

This is easy to knockout and easy to read. I don’t have to know how I am going to accomplish this behavior but get’s things out of my head. Making me less likely to forget the big picture.

Pending goes further into detail. We can mark a test with pending and provide a helpful message.

it​ ​'is light in color'​ ​do​
  pending ​'Color not implemented yet'​
  expect​(coffee.color).to be(​:light​)
end​

#Rspec output
1) A cup of coffee with milk is light in color
  # Color not implemented yet
​  Failure/Error: expect(coffee.color).to be(:light)”

#Rspec output once completed
1) A cup of coffee with milk is light in color FIXED
  Expected pending ’Color not implemented yet’ to fail. No error was raised.
​   # ./spec/coffee_spec.rb:42”

# Excerpt From: Myron Marston, Ian Dees. “Effective Testing with RSpec 3.” iBooks. 

If we forget to remove pending after completing a feature, RSpec will remind us to clean up our test. The authors point out that this is handy for bugs. An unexpected failing test can be marked as ‘pending’ with the issue tracker in the description.

Example Filter & Dry Run

In ‘Running Just What You Need’ we learn about the −−example or -e flag. This is exciting because combined with −−dry-run we can easily explore our specs as documentation. In the last chapter, the authors pointed out that let safeguards us from memoization gotchas. Curios about what let is expected to do I can quickly find out.

rspec --example let --dry-run
...
#let
  raises an error when referenced from `before(:all)`
  yields the example
  raises a useful error when called without a block
  caches a nil value
  caches the value
  raises an error when attempting to define a reserved method name
  generates an instance method
  raises an error when referenced from `after(:all)`
  does not pass the block up the ancestor chain
  when the declaration uses `return`
    can get past a conditional `return` statement
    can exit the let declaration early
  when overriding let in a nested context
    can use `super` to reference the parent context value
  when included modules have hooks that define memoized helpers
    allows memoized helpers to override methods in previously included modules
...

47 examples are returned. This is more than we wanted, but it’s much easier to scan 47 examples instead of thousands. It’s also easier to read as it is just the authors intentions. There are no stack traces, profiling, or meta-data.

Effective Testing with RSpec 3, Getting Started

Notes from Effective Testing with RSpec 3, chapter 1.

Effective Testing opens with explaining that Behavior & Test Driven Development provide us with design guidance, a safety net, and documentation. All of these are great reasons but my favorite is something I heard from the Ruby Rouges podcast,

“You are not paying me to write tests. You are paying to guarantee my code is doing what you are paying for.”

A part of Getting Started is installing Ruby & the RSpec libary, which you would expect is a single gem. RSpec has separated it’s behavior into 3 gems: Core, Expectations, & Mocks. The idea is that you can mix and match libraries with other test frameworks and mocking tools. I was curios if anyone had taken advantage of this and found ‘Zverok with Ruby’ who wrote about using RSpec libraries for easier to read conditionals, exploring a code base, method validations, and more.

By the end of Getting Started we will have coded out a spec to describe the Sandwich class. We will also have explored three different ways to condense our code and make our tests easier to read:

Hooks are a way to run setup code before, after, and around a test block. The idea is to create a standard environment where each example, context, description, etc. runs with the same values and state. One thing I didn’t see in this chapter was filtering hooks, which seems really useful.

Helper methods, the second way of condensing our code, is just plain ruby. The authors remind us that we’re not forced into RSpec and if it’s easier to hack something out we should just do that.

This section then ends on let. A useful way to initiate and reuse an instance variable: let(:sandwich) { Sandwich.new("delicous', []) }. This is the most common way to condense code. It protects us from misspelling the sandwich variable in our examples, it’s only run when called, and easy to refactor.

Getting Started concludes by suggesting we look at rspec --help to discover available options. Here are a few things I explored:

  • dry-run: We can dry-run tests to print out expectations. This is really useful for documentation. If I’m curios about a service or object I can run it’s specs dry and read them as the intended use without waiting for tests to run.
  • formats: Results can be printed in a variety of ways. Not just for communicating to me, the developer, but to another audience viewing them through a browser or JSON API.
  • filter: Tests can be filtered when run. This is essential for rapid testing. When I’m working on a section of code, I only want to run the relevant tests. I won’t run the entire suite until after I’ve completed a task.

Nand2Tetris – The ALU Chip

In Chapter 2 of Nand2Tetris we build the Arithmetic Logic Unit chip. It has 2 16-bit inputs and 1 16-bit output. While providing the inputs you also specify the arithmetic operation you want to perform; . This ALU chip is designed to become the centerpiece of HACK.

In addition to ALU’s there are FPU’s (Floating Point Unit) and CPU’s (Central Processing Unit). ALU’s are used to build these more complex chips, which may contain many ALU’s.

John von Neumann proposed ALU’s in 1945 but components were large and expensive for the type of ALU we are building now.
Initially, they performed operations on single bit data. Now that we have integrated circuit (IC) transistors we are able to build ALU’s so complex they can handle operations in a single clock cycle.

Someone in my group documented errors you may come across building the ALU. Hope this helps, https://github.com/SeaRbSg/nand2tetris2017/tree/master/jwfearn/tips.

SearchKick – Remapping an Index

Recently someone updated a Rails Model ID datatype from int to uuid. This has caused issues with ElasticSearch in my local instance:

:exception:
- Searchkick::ImportError
- '{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [id]", "caused_by"=>{"type"=>"number_format_exception",
"reason"=>"For input string: \"03b42e85-1f09-402c-8f51-89d61c128ffc\""}} on item
with id ''03b42e85-1f09-402c-8f51-89d61c128ffc'''
:exception_object: '{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse
[id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string:
\"03b42e85-1f09-402c-8f51-89d61c128ffc\""}} on item with id ''03b42e85-1f09-402c-8f51-89d61c128ffc'''

After some digging I around I was able to resolve this by deleting the Indexes. Rails/SearchKick rebuilt them afterwords.

You can view the available entries with a GET to elasticsearch with “*” as the path.

curl 'localhost:9200/*'

It’s easier to read if you pipe it into jq.

brew install jq
curl 'localhost:9200/*' | jq keys

From the list, identify the object that is failing, mine was organizations_development. Delete the mapping with

curl -XDELETE 'localhost:9200/organizations_development'

From there, elasticsearch will rebuild it and index your data.

Silver Bay Saved My Bacon!

This morning I was attempting to customize DoorKeeper. After an hour of prodding I wanted to wipe all my changes.

1. Git checkout .
* removes changes to known files
2. Git clean -fd
* Removes unchanged files and directories

Except that’s not what I ran. I ran clean with the flags ‘fdx’ which also removes hidden files. So I lost all my .env and config files. Sure I could spend the next hour or two hunting those down.

I though Time Machine tracked these files but I didn’t see them. Silver Bay Tech has a simple blog to follow along and have Time Machine restore these files.

Mac OS X – Restoring Hidden Files and Folders with Time Machine

Thanks Silver Bay!

Papers We Love: Feral Concurrency Control

This months PWL meeting discussed ‘Feral Concurrency Control: An Empirical Investigation of Modern Application Integrity’. The intended audience is database system researchers. It explains how popular application frameworks lean on ORM’s that disregard the database concurrency control solutions. Instead of using the databases system for foreign keys, constraints, and transactions they create a set of rules that perform in the application layer. The paper demonstrates how validations in the application layer give us inconsistent results.

Unique constraints are given as an example. When a field with a unique validation is created or modified in Rails, it will query the database to ensure that value does not already exist.

Applications with minimal traffic can be run with only one Ruby on Rails instance.

Initial Rails Application
User --> Ruby on Rails Application --> Database

As requests against the applicaion increase, we can add more Ruby on Rails instances to handle the traffic.

Ruby on Rails applications scaled to handle growing requests
         Ruby on Rails Application #1
User --> Ruby on Rails Application #2 --> Database
         Ruby on Rails Application #3 

In the experiment an application is bombarded with redundant data. In Figure 2 we see a blue line, the constant. This is the amount of duplicates without validation. The green line is the amount of duplication with validation. We can see with one running instance we have 0 duplicates. As soon as we start adding more instances the amount of duplicates jump.

As a Ruby developer with a history of Rails I was surprised at the accusations coming from the paper and discussion. Rails doesn’t prohibit us from using database concurrency control solutions. In my experience we have always used them.

This was really eye opening. I’m glad I went to this discussion and I’m thankful for the Seattle Chapter of Papers We Love. Many people come to Rails from boot camps or are self-taught. While ORM’s don’t prevent you from using the database it definitely encourages it to be blindly used as a storage system. It’s changed my thinking in how to help grow jr developers and the importance that needs to be placed on what’s happening in db/schema.rb.

Here is a list of resources I will be referencing in future conversations:
* Transactions
* Constraints vs Validations
* Postgres Explain

As a side note, validations where only inconsistent when compared to other records (am I unique, does this key exist). Validations performed consistently when evaluating data in the current record (is this a string and does it have a pattern?).

Nand2Tetris and HDL

In my local ruby group we’ve been working through Nand2Tetris. A program that walks through building a computer from logic gates to programing Tetris.

I’m struggling with the Hardware Description Language(HDL). I have built up assumptions about languages and their use of variables and functions. Below are my notes that have helped me straighten out how to work with HDL’s.


HDL is a markup language, like HTML or XML. We use this markup language to design logic gates. Logic gates will be described in syntax familiar to you as pseudocode. For example:

Chip Name: Or
 Inputs: a, b
 Outputs: out
 Function: If a=b=0 then out=0 else out=1.

HDL doesn’t have if/else syntax. This is telling us to use boolean arithmetic to create out=1 if a or b is 1.

Chip Body

The chip body is self explanatory. Here is the Or gate HDL file.

CHIP Or {
 //Declaring a Chip named 'or'. The definition is between the brackets.

IN a, b;
  OUT out;
  //Header of the chip. It declares the expected wires IN and OUT.

PARTS:
 //Here are the logical statements that describe how IN gets to OUT.
 Not(in=a, out=nota); // Invert a into nota
 Not(in=b, out=notb); // Invert b into notb
 Nand(a=nota, b=notb, out=out); // Nand's truth table will now output Or's truth table
 }

Statements

Statements will be a single line representing a chip and how it is used.

Not(in=a, out=nota);

If you looked at Not.hdl you would see in the Chip declaration an IN of in and and OUT of out. These names represent physical wires that are coming in and out of the chip.

in=a

Everything on the left side of ‘=’ is the name of the pin going in. Everything on the right is the name of the pin going out.

When you name a pin you can use characters (uppers & lowers), numbers, and ‘-‘. Pin names have to start with an lowercase letter.

In addition to pins, we can provide boolean values using ‘true’ or ‘false’.

Nand(a=in, b=true, out=out);

If we are only concerned with the input of ‘a’ we can supplement the input of ‘b’ with a boolean.

Not declaring a pin is not a problem. The HardwareSimulator will supplement it with false (0).

Nand(out=alwaystrue);
 //The a and b pins are not declared so their values
 // default to 0. Nand returns 1 if a and b are both 0.

We can also declare additional pins. Say we have a statement that declares the out pin but we also need that value for another statement.

Xor(a=a, b=b, out=sum);
 Or(a=a, b=sum, out=carry);

In this chip, ‘sum’ and ‘carry’ are the out pins. If we run this in the Hardware Simulator it will result in an error: “Can’t connect gate’s output to part:”.

Remember, ‘sum’ is a declared physical wire coming out of the chip. We cannot re-route it. However, we can declare additional outs:

Xor(a=a, b=b, out=ab, out=sum);
 Or(a=a, b=ab, out=carry);

Easily store hard coded values with Active Hash

Hard coded data is usually stored in the class that uses it.

#app/models/camper.rb
class Camper
  MEALS = {1: 'breakfast', 2:'lunch', 3: 'supper'}
 
end
 
Camper::MEALS

Active Hash is a library that helps Rails projects abstract this data out into an ActiveRecord like model:

#app/models/meals.rb
class Meals < ActiveHash::Base
  belongs_to :camper
 
  self.data = [
    { id: 1, name: 'breakfast'},
    { id: 2, name: 'lunch'},
    { id: 3, name: 'supper'}
  ]
end
 
#app/models/camper.rb
class Camper
  # required for ActiveRecord 3.1 or later
  extend ActiveHash::Associations::ActiveRecordExtensions
  has_many :meals
 
end
 
Camper.meals

This benefits our classes by removing data it’s not meant to represent. It helps the code architecture by not littering the application with constants. If someday we want that data to be an ActiveRecord model the code is already organized for it.

ActiveHash also works with FactoryGirl

FactoryGirl.define do
  factory :camper do
    name { 'Sally' }
    meals
  end
end

No Tests? No Excuses!

There is a strong testing vibe in our community. It’s a frequent topic in local groups, conferences, and interviews. Even with that drive in our community it’s common to land projects that have no working tests.

If you are only used to unit testing then it may seem overbearing to trash the existing test suite and start fresh. That’s because it is! It is impractical to delay your project to comb through every class and define it’s behavior in a test.

Getting a test suite quickly is possible when approaching it from the angle of acceptance testing. Ensuring the details are working together instead of testing them independently.

For example, when you hit a piñata, do you get candy? We’re not ensuring the piñata is mounted, that it can take a couple of whacks, or that the candy is delicious. We have a working example and can assume those attributes are true. If one of them is off the entire process fails.

In a Ruby on Rails project I can spend an afternoon and get a decent coverage of an applications expected behavior. Here is a sample of testing a home page using Rspec.

I’m not testing for every working component . This asks if the someone requests the homepage, do they get the elements that make the homepage. It is much easier to do a quick analysis and implementation with this method. I don’t even have to look at the source code to define expected behavior and ensure it doesn’t change.

This has an advantage in a refactoring scenario. It doesn’t matter how much we change the underlying code, the results of the homepage are expected to the same. Having integration tests gives us the confidence to strip out any code and give us quick feedback on the consequences.