How to Not Test RabbitMQ Part 2

This is Part 2 of my two part series on working with queues in Ruby. If you want some context please head over to part 1. In this post I’ll touch on Moqueue, using RSpec to stub out Bunny, and a few other hurdles along the way.

So getting started, we didn’t want our unit tests to hit RabbitMQ because we feel strongly that our unit tests shouldn’t depend on outside services any more than necessary (disclaimer: I do feel strongly that our integration tests should depend on those services to recreate the state of the world as accuratly as possible). To do this, there’s a great mock called Moqueue which Chris has kept in our field of vision.

In the example spec for what’ll become our subscriber daemon, it looks like something like this:

require "subscriber_daemon"
require 'moqueue'

describe SubscriberDaemon do
  before :all do
    overload_amqp
  end
  ...
end

The key is this line:

overload_amqp

The way I think about this is that it injects a overgrown array where I call MQ.new. Sweet, now I can leave RabbitMQ out of my tests.

Next, my SubscriberDaemon looks something like this:

require 'simple-daemon'
require 'mq'

class SubscriberDaemon < SimpleDaemon::Base
  SimpleDaemon::WORKING_DIRECTORY = "#{BOX.log_dir}"
  READ_QUEUE = "MyQueueOfWork"

  def self.start
    puts "STARTING #{classname} #{Time.now}"
    STDOUT.flush
    EM.run do
      subscribe_to_read_queue
    end
  end

  def self.stop
    puts "STOPPING #{classname} #{Time.now}"
    STDOUT.flush
  end

  def self.subscribe_to_read_queue
    amq = MQ.new
    queue = amq.queue(READ_QUEUE, :durable => true, :auto_delete => false, :exclusive => false)
    queue.subscribe(:ack => true) do |header, message|
      #stuff the message in the database
      ...
      header.ack
    end
  end

Some things to note: We used SimpleDaemon to get us daemon functionality. The start and stop methods are part of that interface. It needs you define a working directory for the log file and pid file it generates for you. Works great with Monit (make sure to use the latest version or Monit will start up multiple instances of your daemon). It also likes to spool up the messages so if you want feedback in your log that resembles real time events, make sure to flush your output.

You’ll also see that we created a durable queue, meaning stuff stays on the queue (in a “being worked on state” until it gets an ack back. you’ll also see that our subscribe method has to be called inside an EM.run. This is because of the async nature of the AMQP gem’s client queue subscribe method.

First potential pitfall: Sticking your subscribe code inside the EM.run. You’ll never know when it completes. It is also hard to stop. Just extract that logic out into a method and you’ll be as happy as two rabbits on their “bunnymoon”.

Onward, now lets look at some tests around the SubscriberDaemon.

before :all do
  overload_amqp
end

it "should read a item from the queue and stuff it in the database" do
    #put some stuff on the queue
    amq = MQ.new
    queue = amq.queue(SubscriberDaemon::READ_QUEUE)
    queue.publish("My message")

    #do the subscription
    SubscriberDaemon.subscribe_to_read_queue

    #assert on the expected outcome (eg look it up in the database)
    ...
  end

In this case, we’ve written the test to exercise the queue client, but not the server. We fake out RabbitMQ using Moqueue, and we never start the daemon, so we don’t have to worry about stopping it, we simply call the same method the daemon calls in the loop. You may also notice we explicitly put “My message” on the queue in the spec.

Done. SubscriberDaemon unit test is written.

On the other end, we want to publish messages with the string reversed to the queue. So here’s a MessageMaker:

require 'subscriber_daemon'

class MessageMaker
  SUCCESS_QUEUE = SubscriberDaemon::READ_QUEUE

  def self.log(message)
    puts caller.first
    puts Time.now
    puts message
    STDOUT.flush
  end

  def self.make_reversed_message(message="Hello Readers!")
    amqp_client = Bunny.new(:logging => false)
    amqp_client.start
    queue = amqp_client.queue(SUCCESS_QUEUE, :durable => true, :auto_delete => false, :exclusive => false)
    queue.publish(message.reverse, :persistent => true)
    MessageMaker.log("PUBLISHED #{message.reverse}")
    amqp_client.stop
  end
end

In our spec for MessageMaker, let us assume that we just want to test the make_reversed_message() logic and leave all the queue nonsense out of it. This means we’ll have to mock Bunny. For a little extra excitement, let’s call make_reversed_message() twice to ensure that making a message doesn’t pollute the next message that gets made.

require 'bunny'

describe MessageMaker, "#make_message" do
  it "doesn't pollute subsequent messages"
    bunnies = []
    bunnies << mock('bunny')
    bunnies[0].stub(:start)
    bunnies[0].stub(:stop)
    bunnies[0].stub(:exchange)
    bunnies[0].stub(:queue).and_return do |*args|
        queue = mock('queue', :name => 'my queue')
        queue.should_receive(:publish).with("?ykcits dna nworb s'tahW", :persistent => true)
        queue
    end

    bunnies << mock('bunny')
    bunnies[1].stub(:start)
    bunnies[1].stub(:stop)
    bunnies[1].stub(:exchange)
    bunnies[1].stub(:queue)
    bunnies[1].stub(:queue).and_return do |*args|
      queue = mock('queue', :name => 'my queue')
      queue.should_receive(:publish).with("!kcits A", :persistent => true)
      queue
    end

    Bunny.stub(:new).and_return do |*args|
      bunnies.shift #shifts off the first instance in the array and returns it
    end

   MessageMaker. make_reversed_message("What's brown and sticky?")
   MessageMaker. make_reversed_message("A stick!")
  end
end

The assertions happen in the mocked Bunny instances where you see “queue.should_receive(…)” We have two bunnies because each time you call make_reversed_message() it instantiates a new Bunny. We stubbed Bunny.initialize() to return different instances with different expectations when queue() is called.

So now we have a very focused unit test which only tests the logic of make_reversed_message(). It doesn’t test any queue related code because we mocked out Bunny, our AMQP Client that we use for adding our messages to the queue synchronously, but demonstrates that the message does get reversed. And that, is the way to not test your queues.

About the Author

Biography