Cedar device specs and CI

July 7, 2010 Adam Milligan

One of the most common complaints I’ve read about OCUnit, the unit testing framework built into Xcode, is that the tests you write with it won’t run on the device. In addition, I personally have found the process of setting up a target for tests that depend on UIKit confusing and onerous. So, one of our goals for Cedar was to make testing UI elements easy (or easier), by making it easy to run specs in the simulator or on the device.

Probably the second most common complaint I’ve read about OCUnit is that the tests run as part of the build. This makes the test output difficult to separate from the build output, and makes it impossible to use the debugger when running tests. So, in addition to making it easy to run specs on the device, we wanted to be able to run them as a separate, debuggable executable.

Finally, we consider it important that our specs run in our CI system. That means we wanted to be able to run Cedar specs from the command line, and get an exit code signifying success or failure. At the same time, some of us appreciate the value of the green/red feedback for specs passing and failing, so sometimes we like a nice UI. As of today, Cedar will accommodate all of these various requirements.

If you have simple tests that don’t depend on iOS, you can simply link the Cedar dynamic framework into an OS X command line executable build target, write your specs, and run. Running from inside Xcode should give you output that looks something like this:
Xcode spec run pass

For specs that do depend on iOS frameworks, such as custom views or view controllers, you can link the Cedar static framework into an iOS executable, use the Cedar-specific application delegate (CedarAppDelegate), and you get something that looks like this:
Xcode UI spec run pass

The elements in the table represent the top-level specs that you have written. You can click on each one to drill into the state of each of its children, and its childrens’ children, etc.

Of course, those nice green bars aren’t terribly useful for the CI build, so if you’d prefer to skip the UI you can set the CEDAR_HEADLESS_SPECS environment variable, and then running the specs on the simulator (or device) reverts back to the dots:
Xcode headless UI spec run pass

Notice that we’ve created rake tasks for running the specs on the command line, for the sake of simplicity and ease of integration with CCRB. The Rakefile is available in the Cedar project, and does little more than set environment variables and execute bash shell commands. If you’d prefer to use your own bash script, or something similar, feel free to copy and paste your way to happiness.

I’ll spare you another screenshot, but you can see the results and build logs for the Cedar CI build here, and its current status is always displayed on our public CI Monitor page.

The last piece of the testing puzzle for iOS is full-stack integration and lifecycle testing (à la Selenium), which should be possible with Apple’s new UI Automation tool. Unfortunately, Apple hasn’t yet provided a way to run UI Automation tests, or get test results, from the command line. Hopefully we’ll find a way soon; their JavaScript DSL for iOS testing combined with Jasmine could be a potent combination.

About the Author


TaskRabbit looking for early stage Rails Developer in San Francisco
TaskRabbit looking for early stage Rails Developer in San Francisco

At Pivotal Labs, one of the services we provide our clients is helping them interview and hire. Pivotal Lab...

Github service hook for Pivotal Tracker
Github service hook for Pivotal Tracker

With Github's help, we've added Pivotal Tracker to the list of Github's built in service hooks, making it e...

SpringOne 2021

Register Now