User Research As A Commodity

February 9, 2008 Alex Chaffee

The intersection of development and UI is an area I’ve got some interest in.
Arena just shared this amazing article about a team’s experience merging Agile project management with User Research and testing.

User Research as a Commodity (part 3 of 7) by Tim Kieschnick

Here (below the fold) are some of the things in the article I found eye-opening.

  • “When asked, the sponsors and most of the team thought the best thing to do would be to crank out an initial site on a tight timeline, doing the best we could based on intuition and a few heuristics, and then test it with users after we went live. …Clearly, the UX people didn’t understand agile, and the software people didn’t understand UX.” It’s interesting to me, standing mostly on the software side of the fence, that — much like there are developers who don’t get agile, or, getting it, don’t make time for it — there are UI designers who don’t get User Testing or don’t make time for it.
  • Despite initial fears over wasting time or slowing progress, what mattered in the end was that they set up a regular time once a week to meet with users and do user testing. “We virtually eliminated arguments within the team about what would work best for users. Instead of pressing the point, the Product Manager could just say, “I’ll ask them this Thursday.”” This is the key to the agile process: if you care about something, pay attention to it, and then (the hard part) keep paying attention to it. Have a standup every day, a demo every week, a retrospective every release — and a user test every Thursday.
  • Leveraging slack. “What if teams don’t generate enough tasks to fill up the time this week? No problem–our “Platform UI Team” will maintain a backlog of non-urgent areas to test with each audience, so they can fill in as needed.” This is something I see a lot with teams that are successful at what I will somewhat pompously call Journeyman Agile: they are so good at pumping out features that that’s all they do. They’re addicted to points. They often forget to take time for chores like large-scale refactorings or infrastructure improvements or even retrospectives or water-cooler time or “sustainable pace” (i.e. actually leaving the office). And since Agile developers are such nice guys, they rarely push back when their managers — who aren’t confronted directly with the pain of crusty code or communication — keep asking for higher and higher velocity.
  • At one point (in the speculative conclusion) the author mentions the metrics mantra: “What gets measured gets managed.” I am struck by how insidiously similar this is to one of my Alexisms: “If you pay attention to something, it gets better.” Why insidious? Because if you happen to pick the wrong metric, you’re screwed. But “pay attention”
    implies you’re not a slave to the digits, but are actually having intelligent conversations about the thing, and maybe picking or discarding metrics as you go.

About the Author


Screen Scrape No More…Seriously!
Screen Scrape No More…Seriously!

This week I had the pleasure of attending Dapper Camp put on by the folks at Dapper for their user communit...

Happy Path Testing With Selenium RC Fu
Happy Path Testing With Selenium RC Fu

Selenium RC Fu is a fantastic system for testing Ruby On Rails applications. It is the blending of xUnit te...

SpringOne. Catch all the highlights

Watch now