Coming off of a very successful online card sorting exercise, a committee asked me to implement a survey of humanities students and faculty on their use of different collections in our main stacks.
I decided to structure the exercise as another card sort, asking the participants to locate each collection in a sequence of priority from 1-9. It was a huge mistake and I heard about it from many of the participants. Some wouldn’t even complete the survey because the experience of participation was so infuriating.
In retrospect I should have made the survey a series of Likert scales which would have been more intuitive to the participants, simpler to analyze, and a more natural fit to the underlying questions the committee was trying to address.
It was incredibly embarrassing, but it was a valuable lesson in remembering to assess the underlying problem and audience when considering the method of testing.