Early this week we pushed out an initial Alpha version of our current project at work (an online system to assist in Schedule Proofing), and we met with the principal users of the system in the future yesterday to get feedback. Unfortunately, the meeting wasn’t nearly as productive as we’d hoped.
In many organizations, developers aren’t allowed to conduct these meetings. We don’t have that luxury (and neither do most companies), but the principle is really solid. Developer’s want to educate users on how to use their software, but when trying to get user feedback, education is exactly what you don’t want to provide.
I wasn’t the meeting organizer, so I left the running of the meeting to the other developer on the project, who incidentally has also done most of the UI work. The role that I was taking was to sit back and take notes, so that we would know what we needed to improve. The meeting organizer began the meeting innocuously enough, asking for any feedback that the users had requested. The first item was a feature that was already implemented, and rather than waiting for the user’s to finish providing their feedback, the meeting lead immediately jumped into a miniature training session on the UI. Walked the two users we were talking with through the entire UI, both elements that were already implemented, and elements that we were planning.
Because of this the first half of the feedback meeting, turned into a training meeting. I did nothing, I’d already decided that the meeting wasn’t my responsibility to run, and continued to take notes on what was being said. However, at this point, the feedback we were getting had changed in an incredibly un-useful manner. The feedback we were getting was now regarding things that they really liked the sound of, and perhaps one or two issues which were generally more of a matter of us misunderstanding business-layer needs, rather than the UI feedback we were hoping for.
User Feedback sessions are meant for exactly that. Getting feedback from the users. If the user doesn’t do 90% of the talking, you’re likely approaching the meeting wrong. Because of the impromptu training session our meeting turned into we got basically no feedback on the usability of the application, once we taught how to use the application, things made sense. Problems they had had initially were no longer problems. We learned basically nothing about what we needed to improve.
Due to the specialized nature of the application, and our ease of access to the people who will be using it, we are in a nice position in that we can plan to train the users. However, there is always turnover, and the more discoverable the application is, the less ongoing training will be required. But to figure out where the user’s struggle, we have to let them talk.
Developers are terrible at this. Developers want to help users they see struggling, but that struggle is exactly what the developers need. Let your users struggle, then find out why they struggled, and what you can to prevent it. Resist the urge to help the user. You may be helping them now, but you really aren’t helping yourself in the long run.