This week at work has taught me an important lesson about making (and asking for) effective bug reports: screenshot screenshot screenshot.

Bug reports are tricky because of two enormous gaps between the user making the report and the developer reading it. Formally speaking you might call these “information asymmetry” and “lack of a shared vocabulary”. In plain terms: the user doesn’t know what they need to tell the developer, the developer doesn’t know what questions to ask, and they don’t speak the same language anyway.

The symptom of these problems is the dreaded “Could not reproduce” (otherwise known as “It works fine for us, problem must be on your end”), and the cure is: send a screenshot.

You the user

Suppose you hit a bug in the search system of my company’s web application. You send a bug report that ends up with me. As a developer on this system, I happen to know that the search implementation is different for your Inbox and for folders you’ve made yourself. But you didn’t know that was important, so you didn’t include in your bug report which folder you were searching inside. You shouldn’t have to know that this might be important, but now I’m going to mail you to ask, or do extra work trying to reproduce your bug under all the possible conditions.1 Send me a screenshot, and I’ll see the difference immediately.

It turns out that you were searching in your Inbox, but now I’m doing the same and nothing goes wrong. What I don’t realise is, you used the More Search Options popup to build your query, while I copy/pasted it from your bug report straight into the search field. Shouldn’t make a difference, but in fact that’s where the bug lives. Now the information asymmetry goes the other way: I don’t know the question I need to be asking you. Again, though, a screenshot of the moment you hit the “search” button will show the MSO popup, and (if I’m doing my job properly) I’ll do exactly what you did before deciding that I can’t reproduce the bug.

You the developer

I ran into this problem, without recognising it, reporting an issue with the music player Amarok. My mouse wheel control of the volume control was inverted: scrolling “up the page” lowered the volume, and vice versa. (OS X users will recognise the phenomenon.) I explained the problem carefully, and conscientiously gave OS details and version numbers and so on. The developer said she couldn’t reproduce and thought I probably had a non-standard KDE library somewhere on my system.

Turns out that Amarok has more than one volume control, and different ones show depending on choice of a theme or a skin or some such configuration. We weren’t even talking about the same widget: “the volume control” meant something completely different to her than to me.

The words you, the developer, use to describe your system are likely to be very different from the ones that your users choose. You can try to force users to adopt your terminology; it won’t work. You can spend a lot of time mailing back and forth pinning down exactly what they’re talking about.

Or you can ask for a screenshot.

Full window please

Some of our bug reports come with nicely cropped shots of exactly the widget or panel that is behaving oddly, with a big red arrow pointing to the bit that doesn’t work. It’s cute, and I appreciate the effort, but a screenshot of the entire window is much more helpful. The reason, of course, is context: the surrounding context (missing from the cropped version) is where we find all the answers to the questions we didn’t know to ask, and you didn’t know to answer. Developers, the surrounding context is where you’ll see the factor you didn’t include in your test setup and didn’t imagine could possibly be relevant. A full-window screenshot of a web app shows at least part of the URL; it shows what kind of browser is being used. (Yes, those details should always be included in a bug report. It’s when neither you nor the user realises that they’re relevant that they come out and bite you.) It shows the user has his window resized to some weird aspect ratio that nobody on the dev team ever imagined, and it turns out it’s because he’s got another window watching live snooker open and also a chat client and he didn’t think to mention it because isn’t this how everyone uses the site?

It shows the things you don’t know to ask about, and that’s why you should always ask for a screenshot.

Notes:

  1. Sure, two variants is easy. But three yes/no questions I don’t ask gives me eight trials I have to run, and four gives me sixteen. That gets old pretty fast. [↪]