Automated testing is definitely a good thing. Nearly all development teams should do more of it. But it’s still important for humans to test, because people are better than machines at lots of things, and they’re more flexible.
Having said that, testing is boring, and testers often forget that what they should be doing is this:
- Define a test script that includes all reasonable test cases.
- Include cases where the users are doing stupid things.
- Include edge cases, but don’t fixate on them.
- Run through your test cases.
- While you test, look at the whole page, not just the specific items scheduled to be tested.
- Test in all the browsers you need to support. Try to avoid testing in really old annoying browsers if you don’t need to support them.
If something does not work as expected:
- Does the same issue exist in production?
- Can you recreate the issue?
If it is a bug, you should create a bug report. Writing bug reports should be easy, but a lot of people get it wrong, because they don’t understand how the reports are used.
Joe Strazzere has written a good article on this and has a good answer on Stack Exchange.
If you can see more than one problem, you should create a bug report for each problem.
All bug reports should include answers to the following questions:
- Where can I find the problem?
- What is the problem?
- How do I know what would be regarded as fixing the problem?
And they should have these:
- A meaningful summary
- One sentence that explains what the issue is and differentiates it from other issues.
- i.e. not “significant difference between site and spec” or “PLEASE FIX THIS”
- A longer description, which includes:
- The steps to recreate the issue
- The URL where the issue occurs
- The expected result
- The actual result
- A screenshot
- This should be a JPG or PNG, not an enormous BMP file, or a screen print pasted into a Word document
Incidentally, there’s no need to say “please”. Politeness is best saved for real interaction - in a bug tracker it’s just noise. Also, don’t bother explaining why it’s important that the problem is fixed. That’s what the priority field is for. The signal to noise ratio is important.
Also, these things don’t count as screenshots, although over the years I’ve had clients who thought they did:
- a printout of the screen, with hand-written comments on, scanned
- a photo taken from your phone
All of this advice can be boiled down to three sentences:
- Use your eyes.
- Use your brain.
- Be better than a robot.
And now a song: