Handling Non-Required Data in Bug Reports

Congratulations, you just found a bug in an office supply commerce application!  It appears that any time you submit an order with more than one line item; an “object reference not found” error is thrown.  Cool!  Here are the repro steps you logged in the bug report:

1. Create a new order.
2. Add Pencils to the new order.
3. Add Pens to the new order.
4. Select UPS for shipping method.
5. Select Credit Card for payment method.
6. Submit the new order.

Expected Result: No errors are thrown.

Actual Results: “Object reference not found” error is thrown.

Those are pretty solid repro steps.  Every time one performs them, the Actual Results occur.  Nevertheless, do you see any problems with the above repro steps?

Hmmmmm….

What if you depend on just the repro steps to understand the bug?

Does this bug only repro if pens and pencils are ordered, if the UPS shipping method is selected, if the Credit Card payment method is selected?

Those details capture what the tester did, but might they mislead?  Anytime you add specific data to your repro steps, you may be implying, to someone, that said data is required to repro the bug (e.g., perhaps the “pens” data in our DB is bad).  A more skilled tester may log the bug like this:

1. Create a new order.
2. Add at least two line items to the new order.
3. Select a shipping method.
4. Select a payment method.
5. Submit the new order.

Okay, that seems considerably better to me.  However, we can still improve it.  Is it reasonable to assume the programmers, testers, and business analysts on our project team know how to submit an order?  Do they already understand that in order to submit an order, one must specify shipping and payment methods?  Yes!

Thus, an even more skilled tester may end up with these repro steps:

1. Create a new order.
2. Add at least two line items to the new order.
3. Submit the new order.

There is nothing extra to cloud our interpretation.  The steps look so minimal and easy, anyone could do them!  Now that’s a set of repro steps we can be proud of.