Daily Archives: December 22, 2010

White Box QA Test of a 3 text-field and 2 pushbutton web panel…

In an interview, I was asked how I would construct a White Box QA test for a screen with

a Name field,
a password field,
a passwordConfirm field,
an OK button and
a Cancel button.:

I set down my best effort, below, on paper, and then checked it against a software testing book I have- “How to Break Software”. I was fairly pleased with what I’d come up with, since what the book says is to concentrate on generating each and every error message. More or less the same as what I had come up with:

name Password Password Screen

Name: [ ______________________]
Password: [_______________]
Password conf: [_______________]
[ OK ] [ Cancel ]

(Implied outputs:
Error Message: [==============];
Dialog box: msg [=============], [ OK ];
Status message: [============]; )

Note that the “Name”, “Password” and “Password conf” lines are explicitly outputs as well as inputs. They are set to something when the screen comes up. The Passwords might or might not be replaced with “*”s or big black dots. [Cancel] might or might not clear the strings

Inputs: Name, Password, PasswordConf, Ok_button, Cancel_button

(I’m conflicted on one point- Password1 and Password2 are useful definitions *inside* where keeping track that they are two, very similar, pieces of information is important. For a user, “Password” and “Password Confirm” are better definitions. I have to pick one nominclature and use it consistently.)

In addition to text in the Name, Password and Password Confirm fields, at least one or two of the following should be available:

Error Message: I assume that we have some space to output a message on error, Have to remember to blank this once we’re out of the error state.

Dialog box: If there’s a message that requires acknowledgement, then a dialog box is really what we’re talking about (modal use of the buttons in the ‘normal’ interface would be evil.) Of Necessity, a dialog box includes both an output line (which may be an error mentioned OR something else..) and an acknowledgement pushbutton.

Status Message: (blank), (ok), (error), (reset), ? – a complex screen, interface or underlying program might have an internal state, and for testing, it would be nice to show it.

By “White Box” I presume we mean we know something about how it works inside, so we know if the screen is well defined and not subject to sequence effects… or we know we DO have to address sequence effects..

For each input, there are relevant lengths, because Name and Password typically have minimum and maximums, and there are error messages relating to them:
For any text input: Name, Password, PasswordConfirm, etc., the 5 possible lengths are:
(1 character,
minimum number of characters – 1,
minimum number of characters,
maximum number of characters,
maximum number of characters + 1)

For each input, there is an acceptable character set and characters which are not allowed:
Alpha: [A-Z,a-z]
numeric: [0-9]
Alpha no case: [Aa]-[Zz]
Shift-numeric: [0-9,[!@#$%^&*()]
Special chars: [`-=[]\;’,./], [~_+{}|:”?].

Out of these categories, or a literal list of values, we can make acceptedNameChar and acceptedPasswordChar sets. We can also make unAcceptedNameChar and unAcceptedPasswordChar sets. Obviously overkill for testing the little screen this started with but worth considering in the case of algorithmically generating a bunch of non-identical test cases for a large set of screens or other objects-under-test.

name, passWd and passWdConf strings of characters are sets of the 5 possible lengths of the acceptable characters, or acceptable + unAcceptable

PRESUMING sequence doesn’t matter, before the Ok_Button is clicked, the test input and output vectors would look like:

Name PwdA PwdB , ErrorOut
—– ——- ——– ———— ————
“a”, “b”, “c” expect errors: Less than minimum , b & c don’t match…
minName-1, minPwdA, minPwdA (expect error: Less than minimum name error)
minName, minPwdA-1, minPwdA (expect error: Less than minimum passwd1)
minName, minPwdB, minPwdB-1 (expect error: Less than minimum passwd2)
minName, minPwdA, minPwdB Ok (expect error: passwds don’t match)
minName, minPwdA, minPwdA Ok (expect success)
minName, minPwdB, minPwdB Ok (expect error, name used already with different passwd- or does this just reset the password to the new one?
maxName, maxPwdB maxPwdB Ok (expect success)
maxName, maxPwdB, minPwdB Ok (expect error, name used already)
maxName, maxPwdA, maxPwdA Ok (expect success)
maxName + 1 , maxPwdB , maxPwdB error More than maximum error.
maxName, maxPwdB +1 , maxPwdB error More than maximum, pwds don’t match.
maxName1 , maxPwdA , maxPwdA+1 error More than maximum, pwds don’t match.

The vectors of input values and single outputs might expand to walking a blank through valid data. walking valid data through blanks. etc.

I prefer using arrays, arrays of structs and enums as indexes to handle this kind of thing. Because I find it easier to inspect and see whether its correct or not. Since the same values are used over and over again, it seems better controlled with the repeated values coming from a single variable.

enum lengths={ aChar, minM1, min, max, maxP1, numLengths };

String[] name=new String( “a”, “minim”, “minimu”,


“minimum_minimum_minimum…minimum_minimum_minimumX” );

String[] passWd=new String( “a”, “minim”, “minimu”,


“minimum_minimum_minimum…minimum_minimum_minimumX” );

String[] passWdB=new String( “b”, “binim”, “binimu”,


“binimum_minimum_minimum…minimum_minimum_minimumX” );

Name, PwdA, PwdB , ErrorOut?
—– —— —— ———— ————
name[ aChar ], passWd[ aChar ], passWd[ aChar ], errors: Less than minimum.
name[ aChar ], passWd[ aChar ], passWdB[ aChar ], errors: Less than minimum , Password and PasswordConf don’t match…
name[ minM1 ], passWd[ min ], passWd[ min ], error: Less than minimum name error
name[ min ], passWd[ minM1 ], passWd[ min ], errors: Less than minimum passwd1, passwords don’t match
name[ min ], passWd[ min ], passWd[ minM1 ], errors: Less than minimum passwd2, passwds don’t match
name[ min ], passWd[ minM1 ], passWd[ minM1 ], errors: Less than min. chars, both pwds.
name[ min ], passWd[ minM1 ], passWdB[ minM1 ], error: passwds don’t match
name[ min ], passWd[ min ], passWd[ min ], expect success
name[ min ], passWdB[ min ], passWdB[ min ], error, name used already with different passwd- or does this just reset the password to the new one?
name[ max ], passWd[ max ], passWd[ max ], expect success
name[ max ], passWdB[ max ], passWdB[ max ], error, name used already?
name[ maxP1 ], passWd[ max ], passWd[ max ], error More than maximum name error.
name[ max ], passWd[ maxP1 ], passWd[ max ], errors More than max pwd, pwds don’t match.
name[ max ], passWd[ max ], passWd[ maxP1 ], errors More than max pwd, pwds don’t match.
name[ max ], passWd[ max ], passWdB[ max ], error, pwds don’t match.
name[ max ], passWd[ maxP1 ], passWd[ maxP1 ], error More than max pwd.

For symmetry, I suppose name, passWd and passWdB could include a second set of strings which incorporate unacceptable characters and 5 more enums to pick them (or an “un” value to addd… or there should be three more arrays named unName, unPassWd, unPassWdB.

Point is, there are a lot of different error messages implied by this relatively simple panel, and to be certain that it behaves for each and every build.

Generally, I think its a good thing to get each error by itself from each possibly position, but not to get too worked up about combinatorials…. so walking a too short or too long string through the three inputs is fine, but having to wrangle TWO too long or too short or one of each is probably overkill- Its worth having ALL the inputs be errors to be sure that arbitrary collections of errors work, together, and you’re done, unless there’s some reason to think errors interact.