“Testing – How does one learn QA?” – An answer I posted on the StackOverflow “Programers” forum


Ziv, the questioner asks: ” … how would one proceed if he wants to learn QA?

More specifically, a programmer who wants to learn about the QA process and how to manage a good QA methodology. I was given the role of jumpstarting a QA process in our company and I’m a bit lost, what are the different types of testing (system, integration, white box, black box) and which are most important to implement first? How would one implement them?

I wrote:

There are simple rules of thumb.

Try what the manual says. Install and run on a clean target, user license, the works. Does it work? Did you have to add anything not covered in the manual?

Are all the default control values usable? Or is there something that’s wrong, or blank, by default and always has to be changed?

Set every value in the user interface to something other than its default. Can you detect a difference caused by the change? Is it correct? Do them one at a time, or in the smallest sets possible, to make the results clear.

Set every value in the user interface to a second, non-default, value. Change everything at once. Can you detect the difference? Is it correct?

One by one, do something to cause every error message to be generated. Do something similar, but correctly, so that no error message is generated.

All of the above depend on changing a condition, between an “A” case and a “B” case, and that change having a detectable result. Then the “C” case produces another change, another result, and so forth. For 10 tests, you need 11 conditions. Using defaults as much as possible is a good first condition.

By now you’ve got a list of things to test, that you recorded, and results, that you recorded, and maybe some new bugs. Throw something big and complicated at the solution. Give it a file of 173000 words to sort, paste a Jane Austin novel or some telecommunications standard 100 pages long, a 50MB bitmap graphic, 3 hours of streaming video. Open the performance monitor and get CPU-bound, or I/O bound. For an hour. Check memory use: always increasing? Rises and falls?

Take the list of bugs closed in the last week, month, sprint, etc. Check them. All. Are they really fixed?

Keep track of what to do, how it worked on what version/release/build/configuration, open and closed bugs, what controls have been set or changed, what data, test files or examples have been used, etc. is all part of Quality world. Keep results as tables in a spread sheet, make version controlled backups / saves.

Someone writing software, or any one creating anything, has an idea of what they’re trying to make. The quality process starts with expectations. Requirements, specifications, rules, or another articulation of what’s expected. Then there’s the solution, the thing offered to perform, assist, enable or automate what’s expected. Then there are tests, operations, examples, inspections, measurements, questionnaires, etc., to relate one or more particular solution(s) to (relevant) expectations. Finally, there’s an adjustment, compensation, tuning, correction or other positive action that is hoped to affect the solution(s).

When one writes software, one has a goal of it doing something, and to the extent that’s expressed, the behavior can be checked. Hello.exe displays “Hello World” on a screen. “2**150″ in the Python interpreter displays, “1427247692705959881058285969449495136382746624L”. Etc. For small problems and small solutions, its possible to exhaustively test for expected results. But you wouldn’t test a word processor just by typing in some words, or even whole documents. There are limits of do-ability and reason. If you did type in all of “Emma” by Jane Austin, would you have to try her other four novels? “Don Quixote” in Spanish?

Hence an emphasis on expectations. Meeting expectations tells you when the solution is complete. My web search for “Learn Quality Assurance” just returned 46 million potential links, so there’s no shortage of opinions. Classic books on the subject (my opinion, worth what you paid for it:) include

  • Quality is Free” by Philip Crosby,
  • Zen and the Art of Motorcycle Maintenance” by Robert Pursig
  • Managing the Software Process” by Watts Humphrey
  • “The Mythical Man Month” by Fred Brooks
  • Code Complete” by Steve McConnell

Take 5 minutes to read some of the Amazon reviews of those books and you’ll be on your way. Get one or more and read them. They’re not boring. Browse ASQ, Dr. Dobbs, Stack Overflow. Above all, just like writing software. DO it. Consider the quality of some software under your control. Does it meet expectation? If so, firm hand-shake and twinkle in the eye. Excellent!. If not, can it be corrected? Move to the next candidate.

I like the Do-Test-Evaluate-Correct loop, but its not a Universal Truth. Pick a process and follow it consciously. Have people try the testing, verification and validation steps described in the language manual they use most frequently. Its right there on their desk, or in their phone’s browser.

Look at your expectations. Are they captured in a publicly known place? With revision control? Does anyone use them? Is there any point where the solutions being produced are checked against the expectations they are supposed to be meeting?

Look at your past and current bug reports. (You need a bug tracking system. If you don’t have one, start there.) What’s the most common catastrophic bug that stops shipment or requires an immediate patch? Whats the most commonly reported customer bug? What’s the most common bug that doesn’t get fixed?

Take a look at ISO 9000 process rules. Reflect on value to your customers/users. Is there’s a “customer value statement” that explains how some change affects the customer’s perception of the value of the solution? How about in the requirements?

By “the QA process”, you could mean “Quality Assurance”, versus “QC”, “Quality Control”? You might start with the http://www.ASQ.org web site, where the “American Society for Quality” dodges the question by not specifying “Control” (their old name was “ASQC”) or “Assurance”.

Quality; alone, “assured” or “controlled”, is a big idea with multiple, overlapping definitions and usages. Some will tell you it cannot be measured in degrees- its present or not, no “high quality” or “low quality” for them. Another famous claim is that no definition is satisfactory, so its good to talk about it, but avoid being pinned down in a precise definition. How do you feel about it?

 

The original posting is at http://programmers.stackexchange.com/questions/255583/how-does-one-learn-qa/255595#255595

A sad, harsh, somewhat funny (ironically) reminder that all is not well, and never will be. Sheesh.


A reader offered the following comment, on a small technical article, here. I checked out his site and its pretty consistent. I choose to regard it as a cry for help, but that could be overgenerous. Count your blessings, friends. The power of youth and confusion are always around the corner…  and yet, I neither wanted to just delete this or pass it along as sent, you know?

[Edited (###### added) by me]

 

New comment waiting approval on bill abbott’s weblog

###########  commented on SPOILER ALERT! More about the NetBeans Anagram game

If you want to read about the NetBeans Anagram.java program but do NOT want a hint, just scroll down past this post to the one …

f##k you bill, you suck, monkey

Approve  Trash | Mark as Spam

More information about ########

IP: 83.244.###.### 83-244-###-###-cust-83.exponential-e.net
E-mail: #############@gmail.com
URL: http://sosickofthislife.blogspot.com
Whois: http://whois.arin.net/rest/ip/83.244.###.###

Image

Father’s day tides at Moss Beach:


Here’s the tide table for this coming weekend at Moss Beach, just north of Princeton By The Sea, at the north edge of Half Moon Bay. High tide, +6 feet, at Midnight between Friday and Saturday, 1:00am between Saturday and Sunday. Low, low, tides at 7:00am, -1.5 feet!! on Saturday, -1.25 feet, at 7:48am, Sunday.
So, by crackie, we’ll be there as early as we an on Sunday. Sunrise is before 6:00am, so no shortage of light. Do a web search and you’ll discover this place has the best tidepools that ever existed- perhaps 1/4 mile or more along the coast, as much as 200 yards off shore of the normal high tide mark. A huge shelf of very low quality rock, normally around or perhaps a bit below the 0 foot level, that will be a good foot above sea level on Sunday Morning.

Ha! I now have a trivial python program the works interactively! momdad.py:


“””
“””

import os

print “os.listdir(os.getcwd())”
print os.listdir(os.getcwd())

for fname in os.listdir(os.getcwd()):
print fname
text=open(fname).read()
print “text.count( hi mom )”
print text.count(‘hi mom’)

Link

A philosophic reason to sets pointers to NULL after free’ing them.


A philosophic reason to sets pointers to NULL after free’ing them.

On reflection, I think I got this one right, and I kinda like it! I have experienced other people’s double-free errors, surprisingly, but the value of crashing-on-access-via-null is persuasive to me. Making double-free happen silently and without error is a small cost for positively crashing on a rogue access through a freed pointer.

Hawker Hurricane Mk I, 4/27/1939-6/5/1940 dH 2-position prop, “B” pattern camo, starboard profile, 1/2 white underside, v.10


Mid-production Hawker Hurricane, Mk I, with Rotol constant-speed prop and a Spitfire spinner (the dedicated Hurricane spinner arrived in late 1940). This is what one would look like if it was painted all together, and they stopped before applying the national and service markings. Just “B” pattern camouflage on top, Dark Green and Dark Earth. Starboard side white, port side black, underneath, meeting at the centerline. Nothing of the original “Aluminum” finish remains outside, but the wheel wells and inside of the undercarriage doors might well be “Aluminum”, still.
The “A” and “B” camouflage patterns were mirror images, so this starboard, “B” scheme, is the same as a port-side “A” scheme, except reversed left to right
The black/white underside recognition features was ordered in August 1938. While the requirement for black under the port wing was made known fairly readily, how to treat the starboard wing, horizontal stabilizers and fuselage underside was somewhat to very unclear between 8/38 and 4/39. The intent was to have the port side black up to the center of the fuselage, and the starboard side white, up to the center of the fuselage, as this drawing and its companion show.

Hawker Hurricane, 1939, dH 2 position prop, “B” scheme camo, port profile. Black and White meet halfway under the wing. v.11


Another mid-period Hurricane. Between Munich (August, 1938) and the invasion of Poland (September, 1939), the, still, “peacetime” RAF expanded dramatically and was flooded with new technology. While everyone more or less understood the written instructions to paint the underside of the port wing black, for recognition when airborne, what to do with the rest of the plane was not always grasped. Black and white across the undersurface of the wing, meeting at the center line, was a durable interpretation, frequently seen, with the rest left as built, in Aluminum finish. Here’s what that would look like with no national or service markings.

Note, the “B” scheme camouflage is the mirror image of the “A”, same blobs of color, but on the other side. So that a row of airplanes would not stand-out by looking identical.