Continued from page 1
Expecting an unsupervised beta testing group to do anything meaningful - Beta testers need well defined goals, constant supervision and strong leadership to be successful. Without these things beta testing is simply a numbers game which does nothing useful at all.
Testing is not appropriate time to make design decisions - Something that I commonly see from Microsoft and other large companies is they create a product and send it out to their beta testers for feedback. Guys, come on. Beta testing is not place for this. Design decisions need to be made well before a product is sent out for testing. You want to find out if your users will like a feature? Create a prototype, send it to a statistically valid sample audience and ask them for their opinions. Clearly define it to audience as a prototype and survey them for their opinions afterwards. Don't send out a poorly defined "beta test" to a hundred thousand people and try and get their opinions on features. The only thing you are going to accomplish is to get yourself slammed in media. You also find yourself making design changes in a product at wrong stage of product life cycle. Design changes need to be made during analysis and design phases of project, not AFTER implementation.
So how is good testing done?
Analysis and design must be done first - No matter how large project, you will be much more likely to succeed if you do these two steps thoroughly and completely before implementation and testing. Many years ago I had a boss, name of Gary, who didn't understand this simple rule. He asked me to start implementing a warehousing system for a client without writing a specification (over my objections). His concept of design was to spend an hour or two asking customer what was needed, then to start coding, then to show customer, make changes, code some more, show customer, make changes and so on until customer said "it's fine". Needless to say, project took far longer than necessary and did not do a great job of meeting customers needs. It was also very buggy and required an immense amount of support during first couple of years of it's life cycle.
The only phase where marketing department should be involved is analysis - A well trained analyst understands that marketing department is a customer and must be included in analysis phase of project. This is only time (until product is through testing) that marketing should have any input. If you don't follow this rule you will wind up with a product which changes direction during testing, and thus invalidates your test.
Understand that a specification is a contract - The goal is to implement something that meets specification. This is only way that I know of to produce a software project that (a) gets finished at all and (b) meets customers expectations. Of course, this assumes that your analysis and design is top notch. What does this really mean? It simply means that changes to design are only allowed during analysis and design phase. Period. If your customer changes anything at all after analysis and design, you must reanalyze, redesign and renegotiate - always and without fail.
Let's say you are contractor who has been hired to create a new warehouse system. You do your analysis and design and it is approved by customer. You now have a contract and it is important that your customer understands this from beginning. Okay, you begin project and your customer decides he wants to add bar coding. This seems pretty simple so you say "sure". Wrong thing to do. You should say either "let's finish project as designed then add things" or "okay, we will need to stop, see how that effects project (at customers cost), then we will submit a cost and new delivery date".
Maintain standards - Testing measures implementation against specifications and standards. Standards should be made known to customer as part of entire package. These might include things like all fields will be validated in specific ways, all buffers will not overflow, screens will have a certain look and so on.
Remember purpose of testing - Testing should prove implementation meets everything included in specifications and standards. Testing does NOT mean product is measured against customer expectations (that's a marketing function which should have been nailed down during analysis and design phase). You see, specification MUST meet customer expectations before implementation beings. Then final product WILL meet customer expectations as specification is expectation.
Richard Lowe Jr. is the webmaster of Internet Tips And Secrets at http://www.internet-tips.net - Visit our website any time to read over 1,000 complete FREE articles about how to improve your internet profits, enjoyment and knowledge.