1、Part II: Testing Fundamentals Software Testing Part II: Testing Fundamentals Examining the Code ContentsStatic White-Box Testing: Examining the Design and CodeFormal ReviewsCoding Standards and GuidelinesGeneric Code Review ChecklistStatic White-Box Testing: Examining the Design and CodeStatic White

2、-Box Testing: Examining the Design and Code Static white-box testing is the process of carefully and methodically reviewing the software design, architecture, or code for bugs without executing it. Its sometimes referred to as structural analysis.The obvious reason to perform static white-box testin

3、g is to find bugs early and to find bugs that would be difficult to uncover or isolate with dynamic black-box testing. 找到黑盒測(cè)試難以發(fā)現(xiàn)的軟件缺陷A side benefit of performing static white-box testing is that it gives the teams black-box testers ideas for test cases to design and apply when they receive the soft

4、ware for testing. Formal Reviews 正式審查A formal review is the process under which static white-box testing is performed. A formal review can range from a simple meeting between two programmers to a detailed, rigorous inspection of the softwares design or its code.There are four essential elements to a

5、 formal review:Identify Problems. Follow Rules. Prepare. Write a Report. Formal ReviewsWhat makes formal reviews work is following an established process. If the reviews are run properly, they can prove to be a great way to find bugs early. In addition to finding problems, holding formal reviews has

6、 a few indirect results:Communications. Quality. Team Camaraderie. 小組同志化Solutions. Formal ReviewsPeer Reviews 同事審查The least formal method. Sometimes called buddy reviews, this method is really more of an Ill show you mine if you show me yours type discussion.Peer reviews are often held with just the

7、 programmer who designed the architecture or wrote the code and one or two other programmers or testers acting as reviewers. Formal ReviewsWalkthroughs 走查Walkthroughs are the next step up in formality from peer reviews. the programmer formally presents (walks through) it to a small group of five or

8、so other programmers and testers. The reviewers should receive copies of the software in advance of the review so they can examine it and write comments and questions that they want to ask at the review. Having at least one senior programmer as a reviewer is very important.The presenter reads throug

9、h the code line by line, or function by function, explaining what the code does and why. The reviewers listen and question anything that looks suspicious. Formal ReviewsInspections 檢驗(yàn)Inspections are the most formal type of reviews. They are highly structured and require training for each participant

10、. The presenter or reader, isnt the original programmer. The other participants are called inspectors. Some inspectors are also assigned tasks such as moderator and recorder to assure that the rules are followed and that the review is run effectively.Inspections have proven to be very effective in f

11、inding bugs in any software deliverable, especially design documents and code, and are gaining popularity as companies and product development teams discover their benefits.Coding Standards and Guidelines Guidelines are the suggested best practices, the recommendations, the preferred way of doing th

12、ings. Standards have no exceptions, short of a structured waiver process. Guidelines can be a bit loose. There are three reasons for adherence遵守 to a standard or guideline:Reliability. 可靠性Readability/Maintainability. 可讀性，維護(hù)性Portability. 移植性Coding Standards and GuidelinesThe standard has four main pa

13、rts:Title describes what topic the standard covers.Standard (or guideline) describes the standard or guideline explaining exactly whats allowed and not allowed.Justification解釋說(shuō)明 gives the reasoning behind the standard so that the programmer understands why its good programming practice.Example shows

14、 simple programming samples of how to use the standard. This isnt always necessary.（加圖）A sample coding standard explains how several language control structures should be used. An example of a programming guideline shows how to use certain aspects of C in C+. Coding Standards and GuidelinesObtaining

15、 Standards National and international standards for most computer languages and information technology can be obtained from:American National Standards Institute (ANSI), International Engineering Consortium (IEC), International Organization for Standardization (ISO), www.iso.chNational Committee for

16、 Information Technology Standards (NCITS), There are also documents that demonstrate programming guidelines and best practices available from professional organizations such asAssociation for Computing Machinery (ACM), Institute of Electrical and Electronics Engineers, Inc (IEEE), Generic Code Revie

17、w ChecklistSome problems you should look for when verifying software for a formal code review. Data Reference Errors Data reference errors are bugs caused by using a variable, constant, array, string, or record that hasnt been properly declared or initialized for how its being used and referenced.Is

18、 an uninitialized variable referenced? Are array and string subscripts integer values and are they always within the bounds of the arrays or strings dimension?Are there any potential off by one errors in indexing operations or subscript references to arrays? Generic Code Review ChecklistData Referen

19、ce Errors (continued) Is a variable used where a constant would actually work betterIs a variable ever assigned a value thats of a different type than the variable? Is memory allocated for referenced pointers?If a data structure is referenced in multiple functions or subroutines, is the structure de

20、fined identically in each one?Generic Code Review ChecklistData Declaration Errors Data declaration bugs are caused by improperly declaring or using variables or constants. Are all the variables assigned the correct length, type, and storage class?If a variable is initialized at the same time as its

21、 declared, is it properly initialized and consistent with its type?Are there any variables with similar names? Are any variables declared that are never referenced or are referenced only once?Are all the variables explicitly declared within their specific module? Generic Code Review ChecklistComputa

22、tion Errors Computational or calculation errors are essentially bad math. The calculations dont result in the expected result. Do any calculations that use variables have different data types, such as adding an integer to a floating-point number?Do any calculations that use variables have the same d

23、ata type but are different lengths adding a byte to a word, for example?Are the compilers conversion rules for variables of inconsistent type or length understood and taken into account in any calculations?Is the target variable of an assignment smaller than the right-hand expression?Is overflow or

24、underflow in the middle of a numeric calculation possible?Generic Code Review ChecklistComputation Errors(continues)Is it ever possible for a divisor/modulus to be zero?For cases of integer arithmetic, does the code handle that some calculations, particularly division, will result in loss of precisi

25、on?Can a variables value go outside its meaningful range? For example, could the result of a probability be less than 0% or greater than 100%?For expressions containing multiple operators, is there any confusion about the order of evaluation and is operator precedence correct? Are parentheses needed

26、 for clarification?Generic Code Review ChecklistComparison Errors Less than, greater than, equal, not equal, true, false. Comparison and decision errors are very susceptible to boundary condition problems. Are the comparisons correct? It may sound pretty simple, but theres always confusion over whet

27、her a comparison should be less than or less than or equal to.Are there comparisons between fractional or floating-point values? If so, will any precision problems affect their comparison? Is 1.00000001 close enough to 1.00000002 to be equal?Does each Boolean expression state what it should state? D

28、oes the Boolean calculation work as expected? Is there any doubt about the order of evaluation?Are the operands of a Boolean operator Boolean? For example, is an integer variable containing integer values being used in a Boolean calculation?Generic Code Review ChecklistControl Flow Errors Control fl

29、ow errors are the result of loops and other control constructs in the language not behaving as expected. They are usually caused, directly or indirectly, by computational or comparison errors. If the language contains statement groups such as begin.end and do.while, are the ends explicit and do they

30、 match their appropriate groups?Will the program, module, subroutine, or loop eventually terminate? If it wont, is that acceptable?Is there a possibility of premature loop exit?Generic Code Review ChecklistControl Flow Errors (continues)Is it possible that a loop never executes? Is it acceptable if

31、it doesnt?If the program contains a multiway branch such as a switch.case statement, can the index variable ever exceed the number of branch possibilities? If it does, is this case handled properly?Are there any off by one errors that would cause unexpected flow through the loop?Generic Code Review

32、ChecklistSubroutine Parameter Errors Subroutine parameter errors are due to incorrect passing of data to and from software subroutines. Do the types and sizes of parameters received by a subroutine match those sent by the calling code? Is the order correct?If a subroutine has multiple entry points (

33、yuck), is a parameter ever referenced that isnt associated with the current point of entry?If constants are ever passed as arguments, are they accidentally changed in the subroutine?Does a subroutine alter a parameter thats intended only as an input value?Do the units of each parameter match the uni

34、ts of each corresponding argument English versus metric, for example?If global variables are present, do they have similar definitions and attributes in all referencing subroutines?Generic Code Review ChecklistInput/Output Errors These errors include anything related to reading from a file, acceptin

35、g input from a keyboard or mouse, and writing to an output device such as a printer or screen. Does the software strictly adhere to the specified format of the data being read or written by the external device?If the file or peripheral isnt present or ready, is that error condition handled?Does the

36、software handle the situation of the external device being disconnected, not available, or full during a read or write?Are all conceivable errors handled by the software in an expected way?Have all error messages been checked for correctness, appropriateness, grammar, and spelling?Generic Code Revie

37、w ChecklistOther ChecksWill the software work with languages other than English? Does it handle extended ASCII characters? Does it need to use Unicode instead of ASCII?If the software is intended to be portable to other compilers and CPUs, have allowances been made for this? Portability, if required

38、, can be a huge issue if not planned and tested for.Has compatibility been considered so that the software will operate with different amounts of available memory, different internal hardware such as graphics and sound cards, and different peripherals such as printers and modems?Does compilation of

39、the program produce any warning or informational messages? They usually indicate that something questionable is being done. Purists would argue that any warning message is unacceptable.Part II: Testing Fundamentals Testing the Software with X-Ray GlassesContentsDynamic White-Box TestingDynamic White

40、-Box Testing Versus DebuggingTest Adequacy Measurement and Enhancement ：Control and Data flowData flow coverageWhat is test adequacy? What is test enhancement? When to measure test adequacy and how to use it to enhance tests?Control flow based test adequacy; statement, decision, condition, multiple

41、condition, LCSAJ, and MC/DC coverageStrengths and limitations of code coverage based measurement of test adequacyTools for the measurement of code coverageThe “subsumes” relation amongst coverage criteria LEARNING OBJECTIVESWhat dynamic white-box testing isThe difference between debugging and dynami

42、c white-box testingDynamic White-Box TestingDynamic White-Box TestingDynamic white-box testing is using information you gain from seeing what the code does and how it works to determine what to test, what not to test, and how to approach the testing. Another name commonly used for dynamic white-box

43、testing is structural testing because you can see and use the underlying structure of the code to design and run your tests. Dynamic White-Box TestingWhy would it be beneficial for you to know whats happening inside the box, to understand how the software works? Dynamic White-Box TestingDynamic whit

44、e-box testing encompassesDirectly testing low-level functions, procedures, subroutines, or libraries. Testing the software at the top level, as a completed program, but adjusting your test cases based on what you know about the softwares operation.Gaining access to read variables and state informati

45、on from the software to help you determine whether your tests are doing what you thought. And, being able to force the software to do things that would be difficult if you tested it normally.Measuring how much of the code and specifically what code you hit .Dynamic White-Box Testing Versus Debugging

46、The two techniques may appear similar because they both involve dealing with software bugs and looking at the code, but theyre very different in their goals. The goal of dynamic white-box testing is to find bugs. The goal of debugging is to fix them. They do overlap, however, in the area of isolatin

47、g where and why the bug occurs. Test adequacySuppose now that a set T containing k tests has been constructed to test P to determine whether or not it meets all the requirements in R . Also, P has been executed against each test in T and has produced correct behavior. T是關(guān)于P測(cè)試用例的集合，而且所有測(cè)試用例都能檢測(cè)到所有需求。

48、P執(zhí)行了所有測(cè)試，而且都有正常的行為。Consider a program P written to meet a set R of functional requirements. We notate such a P and R as ( P, R). Let R contain n requirements labeled R1, R2, Rn . 所有需求We now ask: Is T good enough? This question can be stated differently as: Has P been tested thoroughly?, or as: Is T

49、adequate? What is adequacy? Adequacy is measured for a given test set designed to test P to determine whether or not P meets its requirements. In the context of software testing, the terms “thorough,” “good enough,” and “adequate,” used in the questions above, have the same meaning. This measurement

50、 is done against a given criterion C .標(biāo)準(zhǔn) A test set is considered adequate with respect to criterion C when it satisfies C. The determination of whether or not a test set T for program P satisfies criterion C depends on the criterion itself and is explained later.Measurement of adequacyProgram sumPr

51、oduct must meet the following requirements:R1 Input two integers, say x and y , from the standard input device. R2.1Find and print to the standard output device the sum of x and y if xy .R2.2 Find and print to the standard output device the product of x and y if x y.ExampleSuppose now that the test

52、adequacy criterion C is specified as: C : A test T for program ( P, R ) is considered adequate if for each requirement r in R there is at least one test case in T that tests the correctness of P with respect to r .Obviously, T=t: is inadequate with respect to C for program sumProduct. The lone（singl

53、e） test case t in T tests R1 and R2.1, but not R2.2. Example (contd.)For each adequacy criterion C , we derive a finite set known as the coverage domain and denoted as Ce . A criterion C is a white-box test adequacy criterion if the corresponding coverage domain Ce depends solely單獨(dú)地 on program P und

54、er test.A criterion C is a black-box test adequacy criterion if the corresponding coverage domain Ce depends solely on requirements R for the program P under test.Black-box and white-box criteriaWe want to measure the adequacy of T. Given that Ce has n 0 elements, we say that T covers Ce if for each

55、 element e in Ce there is at least one test case in T that tests e. The notion of “tests” is explained later through examples. T is considered adequate with respect to C if it covers all elements in the coverage domain. T is considered inadequate with respect to C if it covers k elements of Ce where

56、 kn . The fraction k/n is a measure of the extent to which T is adequate with respect to C . This fraction is also known as the coverage of T with respect to C , P , and R . 測(cè)試、程序、要求CoverageLet us again consider the following criterion: “A test T for program ( P, R ) is considered adequate if for ea

57、ch requirement r in R there is at least one test case in T that tests the correctness of P with respect to r.”In this case the finite set of elements Ce=R1, R2.1, R2.2. 測(cè)試用例 T covers R1 and R2.1 but not R2.2 . Hence T is not adequate with respect to C . The coverage of T with respect to C, P, and R

58、is 0.66.ExampleConsider the following criterion: “A test T for program ( P, R ) is considered adequate if each path in P is traversed at least once.”Assume that P has exactly two paths, one corresponding to condition xy and the other to x y. We refer to these as p1 and p2, respectively. For the give

59、n adequacy criterion C we obtain the coverage domain Ce to be the set p1, p2. 兩個(gè)路徑Another ExampleTo measure the adequacy of T of sumProduct against C , we execute P against each test case in T . As T contains only one test for which xy , only the path p1 is executed. Thus the coverage of T with resp

60、ect to C, P , and R is 0.5 and hence T is not adequate with respect to C. We can also say that p1 is tested and p2 is not tested.Another Example (contd.)In the previous example we assumed that P contains exactly two paths. This assumption is based on a knowledge of the requirements. However, when th