1、Outline1TDD and xUnit FrameworkTest FixtureResult VerificationTest DoublesTest OrganizationThe Value of Self-Testing CodeIf you look at how most programmers spend their time, youll find that writing code is actually a small fraction. Some time is spent figuring out what ought to be going on, some ti

2、me is spent designing, but most time is spent debugging. Fixing the bug is usually pretty quick, but finding it is a nightmare. And then when you do fix a bug, theres always a chance that another one will appear and that you might not even notice it until much later. Then you spend ages finding that

3、 bug. Martin Fowler2Automated Testing is NecessaryWriting good test code is hard, and maintaining obtuse code is even harder.Because test code is optional, there is a strong temptation to abandon testing when the tests become difficult or expensive to maintain. 3How Tests Support S/W DevelopmentBefo

4、re code is writtenTests as SpecificationWhat the code should dClarification of the RequirementsAfter code is writtenTests as DocumentationWhat the code doesTests as Safety NetProvide rapid feedback on qualityDefect LocalizationReduce cost to fix by Minimizing Debugging4Automated Testing is Hard - Fr

5、agile TestsInterface Sensitivity Tests depend on API or the user interface. Even minor changes to the interface can cause tests to fail.Behavior Sensitivity A behavior change should affect only a few tests. Data SensitivityParticular problem with database systems. Tests are broken by changes to the

6、data. Context SensitivityTests are broken by differences in the environment surrounding the SUT. 5xUnit The xUnit family of test automation frameworks is designed for use in automating programmer tests.Make it easy for developers to write tests without needing to learn a new programming language. Ma

7、ke it easy to test individual classes and objects without needing to have the rest of the application available. Test software from the insideMake it easy to run one test or many tests with a single action. Minimize the cost of running the tests so programmers arent discouraged from running the exis

8、ting test. 6A Holistic View 7TeardownSUTSUT: Software Under TestDOCDOC: Depended-On ComponentSetupInitializeExerciseDirect Inputs(Control Points)VerifyDirect Outputs(Observation Points)Indirect Inputs(Control Points)Indirect Outputs(Observation Points)Test RunnerTo run tests and provide the interfac

9、es for the users to specify, invoke, and observe the results of running tests. Graphical Test RunnerCommand-Line Test Runner 8RunnertestMethod_1TestCasetestMethod_nOrganizeCreateRunDisplayTest FixtureEverything we need to have in place to exercise the SUT. SUT: An instance of the class under testDOC

10、: the objects which the method interacts withMay be called differently in different tools. 9FixtureTest ResultsSuccessTest runs without any errors or failuresFailThe test assertion fails. Error / ExceptionEither the SUT or the test itself fails in an unexpected way.10Runtime Test Structure11Test Sui

11、teFactory Test SuiteObjectTestCaseObjectTestMethodTestCaseObjectTestMethodRunRunFixturevRunnerTest Fixture12Test Fixture in GeneralElectronicsIn testing electronic equipment such as circuit boards, a test fixture is a device or setup designed to hold the device under test in place and allow it to be

12、 tested by being subjected to controlled electronic test signals. Physical testingA physical fixture is a device or apparatus to hold or support the test specimen during the test. The influence of test fixture on test results is important. 13Test structure fixtures on large seismic table for earthqu

13、ake tests. A functional test fixture is a complex device to interface the DUT (Device Under Test) to the ATE (Automatic Test Equipment). Software Test FixtureA fixed state of the software under test used as a baseline for running tests. It may also refer to the actions performed in order to bring th

14、e system into such state. For examplesLoading a database with a specific, known set of dataErasing a hard disk and installing a known clean operating system installationCopying a specific know set of filesPreparation of input data and set-up/create of fake or mock objects. In xUnitSet up: to create

15、the expected state for the testTear down: to clean up what had been set up. 14Fresh Fixture Strategy15A large part of making tests repeatable and robust is ensuring that the test fixture is torn down after each test and a new one created for the next test run. SetupExerciseVerifyTeardownFixtureIn-li

16、ne Fixture SetupThe test handles all of the fixture setup within the body of the Test method. Construct SUT objectsCall methods of SUT objectsConstruct SUTCall SUT methods to put into a specific state16In-line Fixture Setup17Public void testStatus_initial () / In-line setup Airport departureAirport

17、= new Airport (“Calgary”, “YYC”); Airport destinationAirport = new Airport (“Toronto”, “YYZ”); Flight flight = new Flight (flighNumber, departureAirport, destinationAirport); / Exercise SUT and verify outcome assertEquals (FlightState.PROPOSED, flight.gestStatus (); / tearDown- Duplicated Code - Dev

18、elopment cost is high- Hard to maintain- Lead to hard-coded test dataDelegated SetupTo reduce test code duplication, we can refactor the sequence of statement into a utility method. 18Public void testStatus_initial () / Delegated setup Flight flight = createAnomymousFlight (); / Exercise SUT and ver

19、ify outcome assertEquals (FlightState.PROPOSED, flight.gestStatus (); / tearDown19Test_1Test_2Test_n.Test CaseSetupSetupSetupSUTDOCFixtureIn-LineFixtureSetupTest_1Test_2Test_n.Test CaseTest MethodSUTDOCFixtureDelegatedFixtureSetupVariations of Creation MethodAnonymous creation methodParameterized cr

20、eation methodAllows the test to pass in some attributes to be used in the creation of the object. Should pass only those attributes that are expected to affect. Named state reaching methodWhen the SUT is state-rich and the validity or behavior of methods is affected by the state of the SUT, it is im

21、portant to test each method from each possible starting state. Attachment methodWe already have a test object and we want to modify it in some way. 20Motivating Example21Public void testPurchase_firstPurchase_ICC () Customer buyer = new Customer (17, “FirstName”, “LastName”, “G”, “ACTIVE”); / Public

22、 void testPurchase_subsequentPurchase_ICC () Customer buyer = new Customer (18, “FirstName”, “LastName”, “G”, “ACTIVE”); / 22Public void testPurchase_firstPurchase_ICC () Customer buyer = createAnonymousCustomer (); / Public void testPurchase_subsequentPurchase_ICC () Customer buyer = createAnonymou

23、sCustomer (); / public Customer createAnonymousCustomer () int uniqueid = getUniqueCutomerId (); return new Customer (uniqueid, “Firstname” + uniqueid, “Lastname” + uniqueid,“G”, “ACTIVE”);Anonymous creation method23Public void testPurchase_firstPurchase_ICC () Customer buyer = createCreditworthyCus

24、tomer (“FirstName”, “LastName”); / Public void testPurchase_subsequentPurchase_ICC () Customer buyer = createCreditworthyCustomer (“FirstName”, “LastName”); / public Customer createCreditworthyCustomer (String firstName, String lastName) int uniqueid = getUniqueCutomerId (); Customer customer = new

25、Customer (uniqueid, firstName, lastName, “G”, “ACTIVE”); customer.setCredit (CreditRating.EXCELLENT); customer.aproveCredit (); return customer;24public void testPurchase_relatedCustomerDiscount_AM () Customer buyer = createCreditworthyCustomer (“Related”, “Buyer”); Customer discountHolder = createC

26、reditworthyCustomer (“Discount”, “Holder”); creatRelationshipBetweenCustomers (buyer, discountHolder); / private void createRelationshipBetweenCustomers (Customer buyer, Customer discountHolder) buyer.addToRelatedCustomersList ( discountHolder ); discountHolder.addToRelatedCustomersList ( buyer );Fi

27、xture TeardownTeardown strategyGarbage-collected teardownAutomated teardownCode organizationIn-Line TeardownImplicit Teardown25Garbage-Collected TeardownWe let the garbage collection mechanism provided by the programming language clean up after our test. 26public void testCancel_proposed_UIT () Flig

28、ht proposedFlight = createAnonymousProposedFlight (); / fixture setup proposedFlight.cancel (); / exercise SUT try assertEquals (Flightstate.CANCELLED, proposedFlight.getStatus (); finally / tear downproposedFlight.delet ();propsedFlight = null; / verify outcomeassertEquals (Flightstate.CANCELLED, p

29、roposedFlight.getStatus ();/ teardown/ automated garbage collectionAutomated Exercise Teardown27Test_1Test_2Test_n.Test CaseSetupMethodSUTDOCFixturesetupTest Object RegistryRegisterTest Obj.Teardownpublic void setup () tracker = new ResourceTracker (); ObjectFactory.addObserver (tracker);public void

30、 tearDown () tracker.cleanup (); ObjectFactory.removeObserver (tracker);setupDestroyAllFixture Teardown Variations28public void teardownFlightAndAireports (BigDecimal FirstAirport, BigDecimal secondAirport) throws FlightBookingException try if (firstAirport != null) faade.removeAirport (firstAirport

31、); finally if (secondAirport != null) faade.removeAirport (secondAirport); Delegated Teardown MethodMulti-resourceTeardownTeardownGuard ClauseResult Verification29SetupExerciseVerifyTeardownFixtureResults VerificationVerification strategyState verification Behavior verificationAssertion method style

32、sCustom assertionDelta assertionGuard assertionUnfinished test assertion30State Verification We inspect the state of the system under test after it has been exercised and compare it to the expected state. Where is the state of the SUT stored?Within the SUTStored in another component such as a databa

33、seHow to verify the state?Procedural state verificationExpected state specification31Example Procedural State Verification32public void setUp() product = createAnonProduct (); anotherProduct = createAnonProdct (); inv = createAnonInvoice ()public void testInovice_addOneLineItem_quantity1() / exercis

34、e inv.addItemQuantity (product, QIANTITY);public void testInovice_addOneLineItem_quantity1() / exercise inv.addItemQuantity (product, QUANTITY); / verify List lineItems = inv.getLineItems (); assertEquals ( “number of items”, lineItems.size (), 1 ); / verify items LineItem actual = (LineItem) lineIt

35、ems.get (0); assertEquals (inv, actual.getInv(); assertEquals (product, actual.getProd(); assertEquals (QUANTITY, actual.getQuantity ();Example Expected ObjectUse the expected object with a single quality assertion instead of a series of assertions on individual attributes. 33public void testInovice

36、_addLineItem () LineItem expItem = new LineItem (inv, product, QUANTITY); / exercise inv.addItemQuantity ( expItem.getProd(), expItem.getQuantity() ); / verify List lineItems = inv.getLineItems (); assertEquals ( “number of items”, lineItems.size (), 1 ); LineItem actual = (LineItem) lineItems.get (

37、0); assertEquals ( “Item”, expItem, actual );Behavior VerificationWe capture the indirect outputs of the SUT as they occur and compare them to the expected behavior. Behavior verification involves verifying the indirect outputs of the SUT as it is being exercised. Procedural behavior verificationExp

38、ected behavior specification34Example35public void testRemoveFlightLogging_ESS() throws Exception / fixture setup FlightDto expectedFlightDto = createAnUnregFlight (); FlightManagementFacadeImplTI faade = new FlightManagementFacadeImplTI (); / exercise faade.removeFlight (expectedFlightDto.getFlight

39、Number (); / verify assertFalse (“Flight still exists after being removed”, faade.flightExists (expectedFlightDto.getFlightNumber ();Example Procedural Behavior Verification36public void testRemoveFlightLogging_recordingTestStub() throws Exception / fixture setup FlightDto expectedFlightDto = create

40、AnUnregFlight (); FlightManagementFacadeImplTI faade = new FlightManagementFacadeImplTI (); / Test double setup AuditLogSpy logSpy = new AuditLogSpy (); faade.setAuditLog (logSpy); / exercise faade.removeFlight (expectedFlightDto.getFlightNumber (); / verify assertEquals ( “number of calls”, 1, logS

41、py.getNumberOfCalls () ); assertEquals ( “action code”, Helper.REMOVE_FLIGHT_ACTION_CODE, logSpy.getActionCode () ); assertEquals ( “date”, helper.getTodaysDateWithouttime(), logSpy.getDate () ); assertEquals ( “user”, Helper.TEST_USER_NAME, logSpy.getUser () ); assertEquals ( “detail”, expectedFlig

42、htDto.getFlightNumber(), logSpy.getDetail () );Example Expected Behavior Specification37public void testRemoveFlightLogging_JMOCK() throws Exception / fixture setup FlightDto expectedFlightDto = createAnUnregFlight (); FlightManagementFacadeImplTI faade = new FlightManagementFacadeImplTI (); / mock

43、configuration Mock mockLog = mock (AuditLog.class); mockLog.expects (once().method (“logMessage”).with (eq(helper.getTodaysDateWithoutTime(),eq(Helper.TEST_USER_NAME),eq(Helper.REMOVE_FLIGHT_ACTION_CODE),eq(expectedFlightDto.getFlightNumber(); / mock installation faade.setAuditLog ( (AuditLog) mockL

44、xy() ); / exercise faade.removeFlight (expectedFlightDto.getFlightNumber (); / verify / verify () method called automatically by Jmock.Delta AssertionWe specify assertions based on differences between the pre- and post- exercise state of the SUT. 38SetupExerciseVerifyTeardownFixtureBeforeAfter

45、Delta1Get Pre-test State2Get Post-test State3Compare with Expected differencesExample39public void testGetFlightsByOriginAirport_OneOutboundFlight () throws Exception FlightDto expectedFlightDto = createNewFlightBetweenExistingAirports (); / exercise faade.createFlight (expectedFlightDto.getOriginAi

46、rportId (), expectedFlightDto.getDestinationAirportId (); / verify List flightAtOrigin = faade.getFlightsByOriginAirport (expectedFlightDto.getOriginAirportId(); assertOnlyFlightInDtoList (“Outbound flight at origin”, expectedFlightDto, flightsAtOrigin);Example Delta Assertion40public void testGetFl

47、ightsByOriginAirport_OneOutboundFlight () throws Exception FlightDto expectedFlightDto = createNewFlightBetweenExistingAirports (); / Remember prior state List flightsBeforeCreate = faade.getFlightsByOriginAirport (expectedFlightDto.getOriginAirportId(); / exercise faade.createFlight (expectedFlight

48、Dto.getOriginAirportId (), expectedFlightDto.getDestinationAirportId (); / verify outcome relative to prior state List flightAfterCreate = faade.getFlightsByOriginAirport (expectedFlightDto.getOriginAirportId(); assertFlightIncludedInDtoList ( “new flight”, expectedFlightDto, flightsAfterCreate); as

49、sertAllFlightIncludedInDtoList ( “previous flight”, flightsBeforeCreate, flightsAfterCreate); assertEquals ( “Number of flights after create”, flightBeforeCreate.size()+1, flightsAfterCreate.size();Test Doubles41Test DoubleSUT depends on other components (DOC) that cannot be used in the test environ

50、ment. Code unavailable yetWill not return the results needed for the testExecuting them would be have undesirable side effectsTest needs more control over or visibility of the internal behavior of the SUT42Use a Test Double to replace the DOC! The Test Double doesnt have to behave exactly like the r

51、eal DOC; it merely has to provide the same API as the real DOC so that the SUT thinks it is the real one!Test Double43How can we verify logic independently when code it depends on is usable?We replace a component on which the SUT depends with a “test-specific” equivalent. SetupExerciseVerifyTeardown

52、FixtureSUTDOCTest DoubleImplementation Alternatives44Test DoubleDummy ObjectTest StubTest SpyMockObjectFakeObjectTo provide an alternative to the value. To verify indirect inputsTo verify indirect outputsTo provide an alternative to the implementation Indirect Input - Test StubWe use Test Stub to re

53、place a real component on which the SUT depends so that the test has a control point for the indirect inputs of the SUT. Its inclusion allows the test to force the SUT down paths it might not otherwise execute. We can further classify Test Stubs by the kind of indirect inputs they are used to inject

54、 into the SUT.Responder injects valid valuesSaboteur injects errors or exception45ExampleThe following test verifies the basic functionality of a component that formats an HTML string containing the current time. Unfortunately, it depends on the real system clock so it rarely ever passes!46public vo

55、id testDisplayCurrenttime_AtMidnight() / fixture setup TimeDisplay sut = new TimeDisplay (); / exercise SUT String result = sut.getCurrentTimeAsHtmlFragment (); / verify direct output String expectedTimeString = “Minight”; assertEquals ( expectedTimeString, result);Example Responder (as Hand-Coded T

56、est Stub)It verifies one of the happy path test conditions using a Responder to get control over the indirect inputs of the SUT. Based on the time injected into the SUT, the expected result can be hard-coded safely. 47public void testDisplayCurrenttime_AtMidnight() / fixture setup / Test double conf

57、iguration TimeProviderTestStub tpStub = new TimeProviderTestStub (); tpStub.setHours (0); tpStub.setMinutes (0); / instantiate SUT TimeDisplay sut = new TimeDisplay (); / Test double installation sut.setTimeProvider (tpStub); / exercise SUT String result = sut.getCurrentTimeAsHtmlFragment (); / veri

58、fy direct output String expectedTimeString = “Minight”; assertEquals ( expectedTimeString, result);Example Saboteur (as Anonymous Inner Class)48public void testDisplayCurrenttime_exception() throws Exception / fixture setup / Define and instantiate Test Stub TimeProvider testStub = new TimeProvider

59、() / Anonymous inner Test Stubpublic Calendar getTime () throws TimeProviderEx throw new TimeProviderEx (“Sample”); / instantiate SUT TimeDisplay sut = new TimeDisplay (); sut.setTimeProvider (testStub); / exercise SUT String result = sut.getCurrentTimeAsHtmlFragment (); / verify direct output Strin

60、g expectedTimeString = “Minight”; assertEquals ( expectedTimeString, result);The Saboteur injects invalid inputs into the SUTIndirect Output - Test Spy and Mock ObjectTest Spy To capture the indirect output calls made to another component by the SUT for later verification by the test. Mock ObjectTo