Regression Testing
Regression Testing
Srinivasan Desikan
Regression Testing
Regression Testing
Doctor: Congratulations! The stomach ulcer that was bothering you and preventing digestion is now completely cured!
08/27/13
Patient: That is fine doctor, but I have got such a bad mouth ulcer that I cant eat anything and hence there is nothing to digest!
2
Regression Testing
Regression Testing
Regression Testing - Definition Regression testing is selective retesting of the system with an objective to ensure the bug fixes work and those bug fixes have not caused any unintended effects in the system This testing is done to ensure The bug-fixes work And bug-fixes do not create any side-effects
08/27/13 Regression Testing 3
Regression Testing
Regression Testing Types
Dev. team
Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes Bug fixes
Build 1
New Features
Build 2
Build 3 Build 4
New Features
Build 5 Build 6
New Features
Test team test Cycle 1
Reg. 1
test Cycle 2
Time line
Reg. 2 test Cycle 3
Final Reg.
4
08/27/13
Regression Testing
Regression Testing
Regression Testing Types I. Final regression Testing Unchanged build exercised for the min period of cook time (Gold master build) To ensure the same build of the product that was tested reaches the customer More critical than any other type of testing Used to get a comfort feeling on the product prior to release II. Regression testing To validate the product builds between test cycles Unchanged build is recommended but not mandatory Used to get a comfort feeling on the bug fixes, and to carry on with next cycle of testing Also used for making intermediate releases (Beta,Alpha)
08/27/13
Regression Testing
Regression Testing Types
Regression Testing
II. Regression testing To validate the product builds between test cycles Unchanged build is recommended but not mandatory Used to get a comfort feeling on the bug fixes, and to carry on with next cycle of testing Also used for making intermediate releases (Beta,Alpha)
08/27/13
Regression Testing
Regression Testing
Methodology explained
1: What is needed for selecting regression test cases? 2. Performing smoke tests 3: Classifying test cases 4: Selecting test cases 5: Resetting test cases for regression testing & phases of testing 6: How to conclude results 7: Popular strategies 8: Some guidelines 9: exercise
08/27/13 Regression Testing 7
Regression Testing
Types Regular regression Final regression
What? Selective re-testing to ensure Defect fixes work No side effects
Why? Defects creep in due to Changes Defect fixes may cause existing functionality to fail
When? When set of defect fixes arrive after formal testing for those areas completed Performed in all test phases
8
08/27/13
Regression Testing
Regression Testing What is needed for selecting test cases? Bug fixes and how it affect the system Area of frequent defects Area which has undergone many/recent code changes Area which is highly visible to the users Area which have risks are more Core features of the product which are mandatory requirements of the customer
08/27/13
Regression Testing
Regression Testing Points to remember
Selection depends more on the criticality of bug fixes than the criticality of the defect itself More positive test cases than negative test cases for final regression Constant set of regression test cases may not work always
08/27/13
Regression Testing
10
Classifying test cases Regression Testing
Note The order of test execution is priority 0, 1 & 2 , Priority helps in entry, exit criteria Priority0 Test cases Sanity test cases which checks basic functionality Run for acceptance tests and for major change Delivers very high project value
08/27/13
Priority 1 Test cases
Uses
Priority 2 Test cases
Uses
the basic and normal setup Delivers high project value High probability of getting selected for regression
Regression Testing
extended
setup Moderate project value Some functional and all nonfunctional test cases fall into this category
11
Regression Testing Performing initial smoke test
Smoke testing ensures that the basic functionality works and indicate that the build can be considered for further testing
Defect is cumulative effect of people, product, process and roles. Regression should take care of all these aspects Regression Testing 08/27/13 12
Criteria
Regression Testing Selecting test cases
a. Bug fixes work b. No side effects
Defect fixes Legend
P0 Priority 0 P1 Priority 1 P2 Priority 2 - Very few - All - Subset
Criticality
Impact
Low
Med
High
P0 P1 P2
08/27/13
P0 P1 P2
Regression Testing
P0 P1 P2
13
Regression Testing How to conclude results
Test case result (Current build) FAIL Test case result (Previous build) PASS
Remarks
Regression failed and Apply RESET guidelines and proceed after getting new build Bug fixes are working and Continue your regression to find side effects Bug fixes not working or not provided or Wrong selection If you are satisfied with work around then result marked as PASS else FAIL (Check with Prog mgmt) This test case could have been included for finding side effects or Wrong selection
PASS
FAIL
FAIL
FAIL
Pass with work around PASS
FAIL
PASS
08/27/13
Everyone monitors regression test results as it not only talk about defects and but also their fixes
Regression Testing
18
Regression Best Testing Practices 5. Detect defects, protect your product from
defects and defect fixes
08/27/13
Regression Testing
23
Regression Testing
08/27/13
Regression Testing
24
Regression Testing
08/27/13
Regression Testing
25
