Download as PDF, PPTX
![Automated testing
#!/bin/bash
out=`python script.py`
if [ $out = 1 ]; then
echo "test passed"
else
echo "test failed"
fi
Same principle, but repeatable
Useful for regression testing
Requires tooling](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/85/DSR-Testing-Part-1-81-320.jpg)
![As a [user]
I want [feature]
so that [benefit]](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/85/DSR-Testing-Part-1-143-320.jpg)
![Given [some initial context]
When [an event occurs]
Then [ensure some outcomes]](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/85/DSR-Testing-Part-1-145-320.jpg)
![Should you use BDD?
More tooling, organisational requirements
Might not be applicable in all cases
Pick and choose!
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when value is not present', function() {
assert.equal(-1, [1,2,3].indexOf(4));
});
});
});](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/85/DSR-Testing-Part-1-165-320.jpg)
![Automated testing
#!/bin/bash
out=`python script.py`
if [ $out = 1 ]; then
echo "test passed"
else
echo "test failed"
fi
Same principle, but repeatable
Useful for regression testing
Requires tooling](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/75/DSR-Testing-Part-1-81-2048.jpg)
![As a [user]
I want [feature]
so that [benefit]](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/75/DSR-Testing-Part-1-143-2048.jpg)
![Given [some initial context]
When [an event occurs]
Then [ensure some outcomes]](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/75/DSR-Testing-Part-1-145-2048.jpg)
![Should you use BDD?
More tooling, organisational requirements
Might not be applicable in all cases
Pick and choose!
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when value is not present', function() {
assert.equal(-1, [1,2,3].indexOf(4));
});
});
});](https://image.slidesharecdn.com/dsrtestingpart1-170208231129/75/DSR-Testing-Part-1-165-2048.jpg)
Uploaded bySteve Upton
DSR Testing (Part 1)
This document provides an overview of software testing concepts. It discusses different types of testing like unit testing, functional testing, error path testing, boundary value testing, and equivalence partitioning. It also covers test strategies like golden path testing, black box testing, and white box testing. The purpose of a tester is explained as quantifying risk to make decisions that improve confidence and quality.
More Related Content
DSR Testing (Part 1)
- 1.
- 2. Who am I? ●Steve Upton ● English / Welsh / British / Irish(?) ● BSc. Computer Science (Cardiff University) ● Physics & Astronomy (Open University) ● IBM (6 years) ○ Messaging ○ OASIS MQTT TC member ○ Working with clients on Microservice systems ○ London μService user group ● HERE Berlin (9 months) ○ Microservices ○ Robots!
- 3.
- 5. Exclusive use ofwhite box testing in a test-phase will: a) Ensure the test item is adequately tested b) Make the need for black-box testing redundant c) Run the risk that the requirements are not satisfied d) Suffice for the unit testing phase
- 6. Which is notin sequence in Step 11 of the Software Testing process? a) Assess development plan and status b) Develop the test plan c) Test software design d) Test software requirement
- 7. Software Test Documentationis described in which standard? a) IEEE 730-2014 b) IEEE 829-2008 c) IEEE 830-1984 d) IEEE 1012-2012
- 10. Understand importance testing Understandhow to test well Be ready for a testing role Be ready for a test interview!
- 11. Why do wetest?
- 18.
- 23.
- 24. So, why dowe test?
- 25. What is thepurpose of a tester?
- 26. To write lotsof tests?
- 27. To write lotsof tests? No
- 28. To run lotsof tests?
- 29. To run lotsof tests? Nooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
- 30. To run lotsof tests? Nooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
- 31. Find all thebugs?
- 32. Find all thebugs?
- 33. Find all thebugs?
- 34.
- 35. Find (lots of)bugs?
- 36. Impact of thebug matters! Find (lots of) bugs?
- 37.
- 38. Improve quality? Better... buta bit vague
- 39.
- 40. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality?
- 41. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality? You are a code monkey
- 42. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality? You are a code monkey You are a code monkey
- 43. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality? You are a code monkey You are a code monkey Your job is futile
- 44. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality? You are a code monkey You are a code monkey Your job is futile Your job never ends
- 45. Implications? Write lots oftests? Run lots of tests? Find all the bugs? Find bugs? Improve quality? You are a code monkey You are a code monkey Your job is futile Your job never ends Everyone has that job
- 46. What is therole of a tester?
- 47.
- 53. The role ofa tester is to quantify risk Then use that information to make decisions and improve confidence and quality
- 54. The role ofa tester is to quantify risk Then use that information to make decisions that improve confidence and quality
- 55. How do wetest?
- 56. Testing Lifecycle Requirements analysis Testplanning Test development Test execution Test reporting Test result analysis Defect Retesting Regression testing Test Closure
- 57. Test strategy Test techniques Defectmanagement Test metrics
- 58.
- 60.
- 66.
- 68.
- 69.
- 70.
- 71.
- 72. Business Facing Functional tests Storytests Prototypes Simulations Technology Facing Supportingtheteam Critiquetheproduct Auto/Manual Unit tests Component tests Scenarios Exploratory tests Usability tests UAT Alpha/Beta Performance/Load Security testing -ility testing Automated Tools Manual
- 73. Unit testing Functionaltesting
- 74. Unit testing Testing individualunits of code Functions, methods, classes etc. Generally done by devs Tests that the code is doing things right Testing functionality (not a function!) User interfaces Generally done by testers Tests that the code is doing the right thing Functional testing
- 75.
- 76. Business Facing Functional tests TechnologyFacing Supportingtheteam Critiquetheproduct Auto/Manual Unit tests Automated Tools Manual
- 77.
- 78.
- 79.
- 80. Manual testing Open acode prompt… Quick to get going... … but slow in the long run >>> "abcd".index("b"); 1
- 81. Automated testing #!/bin/bash out=`python script.py` if[ $out = 1 ]; then echo "test passed" else echo "test failed" fi Same principle, but repeatable Useful for regression testing Requires tooling
- 82. Standard library xUnit based Supportedby other test runners unittest pip install pytest xUnit based No boilerplate, simpler syntax py.test
- 83. xUnit Collection of unittesting frameworks that share structure and functionality Started with SUnit for Smalltalk JUnit (Java), RUnit (R), NUnit (.NET) etc. Object Oriented
- 84. xUnit components Test Case TestSuite Test Runner Test Fixtures
- 85. xUnit components -Test Case
- 86. “A set oftest inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” IEEE Standard 610 (1990) xUnit components - Test Case
- 87. “A set oftest inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” IEEE Standard 610 (1990) xUnit components - Test Case
- 88. “A set oftest inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” IEEE Standard 610 (1990) xUnit components - Test Case Inputs and expected outputs Don’t get expected results = fail
- 89. “A set oftest inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” IEEE Standard 610 (1990) xUnit components - Test Case Inputs and expected outputs Don’t get expected results = fail All tests inherit from TestCase class
- 90. “A set oftest inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” IEEE Standard 610 (1990) xUnit components - Test Case Inputs and expected outputs Don’t get expected results = fail All tests inherit from TestCase class Test methods start with test Has at least one assert statement
- 91. xUnit components -Test Suite Collection of test cases All test cases should be independent Allows us to run specific sets of tests
- 92. xUnit components -Test Fixture Pre-requisites and cleanup need by one or more tests Generate fake data, temporary databases etc. (context) Includes cleanup! unittest provides setUp() and tearDown() methods
- 93. xUnit components -Test Runner Runs test cases or test suites and returns results unittest provides TextTestRunner but many others available HTMLTestRunner outputs a HTML report XMLTestRunner outputs XML
- 94. class TestIndex(unittest.TestCase): # Testsgo here if __name__ == '__main__': unittest.main()
- 95.
- 96. def test_my_first_test(self): alphabet ="abcdefghijklmnopqrstuvwxyz" str1 = "ab" self.assertEqual(alphabet.index(str1), 0)
- 97.
- 98.
- 99. def test_value_error(self): with self.assertRaises(ValueError): alphabet= "abcdefghijklmnopqrstuvwxyz" str2 = "not_in_the_alphabet" alphabet.index(str2)
- 100. # Run beforeeach test def setUp(self): self.alphabet = "abcdefghijklmnopqrstuvwxyz" # Run after each test def tearDown(self): return
- 101.
- 102.
- 103. Golden Path Testing ErrorPath Testing Boundary Value Testing Equivalence Partitioning Defect clustering Black Box/White Box
- 104.
- 105. Golden Path Testing Testthe ideal data Don’t do anything to induce errors Don’t rock the boat
- 106. Golden Path Testing Testthe ideal data Don’t do anything to induce errors Don’t rock the boat Danger of writing ‘easy’ tests
- 107. Error Path Testing Extremevalues (+∞, -∞ etc.) Invalid values (string in place of int etc.) Out of date data (expired password etc.) Unusual values, characters ( Ю 屁) Null values (“”, “ “, 0, null, undefined)
- 108.
- 109. Let’s test... Exhaustive testingis impossible
- 110. Equivalence Partitioning Identify partitionsof inputs eg. - Age 0-18, 18-26, 26+ - 2n (32-64, 64-128, 128-256…) - ≤1, 0, ≥1
- 111. Equivalence Partitioning Identify partitionsof inputs eg. - Age 0-18, 18-26, 26+ - 2n (32-64, 64-128, 128-256…) - ≤1, 0, ≥1 Consider each group to be the equivalent Only need to test 1 value per partition
- 112. Equivalence Partitioning Identify partitionsof inputs eg. - Age 0-18, 18-26, 26+ - 2n (32-64, 64-128, 128-256…) - ≤1, 0, ≥1 Consider each group to be the equivalent Only need to test 1 value per partition 30 31 32 33 34 35 ... ...
- 113. Boundary value testing Identifypartitions of inputs again Test values on the edges of the partitions
- 114. 2 Boundary value testing Identifypartitions of inputs again Test values on the edges of the partitions -2 -1 0 1 ... ...
- 115. Defect clustering Defects tendto be clustered together 80% of bugs in 20% of code (Pareto principle)
- 116. Defect clustering Defects tendto be clustered together 80% of bugs in 20% of code (Pareto principle) Adapt testing plans based on results!
- 117. White Box testingBlack Box testing
- 118. White Box testing Testsbased on external interfaces No knowledge of internals Intuit error paths (test data selection) Generally done by testers Black Box testing
- 119. White Box testing Writingtests based on code Uses knowledge of internals Identify and test error paths in code Generally done by developers Tests based on external interfaces No knowledge of internals Intuit error paths (test data selection) Generally done by testers Black Box testing
- 120. Business Facing Technology Facing Supportingtheteam Critiquetheproduct Auto/Manual Unittests Automated Tools Manual
- 122. Let’s test… theFibonacci sequence
- 123. Let’s test… theFibonacci sequence import unittest # Returns the nth Fibonacci number def fib(n): return;
- 124. Let’s test… theFibonacci sequence class TestFibonacci(unittest.TestCase): def test_fib_1(self): self.assertEqual(fib(1), 1) if __name__ == '__main__': unittest.main()
- 125. Let’s test… theFibonacci sequence class TestFibonacci(unittest.TestCase): def test_fib_1(self): self.assertEqual(fib(1), 1) if __name__ == '__main__': unittest.main()
- 126. Let’s test… theFibonacci sequence $ python main.py FFFFF ====================================================================== FAIL: test_fib_0 (__main__.TestFibonacci) ---------------------------------------------------------------------- Traceback (most recent call last): File "main.py", line 15, in test_fib_0 self.assertEqual(fib(0), 0) AssertionError: None != 0
- 127. Let’s test… theFibonacci sequence def fib(n): if n == 0: return 0 elif n == 1: return 1 else: return fib(n-1)+fib(n-2)
- 128. Let’s test… theFibonacci sequence $ python main.py ..... ---------------------------------------------------------------------- Ran 5 tests in 0.000s OK
- 129. Let’s test… theFibonacci sequence def fib(n): a,b = 0,1 for i in range(n): a,b = b,a+b return a
- 130.
- 131.
- 132.
- 133.
- 134.
- 135.
- 136. Where do westart testing?
- 137. Where do westart testing? What do we test?
- 138. Where do westart testing? What do we test? (and what don’t we test?)
- 139. Where do westart testing? What do we test? (and what don’t we test?) How much do we test in one go?
- 140. Where do westart testing? What do we test? (and what don’t we test?) How much do we test in one go? How do we name our tests?
- 141. Where do westart testing? What do we test? (and what don’t we test?) How much do we test in one go? How do we name our tests? How do we understand why a test fails?
- 142.
- 143. As a [user] Iwant [feature] so that [benefit]
- 144. As a customer Iwant to get coffee from the machine so that I don’t have to make my own
- 145. Given [some initialcontext] When [an event occurs] Then [ensure some outcomes]
- 146. Given the coffeemachine is installed And the coffee machine has coffee And I have deposited €1 When I press the coffee button Then I should be served a coffee
- 147. Given the coffeemachine is installed And the coffee machine has no coffee And I have deposited €1 When I press the coffee button Then I should be shown an error And I should be have my €1 returned
- 148. Feature: Some terseyet descriptive text of what is desired Textual description of the business value of this feature Business rules that govern the scope of the feature Any additional information that will make the feature easier to understand Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action And yet another action Then some testable outcome is achieved And something else we can check happens too Scenario: A different situation ...
- 149. Feature: Some terseyet descriptive text of what is desired Textual description of the business value of this feature Business rules that govern the scope of the feature Any additional information that will make the feature easier to understand Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action And yet another action Then some testable outcome is achieved And something else we can check happens too Scenario: A different situation ...
- 150. Feature: Some terseyet descriptive text of what is desired Textual description of the business value of this feature Business rules that govern the scope of the feature Any additional information that will make the feature easier to understand Scenario: Some determinable business situation Given some precondition And some other precondition When some action by the actor And some other action And yet another action Then some testable outcome is achieved And something else we can check happens too Scenario: A different situation
- 151.
- 153. Given Put the systemin a known state Given there is coffee in the machine ↓ @given('the coffee machine is installed') def step_impl(context): context.coffee_machine = CoffeeMachine()
- 154. Given Given there iscoffee in the machine ↓ @given('the coffee machine is installed') def step_impl(context): context.coffee_machine = CoffeeMachine() Put the system in a known state
- 155. When When I pressthe coffee button ↓ @when('I press the coffee button') def step_impl(context): context.coffee_machine.push_coffee_button Describe the key action
- 156. When When I pressthe coffee button ↓ @when('I press the coffee button') def step_impl(context): context.coffee_machine.press_coffee_button() Describe the key action
- 157. Then Observe outcomes Then Ishould be served a coffee ↓ @then('I should be served a coffee') def step_impl(context): assert context.coffee_machine.served_drink is "coffee"
- 158. Then Then I shouldbe served a coffee ↓ @then('I should be served a coffee') def step_impl(context): assert context.coffee_machine.served_drink is "coffee" Observe outcomes
- 159. $ behave Feature: Acoffee machine that dispenses coffee # coffee_machine.feature:1 Scenario: run a simple test # coffee_machine.feature:3 Given the coffee machine is installed # steps/steps.py:4 0.000s And the coffee machine has coffee # steps/steps.py:8 0.000s And I have deposited 1 euro # steps/steps.py:12 0.000s When I press the coffee button # steps/steps.py:16 0.000s Then I should be served a coffee # steps/steps.py:20 0.000s 1 feature passed, 0 failed, 0 skipped 1 scenario passed, 0 failed, 0 skipped 5 steps passed, 0 failed, 0 skipped, 0 undefined Took 0m0.000s
- 160. behave pip install behave Featuresgo in a .feature file Steps (Givens, Whens and Thens) go in steps/
- 161.
- 162. Should you useBDD?
- 163. Should you useBDD? More tooling, organisational requirements
- 164. Should you useBDD? More tooling, organisational requirements Might not be applicable in all cases
- 165. Should you useBDD? More tooling, organisational requirements Might not be applicable in all cases Pick and choose! describe('Array', function() { describe('#indexOf()', function() { it('should return -1 when value is not present', function() { assert.equal(-1, [1,2,3].indexOf(4)); }); }); });
- 166. Should you useBDD? More tooling, organisational requirements Might not be applicable in all cases Pick and choose! $ mocha Array #indexOf() ✓ should return -1 when value is not present 1 passing (9ms)
- 167.
- 168.
- 173.
- 175. Integration testing Testing morethan individual components
- 176. Integration testing Testing morethan individual components Multiple approaches - Big Bang - Top Down - Bottom Up
- 177. Integration testing Testing morethan individual components Multiple approaches - Big Bang - Top Down - Bottom Up May use fixtures (stubs and mocks)
- 178. Test fixtures -Stubs ‘Stub’ out functionality Reply to calls with canned responses Allows testing without dependencies
- 179. Test fixtures -Stubs ‘Stub’ out functionality Reply to calls with canned responses Allows testing without dependencies pip install mock (Python 2) from unittest.mock import MagicMock dependency.method = MagicMock(return_value=5) ... dependency.method(10) >>> 5
- 180. Test fixtures -Mocks Basically stubs with expectations from unittest.mock import MagicMock dependency.method = MagicMock(return_value=5) ... dependency.method(10) ... dependency.method.assert_called_with(10)
- 181. Let’s test …Conway’s Game of Life Cellular automaton Set initial state and watch the system evolve Complicated behavior from simple rules
- 182. Let’s test …Conway’s Game of Life Grid of cells
- 183. Let’s test …Conway’s Game of Life Grid of cells Each cell can be on (alive) or off (dead)
- 184. Let’s test …Conway’s Game of Life Grid of cells Each cell can be on (alive) or off (dead) Each ‘step’, apply rules: ● Live cells with < 2 neighbours die (under population) ● Live cells with 2 - 3 neighbours live (stable) ● Live cells with > 2 neighbours die (over population) ● Dead cells with 3 neighbours live (reproduction)
- 185. Let’s test …Conway’s Game of Life Grid of cells Each cell can be on (alive) or off (dead) Each ‘step’, apply rules: ● Live cells with < 2 neighbours die (under population) ● Live cells with 2 - 3 neighbours live (stable) ● Live cells with > 2 neighbours die (over population) ● Dead cells with 3 neighbours live (reproduction)
- 186. py.test pip install pytest Similarto unittest Less boilerplate Supports running unittest tests
- 187. py.test $ pytest -vtest_game_of_life.py ============================= test session starts ============================= platform linux2 -- Python 2.7.12, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 -- /home/steve/Dev/dsr/dsr-testing-lab/exercise-pytest/venv/bin/python cachedir: .cache rootdir: /home/steve/Dev/dsr/dsr-testing-lab/exercise-pytest, inifile: collected 1 items test_game_of_life.py::test_advance_should_return_a_set PASSED ========================== 1 passed in 0.00 seconds ===========================
- 188.