Introduction To
Agile Testing
Welcome
Philip Juhl
INTRODUCTION & EXPECTATIONS
Overview of different test-practises for Agile teams
Role for a tester and QA manager in an Agile development team
Recommendation and overview of test-practises and test automation
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Input for Test-strategy & Test-automation
WHAT ARE THE VALUES OF TESTING?
Align expectation
Get confirmation
Measure progress
Measure quality
Learn about the system
PART OF THE FEEDBACK LOOP
What do we want?
Adjust or What do we
Continue want?
Build it Describe test
Test Driven
Execute test Describe test
Execute test
Adjust or Continue Build it
DEMING’S CYCLE
MANY FEEDBACK LOOPS
What do we want feedback on How we test it
Code-change integrate with latest source code Check-in, Merge & CI Build
New functionality is correct Functional testing
Existing functionality still works Regression test
We haven’t missed anything Exploratory testing
Integrates correct with other systems System integration testing
Complete workflow is correct End-2-End testing
Implementation is user friendly Usability testing
Implementation is secure Security testing
Works as user expects Acceptance testing
Implementation has the desired effect Validation testing
FEEDBACK LOOPS INSIDE LOOPS
Short delay is better
ALWAYS PUT THE MARSHMALLOW ON TOP
YOUR FEEDBACK LOOPS AND LEAD TIME?
1. What are the feedback loops in your systems(s)
2. What are the lead time?
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Input for Test-strategy & Test-automation
PLAN-DRIVEN IN A REAL ENVIRONMENT
Feedback-time?
Requirement analysis Test report Acceptance test
System Design System/Integration test
Change- Code
advisory board freeze
Component Component test
Implementation
Requirement Implementation
Phase Phase Test Phase
Time
PROJECT SIZE AND SUCCESS-RATE
59%
56%
31%
18% 19%
9%
Large projects Medium projects Small projects
Agile Waterfall
Source: Standish Group, Chaos Studies 2013-2017
COMPLICATED VS COMPLEX
Complex Complicated
Loosely coupled Tightly coupled
Emergent practice Good practice
th o d
Probe – Sense - Respond Sense – Analyse - Respond
e n me
h o d r i v
em e t
Pla n -d
Agil
Chaos Obvious
De-coupled No degree of freedom
Novel practice Best practice
Act – Sense - Respond Sense – Categorize - Respond
Cynefin framework
AGILE ONION
- Mindset
- Values
- Principles
- Technics
- Tools
MANIFESTO FOR AGILE SOFTWARE DEV.
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right,
we value the items on the left more.
PRINCIPLES FROM THE AGILE MANIFESTO
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to
the shorter timescale.
Business people and developers must work together daily throughout the project.
Continuous attention to technical excellence and good design enhances agility.
The most efficient and effective method of conveying information to and within a development team is
face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
HOW DOES THESE PRINCIPLES AFFECTS TEST
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference
to the shorter timescale.
Business people and developers must work together daily throughout the project.
Continuous attention to technical excellence and good design enhances agility.
The most efficient and effective method of conveying information to and within a development team is
face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
SCRUM
The Scrum Team is responsible for all product-related activities from stakeholder collaboration,
verification, maintenance, operation, experimentation, research and development,
Scrum Teams are cross-functional, meaning the members have all the skills necessary to create value
each Sprint.
Developers are the people in the Scrum Team that are committed to creating any aspect of a usable
Increment each Sprint.
HOW DOES AGILE AND SCRUM AFFECT TEST?
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Input for Test-strategy & Test-automation
WHAT IS THE IDEAL RATIO?
What’s the ratio between
Software-testers and
Software developers for …
1. The successful projects?
2. The worst projects?
TESTER & SOFTWARE-DEVELOPER RATIO
1:2
2:1
1:1
Testers Software-developers
TESTER & SOFTWARE-DEVELOPER RATIO
Testers Software-developers
EXPERIENCE FROM DIFFERENT PROJECTS
Successful projects do not seem to have an ideal
ratio.
Unsuccessful projects seems to have a correlation
with a high number of software testers per
software developer.
Correlation not Causality.
Elisabeth Hendrickson: testobsessed.com
BETTER TESTING, WORSE TESTING
Better testing at one level tends to
result in worse testing in another
given no other changes in the system.
Elisabeth Hendrickson: testobsessed.com
Can be managed
TEST AFFECTS TEST Unmanaged effect
Unmanaged negative effect
Perceived
Quality
Unknown Software
Known defects Defect Testers
defects Developers
released Prevention Test Effort
released Test Effort
Defects
Delivered to
System test
Defects Found
in System test
Can be managed
MULTIPLE FOCUS FOR TESTER Unmanaged effect
Unmanaged negative effect
Perceived
Quality
Unknown Software
Known defects Defect Testers
defects Developers
released Prevention Test Effort
released Test Effort
Defects
Delivered to
System test
Defects Found
in System test
A TRUE STORY ABOUT A DEV. TEAM
SHARED RESPONSIBILITY
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Input for Test-strategy & Test-automation
FOCUS AND TECHNIQUES FOR TESTER
Testable acceptance criteria 3 amigos
Defect
Prevention Discuss how to test/verify before implementation
Transparency for what is unit-tested Encourage unit-testing
Software
Developers Share responsibility
Test Effort Pair-testing Include in test-strategy
Testers Include software-developers to build test-data and test-environments
Test Effort
Exploratory testing Test that critize the product
Risk-based testing
Known defects
released
Handling bugs that are released as other backlog items
TEST QUADRANT
Business
Support the team Criticize the product
Progress/Verification Validation
Technical
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Input for Test-strategy & Test-automation
TEST-DRIVEN DEVELOPMENT
TDD BDD/SBE ATDD
The correct behaviour
Developer trust the
Focus for the system. Combine Writing Acceptance test
code
Documentation, test & code
Participants Software-developer, Tester, Software-developer &
Tester, Software-developer &
Technical tester Business/Customer
Business/Customer
Natural language,
Language Programming language
Gherkin syntax
Natural language
Xunit, Junit, MSTest, SpecFlow, Gherkin, Microsoft teams,
Tools Jasmine, Karma Cucumber, JBehave Jira, Word/Excel
BDD / SPECIFICATION BY EXAMPLE
§ Early Clarification through deliberate discovery
§ Common Domain language
§ Foster collaboration across domain-knowledge
§ Support iterative development
§ Documenting functional requirements
§ Possible to automate
SCRUM WATER FALL
Sprint board
Stories ToDo Doing Done
Story 1 T T C A
C
Analyse
Story 2 T C A
C A
Code
Story 3 A T
C Test
PROCESS WITH BDD
Scope
l
goa
pe from User User User User
sc o
ve story story story story
Deri
3 Amigos
Business goal
ples Illustrate using
m
exa
ine Examples Specifying
Ref
collaboratively
Key examples
Specification with
Examples Validate
Automate frequently
validation Living
Executable Documentation
Specification
From Gojko Adzic - Specification By Example
SPECIFICATION IN GHERKIN-FORMAT
§ Readable for humans
§ Describe behaviour not implementation
§ What we tests and not how we test it
§ Executable for machines
EXAMPLES IN GHERKIN
Feature: Delivery cost
Scenario: VIP customers gets free delivery for orders with 3 or more items
Given the customer is VIP
And the customer has 4 items in the shopping basket
When the customer goes to payment
Then the delivery cost is free
Link
SPECIFICATION EXERCISE
We want to increase the awareness and behaviour of using
Apple Pay for payment transactions. We would like to start
a campaign where the user get a bonus of 50,- DKR if
(s)he makes an Apple Payment with one of his/her’s credit
cards. The campaign is only active for payments that are
completed the first day of the month.
Info:
A user with Apple Pay can assigned more than one credit
card to Apple Pay.
A user can have multiple devices that can be used to pay
with Apple Pay.
SPECIFICATION EXERCISE
1. Assign one to take the Product Owner
role for the Apple Pay story
2. As a team write acceptance criteria for
the functional requirements - You can use
the Gherkin syntax writing scenarios
3. Can your acceptance criteria be used to
split the story?
HOW TO START USING BDD
Establish place Merge Test-scenarios Build structure for
to communicate with requirement- Test automation
Test-scenarios documentation
AGENDA
Purpose of Testing
Testing in an agile vs traditional context
Quality assurance
The testers roles
Test-driven development
Test-strategy & Test-automation
TRANSACTION AND HOLDING COST
RELEASE NEW FUNCTIONALITY
KEEP NEW FUNCTIONALITY
TRANSACTION AND HOLDING COST
Total cost
Optimal batch size
Holding cost
Cost
Automation
Transaction cost
Items (featues) per batch (release)
FREQUENT RELEASES &
SUSTAINABLE PACE
Iterative development Test activities
Iterations 1 2 3 4
New functionality Acceptance test
Acc.
test
Existing Functions Integration test
Integration test
Unit test
Unit test
TEST AUTOMATION
Functional Business
End-2-End testing
testing A/B-
testing
Usability
testing
Acceptance
Guide the team testing Exploratory Criticize the product
Progress/Verification testing Validation
Component Performance
testing Security
testing
testing
Unit-level
testing Manual test
Automated test
Technical
CONSIDERATIONS
d at ing
Up
test
n
entatio t est
m p
not imple Setu
t
i rem
en
m a tisk
Test
requ
t r uc ture Auto kation
e d for le t est-s verif
i
N e s ab
t es ting Reu em ent
r
re
th r equi
New lity ple wi
n c t iona Cou
fu
a n uel
M tion
if ic a
ver
Automate
or
manual
VALUE IN THE SHORT AND LONG TERM
Execution time/cost
value
Easier to maintain than to implement
Time
Fragility
Maintainability
Debug time
AUTOMATION
Cheap
Blinders Exo Skeleton
Maintenance cost
Expensive
Fast Food Cement
Poor Important
Risk Detection
HOW TO GET STARTED
§ Build test strategy together with the team
§ Make Test-automation a team responsibility
§ Avoid “Fast-food” test- tool
§ Encourage test-automation-exercises
§ Test strategy in a scaled setup
RECOMMENDED BOOKS
§ Agile Testing
§ Specification By Example
AGILE TESTING PRACTICES FOR TEAMS
Exercises with
§ How to split stories an test early
§ How to use Specification By Example
