Agile Testing: Methods and Models

Agile Testing:
Methods &
Models
Rad Butalid, ICP-ACC, CKC
May 31, 2019

• IBM Recognized Agile Thought Leader
and Agile Champion
• International Consortium for Agile -
Agile Certified Coach (ICP-ACC)
• International Business and Quality
Management Institute – Certified
Kanban Coach (CKC)
• Project Management Institute –
“Incoming” Agile Certified Professional
(PMI-ACP)
Hello!

Agenda
o What is Agile Testing?
o Testing Methods and Models
o What’s next for Agile Testing?
o Q&A

“Agile development has
always centered around
testing and quality. That
characteristic drew us to
agile: Test first, test all the
time, build in quality. ” – Lisa
Crispin & Janet Gregory

6
What is Agile
Testing?
Agile testing is a software testing practice that follows the
principles of agile software development. Agile testing not
only facilitates the early detection of defects but
also reduces the cost of bugs by fixing them early. This
approach also yields a customer-centric approach by
delivering a high-quality product as early as possible.
Both development and testing activities are concurrent
unlike the Waterfall model.

Agile Testing
Principles
o Test Driven: Test First vs. Test Last
o Continuous Testing = Continuous Progress
o Tests performed by the whole team
o Continuous feedback
o Decrease time of feedback response
o Bug fix within iteration
o Lightweight documentation
7

Advantages of Agile
Testing
o It saves time and money
o Agile testing reduces documentation
o It is flexible and highly adaptable to changes
o It provides an easier way for receiving regular feedback
from the end user
o Better determination of issues through daily
standups/meetings
8

Agile Testing:
Methods and
Models
10

Agile Test Plan
o The scope of the testing
o Consolidating new functionalities
to be tested
o Types of testing/Levels of testing
o Performance & load testing
In agile testing, the test plan is written as well as updated for
every release. A test plan in agile includes:
11
o Consideration of infrastructure
o Risks Plan
o Planning of resources
o Deliverables & Milestones

Agile Testing
Lifecycle
The agile testing lifecycle includes the following 5 phases:
o Impact assessment
o Agile Testing Planning
o Release Readiness
o Daily Scrums
o Test Agility Review
12

15
Behavior
Driven
Development
(BDD)

Behavior Driven
Development (BDD)
16
o Who is it for? The BDD methodology is ideal for teams that
work on feature-focused software and/or teams that put
the user experience first.
o BDD starts with a functional specification using the
Gherkin Given/When/Then syntax. This specification then
guides developers, testers and product owners who move
across features. As they do, they use automated test
functions to determine completeness, refining code until it
passes the test, much like in the TDD approach except at
the team level.

How is it different from
standard Waterfall testing?
17
o Waterfall testing requires test cases to be written early against
requirements and calls for those tests to be executed toward the end of the
development cycle. With BDD, tests are not based on requirements (but on
“end user behavior”) and testing happens in conjunction with development
of the features.
o Additionally, within a Waterfall methodology, testers are the ones writing
the test cases. A BDD approach, on the other hand, lends itself to the
business owners writing the tests. This switch reduces communication (or
miscommunication) between the business analysts, developers and testers.

What does adoption entail?
18
o Changing to a BDD methodology can be challenging when the team is used
to a traditional style of testing. It requires a BA to write tests up front and
for the developers to write the test specification in code to match. It’s a
new type of coordination within the team, but it’s extremely positive in that
the team works together as one unit, business users included.

Best Practices
19
o Streamlining documentation in order to keep the entire process lean
o Embracing a “three amigos” model where the product owner, developer
and tester form a cohesive team
o Using a test framework like Cucumber to define criteria
o Building automated tests in a way that make them easy to reuse as much as
possible
o Having business analysts learn the Gherkin syntax and write test cases
directly

20
Acceptance
Test Driven
Development
(ATDD)

21
o Who is it for? Due to its emphasis on user perception, it’s best
suited for teams that are focused on the user experience, have
goals around high rates of adoption and want to minimize the
number of feature changes in future releases.
o The idea behind ATDD is that user perception of the product is
just as important as functionality, and should drive product
performance in order to help increase adoption. ATDD collects
and uses input from customers to develop acceptance criteria,
translates that criteria into manual or automated acceptance
tests and then develops code against those tests.
Acceptance Test Driven
Development (ATDD)

22
o ATDD is different because it is a test-first methodology.
Standard Waterfall testing calls for test cases to be written
upfront based on requirements, whereas ATDD is not a
requirements driven testing process.

23
o Because ATDD represents such a departure from traditional
methods, going from one to the other isn’t easy for teams to
do. To be in the best position to adopt an ATDD methodology,
teams need to get stakeholder buy-in, which can prove
challenging at times.

24
o Interacting closely with customers, for example through focus
groups, in order to determine expectations
o Leaning on customer-facing team members, such as sales
representative, customer service agents and account managers,
to understand customer expectations
o Developing acceptance criteria based on customer expectations
o Prioritizing two questions:
1. Will customers use the system if it does X?
2. How can we validate if the system does X?
Best Practices

26
o Who is it for? Best suited for teams that are under time
constraints, teams that need help identifying the best types of
tests to run (especially in cases where there are no specifications
from developers) and those who want to make sure they didn’t
miss anything from previous tests.
o Exploratory testing can help reduce time spent testing, find
more defects and improve code coverage. Because of its
unscripted approach, exploratory testing often mimics how
users will interact with the software in real life.
Exploratory Testing

27
o Exploratory testing can actually be done in both Waterfall and
Agile environments, but the tight integration between testers
and developers in an Agile environment helps ease any
bottlenecks that might surface while running exploratory testing
in a Waterfall environment.

28
o Embracing exploratory testing is relatively easy, as it is quick to
launch (and scale), simple to learn and provides benefits for the
entire team. That said, it’s important to keep in mind that it
should not be the only form of testing (rather, it should inform
what type of testing happens next). Additionally, even though it’s
unscripted, exploratory testing should not be unstructured
(testers still need to set a goal, log your activities and take on the
mindset of a specific user persona).

29
o Organizing the functionality in the application using something
like a Mindmap or a spreadsheet
o Focusing on certain areas or certain scenarios
o Tracking what gets tested to help reproduce any errors
o Documenting results for accountability on what got tested
Best Practices

31
o Session based testing builds on exploratory testing by providing
more structure. It provides this structure by conducting testing
during time-boxed, uninterrupted sessions, testing against a
charter and requiring testers to report on the testing that took
place during each session.
o Additionally, session based testing should be capped off with a
“debrief” between the tester(s) and the manager that covers the
five PROOF points: What happened(Past), what was achieved
(Results), what got in the way (Obstacles), what still needs to be
done (Outlook) and how does the tester feel about it (Feelings).
Session Based
Testing

32
o The same as exploratory testing, session based testing can be
run in both Agile and Waterfall environments, but it is more
conducive to the tight collaboration between testers and
developers that is typically found in Agile environments.

33
o Much like with exploratory testing, adopting session based
testing proves relatively easy because it is easy to pick up
and launch quickly. For testers already accustomed to
exploratory testing, the biggest hurdle is embracing the
additional structure for which session based testing calls.
o Also like with exploratory testing, teams running session
based testing should remember that it is not a final stop,
but rather a method to help determine the best type of
testing to conduct next.

34
o Outlining a mission so that testers are clear about the software they’re testing
o Developing a clear charter that indicates the mission, the areas of the
software to test, which tester(s) will run the session, when the session will take
place and for how long, the tests designed and executed, the bugs found and
overall notes.
o Running the testing session without any interruptions
o Clearly documenting activities and notes during the session in a session report
o Holding a debrief between the tester(s) and the manager to review findings
from the session and discuss next steps for testing
Best Practices

Marick Test Matrix
35
o Back in 2003, Brian Marick published an
influential series of articles on agile testing. He was
attempting to point the way forward for agile testers.
But, in the process, he came up with an elegant method
of cataloguing testing methods that has become known
as the “Marick Test Matrix.”
o Marick’s work categorized tests by asking two questions:
1. Is the test business facing or technology facing?
2. Does the test support programming or critique a
product?

37
o The Agile test (automation) pyramid was introduced by
Mike Cohn in his book ”Succeeding with Agile”, back in 2009.
o The model splits types of testing into three layers based on
the return on investment (ROI) offered by automating that
particular type: Unit tests involve testing small units of
code. They are the least expensive to write and maintain,
and they provide value to the team multiple times per day.
Integration or Acceptance tests, provide the next greatest
benefit with User Interface testing having the least.
The Agile Test
(Automation) Pyramid

39
o DevOps shift-left testing is an approach to software and
system testing in which testing is performed earlier in the
software lifecycle, with the goal of increasing quality,
shortening long test cycles and reducing the possibility of
software defects making their way into production code.
o One way that DevOps teams do Shift Left testing is by
shifting integration testing to the left of its usual position in
the delivery pipeline so that it occurs as close as possible to
the build process. This allows teams to receive feedback on
code quality faster, with more accurate results.
Shift Left Testing

What’s next for
Agile Testing?
41

42
In the future, three key tenets will become particularly
important for testers working in Agile environments:
1) Communication
Agile requires a tight collaboration between testers and developers, and that collaboration
makes communication a top priority for testers. Additionally, in a world where quality
becomes everyone’s responsibility, testers will become “quality champions” that serve as
internal experts, which will put their ability to clearly communicate testing needs and
reasoning under the spotlight.
2) Skill Diversity
In an Agile environment, everything can change on a dime, and that requires testers to be
adaptable. Part of this adaptability is having a diverse skillset so that testers can change course
as needed. For instance, functional testers need to expand their skills beyond manual scripted
execution. This diverse skillset will be a must as different sprints require different types of
testing to be executed in a short timeframe.
3) Business Mindset
Finally, Agile takes on a very customer-centric approach in order to ensure customers receive
as much value as possible as quickly and as early on as possible. Testers have a big role to play
in delivering this value, but it requires them to take on a business mindset so that they can
understand customer expectations, desires and concerns and develop their testing strategies
accordingly.

43
Although Agile has already made significant
inroads into the software development
lifecycle, there’s still a long way to go,
especially among testing teams..
Going forward, more widespread adoption and greater maturity
of Agile methodologies will require testers to go beyond test
creation and execution and begin to focus on code delivery and
integration too. At the same time, testers will need to hone their
automation skills, become more involved in the entire software
development process and continue to develop a collaborative
relationship with developers. Ultimately, these changes will also
require testers to become experts at development and product
usage in order to provide more holistic testing strategies
and take on the role of “quality champions.”

× “Lots of methodologies
have come and gone,
paradigms have changed
but the requirements are
always the same; Make it
good and make it fast.” -
Anonymous
44

Agile Testing: Methods and Models

More Related Content

What's hot

Similar to Agile Testing: Methods and Models

Recently uploaded

Agile Testing: Methods and Models

Editor's Notes