The Best Pairwise Testing Tool / Best Orthogonal Array Tool Just Got Better

Powerful New
Hexawise Features

1

Agenda
• Intro and “Quickie Demo”
- What is Hexawise? What doesn’t it do?
- How does pairwise testing and related methods work?
- Advantages and Limitations of the “Old” Hexawise
• Hexawise “2.0” - The Most Powerful New Features We’ve Ever Released:
- Requirements Traceability
- More Powerful Test Design Capabilities
- Faster Test Script Documentation (including Expected Results)
- Better Auto-Scripting
• Demo of New Features
• Wrap Up with Questions and Answers
- Suggestions for Good Sources of Information
2

What is Hexawise?
It’s a test design tool.
Testers use it to do these two things better & faster.

Define
Requirements

Export Test
Scripts into Test
Mgmt. Tool

Select Test
Conditions

Execute
Test Scripts

Document
Test Scripts

Find Bugs
and Verify
Behavior

But that’s definitely not all Hexawise does….
3

What is Hexawise? (Cont’d)
… Hexawise also improves test EXECUTION efficiency &
effectiveness.
The tests you generate with Hexawise are more powerful than the tests
you would select by hand. Specifically, tests generated using Hexawise
are optimized to:
• Be as varied as possible,
• Minimize wasteful repetition, and
• Maximize test coverage in a manageable number of tests.

Export Test
Scripts into Test
Mgmt. Tool

Execute
Test Scripts

Find Bugs
and Verify
Behavior

4

What DOESN’T Hexawise Do?

Hexawise is not a test management tool (like Quality Center).

It’s also not a tool that automates test execution (like QTP).

5

When and Where is Hexawise used?

After testers generate
tests in Hexawise, they
usually export them to a
tool like Quality Center.

Export Test
Scripts into Test
Mgmt. Tool

Hexawise generates tests
for teams that do both:
- Manual testing, and
- Automated testing.

Execute
Test Scripts

6

Poll Results:
Most Popular Hexawise Benefit?
1. You document
tests faster…

2. You complete test
execution faster…

3. You consistently
find more bugs.

Select Test
Conditions

Document
Test Scripts

How? Hexawise automates
many test design and test
documentation steps.

Execute
Test Scripts

How? If you want to, you can
execute fewer, more powerful tests.

Find Bugs
and Verify
Behavior

How? Test generation algorithms
maximize testing coverage.

4. Your team makes more informed,
data-driven decisions.

How? From much more precise
coverage analysis reporting.
7

Where Do Testers Use Hexawise?

Everywhere.

8

Is Hexawise Applicable to My System?

Yes.

“When testing anything: "No matter what kind of interface it has, you can
always identify interesting things to vary."
- Elisabeth Hendrickson – in her brand new book “Explore It!”

9

Agenda
• Same Easy to Use Features as Always, to Increase Your…

- Test selection speed
- Test documentation speed
- Testing thoroughness
• “Quickie Demo”

- Advantages and Limitations of the “Old” Hexawise
• Plus The Most Powerful New Features We’ve Ever Released:

- Requirements Traceability
- More powerful Test Design Capabilities
- Faster Documentation with Expected Results
- Better Auto-Scripting
• Demo of New Features
10

Charlie’s Chocolates – Release 7
The “interesting things to vary”…

11

As
before,
to do
this

Select Test
Conditions

Simply (A) Enter test conditions, and (B)
click here.

12

How does Hexawise prioritize tests?
Short version:
1)

Pairwise: In this example – using the default “pairwise” coverage strength Hexawise will select 16 tests from 576 total possible ones. Those 16 tests
include at least one test of every possible pair of Values.

2)

Rationale: The vast majority of your defects that sneak by undetected into
production will be triggered if you tested for every possible pair of Values.

3)

Manually-Selected Tests Have Many Holes: If you started to test this way
tomorrow, even with fewer tests, you would find many more defects than you
currently do because you would be closing thousands of gaps that exist today.

4)

First Draft vs. Final: These 16 are a “first draft” set of powerful tests. Not a
final set. You can and should modify them based on your knowledge.

5)

Turn the Coverage Dial Up or Down: You can easily generate however
many tests you want depending upon your time vs. thoroughness preferences:
a dozen?, a hundred?, all 576?

For more information, data, and real-world case studies, see: pairwisetesting.com
13

Before,
to do
this

Document
Test Scripts

… testers would transform powerfully
optimized sets of test conditions like these…

14

…into “test scripts” like these below using the Auto-Script feature. (Note
that these have both: (A) detailed instructions and (B) test conditions).

15

Advantages & Limitations: “Old” Hexawise
Test Script Documentation:
While we were able to
Quickly select and document new test scripts…

And re-write the tester instructions “instantly” for each test…
Those test scripts…

Did not include Expected Results.

16

Advantages & Limitations: “Old” Hexawise
Requirements / Ability to Include Specific Complex Combinations:
While we were able to
Quickly generate new sets of highly-varied test scripts…

And those tests had maximum coverage possible with
minimum repetition…
We did not have the ability to…
Include specific combinations of Values to test for our
complex requirements.

17

So What’s New in “Hexawise 2.0”?

18

Let’s see how “Hexawise 2.0” handles changes for the next release:

New Features to Test in Release 8 of Charlie’s Sales Platform:
1. Charlie now accepts MasterCard
2. Charlie now offers 2-day shipping

Additional “Trouble Area” to Test More Thoroughly in Release 8:
1. Single item orders are working well; when multiple items of an item are
ordered, odd things have been happening in Release 7.

19

We need to add some new inputs (as usual)…

20

… which will lead to an entirely new set of
“scrambled” tests which need new ERs.

… tons of new Expected Results!
21

Good news: we can now generate
Expected Results automatically!

22

Which means creating test scripts like
these is now easy and fast:
Once you generate your simple rules, hundreds (or thousands!) of
these Expected Results will be generated with the push of a
button… without a single mistake.

23

So far so good, but how can we test a
convoluted warning message like this?!

This
“warning”
feature can
only be
tested by
specifying
many
specific
Values to
appear in a
test.

This example (a result of both great customer service and good testing) was “seen in the wild” by one of my heroes:
http://37signals.com/svn/posts/3239-a-great-customer-experience-nutscom-goes-the-extra-mile
24

And how does “Hexawise 2.0” handle these additional testing goals?

Two Requirements to Test (from new Business Rules):
1. Fizzy Lifting Drinks ordered on November 11 should get an 11% discount.

2. “The complex conditional melted chocolate notification”
• If customer selects 2-day delivery,
• … and the type of food being shipped is made of chocolate,
• … and it would actually take 3-days to deliver (b/c of a Sunday or Holiday),

• … and it is predicted to be hot enough to melt chocolate outside,
• …. Then notify the customer there is a risk the chocolate might melt.

25

Here’s how to think about entering
Requirements in Hexawise:

First –

Ask “what is the Expected Result of the requirement?”

Second –

Ask “What is the smallest number of test inputs you
need to specify in order to see if the requirement is true
or not?”

Third –

Enter those test inputs (and only those) into the
Requirements screen.

26

Requirements
“Fizzy Lifting Drinks ordered on November 11 should get an 11% discount.”

1. Expected Result =?
“should get an 11% discount.”
2. Smallest Number of Values to trigger the discount = ?
2:
Type of Candy must = Fizzy Lifting Drinks
Date Ordered must = November 11th.
3. Now just enter those two Values (and only those two
values!) into the Requirements screen and enter the ER!

27

Add the Values in

Screen:

28

Requirements Feature (Cont.)
To add a test for the second “complex melting chocolate notification”
requirement, we follow the same process:

What is the Expected Result we want to trigger?
“Then notify the customer there is a risk the chocolate might melt.”

How many specific Values do we need to trigger it?
4. So simply select these four values from the drop down menus
on the Requirements screen and add your Expected Result:
1. Customer selects 2-day delivery, and
2. The type of food being shipped is made of chocolate, and
3. It would actually take 3-days to deliver (b/c of a Sunday or
Holiday), and
4. A weather prediction system predicts it will be hot.
29

“Traditional” Recommended Sources
Explanations of how Pairwise Testing (and similar, more
sophisticated test design methods) consistently pack more testing
coverage into a manageable set of tests than hand-selected tests
• pairwisetesting.com

Questions about features (e.g., “How do I…?”)
•

help.hexawise.com

Training / understanding test design concepts
•

training.hexawise.com

An introductory, fact-based article for skeptical managers
•

A scholarly, evidence-based article and explanation of benefits from a 10
project study that I co-wrote with 3 PhD’s

30

Fun Recommended Sources
•

A fun presentation about pairwise software testing - powerful test design

hexawise.com/?p=531

•

Nutty video announcement of our new “Hexawise 2.0” features

http://www.youtube.com/watch?v=3F1eQag6XUg
31

Sign up for a free account now:
Try Hexawise out yourself.

hexawise.com/free

32

The Best Pairwise Testing Tool / Best Orthogonal Array Tool Just Got Better

More Related Content

What's hot

Viewers also liked

Similar to The Best Pairwise Testing Tool / Best Orthogonal Array Tool Just Got Better

Recently uploaded

The Best Pairwise Testing Tool / Best Orthogonal Array Tool Just Got Better

Editor's Notes