Automated Acceptance Testing from Scratch

Automated Acceptance Tests
from Scratch
Daniel Davis

• Daniel Davis
• Software Developer for 8 years
• Fun Fact:
– I recently went to see “Laser Cat” in DC
Who Am I?

Really though, who are you?
• Came from Java world
• Python developer for 2 years
• DevOps
– Lots of work with automation and quality
• First introduced to AAT about 2 years ago
– Now I use it at my client!

Now that it’s up…
how do I test that it works?
6

• Go to the website, click around, make sure
things look right…
• Problem: Scaling
– What happens as the site continues to grow?
– What happens as we deploy more frequently?
• We need a way to test the functionality of
the site
Simple!

Automated Acceptance
Testing Can Help!!!

• Tests written at the functional/business
level
• Can be read by non-technical people
• Doesn’t have to be automated to provide
value!!!
What is Acceptance Testing

Unit – 70%
Integration
– 20%
AAT
10%
Where does AAT fit in?

Consider it this way…
11
Unit Testing Integration Testing

Automated Acceptance Testing
• Great for covering a lot of ground
• Not so good for very specific things

Ok, so how do I get
started???

• Java/JVM world: Cucumber-JVM/JBehave
• Ruby: Cucumber-Ruby
• Ruby on Rails: Cucmber-Rails
• Node.js/Javascript: Cucumber-Javascript
– AngularJS: Protractor  not the same
• C#/.NET: SpecFlow
• Python: Behave
Cucumber
16

• Cucumber’s syntax and style is virtually
the same across all the platforms!
• Tonight’s examples will be in Python
Good News Everyone…
17

• One example, from start to finish
– This is how we solve problems
• Learning Objectives:
– Write an acceptance test
– Automate that test using Selenium
– Incorporate that test into your Continuous
Integration
• Stay high level
– Not enough time to go deep
Let’s get started!
18

• Objective: Want to test Upvote/Downvote
functionality
• Follow Along:
– https://github.com/Ooblioob/aat_demo/
theCatAPI.com

Step #1: Write an
Acceptance Test
22

• Install your cucumber tool of choice
• Directory structure:
project-root/
|-- features/
|-- steps/ (we’ll talk about this soon)
|-- pages/ (we’ll talk about this soon)
Project Setup
23

3 Layers to Cucumber
24
Gherkin (Feature)
“Glue” (Step)
Selenium Code

• A Human-readable syntax describing the
test
• Consists of a main “feature” and scenarios
describing the feature
Writing Gherkin

Feature: Title (one line describing the story/feature)
As a [role]
I want [feature]
So that [benefit]
Scenario: Title1 (for Behaviour 1)
Given [context or setup]
And [some more context]...
When [event occurs]
Then [expected outcome]
And [another outcome]...
Scenario: Title2 (for Behaviour 2)
…
Writing Gherkin
26

TheCatAPI.com
27
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/
features/upvote.feature

• Value even when not automated
– Clear, concise way to manually test
– Developer friendly
• Not extremely specific…
– What does it mean “a new cat image should
be loaded…”?
– Declarative vs Imperative
– Don’t get bogged down in
precision/verboseness
Observations

Step #2: Automating your
Acceptance Tests
29

3 Layers to Cucumber
30
Gherkin (Feature)
“Glue” (Step)
Selenium Code

• Create an “environment.py” file in the
features directory
– Defines the test runner setup
• (optional) Create an environment.cfg file in
the features directory
– Holds editable configurations for your setup
• Create a “steps.py” file in the steps directory
– Contains the implementation of your steps
Project Setup
31

Environment.py
32
Full Example:
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/fe
atures/environment.py

• Setting up Selenium
– Select a driver, like Chromedriver, PhantomJS
or SauceLabs
• Setting up a logger
• Initializing the context object
• Set defaults
What goes in Environment.py?
33

• Parses each line of Gherkin
• Decide how to implement
Glue code (steps)
home.feature
steps/steps.py

Writing your implementation
Example:
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/featu
res/steps/steps_home.py

• Making references to context
– Way of passing information between steps
• What is “Home” in the context?
– Class representing the homepage
– Part of “Page-object model”
Observations
36

That pretty much it…
When do we talk about Selenium?
37

Not the best practice…
38
Glue (step)
One Giant Selenium Class

Page-Object model
39
Glue (step)
Home
page
Docs
page
Reference
page
Utility
functions
Base Page

So what goes in a
“Page” object?
40

“Hate It” Button
“Like It” Button
Title
Main
Image

• For each page element:
– Selectors: Id, CSS, XPATH, etc
– get() method
– commonly used attributes
• Define navigation to the page
– go()
• Define other actions / helpers
– Storing info, waits, etc.
Writing your pages

• Need to setup the web driver
– Let’s use ChromeDriver
• https://sites.google.com/a/chromium.org/chromedri
ver/getting-started
Environment.py revisited
45

Demo Time!
(Making it all work)
46

• Use page-object model to organize code
• Write simple, reusable methods to
implement pages
• Installed Chromedriver
Recap

Step #3: Put it into Continuous
Integration!
48

• Now that we have tests
– Validate site is up and running
– Validate deployments
– Ensure new code doesn’t break existing
functionality (regression)
Why Continuous Integration?

• PhantomJS
– Headless browser
– Based on Webkit
– Fast
• Works with Selenium
Solving the problem
51

• Black Box
– Can’t see problems when they happen
• Can’t handle certain things
– e.g. alerts
• Buggy, but getting better
Problems with PhantomJS
52
At this time, I don’t recommend it…

• “Cloudified” browsers
Sauce Labs
53

• Test different browser / os configurations
– Don’t need to own that hardware
• Integration with many CI tools
– Jenkins, Travis, CircleCI, Bamboo, Teamcity…
• Video recording/playback
– No longer a black box!!!
Advantages
54

Yes, but what about
my Dev severs???
55

• Sauce Connect: test your Dev server too!
Advantages
56

• Sign Up for an account:
– https://saucelabs.com/signup/trial
• Receive an API key
Sauce Labs Setup

• Modify your environment.py file to use
Sauce instead of Chrome
– Provide username and api key
– Select a browser and OS to use
Sauce Labs Setup
59

Sauce Labs Setup
60
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/f
eatures/environment.py

Sauce Labs Setup
61
https://github.com/Ooblioob/aat_demo/blob/master/browser_tests/f
eatures/environment.py

• Can only test what’s available through the
browser
• Selenium can be painful at times
– May not wait for things to load
– Debugging can be hard
Limitations of AAT

• Data
– Not a great choice for data-heavy applications
– Not impossible, but trickier:
• Launch a separate test server?
• Create a “testing mode” where transactions are
rolled back
– Might be better covered by Integration Testing
Limitations of AAT
65

• https://github.com/Ooblioob/aat_demo
• https://cukes.info/
• http://jenisys.github.io/behave.example/
• https://docs.saucelabs.com/reference/sauc
e-connect/
Resources
66

Automated Acceptance Testing from Scratch

More Related Content

What's hot

Viewers also liked

Similar to Automated Acceptance Testing from Scratch

More from Excella

Recently uploaded

Automated Acceptance Testing from Scratch