KEMBAR78
Automated Testing but like for PowerShell (April 2012) | PPTX
Automated Testing 
but like 
For PowerShell 
Rob Reynolds
Who am I? Who cares… 
• Rob Reynolds 
• Technical Lead for VML 
• OSS 
– Chocolatey / Chuck Norris Framework / NuGet 
– Various contributions to other projects 
• ferventcoder – twitter, gmail, you name it 
• ferventcoder.com / Devlicio.us 
• Automated Testing for 8 years
Topics d 
• Automated Testing… 
– Integration Testing 
– Mocking 
– Unit Testing 
• Behavior Driven 
Development… 
– What does it all mean? 
• ..but like for PowerShell? 
– Pester 
Entire presentation w/examples: 
http://bit.ly/powershellbdd
Back to Basics
Test Automation 
• Test automation is the use of software to 
control the execution of tests, the comparison 
of actual outcomes to predicted outcomes, 
the setting up of test preconditions, and other 
test control and test reporting 
functions.[1] Commonly, test automation 
involves automating a manual process already 
in place that uses a formalized testing process. 
• -Wikipedia
So what is test automation? 
• Making executable tests to determine your 
software works appropriately
Fallacies of automated testing 
• “Automated tests will prove the absence of 
bugs” 
• “’Unit’ testing is easy” 
• “One class – one test class” 
• “Testing takes little time”
Why do automated testing? 
• “We never touch that piece of code. A guy 
wrote it a long time ago and he’s not here 
anymore. It just works.” 
– Translation “We have no idea WTF it does and 
have no test automation on it to help us figure it 
out.”
So why do automated testing? 
• Maintenance – change is a constant 
• Feedback cycles 
• Scenarios / edge cases 
• Gives you confidence in the code 
– Works appropriately
What are the types of automated 
tests? 
• White box (MSTest / Nunit / xUnit / MbUnit) 
– Unit 
– Integration 
• Functionality – Groups of functionality 
• Database Tests 
• Black box (FitNesse) 
– End 2 End – System Integration Testing 
• Performance testing
What are the types of automated 
tests? 
• White box (MSTest / Nunit / xUnit / MbUnit) 
– Unit 
– Integration 
• Functionality – Groups of functionality 
• Database Tests 
• Black box (FitNesse) 
– End 2 End – System Integration Testing 
• Performance testing
What are the types of automated 
tests? 
• White box 
– Unit 
– Integration 
• Functionality – Groups of functionality 
• Database Tests 
Why do we categorize? Is it because we are just 
developers and everything has a place?
Reasons for categorization 
• Feedback cycles 
• Maintenance 
• “The more units put into something the 
greater the feedback cycle and the less 
maintainable the test.”
Unit Testing 
• Test a “unit” of something 
• But WTF is a unit? 
– A near religion around this 
• For purposes of discussion a unit is one class / 
sometimes one method in code
Integration Testing 
• Integration 
– Functionality – Groups of functionality 
– Database Tests 
“Anything that tests more than one unit as part 
of the same test is an integration”
But wait, couldn’t that mean all tests 
are integration?
Integration Testing 
• Integration 
– Functionality – Groups of functionality 
– Database Tests 
“Anything that tests more than one unit as part 
of the same test is an integration” 
Yes, most tests we write are a blend of unit and 
integration but we strive for unit testing
Demo
Where does Mocking come in?
Where does mocking come in? 
• Fakes, mocks, stubs – oh my! 
– Dummies - objects are passed around but never 
used 
– Fake - working implementations with possible 
shortcuts 
– Stubs - provide canned answers (answer state 
questions) 
– Mocks - objects with you can set expectations and 
verify behavior on 
• Known as test doubles
Test Doubles – is it okay to blend? 
• Most of us never write a true 
stub/fake/mock/etc but something that 
blends ideas from each of these
Behavior Driven Development…
This is not BDD… 
• ...but the style of writing for your tests in a 
natural language 
• BDD can be a religious battle and this talk is 
not about that (check out Dan North for more 
info) 
• Describe behaviors and interactions 
• Think not about “tests” but about behaviors 
and specifications
Demo
…but like for PowerShell
Meet Pester 
• https://github.com/scottmuc/Pester 
• BDD style of test writing
Demo
But is Pester enough? 
• Does not get you to unit testing alone 
• Does no mocking/stubbing
How to get there 
• A little ingenuity 
• Maybe a mocking/stub framework for 
PowerShell in the future – what would you 
name it? 
– PowerMock? 
– PSomething 
– PShellshock
PowerShell Unit Testing 
• A PowerShell unit is a function 
• Functions can be overwritten by scoping a 
function of the same name
PowerShell Stubbing 
• Functions can be overwritten by a function of 
the same name and parameters 
• This allows you to interrupt and stub the 
functionality 
• “I want behavior verification. I want my mock”
PowerShell Mocking 
• After you have implemented the stubs you 
allow different behaviors to be returned 
through conventional properties and flags
Demo
Questions? 
• ferventcoder – twitter, gmail, etc 
• Pester – 
https://github.com/scottmuc/Pester#readme 
• Chocolatey – http://chocolatey.org 
• Mocks aren’t stubs - 
http://martinfowler.com/articles/mocksArentS 
tubs.html
Automated Testing but like for PowerShell (April 2012)

Automated Testing but like for PowerShell (April 2012)

  • 1.
    Automated Testing butlike For PowerShell Rob Reynolds
  • 3.
    Who am I?Who cares… • Rob Reynolds • Technical Lead for VML • OSS – Chocolatey / Chuck Norris Framework / NuGet – Various contributions to other projects • ferventcoder – twitter, gmail, you name it • ferventcoder.com / Devlicio.us • Automated Testing for 8 years
  • 4.
    Topics d •Automated Testing… – Integration Testing – Mocking – Unit Testing • Behavior Driven Development… – What does it all mean? • ..but like for PowerShell? – Pester Entire presentation w/examples: http://bit.ly/powershellbdd
  • 5.
  • 6.
    Test Automation •Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions.[1] Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. • -Wikipedia
  • 7.
    So what istest automation? • Making executable tests to determine your software works appropriately
  • 8.
    Fallacies of automatedtesting • “Automated tests will prove the absence of bugs” • “’Unit’ testing is easy” • “One class – one test class” • “Testing takes little time”
  • 10.
    Why do automatedtesting? • “We never touch that piece of code. A guy wrote it a long time ago and he’s not here anymore. It just works.” – Translation “We have no idea WTF it does and have no test automation on it to help us figure it out.”
  • 11.
    So why doautomated testing? • Maintenance – change is a constant • Feedback cycles • Scenarios / edge cases • Gives you confidence in the code – Works appropriately
  • 12.
    What are thetypes of automated tests? • White box (MSTest / Nunit / xUnit / MbUnit) – Unit – Integration • Functionality – Groups of functionality • Database Tests • Black box (FitNesse) – End 2 End – System Integration Testing • Performance testing
  • 13.
    What are thetypes of automated tests? • White box (MSTest / Nunit / xUnit / MbUnit) – Unit – Integration • Functionality – Groups of functionality • Database Tests • Black box (FitNesse) – End 2 End – System Integration Testing • Performance testing
  • 14.
    What are thetypes of automated tests? • White box – Unit – Integration • Functionality – Groups of functionality • Database Tests Why do we categorize? Is it because we are just developers and everything has a place?
  • 15.
    Reasons for categorization • Feedback cycles • Maintenance • “The more units put into something the greater the feedback cycle and the less maintainable the test.”
  • 16.
    Unit Testing •Test a “unit” of something • But WTF is a unit? – A near religion around this • For purposes of discussion a unit is one class / sometimes one method in code
  • 17.
    Integration Testing •Integration – Functionality – Groups of functionality – Database Tests “Anything that tests more than one unit as part of the same test is an integration”
  • 18.
    But wait, couldn’tthat mean all tests are integration?
  • 19.
    Integration Testing •Integration – Functionality – Groups of functionality – Database Tests “Anything that tests more than one unit as part of the same test is an integration” Yes, most tests we write are a blend of unit and integration but we strive for unit testing
  • 20.
  • 21.
  • 22.
    Where does mockingcome in? • Fakes, mocks, stubs – oh my! – Dummies - objects are passed around but never used – Fake - working implementations with possible shortcuts – Stubs - provide canned answers (answer state questions) – Mocks - objects with you can set expectations and verify behavior on • Known as test doubles
  • 23.
    Test Doubles –is it okay to blend? • Most of us never write a true stub/fake/mock/etc but something that blends ideas from each of these
  • 24.
  • 25.
    This is notBDD… • ...but the style of writing for your tests in a natural language • BDD can be a religious battle and this talk is not about that (check out Dan North for more info) • Describe behaviors and interactions • Think not about “tests” but about behaviors and specifications
  • 26.
  • 27.
    …but like forPowerShell
  • 28.
    Meet Pester •https://github.com/scottmuc/Pester • BDD style of test writing
  • 29.
  • 30.
    But is Pesterenough? • Does not get you to unit testing alone • Does no mocking/stubbing
  • 31.
    How to getthere • A little ingenuity • Maybe a mocking/stub framework for PowerShell in the future – what would you name it? – PowerMock? – PSomething – PShellshock
  • 32.
    PowerShell Unit Testing • A PowerShell unit is a function • Functions can be overwritten by scoping a function of the same name
  • 33.
    PowerShell Stubbing •Functions can be overwritten by a function of the same name and parameters • This allows you to interrupt and stub the functionality • “I want behavior verification. I want my mock”
  • 34.
    PowerShell Mocking •After you have implemented the stubs you allow different behaviors to be returned through conventional properties and flags
  • 35.
  • 36.
    Questions? • ferventcoder– twitter, gmail, etc • Pester – https://github.com/scottmuc/Pester#readme • Chocolatey – http://chocolatey.org • Mocks aren’t stubs - http://martinfowler.com/articles/mocksArentS tubs.html