KEMBAR78
Unit Testing | PDF
Scott Leberknight
6/28/2019
(Unit) Testing
Why Test?
Does your code work?
How do you know?
Tests provide some level of confidence
your code does what you think it does…
…which is not the same as saying
the code is correct
from a business perspective
Tests cannot and will not ever
catch or prevent all possible problems
You might have a passing test suite but
still have logic bugs…
e.g. due to poorly defined or
understood requirements, poor
design, etc.
“In the 737 Max, only one of the flight
management computers is active at a time -
either the pilot’s computer or the copilot’s
computer.And the active computer takes
inputs only from the sensors on its own side
of the aircraft.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
“When the two computers disagree, the
solution for the humans in the cockpit is 

to look across the control panel to see

what the other instruments are saying and
then sort it out. In the Boeing system, the
flight management computer does not “look 

across” at the other instruments. It 

believes only the instruments on its side.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
“This means that if a particular angle-of-
attack sensor goes haywire - which happens
all the time in a machine that alternates
from one extreme environment to another,
vibrating and shaking all the way - the flight
management computer just believes it.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
“The primary cause of this discrepancy was that one piece of ground
software supplied by Lockheed Martin produced results in a United
States customary unit, contrary to its Software Interface Specification
(SIS), while a second system, supplied by NASA, expected those
results to be in SI units, in accordance with the SIS. Specifically,
software that calculated the total impulse produced by thruster
firings produced results in pound-force seconds.The trajectory
calculation software then used these results – expected to be in
newton seconds – to update the predicted position of the
spacecraft.”
https://en.wikipedia.org/wiki/Mars_Climate_Orbiter#Cause_of_failure
The act of testing makes you consider more
than just the “happy path”
Many Flavors…
“Unit” Integration
Performance
Functional (QA)
Security
User interface
Other…?Exploratory
Good testing…
Automated
Repeatable
Continuous
(“ARC” testing)
Good unit testing…
Fast (a few milliseconds?)
Isolated (no “real” dependencies)
Consider success & failure modes
What is a “unit” test?
Not always clearly defined
A single class? A single function?
Can it touch external resources?
(e.g. a database, web service, or file system)
@Test
fun `contribution should be sum of salary & Roth deferrals`() {
assertThat(payroll.totalContribution)
.isEqualTo(payroll.salaryDeferral +
payroll.rothDeferral)
}
A simple unit test
Potential Benefits
Find regressions quickly
Confidence
Better design (*)
(*) probably…
You can go faster in the long run
Potential Pitfalls
False confidence
Tests can have bugs too…
(*) debatable, but not in my personal experience…
You might be slower in short run (*)
What to test?
“Business” logic…
Persistence logic…
User interfaces…
“Anything that can possibly break” ???
What to test?
Test your code…
…not your dependencies
What to test?
More generally you should assume
libraries, frameworks, operating systems,
etc. have been tested and work as
advertised…
(obviously there are caveats to the above)
How to test?
TDD (Test-Driven Development aka “test first”)
TAST (test at same time, commit only tested code)
Test after (avoid…or deal with the consequences)
…on “legacy code” this is your only option
“Legacy Code”
“To me, legacy code is simply
code without tests”
- Michael Feathers,
Working Effectively with Legacy Code
“TDD”
Test-driven development
Usually associated with “test first”
Write failing test, write simplest code
to make it pass, repeat…
“Red-Green-Refactor” cycle
“TDD”
Can you be test-driven without test first?
I think you can…
By letting tests drive the design…
And guide towards “testable” code…
Tests & Design
What does “testable” code mean?
* Loosely coupled, e.g. injecting dependencies
* Separation of concerns
* Single Responsibility Principle (SRP)
Tests & Design
What does “testable” code look like?
* Smaller classes in OO languages
* Smaller methods/functions (7-10 lines max? 20? 30?)
* Injected dependencies, e.g. via constructors or
function parameters
* More “pure” functions
Tests & Design
Why do these things make code testable?
* Facilitates “mocking” dependencies, e.g. a credit
card processing web service, or persistence to a
database
* A function that accepts input, has no side effects,
and produces a consistent output is much easier
to test
Testing Techniques
Prefer smaller, targeted tests (less coupling)
Mock objects (but don’t overdo it)
Perform assertions on return values
Verify behavior of mock objects
(correct methods called, expected arguments)
Testing Techniques
Property-based testing
Test “units” and mock dependencies
Generative testing
Measuring code “covered” by tests
Example test using a mock
@ExtendWith(DropwizardExtensionsSupport.class)
class RelationshipResourceTest {
private static final RelationshipService SERVICE = mock(RelationshipService.class);
private static final ResourceExtension RESOURCES = ResourceExtension.builder()
.addResource(new RelationshipResource(SERVICE))
.build();
@AfterEach
void tearDown() {
reset(SERVICE);
}
@Test
@DisplayName("given a valid event should attempt to save the event")
void testRecordEvent() {
ConnectionEvent event =
newConnectionEvent(A_SERVICE_NAME, Direction.OUTBOUND, "some-identifier");
Response response = RESOURCES.target(“/elucidate/event")
.request()
.post(Entity.json(event));
assertThat(response.getStatus()).isEqualTo(202);
verify(SERVICE).createEvent(eq(event));
}
// more tests...
}
Testing Async Code
No sleeps!
Mock the environment
“If we go by the book…hours could seem like days…”
Wait for conditions
(e.g. until a status value is changed)
Unit vs Integration Tests
* A unit test of a single HTTP endpoint accepts
HTTP requests, but uses mock dependencies, e.g.
objects to persist data, or process payments, or
send asynchronous messages
* An integration test of that single HTTP endpoint
accepts HTTP requests, but uses the actual
dependencies…
Code Coverage
How much is “enough”?
Should automated builds fail if
below a threshold? 70%? 80%?
Can/should you test all exceptional
conditions?
Code Coverage
By definition, decoupled, “testable” code
is easier to achieve higher coverage
The goal should not be a number (80%)
The goal is ensuring correct behavior,
output, etc.
Key Takeaways…
Gain confidence that code is “correct”
Automated, repeatable, continuous (ARC)
Tests cannot catch all problems
Better design through “testable code”
Build & change systems faster
(THE END)
Suggested Reading
Clean Code, Robert “Uncle Bob” Martin
Working Effectively with Legacy Code, Michael Feathers
Pragmatic Programmer, Dave Thomas & Andy Hunt
(20th anniversary edition in beta at
https://pragprog.com/book/tpp20/the-pragmatic-programmer-20th-anniversary-edition)
Suggested Websites
JUnit, de-factor Java unit testing framework, https://junit.org/junit5/
Mockito, mocking library, https://site.mockito.org
AssertJ, fluent assertion library, https://joel-costigliola.github.io/assertj/
Awaitility, async testing library, https://github.com/awaitility/awaitility
Jacoco, code coverage, https://www.jacoco.org/jacoco/
Boeing 737 MAX
https://commons.wikimedia.org/wiki/File:WS_YYC_737_MAX_1.jpg
https://creativecommons.org/licenses/by-sa/4.0/deed.en
No changes made
Photos & Images
(All other images were purchased from iStock)
66MHz Pentium Chip with FDIV flaw
https://en.wikipedia.org/wiki/Pentium_FDIV_bug#/media/
File:KL_Intel_Pentium_A80501.jpg
Konstantin Lanzet
[CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0/)]
No changes made
Mars Climate Orbiter
https://en.wikipedia.org/wiki/Mars_Climate_Orbiter#/media/
File:Mars_Climate_Orbiter_2.jpg
Public domain
Photos & Images
(All other images were purchased from iStock)
My Info
sleberknight at
fortitudetec.com
www.fortitudetec.com
@sleberknight
scott.leberknight at
gmail

Unit Testing

  • 1.
  • 2.
  • 4.
    Does your codework? How do you know?
  • 5.
    Tests provide somelevel of confidence your code does what you think it does…
  • 6.
    …which is notthe same as saying the code is correct from a business perspective
  • 7.
    Tests cannot andwill not ever catch or prevent all possible problems
  • 8.
    You might havea passing test suite but still have logic bugs… e.g. due to poorly defined or understood requirements, poor design, etc.
  • 10.
    “In the 737 Max,only one of the flight management computers is active at a time - either the pilot’s computer or the copilot’s computer.And the active computer takes inputs only from the sensors on its own side of the aircraft.” https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
  • 11.
    “When the twocomputers disagree, the solution for the humans in the cockpit is 
 to look across the control panel to see
 what the other instruments are saying and then sort it out. In the Boeing system, the flight management computer does not “look 
 across” at the other instruments. It 
 believes only the instruments on its side.” https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
  • 12.
    “This means thatif a particular angle-of- attack sensor goes haywire - which happens all the time in a machine that alternates from one extreme environment to another, vibrating and shaking all the way - the flight management computer just believes it.” https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
  • 14.
    “The primary causeof this discrepancy was that one piece of ground software supplied by Lockheed Martin produced results in a United States customary unit, contrary to its Software Interface Specification (SIS), while a second system, supplied by NASA, expected those results to be in SI units, in accordance with the SIS. Specifically, software that calculated the total impulse produced by thruster firings produced results in pound-force seconds.The trajectory calculation software then used these results – expected to be in newton seconds – to update the predicted position of the spacecraft.” https://en.wikipedia.org/wiki/Mars_Climate_Orbiter#Cause_of_failure
  • 15.
    The act oftesting makes you consider more than just the “happy path”
  • 16.
    Many Flavors… “Unit” Integration Performance Functional(QA) Security User interface Other…?Exploratory
  • 17.
  • 18.
    Good unit testing… Fast(a few milliseconds?) Isolated (no “real” dependencies) Consider success & failure modes
  • 19.
    What is a“unit” test? Not always clearly defined A single class? A single function? Can it touch external resources? (e.g. a database, web service, or file system)
  • 20.
    @Test fun `contribution shouldbe sum of salary & Roth deferrals`() { assertThat(payroll.totalContribution) .isEqualTo(payroll.salaryDeferral + payroll.rothDeferral) } A simple unit test
  • 21.
    Potential Benefits Find regressionsquickly Confidence Better design (*) (*) probably… You can go faster in the long run
  • 22.
    Potential Pitfalls False confidence Testscan have bugs too… (*) debatable, but not in my personal experience… You might be slower in short run (*)
  • 23.
    What to test? “Business”logic… Persistence logic… User interfaces… “Anything that can possibly break” ???
  • 24.
    What to test? Testyour code… …not your dependencies
  • 25.
    What to test? Moregenerally you should assume libraries, frameworks, operating systems, etc. have been tested and work as advertised… (obviously there are caveats to the above)
  • 26.
    How to test? TDD(Test-Driven Development aka “test first”) TAST (test at same time, commit only tested code) Test after (avoid…or deal with the consequences) …on “legacy code” this is your only option
  • 27.
    “Legacy Code” “To me,legacy code is simply code without tests” - Michael Feathers, Working Effectively with Legacy Code
  • 28.
    “TDD” Test-driven development Usually associatedwith “test first” Write failing test, write simplest code to make it pass, repeat… “Red-Green-Refactor” cycle
  • 29.
    “TDD” Can you betest-driven without test first? I think you can… By letting tests drive the design… And guide towards “testable” code…
  • 30.
    Tests & Design Whatdoes “testable” code mean? * Loosely coupled, e.g. injecting dependencies * Separation of concerns * Single Responsibility Principle (SRP)
  • 31.
    Tests & Design Whatdoes “testable” code look like? * Smaller classes in OO languages * Smaller methods/functions (7-10 lines max? 20? 30?) * Injected dependencies, e.g. via constructors or function parameters * More “pure” functions
  • 32.
    Tests & Design Whydo these things make code testable? * Facilitates “mocking” dependencies, e.g. a credit card processing web service, or persistence to a database * A function that accepts input, has no side effects, and produces a consistent output is much easier to test
  • 33.
    Testing Techniques Prefer smaller,targeted tests (less coupling) Mock objects (but don’t overdo it) Perform assertions on return values Verify behavior of mock objects (correct methods called, expected arguments)
  • 34.
    Testing Techniques Property-based testing Test“units” and mock dependencies Generative testing Measuring code “covered” by tests
  • 35.
  • 36.
    @ExtendWith(DropwizardExtensionsSupport.class) class RelationshipResourceTest { privatestatic final RelationshipService SERVICE = mock(RelationshipService.class); private static final ResourceExtension RESOURCES = ResourceExtension.builder() .addResource(new RelationshipResource(SERVICE)) .build(); @AfterEach void tearDown() { reset(SERVICE); } @Test @DisplayName("given a valid event should attempt to save the event") void testRecordEvent() { ConnectionEvent event = newConnectionEvent(A_SERVICE_NAME, Direction.OUTBOUND, "some-identifier"); Response response = RESOURCES.target(“/elucidate/event") .request() .post(Entity.json(event)); assertThat(response.getStatus()).isEqualTo(202); verify(SERVICE).createEvent(eq(event)); } // more tests... }
  • 37.
    Testing Async Code Nosleeps! Mock the environment “If we go by the book…hours could seem like days…” Wait for conditions (e.g. until a status value is changed)
  • 38.
    Unit vs IntegrationTests * A unit test of a single HTTP endpoint accepts HTTP requests, but uses mock dependencies, e.g. objects to persist data, or process payments, or send asynchronous messages * An integration test of that single HTTP endpoint accepts HTTP requests, but uses the actual dependencies…
  • 39.
    Code Coverage How muchis “enough”? Should automated builds fail if below a threshold? 70%? 80%? Can/should you test all exceptional conditions?
  • 40.
    Code Coverage By definition,decoupled, “testable” code is easier to achieve higher coverage The goal should not be a number (80%) The goal is ensuring correct behavior, output, etc.
  • 41.
  • 42.
    Gain confidence thatcode is “correct” Automated, repeatable, continuous (ARC) Tests cannot catch all problems Better design through “testable code” Build & change systems faster
  • 43.
  • 44.
    Suggested Reading Clean Code,Robert “Uncle Bob” Martin Working Effectively with Legacy Code, Michael Feathers Pragmatic Programmer, Dave Thomas & Andy Hunt (20th anniversary edition in beta at https://pragprog.com/book/tpp20/the-pragmatic-programmer-20th-anniversary-edition)
  • 45.
    Suggested Websites JUnit, de-factorJava unit testing framework, https://junit.org/junit5/ Mockito, mocking library, https://site.mockito.org AssertJ, fluent assertion library, https://joel-costigliola.github.io/assertj/ Awaitility, async testing library, https://github.com/awaitility/awaitility Jacoco, code coverage, https://www.jacoco.org/jacoco/
  • 46.
    Boeing 737 MAX https://commons.wikimedia.org/wiki/File:WS_YYC_737_MAX_1.jpg https://creativecommons.org/licenses/by-sa/4.0/deed.en Nochanges made Photos & Images (All other images were purchased from iStock) 66MHz Pentium Chip with FDIV flaw https://en.wikipedia.org/wiki/Pentium_FDIV_bug#/media/ File:KL_Intel_Pentium_A80501.jpg Konstantin Lanzet [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0/)] No changes made
  • 47.
  • 48.