Testing Metrics - Making your tests visible

Testing Metrics - Making your tests visible

• 1.
  TESTING METRICS MAKING YOURTESTS VISIBLE Alper Mermer 1
• 2.
  WHY WOULD YOUCARE ABOUT METRICS ? 2 Make your team aware of the trends
• 3.
  WHY WOULD YOUCARE ABOUT METRICS ? 3Share the pain
• 4.
  WHY WOULD YOUCARE ABOUT METRICS ? 4Watch production
• 5.
  WHY WOULD YOUCARE ABOUT METRICS ? 5 Improve ownership
• 6.
  WHY WOULD YOUCARE ABOUT METRICS ? 6 Prevent issues
• 7.
  WHY WOULD YOUCARE ABOUT METRICS ? 7Not all metrics are needed in every project
• 8.
  WHICH METRICS SHOULDYOU TRACK ? Measure every service performance seperately Define sets of scenarios to run in every build 8 Performance Measurements Through Time
• 9.
  WHICH METRICS SHOULDYOU TRACK ? Build mechanisms to warn about unexpected variances in services Watch overall system performance as well Keep an eye on trends 9 Performance Measurements Through Time
• 10.
  WHICH METRICS SHOULDYOU TRACK ? Break builds with high security risks For lesser security risks, watch the trend 10 Number Of Security Warnings
• 11.
  WHICH METRICS SHOULDYOU TRACK ? Test coverage (?) Cyclomatic Complexity 11 Code Quality Metrics
• 12.
  WHICH METRICS SHOULDYOU TRACK ? Depth of Inheritance Distance from Main 12 Code Quality Metrics
• 13.
  WHICH METRICS SHOULDYOU TRACK ? Tests that cover your requirements are listed here Increased time in Run Time affects feedback time Serves as a motivation for testers 13 API + GUI Test Count And Run Time
• 14.
  WHICH METRICS SHOULDYOU TRACK ? Should help to increase focus on quality Decreasing number of higher level bugs boosts up morale 14 Defects By Priority/Severity
• 15.
  WHICH METRICS SHOULDYOU TRACK ? Lesser number of bugs increases the confidence in the product An increase in the number of higher level bugs should start an investigation 15 Defects By Priority/Severity
• 16.
  WHICH METRICS SHOULDYOU TRACK ? Trend in number of defects from production is important It affects product and company image Should be closely monitored and analyzed 16 Number of Defects From Production
• 17.
  WHICH METRICS SHOULDYOU TRACK ? Serves as an indicator of problematic areas Should initiate discussions around prioritization 17 Defects From Production by Module
• 18.
  WHICH METRICS SHOULDYOU TRACK ? A team internal metric A nice indicator of your testing process It is expected to be very high in a mature shift left testing culture 18 Mean Time Between Build Failures
• 19.
  LAST WORDS 19 Metrics AreThere to Help Your Brain, Not Replace It
• 20.
  20 THANK YOU Questions ? amermer@thoughtworks.com @alpermermer @test_hive