Best Practices for Accelerating Continuous Testing

Best Practices for Accelerating Continuous Testing

• 1.
  Seven Best Practicesfor Accelerating Continuous Testing Marc Hornbeek Senior Solutions Architect
• 2.
  2Spirent Communications “There isno continuous delivery without testing” Dev Ops
• 3.
  3Spirent Communications Fast CTis required to race through the many test steps in the pipeline from Pre-Flight through to Delivery
• 4.
  4Spirent Communications CT infrastructuremust be stable and accurate because test failures at any point in the pipeline cause delays or roll- backs
• 5.
  5Spirent Communications Consequences of poorCT practices 80% failure rate when using current tools IDC Fortune 1000 survey, December 2014
• 6.
  6Spirent Communications Accelerated ContinuousTesting Best Practices Principles Comprehensive & Results Test fast & Don’t Stop Powerful Relevant
• 7.
  7Spirent Communications CT bestpractices are across the pipeline “Shift Left”
• 8.
  8Spirent Communications
• 9.
  9Spirent Communications Focus oncustomer relevant defects not quantity of defects “Quality is everyone’s Responsibility.” Dr. W. Edwards Deming
• 10.
  10Spirent Communications Pre-Flight Team-workDesignfor CT CT End-to-End Culture
• 11.
  11Spirent Communications CT Processdesign Fault tolerant Obfuscation Intelligent dashboards CT-Ready Infrastructure
• 12.
  12Spirent Communications CT-Ready Tools Restful APIs  Cache and pipeline  Virtual and physical  Program agnostic  Large scale
• 13.
  13Spirent Communications CT ToolsIntegration  Pre-Flight  CI Tests  Integration  Regression  System  Deployment
• 14.
  14Spirent Communications Orchestrate TestTopologies  Physical and Virtual  Utilization measurement  Topologies  Test Automation
• 15.
  15Spirent Communications Accelerate TestExecution  Powerful servers  Test design  Pipeline  Scale horizontally  Thresholds  Dynamic test schedules
• 16.
  16Spirent Communications Fast RelevantCT Analytics  Continuous Monitoring (CM)  Relevance  Dashboards, Alerts, logs, thresholds  Aggregation  Snapshots and trends  Telescope for diagnostics
• 17.
  17Spirent Communications Case Study 360 developers, six sites  36 MLOC+, 100+ features  86 build targets  complex interdependencies  200+ topologies  12K test cases  80+ deployment packages (physical and virtual) 1,280,000,000 test possibilities
• 18.
  18Spirent Communications The primarycontributing factor was CT and CM ! CT Case Results Metric Major release (#months) Minor release (#weeks) # Features Defects Integrations / day Tests / day Automated tests After 3 2 150 10 100 10 85% Before 6 4 113 1260 0.5 0.3 5%
• 19.
  19Spirent Communications Best practicestoolBest Practices Assessment Tool simplifies the assessment process.
• 20.
  20Spirent Communications Summary  7CT best practices emphasize speed, don’t stop, comprehensive and relevant.  Benchmark against Best Practices Blueprint and DevOps Assessment.  Don't fall into the “do-CT- yourself / NIH” trap! “There is no continuous delivery without testing”
• 21.
  21Spirent Communications www.spirent.com/clear 1) CTBlueprint white papers 2) CT eBook 3) CT Assessment Tool Take-aways http://www.spirent.com/go/AMR
• 22.
  22Spirent Communications How canSpirent help?: Test tools CT Orchestration Lab Management Professional Services
• 23.
  23Spirent Communications © SpirentCommunications, Inc. All of the company names and/or brand names and/or product names and/or logos referred to in this document, in particular the name “Spirent” and its logo device, are either registered trademarks or trademarks pending registration in accordance with relevant national laws. All rights reserved. spirent.com Thank you Seven Best Practices for Accelerating Continuous Testing Marc Hornbeek Senior Solutions Architect Marc.Hornbeek@spirent.com Twitter: mhexcalibur