End-to-end Testing for IoT Integrity

End-to-end Testing for IoT Integrity

• 1.
  Parasoft Proprietary andConfidential 1 2016-03-11 End-to-End Testing for IoT integrity Parasoft Mark Lambert (mlambert@parasoft.com) Mirosław Zielinski (mirek@parasoft.com)
• 2.
  Parasoft Proprietary andConfidential 2Parasoft Proprietary and Confidential 2 Agenda  IoT specifics  Challenges in quality assurance  Effectiveness of end-to-end testing  Doing it better  Demonstration
• 3.
  Parasoft Proprietary andConfidential 3Parasoft Proprietary and Confidential 3 IoT specifics  Multi-layered systems  Disparate technologies  Functionality spanning across many layers
• 4.
  Parasoft Proprietary andConfidential 4Parasoft Proprietary and Confidential 4 IoT specifics  Even simple functionality may require different teams to develop  Debugging requires solid understanding of many technolgies  Complex functionality may be hidden behind simple interface
• 5.
  Parasoft Proprietary andConfidential 5Parasoft Proprietary and Confidential 5 Why it is difficult to test IoT  Disparate development technologies hamper introducing consistent testing strategies  “Not enough code to justify testing” syndrom  Limited user interface make the solution to look deceptively simple  Inverted testing pyramid problem  Strong “temptation” to rely entirely on system testing
• 6.
  Parasoft Proprietary andConfidential 6Parasoft Proprietary and Confidential 6 Effectiveness of end-to-end testing  Labor intensive  Difficult to automate  Difficult to start early  Requires inter-teams synchronization  Difficult to test error handlers  Failed tests do not provide precise information about failure points
• 7.
  Parasoft Proprietary andConfidential 7Parasoft Proprietary and Confidential 7 Effectiveness of end-to-end testing  End-to-End testing for diabetes health system example  Difficult to apply stimulation  Difficult to validate test result  Difficult to test error handlers
• 8.
  Parasoft Proprietary andConfidential 8Parasoft Proprietary and Confidential 8 Why is it difficult to test IoT?  Well known problems …  What makes it special?  So far unseen combination of technologies  Applies to high/low budget projects
• 9.
  Parasoft Proprietary andConfidential 9Parasoft Proprietary and Confidential 9 How to do it better?  Limit manual end-to-end testing  Invest more into automated tests at component  Follow testing pyramid rules  Measure:  test results  test effectiveness
• 10.
  Parasoft Proprietary andConfidential 10Parasoft Proprietary and Confidential 10 How to do it better?  Give absolute priority to automated tests  Invest time into designing interfaces (API)  Use API (service) testing tools to cover interfaces  Measure the quality of the test
• 11.
  Parasoft Proprietary andConfidential 11Parasoft Proprietary and Confidential 11 How to do it better?  Use stubs and mocks to improve:  Facilitate test automation  Increase code penetration
• 12.
  Parasoft Proprietary andConfidential 12Parasoft Proprietary and Confidential 12 How to do it better?  Use service virtualization to improve testing automation  Isolate at the message layer  Simulate functional scenarios and performance conditions
• 13.
  Parasoft Proprietary andConfidential 13Parasoft Proprietary and Confidential 13 How to do it better?  Report test results against requirements  Try to get holistic view of your testing process  Aggregate and review test data from different layers  Assess health of given feature across all layers
• 14.
  Parasoft Proprietary andConfidential 14Parasoft Proprietary and Confidential 14 Test scenario example
• 15.
  Parasoft Proprietary andConfidential 15Parasoft Proprietary and Confidential 15 Test Scenario: REQUEST_NEW_BOLUS_UPDATE_PUMP Tested System Tested System
• 16.
  Parasoft Proprietary andConfidential 16Parasoft Proprietary and Confidential 16 Summary: End-to-End Testing for IoT integrity  IoT combines previously isolated technologies  Multiple Tiers = Complexity = Risk  End-to-End Testing = Chain is only as strong as it’s weakest link  Reliable quality process allows organizations to assess modules individually and in combination  Continuously automate testing  Tracability across tiers, technologies and testing techniques
• 17.
  Parasoft Proprietary andConfidential 17Parasoft Proprietary and Confidential 17 Q & A