Automated visual-regression-testing

Automated visual-regression-testing

• 1.
  A Visual Comparison Approachto Automated Regression Testing (PDF to PDF Compare)
• 2.
  Regression Testing • Typeof software testing that seeks to uncover new bugs in existing functional areas of a system after changes have been made to them • Ensure that a change has not introduced new faults and that the intended results are still being produced VisualRegressionTesting 2
• 3.
  Visual Regression Testing •Compares visual output • New file = or ≠ reference file • Different underlying code can produce the same output • Testing is necessary to see if anything changed • Produce wanted results • Produce unwanted results • Break something that was working fine before • Involves testing either • A subset of documents • A large collection of documents (preferred) VisualRegressionTesting 3
• 4.
  The Antenna HouseProblem • The need to regression test new releases of Antenna House Formatter • At least 8 maintenance releases a year • Regression Testing (Visual Method) • Has been done manually by visually comparing output of old and new files • Time consuming • Prone to missing minor (but important) differences • Can lead to delays in product releases • Done only on candidate release versions • Increase in support calls due to missed errors VisualRegressionTesting 4
• 5.
  Going Automated Challenges •Compareoutputs visually, not the underlying code •High speed performance •Support for large collection of documents •Method to generate meaningful reports VisualRegressionTesting 5
• 6.
  What is AHRTS? •Java application • Automated visual comparison tool • Precision pixel-by-pixel comparison of output documents • Scalable & fast regression testing • Regression testing of new releases of AH Formatter • An automated system for comparing versions of rendering software or rendered output • Beyond regression testing • Pre-production system check • Stylesheet development • Installation validation • System(s) certification VisualRegressionTesting 6
• 7.
  Homogenize Properties Compare Homogenize Properties Is the content thesame? PDF2PDF Select PDF documents to be compared PDF 1 PDF 2 Extract PDF code as character strings Extract PDF code as character strings Return XML Report Page No Difference Yes How does it work? VisualRegressionTesting 7
• 8.
  Are pages the same? PDF1 PDF 2 Compare Pixel by Pixel Page by Page Convert to Bitmap Convert to Bitmap Return XML Report Page Difference No Create Composite Image of Differences No Return XML Report Page No Difference Yes Format to PDF using XSL-FO How does it work? VisualRegressionTesting 8
• 9.
  Benefits • Automates regressiontesting • High speed performance • 90%+ reduction in human effort • Handles large document comparisons • Significantly more accurate and reliable • Able to catch more subtle differences • Able to test more often, fewer problems results in overall better product • Fewer support issues VisualRegressionTesting 9
• 10.
  Conclusion Beyond regression testing,a PDF to PDF comparison tool like Antenna House Regression Testing System, can have a significant impact on improving software as it may play many roles in the development cycle for any system generating visual outputs. VisualRegressionTesting 10
• 11.
  Thank you! For moreinformation: www.antennahouse.com celina@antennahouse.com