Uploaded byMek Srunyu Stittri
Visual Automation Framework via Screenshot Comparison
This document discusses visual testing and the challenges of automating it. It introduces Applitools, a framework that handles challenges like anti-aliasing, pixel shifts, and dynamic content when comparing screenshots. Applitools uses flexible verification settings and image processing to identify visual differences while ignoring minor layout changes. The document provides examples of integrating Applitools into test frameworks and executing cross-browser tests by leveraging baseline data. It also outlines next steps like using screen regions and different resolutions.
More Related Content
Similar to Visual Automation Framework via Screenshot Comparison
Visual Automation Framework via Screenshot Comparison
- 1. Visual Automation Framework via ScreenshotComparison Mek Srunyu Stittri Apr 15 2015 Powered by Applitools
- 2. Visual Automation Testing Whatis Visual Test? “A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users”
- 3. This Credit : imagefrom Adam Carmi, co-founder Applitools A Visual Bug
- 4.
- 5. Data Layer Backend Front end Topof the stack checkpointing. Requires the least amount of investment but huge returns. Visually Perfect Why Automate Visual Testing
- 6. Credit : imagefrom Adam Carmi, co-founder Applitools Why Automate Visual Testing
- 7. THE TEST MATRIXIS HUGE.. ■ Web browsers ■ Operating systems ■ Screen resolutions ■ Responsive designs ■ L10n Why Automate Visual Testing
- 8. Many are alreadydoing it (and sharing)… Credit : image from Adam Carmi, co-founder Applitools Why Automate Visual Testing
- 9. Well, what arewe waiting for..
- 10. Credit : imagefrom Adam Carmi, co-founder Applitools Challenges
- 11. If it’s soimportant. How come it’s not widely implemented ? Its not that simple.. Credit : image from Adam Carmi, co-founder Applitools Challenges
- 12. Credit : imagefrom Adam Carmi, co-founder Applitools Anti Aliasing
- 13. Credit : imagefrom Adam Carmi, co-founder Applitools Anti Aliasing
- 14. Credit : imagefrom Adam Carmi, co-founder Applitools Brightness
- 15. Credit : imagefrom Adam Carmi, co-founder Applitools Brightness
- 16. Credit : imagefrom Adam Carmi, co-founder Applitools Sub pixel shifts
- 17. Credit : imagefrom Adam Carmi, co-founder Applitools Sub pixel shifts
- 18. ■ Dynamic content ■1 pixel offsets in element positioning ■ Moving elements ■ Images of different size QE Credit : image from Adam Carmi, co-founder Applitools And more...
- 19. Introducing - APPLITOOLS,A specialized image processing stack designed to compare computer generated UI images Handles.. 1. Anti-aliasing 2. Partial and full pixel offsets 3. Images of different size 4. Dynamic content 5. Moving elements 6. Layout matching How did we do this
- 20. ● Exact -Only windows that are exact, pixel based, replicas of each other will be considered a success. ● Strict - Taking into consideration both layout and content changes while using Applitools Eyes smart image analysis tools to ignore minor changes that might fail a test when using pixel based comparison. ● Content - Only identify content differences while ignoring layout based ● Layout - Applitools Eyes will only identify layout differences while ignoring content differences. Flexible Verification settings built-in
- 21. STRICT - Sameimage : Passed
- 22. CONTENT - Invertedcolor : Passed
- 23. STRICT - Invertedcolor : Failed
- 24.
- 25. Testdata (BaseLine) CreationFlow 1. Identify a Panel and create a new Applitools UI testcase to navigate to the Panel under test, do UI interactions and add Visual Checkpoints 2. Run the test once to generate the BaseLine 3. Go to Applitools to approve the BaseLine and add customizations as necessary Test Execution Flow 1. Run the test 2. Analyze the test result on differences reported Workflow
- 26.
- 27. Jersey dependency conflicts ■http://support.applitools.com/customer/portal/questions/9866576-nosuchmethod-error Integration into existing Framework with our own Implementation of WebDriver Applitools API requires passing in the actual Selenium WebDriver instance ■ http://support.applitools.com/customer/portal/questions/9866542-using-own-implementation-of-webdriver ■ A new testbase ApplitoolsTestBase.java is also created for all new validation tests. More in wiki More Groovy dependency conflicts with the latest RELEASE version ■ http://support.applitools.com/customer/portal/questions/11629509-dependency-conflict-in-the-latest-release-version-getting- error-when-initgremlingroovyscriptengine- Challenges
- 28. public class ApplitoolsTestBaseextends TestBase { protected Eyes eyes; /** * Instantiates new {@link Eyes} object and sets Applitools Key and Testname from TestNG injection */ @BeforeMethod public void setupTestMethod(Method method) { super.setupTestMethod(method); this.eyes = new Eyes(); eyes.setApiKey(APPLITOOLS_API_KEY); String testPreFix = method.getDeclaringClass().getCanonicalName(); // shorten down com.mekdev.qe to c.m.q testPreFix = testPreFix.replace("com.mekdev.qe", "c.m.q"); // Instantiate new driver and set application name and testname WebDriver driver = eyes.open(getWebDriver(), "Visual Test", testPreFix + "::"+ method.getName()); setWebDriver(driver); } } New Applitools specific TestBase
- 29. Test Example @Test public voidtestApplitools() throws Exception { eyes.setMatchLevel(MatchLevel.CONTENT); // Start the UI portion of the test Navigation navigator = new Navigation.Builder() .username(Constants.ADMIN).password(Constants.PASSWORD) .build(); navigator.run(); LoginPage loginPage = WebDriver.update(navigator, LoginPage.class); // Click Run loginPage.clickSignup(); // Check screen #1 loginPage.fillInformation("John doe","password"); eyes.checkWindow("Signup Page"); // Check screen #2 loginPage.clickCancel(); eyes.checkWindow("Back to Login selection"); // finish the test eyes.close(); loginPage.close(); }
- 30. Cross Browser /OS tests using the same BaseLine Data ■ Use - eyes.setBaselineName("baselineName"); Evaluate feasibility and stability of Screen Regioning Capture vs whole screen ■ Via - eyes.checkRegion(selector, matchTimeout, "windowName"); Expand to Different Screen Resolutions ■ Requires some tweaking to the framework to specify viewport resolution at the test level - driver = eyes.open(driver, appName, testName, viewportSize); Next Phases
- 31. Q & A SpecialThanks Adam Carmi - VP Engineering, Co-Founder at Applitools Adam Carmi: Automated Visual Testing that Doesn't Suck! http://www.meetup.com/SeleniumSanJose/events/209029032/ - November 17, 2014 http://www.slideshare.net/adamcarmi/visual-test-automation-that-does-not-suck - Adam’s original slide deck from Selenium Meetup Moshe Milman - VP Operations, Co-Founder at Applitools 24x7 Continuous Support, late night Join.me sessions from Tel Aviv