Download as PDF, PPTX
![self.introduce =>
{
name: "Viktar Karanevich",
title: ["SSTAE", "SSDET", "SQAAE"],
languages: ["Java", "Ruby"],
experience: "iOS Automation 7+ years"
}](https://image.slidesharecdn.com/iosiiautomation-170301181324/85/iOS-Parallel-Automation-run-faster-than-fast-Viktar-Karanevich-SeleniumCamp-2017-3-320.jpg)
![self.introduce =>
{
name: "Viktar Karanevich",
title: ["SSTAE", "SSDET", "SQAAE"],
languages: ["Java", "Ruby"],
experience: "iOS Automation 7+ years"
}](https://image.slidesharecdn.com/iosiiautomation-170301181324/75/iOS-Parallel-Automation-run-faster-than-fast-Viktar-Karanevich-SeleniumCamp-2017-3-2048.jpg)
Uploaded byBadoo
iOS Parallel Automation: run faster than fast — Viktar Karanevich — SeleniumCamp 2017
The document details the evolution and current practices of iOS automation frameworks, particularly focusing on Badoo's approach to optimizing test execution time through parallel testing. Key topics include the history of various frameworks, challenges faced with flaky tests, and the transition to using multiple desktops and simulators. It concludes with a summary of current methodologies and pros and cons associated with the implemented strategies.
More Related Content
Similar to iOS Parallel Automation: run faster than fast — Viktar Karanevich — SeleniumCamp 2017
![[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium](https://cdn.slidesharecdn.com/ss_thumbnails/srijan-webinar-161103125140-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![UiPath Automation Suite Installation (Hands-On) [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/automationsuitecommunitysession2-251015095633-a6d862f1-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
iOS Parallel Automation: run faster than fast — Viktar Karanevich — SeleniumCamp 2017
- 1. iOS Parallel Automation: Runfaster than fast! Viktar Karanevich
- 2. Content: I. History ofiOS Automation Frameworks (5 min) II. Path to Current Approach at Badoo (10 min) III. Summary (5 min) IV. Movie: Test Run (3 min)
- 3. self.introduce => { name: "ViktarKaranevich", title: ["SSTAE", "SSDET", "SQAAE"], languages: ["Java", "Ruby"], experience: "iOS Automation 7+ years" }
- 4. The original, largestand leading dating network: 330M+ registered users 800+ automated tests on iOS platform 50M+ downloads on iOS
- 5.
- 6.
- 7. Jun 2010 UI Automation Sep2014 UI Automation xcrun simctl Apple "xcrun simctl" is introduced in xCode 6 It is command line utility to control the Simulator (create, delete, erase, install, uninstall, ...).
- 8. Jun 2010 UI Automation Sep2015 UI Automation xcrun simctl UI Testing Apple UI Testing is introduced as replacement for UIAutomation in xCode 7 Sep 2014 UI Automation xcrun simctl
- 9. Jun 2010 UI Automation Sep2016 UI Testing xcrun simctl Apple Sep 2015 UI Automation xcrun simctl UI Testing Sep 2014 UI Automation xcrun simctl UI Automation removed in xCode 8
- 11. Jun 2010 UI Automation Sep2014 xcrun simctl Sep 2016 UI Testing Sep 2016 UI Testing Apple Calabash Nov 2009 UI Spec
- 12. Calabash Nov 2009 UI Spec Feb2011 Frank
- 13. Calabash Nov 2009 UI Spec Feb2011 Frank Jan 2012 Calabash Step Definitions Ruby Client Library Features Developer host Calabash server Simulator Your app UIQuery Touch Actions
- 14. Calabash Apr 2013 Calabash + UIAutomation Link:https://github.com/calabash/run_loop Step Definitions Ruby Client Library Features Developer host Calabash server Simulator Your app UIQuery Run loop: UIAutomation
- 15. Calabash Sep 2016 Calabash + DeviceAgent Link:https://github.com/calabash/run_loop/ tree/develop/lib/run_loop/device_agent Developer host Simulator Step Definitions Ruby Client Library XCTest Runner Features Calabash server Your app UIQuery Device Agent
- 16. Calabash Nov 2009 UI Spec Feb2011 Frank Jan 2012 Calabash iOSAutomation Frameworks Apr 2013 Calabash + UIAutomation Sep 2016 Calabash + DeviceAgent Jun 2010 UI Automation Sep 2016 UI Testing xcrun simctl Sep 2015 UI Automation xcrun simctl UI Testing Sep 2014 UI Automation xcrun simctl Apple 3 years 1 year
- 17. Calabash Nov 2009 UI Spec Feb2011 Frank Jan 2012 Calabash iOSAutomation Frameworks Apr 2013 Calabash + UIAutomation Sep 2016 Calabash + DeviceAgent Jun 2010 UI Automation Sep 2016 UI Testing xcrun simctl Sep 2015 UI Automation xcrun simctl UI Testing Sep 2014 UI Automation xcrun simctl Apple 3 years 1 year
- 18. II. iOS Automationat Badoo Approach ~80 flaky tests 80 data fail tests
- 19. II. Before ParallelRun Test Suite Run Nightly Cucumber Report Nightmare! 15h total time 1 simulator 80 data ~500 tests
- 20. II. Before ParallelRun Test Suite Run Nightly Cucumber Report Nightmare! 80 data fail tests 1simulator Key Issue: How to speed up Test Run? 15h total time ~500 tests 1 simulator
- 21.
- 22. Let's Parallel! Blocker Issue: -Calabash does not support Parallel run on 1 desktop
- 23.
- 24. https://techblog.badoo.com Tim's talk atBadoo: "Parallel Calabash Testing on iOS"
- 25. II. Test Result:multi-desktops ~4 hours regression suite run time 5 Desktops
- 26. II. Test Result:multi-desktops Mac PRO (24 cores) ~2 hours regression suite run time 10 Desktops
- 28. X Desktops: hostlimit reached
- 29.
- 30. II. Clustering Hosts -Parallel run on multi (virtual) hosts - Mac Pro and hosts: test suite - ~1h 30 min
- 31. II. Clustering Hosts Issues: -multi-desktops start up is Flaky - simulators can get stuck
- 32. II. Clustering Hosts Isit possible to automate multi-simulators on one desktop?
- 33. III. Facebook: WebDriver Agent
- 34. Developer host Simulator SeleniumDriver WDA Runner Your app FB UIA WDA Key Change: Reimplemented UIAutomation dependency Link: https://github.com/facebook/WebDriverAgent
- 35. Profit: - 7 simulatorson single desktop - More stable! more efficient!
- 36. III. Full Migrationto FB UIA WDA Issues: - all tests have calabash code
- 37. III. Full Migrationto FB UIA WDA Issues: - all tests have calabash code - custom app backdoors
- 38. III. Full Migrationto FB UIA WDA Issues: - all tests have calabash code - custom app backdoors - workarounds and other
- 39. III. Full Migrationto FB UIA WDA Full Migration Costs: ~x hours Is it possible to do Fast Migration?
- 40. III. Calabash +Facebook Fast migration: Facebash! Step Definitions Ruby Client Library Features Developer host Calabash server Simulator Your app UIQuery Run loop: UIAutomation
- 41. III. Calabash +Facebook Fast migration: Facebash! Step Definitions Ruby Client Library Features Developer host Calabash server Simulator Your app UIQuery Run loop: UIAutomation Developer host Simulator Step Definitions Ruby Client Library WDA Runner Features Calabash server Your app UIQuery FB UIA WDA
- 42. IV. iOS 10Update: from UIAutomation WDAto XCTest WDA Link: https://github.com/facebook/WebDriverAgent Developer host Simulator Selenium Driver Your app FB UIA WDA WDA Runner
- 43. IV. iOS 10Update: from UIAutomation WDAto XCTest WDA Link: https://github.com/facebook/WebDriverAgent Developer host Simulator Selenium Driver Your app FB UIA WDA Developer host Simulator Your app FB XCTest WDA WDA Runner Selenium Driver FB simctl
- 44. IV. iOS Hybrid2.0 iOS 10 update Developer host Simulator Step Definitions Ruby Client Library WDA Runner Features Calabash server Your app UIQuery FB UIA WDA
- 45. IV. iOS Hybrid2.0 iOS 10 update Developer host Simulator Step Definitions Ruby Client Library WDA Runner Features Calabash server Your app UIQuery FB UIA WDA Developer host Simulator Step Definitions Ruby Client Library FB simctl Features Calabash server Your app UIQuery FB XCTest WDA
- 46. V. Summary: multi-desktopsvs multi-simulators Test Suite: 500 tests Nightly Cucumber Report Happy morning! 80 data Mac PRO Test Suite Run Time 0 min 225 min 450 min 675 min 900 min 1 sim 10 sims 16 sims 75 min 120 min 900 min Multi Desktops Multi Simulators
- 47. Developer host Simulator StepDefinitions Ruby Client Library FB simctl Features Calabash server Your app UIQuery FB XCTest WDA "Face"book + Cala"bash" = Facebash V. Summary: current approach
- 48. - easy toset up Parallel Run on 1 Desktop - keep using of backdoors - full access to internal AUT state - cross iOS app automation - iOS simulator control with FBSimCtl - FB WDA supports Selenium protocol - FB WDA independent of Calabash/Appium Facebook Calabash Calabash Facebook Facebook Facebook Facebook V. Summary: PROs
- 49. - support frameworkparts: calabash server, FB WDA, fbsimctl, ... - "poor" documentation - using of mixed locator strategies V. Summary: PROs and CONs
- 51. - Parallel Runin Headless Mode - Simulator Start/Shutdown/Reset optimization - Cold/Hot start iOS app with relogin - record video per the each test in Parallel Run V. TODO
- 52.