More Related Content
![[IBM Pulse 2014] #1579 DevOps Technical Strategy and Roadmap](https://cdn.slidesharecdn.com/ss_thumbnails/pulse1579devopstechstrategyroadmap-140303191835-phpapp02-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to Development and QA dilemmas in DevOps
More from Matteo Emili
Development and QA dilemmas in DevOps
- 1. Development and QAdilemmas in DevOps Matteo Emili Microsoft MVP
- 2. Who am I? Visual Studio and DevelopmentTechnologies MVP (ALM) Professional Scrum Master I Systems EngineeringAdvisor @ One Identity Taunton Developers Meetup | London Microsoft DevOps Meetup GetLatestVersion.it | DomusDotNet matteo.emili@live.com || @MattVSTS || mattvsts.blogspot.com
- 3.
- 4. Are we allready for it?
- 5. The typical (legacy) approach Developeverything in a separate environment Countless hours of tests Big-bang migration then new feature for all the users Big release date, people freaking out …lots of fingers crossed!
- 6. Sometimes things work the otherway round… The first thing to do should be creating a MinimumViable Product, including the infrastructure The Pipeline must work from the very first moment Dev andTest environments should be disposable, infrastructure must be idempotent Testing can happen in Production
- 7. It is ashocking change for some Excerpts from interviews with developers, testers, managers… “I was frightened by not feeling a safety net” “Does it actually work?” “The feeling is not good at first” “We don’t want to be accountable for all of this” “It can’t work!” “How do we know what to build?” “It is hard to get the grip with these tools”
- 8. Prerequisite The organisation mustbuy into DevOps!
- 9. “Should I goall-in with ‘the Cloud’ then?”
- 10. Of course not… Itis all about culture and practices, there is little technology you really need to know about This change puts the customer at the centre of the process, where everybody across all the company work towards fulfilling the customers’ requests and demands These practices can easily be reused with a totally on-premise product, there is no need of targeting a technology you do not need
- 11. Use your infrastructure! Leverageon Infrastructure As Code (Azure RM,Amazon CloudFormations, PowerShell DSC) to remove friction when factoring the infrastructure in Spend time on defining what you need to have, and the provider (Azure, AWS, local deployment service) will deploy that for you Idempotent and resilient – if there is something wrong, redeploy
- 12. Package your application for distribution Leverage on packaging and deployment technologies to create a flexible deployment and provisioning system These technologies are usually proven and tested MSDeploy Web Archive format NuGet They take centre stage with a Release Pipeline, together with Infrastructure as Code
- 13. Demo Build once, deployeverywhere
- 14. “How should Ihandle changes?”
- 15. Shhh! Don’t make noise! Each big change must be split in smaller changes They are often invisible to the user Silent deployments can happen across multiple sprints If something breaks, rollback Build incrementally, migrate data before full rollout Start using your MVP straight away!
- 16. An example… Youmust bring a database schema change in production (a new column) Cloud or mission critical web service Transparent for your users, regardless of public-facing or internal change No downtime for migration – it must work!
- 17. Hang on. How aboutbreaking changes? It is essential to factor in some sort of mitigative solution that keeps the service online while the change is deployed and tested A real world example?TFS 2015 introduced a massive change that resulted in large databases being unable to be upgraded in less than 48 hours Microsoft released a tool that performed an online transformation of the affected tables so that the upgrade proper could happen in a reasonable time Pro: feasible upgrade Con: resource cost More on that at https://mattvsts.blogspot.com/2017/10/successfully- handle-disruptive-changes.html
- 18. Demo Handle a databaseschema change in production
- 19.
- 20. A/BTesting, MVT Testing…is it actually about testing? Little changes for QA in DevOps, except for automation and the actual target of the system under test What is really new are practices ofTesting in Production - extremely common especially withWeb Applications You deploy version B of your A application (or a permutation of possibilities), direct a percentage of the traffic towards it and compare the adoption results Implementing them is extremely easy, getting value out of them is hard!
- 21. Feature Flags to therescue Championed by Martin Fowler, Feature Flags are around since at least 2009 Pretty much all the major services nowadays use them Flickr,Amazon, Microsoft, Facebook, Google, Netflix … They are used to enable or disable certain areas of your application to your users Also extremely valuable as a kill switch in case of issues
- 22. “…wait. Does it mean Microservices andall the new stuff around?” No, it doesn’t! You can use Feature Flags with any architecture It is a practice, not necessarily a product It ties in with the previous scenario and makes testing easier
- 23. How? Practically speaking,it is about a Configuration Database coupled with Identifiers The Configuration Database can be a csv file, XML file, key-value model, a database (!) Identifiers are areas you will easily split your applications into They often go well together with Feature/Epics or even User Stories Countless OSS libraries
- 24. It doesn’t come forfree It is expensive (in terms of time, and sometimes refactoring) to build it up from scratch in an existing application Not just enable/disable – logging, kill switch, selective adoption, rollouts are all things to implement that take time OSS libraries help, as there is a choice for each platform, but there is still an implementation cost to keep in mind
- 25. Demo Feature Flags withLaunchDarkly
- 26. “How can Iget better ROI from my R&D investment?”
- 27. A bit ofLean… "There is no such thing as a free lunch“ – technology alone is not going to deliver a successful product My take is to adopt the Build-Measure-Learn loop Start with a MinimumViable Product and measure its usage with appropriate metrics Build on top of it, discard what doesn’t work
- 28. Getting visibility on what is actually happening Is my application performing as expected? What about availability? Are my users having a good experience? Above all, you need better, faster and more direct feedback
- 29. Learn from your usersand improve your product In the Build-Measure-Learn loop the Measure stage is the most important one Metrics will tell if you are going in the right direction, so it is critical to ‘get it right’ Extend your product to include what really matters Get insights on your users before they react Proactively understand potential problems You can analyse your data and understand if your application is behaving as expected
- 30. It is different fromlogging We are not talking about plain application logs What user actions are meaningful to your business goals? Is the telemetry data answering this question? Is there any bottlenecks which is penalising the business? You need to have visibility on user events Actionable insights are mapped into behaviours (what content is used, not how) E.g.: when a user enters in a certain page/area, log on/off, Favourites, social stats, errors…
- 31. You won’t do thatfrom scratch It is too expensive to build something from scratch You need to use an analytics provider The development experience must be cross-platform, unobtrusive at first and then customisable with an SDK Easily add custom telemetry wherever needed
- 32. “Will I geta meaningful insight from data alone?” Telemetry is not just reactive, it can also be proactive You will need to report on top of your data, with your metrics The alternative Easily add custom telemetry wherever needed
- 33. Demo Learn from yourusers
- 34. So? These scenariosare very common when approaching DevOps Remember: DevOps is not the silver bullet, teams must buy-in into the process Collaboration is key to success Use data to take decisions Don’t be afraid!
- 35.