Downloaded 51 times
Uploaded byitSMF UK
Daniel Breston - DevOps metrics that matter
The document discusses the importance of effective DevOps metrics and KPIs that align with business objectives, emphasizing accountability, actionable insights, and continuous improvement. It highlights the necessity of measuring relevant factors rather than easily quantifiable ones and presents a structured approach to KPI design, ensuring they are accessible and comprehensible at all organizational levels. Additionally, it underscores the significance of agile methodologies and frequent assessments to foster better performance and decision-making in IT and business operations.
More Related Content
Similar to Daniel Breston - DevOps metrics that matter
Daniel Breston - DevOps metrics that matter
- 1.
- 2. BritAn Too many yearsin IT Principal Consultant for Virtual Clarity ITSM-DevOps-Lean-AgileITSM DevOps Institute trainer Gamingworks GromIT associate Active BCS & itSMFUK Conference speaker Blogs & webcasts Daniel Breston @DanielBreston
- 3. Just curious ? Is ‘Gamingmetrics’ something that happens where you work? No, our reports accurately reflect facts and actions Yes we “game the numbers since we manage from a distance Not sure! ? Does your business lifecycle match your IT lifecycle from their point of view? Yes, most of the time Yes, some of the time No not really
- 4. Business & Technology Don’tmeasure me on cost or traditional IT metrics, but on the metrics of the business. Jim Fowler, CIO, GE Capital
- 5. Employee view ofa KPI “Tell me how you measure me, and I will tell you how I will behave” Eliyahu M. Goldratt
- 6. Measure, Metric, KPI •Measure: a standard unit used to express the size, amount, or degree of something • Metric: A system or standard of measurement
- 7. KPI: Indicator ofPerformance that is KEY An agreed measure that allows us to: • Assess our performance against a baseline or expectation • Immediately spot issues, problems or waste • Enables a behaviour • Let’s us continuously improve • Is a Metric that Matters • Links to a Value statement Steve Bell” of LeanIT Strategies: “A useful metric that makes the invisible visible”
- 8. Measurement systems failbecause REASONS FOR FAILURE Measure what is easy, not what is important Not created with people involved Information is wrong or misleading Reported too late to be actionable Baseball bat GOOD MEASUREMENTS Minimize quantity Maximize accuracy Minimize effort Minimize ambiguity Maximise behaviours Maximise being SMART
- 9. 9 KPI:Does this make sense? ∏= = n i i E EMSizeAPM 1 xx ∑= += 5 1 x01.0 j jSFBE where Effort kSLOC kiloSourceLinesOfCo de Effort Multiplier: linear effect on effort Scale Factor: exponential effect on effort Constants that can be used to calibrate the formula KPI: so what do you think?
- 10. Better, faster, safervalue as often & early as possible ITSM AgileLean DevOps DevOps
- 11. DevOps: values &principles • Focus on people • Embrace change and experimentation • Focus on people • Embrace change and experimentation CultureCulture • “Continuous Delivery” • “Infrastructure as Code” • “Continuous Delivery” • “Infrastructure as Code” • Focus on producing value for the end user • Small batch sizes • Focus on producing value for the end user • Small batch sizes • Measure everything • Show the improvement • Measure everything • Show the improvement • Open information sharing • Collaboration and communication • Open information sharing • Collaboration and communication AutomationAutomation LeanLean MetricsMetrics SharingSharing
- 12.
- 13. KPI Design An owner Definition ValueGoal Equation Unit of measure Frequency Data location Dependency Most importantly – the required response when checked
- 14. ABCs of KPIs Aligned Providesclear line of sight between strategic goals and individual targets and activities (cascaded targets) Actionable The person using the indicator should be capable of influencing and improving it (accountability aligned with autonomy) Balanced Evaluates performance from multiple perspectives (leading and lagging; customer and business; efficiency, effectiveness, quality, time, cost, etc.) Controllable Small number and owned Consistent Definitions, targets, data collection and analysis methods remain constant from one period to the next (KPIs matrix and definitions) Comparable Measured against a historical baseline, a current plan, and a future target (base, plan, and target) KPI ABCs
- 15. 1. What isthe vision? 3. Where do we want to be? 2. Where are we now? 4. How do we get there? 5. Did we get there? 6. How do we keep the momentum going? Business Vision, mission, goals and objectives Baseline Assessments Measurable targets Service & process improvement Measurements and metrics Audits and reviews Image : © Crown copyright 2007. Reproduced with permission of OGC. Agile ITSM
- 16. Sailing to anisland The journey normally takes 4 days Day-1 Day-2 Day-3 Day-4 Analogy (or the way we really work??)
- 17. The captain checkshis position after 4 days Necessary and time consuming course correction is needed Day-1 Day-2 Day-3 Day-4Day-5&6 (lost) Oops to late!
- 18. The captain checkshis position on a daily basis Daily correction of the position Partly day-5 (lost) Day-1 Day-2 Day-3 Day-4 DevOps: Frequency is KEY
- 19. The captain checkshis position every half hour Continuous and immediate correction with minimal time loss On time Day-1 Day-2 Day-3 Day-4 An important difference between Lagging and Leading KPIs! DevOps: better, faster, safer
- 20.
- 21. DevOps metrics thatmatter Customer Cost of delay Get it back
- 22. KPI tree example Operations Outagevital businessfunctions - % & € (cost impact) Lost opportunities -% & € (cost impact) Release Traced - % Categorized - % (risk based) Decided - % Executed - % Stabilized– T Process, procedures up to date & agreed % Run Availability- % Reliability– T Maintainability– T Resilience– T Process, procedures up to date & agreed % Asset Process, procedures up to date & agreed % Major & Minor Traceable % Planned % Documented % Tested % Profiled % Approved % Emergency Documented % Profiled % Tested % Approved % Infrastructure Traceable % Planned % Documented % Tested % Profiled % Approved % IMAC In time % 1-st time right % Online Vitalbusiness Applicationsmonitored % Failures in execution/performance% Workorder In time % 1-st time right % Batch Jobs under control % Failures in execution % Exceed time window % Wintel Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % &T SAP Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % & T Support 1-st line resolution - % Process, procedures up to date & agreed % Applications Defined - % Compliant - % Scaled - # & % Monitored - % Wintel Defined - % Compliant - % Scaled - # & % Monitored - % SAP Defined - % Compliant - % Scaled - # & % Monitored - % Infrastructure Defined - % Compliant - % Scaled - # & % Monitored - % Infrastructure Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % & T Call take Duration call take % Abandoned call % Overdue Type/Sev % Policies Defined % Agreed % Up to date % Follow up Type/Sev % Back log To assign # Type / sev % Per status in life cycle #
- 23.
- 24. From Lean IT withpermission from Steve Bell From Lean Enterprise with permission from Jez Humble Catch-ball
- 25. Charter of IntentService Management Charter Event Scope Leadership / Coordination (Normal) Schedule Product or Service Compute, Storage, Service Desk, etc. Executive Sponsor VP or C-level Days of Product/Service Specific Conditions What circumstances you're including and excluding? (type of customer, time of year, etc.) I&O Manager If needed - often Director or Manager level Start/End Times Customer Demand How many times is this done per wk, qtr, mo, or yr.? (if helpful to know) Team Leader Skilled person leading the activity Location(s) of service Location of documentation SharePoint location First Step Task on first process block OBP Manager Not always needed Product - Service Date exceptions Always a nice touch; keeps the team from wandering.Last Step Task on last process block Boundaries & Limitations What is the team NOT authorized to change, do, etc.? OP Manager The person arranging logistics (reserving the room, ordering food, sending meeting notices, etc.) Interim Briefing(s) Aid in consensus building and organizational learning. Reporting cycle Daily, weekly, quarterly (add in roles if helpful) Event Drivers Significant Contacts 1 What dependencies in services Providers Name Contact Name Contact Information 2 What dependencies on other partners (significant 1 3 Assumptions 2 4 3 5 4 Goals & Measureable Objectives 5 1 Aim for objectives w/ measurable targets (from X to X). 6 2 7 3 8 4 9 5 10 Planned Deliverables by Output-based Partners On-Call Support 1 Application mgt Function Name Contact Information 2 Compute storage 1 SMEs that may not be needed full time 3 Service Desk 2 4 3 5 4 Planned Deliverables by Operating Partners On-Call Support 1 Application mgt Function Name Contact Information 2 compute storage 1 SMEs that may not be needed full time 3 Service Desk 2 4 3 5 4 Potential Obstacles Approvals 1 Application mgt Executive Sponsor Output-based Partner Operating Partner 2 Compute storage 3 Service Desk 4 Signature: Signature: Signature: 5 Date: Date: Date: Version Control 1 Version no. Reason for change 2 Creation Date 3 Revised Date 4 Author
- 26.
- 27. DO Review KPIs regularlyagainst a target to identify trends and define corrective actions Measure leading KPIs on the shortest interval they an be controlled Cascade targets and align KPIs between all levels of the organization Bring information to life by visualizing KPIs using graphs, charts, and white-boards DON’T Measure too many things or measure just because you can Use complex calculations, percentages, or ratios when a simple number is available Try to roll-up or aggregate every KPI between each level of the organization Use KPIs to punish people – instead, use them as a tool to improve performance Tips
- 28. ITSMF UK Premier Gate,Easthampstead Road, Bracknell, RG12 1JS, United Kingdom Tel: +44 (0) 118 918 6500 | Web: www.itsmf.co.uk Conference KPI: fill in your evaluation please!!