Testing…how…what & when
Simon Evans, Experior
See SAP Differently
www.experiorgroup.com
Agenda
1. Assessing your optimal route
1.1 Where and why
2. What to test
2.1 Understanding risk
2.2 Mitigating risk
2.3 The importance of Business Processes
2.4 Project Mobilisation
3. SAP Testing Suite
3.1 Tools
4. Example
4.1 Phase 1 – High-Level Strategy
4.2 Phase 2 – Detailed Scope
www.experiorgroup.com
Experior Overview
Who we are What we do
Enterprise applications testing experts Testing strategy
Thought leaders for SAP Testing best Assessments
practices Test Management
Partners with key integrators to deliver Functional Testing
on-demand Testing Solutions
Performance testing
Tools agnostic
Test Automation
Only SAP UK partner dedicated to
testing Test Data Management
Customer training and enablement
Our value Proposition How we do it
Improve business process design, development Trusted advisor during the planning,
and production Support through accelerated architecture, implementation and support of
and improved testing processes testing
Reduces risk and lowers TCO Flex specific skills to support project and
Ensures reliable delivery, support, and quality programme needs
Accelerates ROI through the best people, Provide leadership and support through best
processes and tools practice, testing methodology, and expert
Increases efficiency in long-term sustainability consulting
Support regulatory compliance
www.experiorgroup.com
Challenges…
Do you find it hard to find time for testing?
Does your testing get squeezed?
Are you ready to efficiently deploy support or
enhancement packs?
TIME
Do you have all of the testing tools but your
not getting the most out of them?
Point solutions or integrated?
QUALITY COST
www.experiorgroup.com
Assessing your optimal route
Defects - where are they most likely to occur?
Bespoke Code 54%
Infrastructure 16%
Roles & Authorisations 10%
Known Problem/Rejected 10%
SAP Code/Conf 7%
Master Data 3%
>50% of defects from bespoke code
www.experiorgroup.com
Knowing what to test
Implement Risk-Based approach to identify ‘what to test’
– Are you currently challenged when identifying testing scope?
– Risk-Based approach will enable focused & controlled testing in the ‘right’ areas
– Opportunity to streamline testing activities with limited time & resource availability
Risk Based Testing
Understand the Analyse risk based Rate business Test what
business and on knowledge of process scenarios matters
the business the business and in terms of risk Test reality
processes the system to the business
Probability of failure
Virtually •1
certain
•3
High
Probabl
•4 •2
e
High
All business Possible
processes Significant
Impossible
•5
Significant
Highly
likely
Impact Intermediate
Intermediate
•Minor
•Moderate
•Major
•Insignificant
•Catastrophic
Low
www.experiorgroup.com
Knowing what to test
Understanding Risk Analysis Probability of failure
Virtually
1
Business Impact certain
3
Rated on 4 key areas – Probable
4 2
– Legal /Financial implications
– Frequency of occurrence Possible
– Number of users Impossible
– Lead time 5
Highly
Sum of all ratings = Business Impact likely
Impact
Technical Risk
Minor
Moderate
Major
Insignificant
Catastrophic
Rating comes from the event with the highest rating
in the business process
– E.g. If scenario includes ‘Submit Invoice’ with a risk Criticality
rating of 1 – the technical risk for this scenario will
be 1 High
Significant
Intermediate
Low
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 1: BE 1 BE 2 BE 3 BE 5 BE 6 END
Scenario 2: BE 1 BE 2 BE 4 BE 5 BE 7 BE 9 END
Scenario 3: BE 1 BE 2 BE 4 BE 5 BE 8 BE 10 END
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 1: BE 1 BE 2 BE 3 BE 5 BE 6 END
Scenario 2: BE 1 BE 2 BE 4 BE 5 BE 7 BE 9 END
Scenario 3: BE 1 BE 2 BE 4 BE 5 BE 8 BE 10 END
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 4: BE 1 BE 2 BE 3 BE 5 BE 8 BE 10 END
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 1: BE 1 BE 2 BE 3 BE 5 BE 6 END
Scenario 2: BE 1 BE 2 BE 4 BE 5 BE 7 BE 9 END
Scenario 3: BE 1 BE 2 BE 4 BE 5 BE 8 BE 10 END
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 5: BE 1 BE 2 BE 3 BE 5 BE 7 BE 9 END
www.experiorgroup.com
Business Processes & Testing
BE 6
BE 3
BE 1 BE 2 BE 5 BE 7 BE 9 END
BE 4
BE 8 BE 10
Scenario 1: BE 1 BE 2
BE2 BE 3 BE 5 BE 6 END
Scenario 2: BE 1 BE 2
BE2 BE 4 BE 5 BE 7 BE 9 END
Scenario 3: BE 1 BE 2
BE2 BE 4 BE 5 BE 8 BE 10 END
www.experiorgroup.com
Accelerate Timelines
September
November
December
February
October
January
August
March
June
June
April
May
May
July
July
Month 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Traditional Test Project Test Planning Test Prep Test Execution Go-Live
Month 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Best Practice Test Project Test Planning Test Prep Test Execution Go-Live
Go-live 3 months
35% Prep
50% time earlier
saving
saved – execution
in test saving time
(repeatable and
test
time costs
assets)
www.experiorgroup.com
Project Mobilisation – Test Planning
Scope Definition & Planning Key Objectives
Utilise established in-house teams Mitigate risk through quality focused
High-level of existing expertise testing
Experienced testing teams with effective assets Business Assurance
Identify hotspots and business critical activity Defined Business requirements
Target scope against business objectives Defined acceptance criteria
Time, Cost & Quality Utilise appropriate resources
Provide leadership and best practice Data Harmonisation
Data Migration
Data Integrity
Test Asset Assessment
Performance Impact
Maximise utilisation of existing test assets Benchmark current system(s)
Review existing assets
Gap & suitability analysis
Testing Toolset Review
Assess existing testing tools and license availability
Gap analysis
Environments & Data
Align environments & data demands to testing
Identify 3rd party system requirements
Streamline testing activity
www.experiorgroup.com
The SAP Quality Platform
Plan Execute Evaluate
SAP Test
SAP Solution SAP Solution Mgr.
SAP Quality Center Acceleration & SAP LoadRunner
Manager Adapter
Optimization (TAO)
End-to-end project Closed loop from Detailed test planning Further time and cost Performance test and
road-mapping and blueprint to test and management savings via drag-and- tune SAP and non-
blueprinting results like status, drop test case SAP applications
incidents, and defects Test case creation
SAP enterprise management Comprehensive
business process Bi-directional Unique SAP offering performance
modeling integration between Visibility into the enabled by SAP core dashboards
Solution Manager overall testing business process
and Quality Center process expertise Deep diagnostics for
for role-based usage Comprehensive SAP Business Suite
and SAP NetWeaver
dashboards
Create consistent and small QA and development environments
VALUE Extension
Select only the data you need
Test Data Migration Server (TDMS)
Scramble sensitive information
Test Center Implementation Service Performance Automation Quickstart
RAPID Return of Investment
SAP Professional Services Functional Implementation Quickstart Test Strategy & What to Test
www.experiorgroup.com
The road towards SAP Testing Best Practice
Solution Manager Solution Manager
Blueprint BPCA SAP Quality Center
1. Drive all project 2. Analyse impact of
By HP
activities from change & understand 3. Define test
single source of ‘what to test’ requirements
information directly from
Blueprint
4. Apply Risk
Based test
Audit Trail
approach
5. Generate
reusable
11. Synchronise 10. Update project
Audit Trail components:
defects back with test results SAP Quality Center
to SolMan a) Test
Manual Testing Acceleration &
6. Compose test Optimization
SAP Quality Center cases out of (TAO)
Test Execution SAP Quality Center components b) Manual
Compose Processes
12. Performance
Test & Tune
before Go-Live
SAP TAO
SAP LoadRunner
By HP
SAP TDMS
9. Create accurate test
data, without storage 7. Compose test
requirements cases out of TAO
components
www.experiorgroup.com
Building a test strategy…
See SAP Differently
www.experiorgroup.com
Upgrade Testing Approach
Upgrades are very different
Everything has changed but nothing should change
Engage all stakeholders early in the process
Take a Risk-based approach to testing
Understand both the technical and business risks
Don’t worry about Standard SAP
Include Security and Authorisations
Use both Manual and Automated testing
Shape your approach for managing change in SAP
Form the foundation of XX test assets for the future
Focus XX testing on what matters
Ensure key XX business coverage
Focus on areas of track record of failure
www.experiorgroup.com
Phase 1
Activity Deliverable
Documentation Review Agreed XX functional areas of focus and workshops
identified
Review Scope High-level breakdown and process decomposition of
functional areas
Test Asset Assessment Gap Analysis & recommendations for re-usability of
current test assets
Test Tools Assessment of current test tool usage and
recommendations for upgrade
Environments Best practice test environment approach
Upgrade Planning High-Level testing plan
www.experiorgroup.com
Phase 2
Activity Deliverable
Documentation Analysis Detailed view on XX Business Processes to accelerate
& enhance output of workshops
Technical Impact Technical review of high-risk & impact areas of the
Assessment upgrade
Workshop Delivery Workshops to determine detailed scope
recommendations
Scope Definition Agreed & signed-off scope with Business and IS
teams
Upgrade Test Approach Detailed project plan, scope and considerations for
environments, data and tools
Delivery Planning Resource driven plan to deliver testing for the
Upgrade
www.experiorgroup.com
Benefits to XX
Reduced workload for XX Business and IS resources
Business workload reduced by up to 80% across upgrade projects
Allowing XX resources to focus on key areas of the upgrade
XX resources to focus on ‘Next Generation’ projects – realising the business case from Upgrades
Reduce upgrade testing times by 50%
Reducing overall upgrade project timelines and cost by 33%
Proven methodology
Defined functional test scope
Utilise Experior knowledge in best practice testing for Upgrades
Focus on critical XX processes
Gap analysis of current test assets
Support XX now and in the future for all enhancements and changes
Maximise current XX test assets
Implement automated regression pack
www.experiorgroup.com
Thank you
Questions?
See SAP Differently
www.experiorgroup.com
