0% found this document useful (0 votes)
216 views349 pagesOWASP Security Testing
Uploaded by
Amarnath MalliahyagariCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
216 views349 pagesOWASP Security Testing
Uploaded by
Amarnath MalliahyagariCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 349
OWASP TESTING GUIDE
2008 V3.0
(© 2002-2008 OWASP Foundation
This document is licensed under the Creative Commons Attribution-ShoreAlike 3.0 license. You must attribute your version to
the OWASP Testing or the OWASP Foundation. @
Table of Contents
Foreword
Why OWASP?.
Tailoring and Prioritizing
The Role of Automated Tools
Call to Action
1. Frontispiece
Welcome to the OWASP Testing Guide 3.0
‘About The Open Web Application Security Project.
2. Introduction
Principles of Testing
Testing Techniques Explained.
Security Requirements Test Derivation
3. The OWASP Testing Framework
Overview
Phase 1: Before Development Begins.
Phase 2: During Definition and Design,
Phase 3: During Development
Phase 4: During Deployment
Phase 5: Maintenance and Operations.
4 Web Application Penetration Testing.
4.1 Introduction and objectives
4.2 Information Gathering.
4.2.1 Testing: Spiders, robots, and Crawiers (OWASP-IG-001),
4.2.2 Search engine discovery/Reconnaissance (OWASP-IG-002)
4.23 Identify application entry points (OWASP-IG-003)
4.2.4 Testing for Web Application Fingerprint (OWASP-IG-004)
12
14
16
19
25
40
40
a1
41
42
43
44
46
46
51
52
54
56
59 4.25 Application Discovery (OWASP-IG-005)
4.2.6 Analysis of Error Codes (OWASP-1G-006).
4.3 Configuration Management Testing
4.3.1 SSL/TLS Testing (OWASP-CM-001)
4.3.2 DB Listener Testing (OWASP-CM-002)
4.3.3 Infrastructure configuration management testing (OWASP-CM-003}
4.3.4 Application configuration management testing (OWASP-CM-004).
4.3.5 Testing for File extensions handling (OWASP-CM-005)
4.3.6 Old, backup and unreferenced files (OWASP-CM-006)
4.3.7 Infrastructure and Application Admin Interfaces (OWASP-CM-007)
4.3.8 Testing for HTTP Methods and XST (OWASP-CM-008)
4.4 Authentication Testing,
4.4.1 Credentials transport over an encrypted channel (OWASP-AT-001),
4.4.2 Testing for user enumeration (OWASP-AT-002)
4.4.3 Default or guessable (dictionary] user account (OWASP-AT-003)
4.4.4 Testing For Brute Force (OWASP-AT-004),
4.4.5 Testing for Bypassing authentication schema (OWASP-AT-005)
4.4.6 Testing for Vulnerable remember password and pwid reset (OWASP-AT-006)
4.4.7 Testing for Logout and Browser Cache Management (OWASP-AT-007)
4.4.8 Testing for Captcha (OWASP-AT-008)
4.4.9 Testing for Multiple factors Authentication (OWASP-AT-009)
4.4.10 Testing for Race Conditions (OWASP-AT-010)
45 Session Management Testing
4.5.1 Testing for Session Management Schema (OWASP-SM-001),
4.5.2 Testing for Cookies attributes (OWASP-SIM-002).
4.5.3 Testing for Session Fixation (OWASP-SM_003)
4.5.4 Testing for Exposed Session Variables (OWASP-SM-004).
OWASP Testing Guide v3.0
65
n
75
76
82
86
91
95
97
102
108
109
110
a3
uy
120
126
131
133
138
140
148
146
147
156
159
161 @
4.5.5 Testing for CSRF (OWASP-SM-005)
4.6 Authorization testing
4.6.1 Testing for path traversal (OWASP-AZ-001}
4.6.2 Testing for bypassing authorization schema (OWASP-AZ-002)
4.6.3 Testing for Privilege Escalation (OWASP-AZ-003)
4.7 Business logic testing (OWASP-Bt-001)
4.8 Data Validation Testing
4.8.1 Testing for Reflected Cross Site Scripting (OWASP-DV-00}).
4.8.2 Testing for Stored Cross Site Scripting (OWASP-DV-002)
4.8.3 Testing for 00M based Cross Site Scripting (OWASP-DV-003)
4.8.4 Testing for Cross Site Flashing (OWASP-DV-004).
4.8.5 SQL Injection (OWASP-DV-005)
4.8.5.1 Oracle Testing
4.8.5.2 MySQL Testing.
4.8.5.3 SQL Server Testing
4.8.5.4 MS Access Testing
4.8.5.5 Testing PostgreSQL
4.8.6 LDAP Injection (OWASP-DV-006)
4.8.7 ORM Injection (OWASP-DV-007),
4.8.8 XML Injection (OWASP-DV-008}.
4.8.9 SSI Injection (OWASP-DV-009)
4.8.10 XPath Injection (OWASP-DV-010)
4.8.11 IMAP/SMTP Injection (OWASP-DV-011)
4.8.12 Code Injection (OWASP.DV.012)
4.8.13 05 Commanding (OWASP-DV-013)
4.8.14 Buffer overflow Testing (OWASP-DV-014)
4.8.14.1 Heap overflow
164
170
170
174
176
178
184
187
191
197
199
208
219
225
233
236
241
243
245
251
254
255
260
261
264 4.8.14.2 Stack overflow.
4.8.14.3 Format string,
4.8.15 Incubated vulnerability testing (OWASP-DV-015}.
4.8.15 Testing for HTTP Splitting/Smuggling (OWASP.DV-016).
4.9 Denial of Service Testing.
4.9.1 Testing for SQL Wildcard Attacks (OWASP-0S-001)
4.9.2 Locking Customer Accounts (OWASP-DS-002).
4.9.3 Buffer Overtlows (OWASP-0S-003)
4.9.4 User Specified Object Allocation (OWASP-0S-004)
4.9.5 User Input as a Loop Counter (OWASP-0S-005)
4.9.6 Writing User Provided Data to Disk (OWASP-DS-006)
4.9.7 Fallure to Release Resources (OWASP-S-007).
4.9.8 Storing too Much Data in Session (OWASP-DS-008)
4.10 Web Services Testing
4.10.1 WS Information Gathering (OWASP-WS-001)
4.10.2 Testing WSDL. (OWASP-WS-002)
4.10.3 XML Structural Testing (OWASP-WS-003)
4.10.4 XML Content-level Testing (OWASP-WS-004).
4.10.5 HTTP GET parameters/REST Testing (OWASP-WS-005)
4.10.6 Naughty SOAP attachments (OWASP-WS-006)
4.10.7 Replay Testing (OWASP-WS-007)
4.11 AIAX Testing,
4.11.1 AJAX Vulnerabilities (OWASP-AL-001),
4.11.2 Testing For AJAX (OWASP-AL-002)
5. Writing Reports: value the real risk
5.1 How to value the real risk
5.2 How to write the report of the testing.
OWASP Testing Guide v3.0
268
an
275
278
281
282
288
286
287
288
289
290
291
292
293
299
302
307
309
310
312
ais
316
319
325
325
332 @
Appendix A: Testing Tools,
Appendix B: Suggested Reading.
Appendix C: Fuzz Vectors.
Appendix D: Encoded Injection
337
340
3a1
346 (OWASP Testing Guide v3.0
‘The problem of insecure software is perhaps the most important technical challenge of our time. Security is now the key
limiting factor on what we are able to create with information technology. At The Open Web Application Security Project
(OWASP), we're trying to make the world a place where insecure software is the anomaly, not the norm, and the OWASP
Testing Guide is an important piece of the puzzle,
It goes without saying that you can't build a secure application without performing security testing on it. Yet many software
development organizations do not include security testing as part of thelr standard software development process. til,
security testing, by itself, isn'ta particularly good measure of how secure an application is, because there are an infinite
‘number of ways that an attacker might be able to make an application break, and it simply isn’t possible to test them all
However, security testing has the unique power to absolutely convince naysayers that there is a problem. So security
testing has proven itself as a key ingredient in any organization that needs to trust the software it produces or uses.
‘Taken together, OWASP's guides are a great start towards building and maintaining secure applications. The Development
Guide will show your project how to architect and build a secure application, the Code Review Guide will tell you how to
verify the security of your application's source code, and this Testing Guide will show you how to verify the security of your
running application. | highly recommend using these guides as part of your application security initiatives.
WHY OWASP?
Creating a guide like this is a massive undertaking, representing the expertise of hundreds of people around the world.
‘There are many different ways to test for security flaws and this guide captures the consensus of the leading experts on
how to perform this testing quickly, accurately, and efficiently.
It's impossible to underestimate the importance of having this guide available in a completely free and open way. Security
should not be a black art that only a few can practice. Much of the available security guidance is only detailed enough to get
people worried about a problem, without providing enough information to find, diagnose, and solve security problems. The
project to build this guide keeps this expertise in the hands of the people who need it.
‘This guide must make its way into the hands of developers and software testers. There are not nearly enough application
security experts in the world to make any significant dent in the overall problem. The intial responsibility for application
security must fall on the shoulders of the developers. It shouldn't be a surprise that developers aren't producing secure
code if they're not testing for it.
Keeping this information up to date isa critical aspect of this guide project. By adopting the wiki approach, the OWASP
‘community can evolve and expand the information in this guide to keep pace with the fast moving application security
threat landscape.
TAILORING AND PRIORITIZING
You should adopt this guide in your organization. You may need to tailor the information to match your organization's
technologies, processes, and organizational structure. If you have standard security technologies, you should tailor your
testing to ensure they are being used properly. There are several different roles that may use this guide,
"Developers should use this guide to ensure that they are producing secure code. These tests should be a part of
normal code and unit testing procedures. @
"Software testers should use this guide to expand the set of test cases they apply to applications. Catching these
vulnerabilities early saves considerable time and effort later.
"Security specialists should use this guide in combination with other techniques as one way to verify that no
security holes have been missed in an application.
‘The most important thing to remember when performing security testing is to continuously reprioritize. There are an
Infinite number of possible ways that an application could fall, and you always have limited testing time and resources. Be
sure you spend it wisely. Try to focus on the security holes that are the most likely to be discovered and exploited by an
attacker, and that will lead to the most serious compromises.
This guide is best viewed as a set of techniques that you can use to find different types of security holes. But not all the
techniques are equally important. Try to avoid using the guide as a checklist.
THE ROLE OF AUTOMATED TOOLS
‘There are a number of companies selling automated security analysis and testing tools. Remember the limitations of these
tools so that you can use them for what they/re good at. As Michael Howard put it at the 2006 OWASP AppSec Conference
inSeattle, "Tools do not make software secure! They help scale the process and help enforce policy.”
Most importantly, these tools are generic - meaning that they are not designed for your custom code, but for applications in
general. That means that while they can find some generic problems, they do not have enough knowledge of your
application to allow them to detect most flaws. In my experience, the most serious security issues are the ones that are not
generic, but deeply intertwined in your business logic and custom application design.
‘These tools can also be seductive, since they do find lots of potential issues. While running the tools doesn’t take much time,
each one of the potential problems takes time to investigate and verify. If the goal s to find and eliminate the most serious
flaws as quickly as possible, consider whether your time is best spent with automated tools or with the techniques
described in this guide.
Still these tools are certainly part of a well-balanced application security program. Used wisely, they can support your
overall processes to produce more secure code.
CALL TO ACTION
If you're building software, | strongly encourage you to get familiar with the security testing guidance in this document. f
you find errors, please add a note to the discussion page or make the change yourself. You'll be helping thousands of others
who use this guide.
Please consider joining us as an individual or corporate member so that we can continue to produce materials ike this
testing guide and all the other great projects at OWASP.
‘Thank you to all the past and future contributors to this guide, your work will help to make applications worldwide more
Jeff Wiliams, OWASP Chair, December 15, 2006 (OWASP Testing Guide v3.0
WELCOME TO THE OWASP TESTING GUIDE 3.0
“Open and collaborative knowledge: that's the OWASP way"
Matteo Meuec
(OWASP thanks the many authors, reviewers, and editors for their hard work in bringing this guide to where itis today. If
you have any comments or suggestions on the Testing Guide, please e-mail the Testing Guide maillist
te
ists.owasp.org/mailman)
Or drop an e-mail to the project leader: Matteo Meucci
VERSION 3
‘The OWASP Testing Guide Version 3 improves version 2 and creates new sections and controls. This new version has added:
‘+ Configuration Management and Authorization Testing sections and Encoded Injection Appendix;
‘+ 36 new articles (2 taken from the OWASP BSP);
Version 3 improved 9 articles, for a total of 10 Testing categories and 66 controls.
COPYRIGHT AND LICENSE
Copyright(c) 2008 The OWASP Foundation.
‘This document is released under the Creative Commons 2.5 License. Please read and understand the license and copyright
conditions.
REVISION HISTORY
‘The Testing Guide v3 was released in November 2008. The Testing guide originated in 2003 with Dan Cuthbert as one of the
original editors. It was handed over to Eoin Keary in 2005 and transformed into a wiki. Matteo Meucci has taken on the
‘Testing guide and is now the lead of the OWASP Testing Guide Project since v2.
+ 16th December, 2008
"OWASP Testing Guide", Version 3.0 - Released by Matteo Meucc at the OWASP Summit 08
+ December 25, 2006,
“OWASP Testing Guide", Version 2.0
+ July 14, 2004 @
"OWASP Web Application Penetration Checklist", Version 1.1
December 2008
he OWASP Testing Guide”, Version 1.0
EDITORS
Matteo Meucci: OWASP Testing Guide Lead since 2007.
Eoin Keary: OWASP Testing Guide 2005-2007 Lead.
Daniel Cuthbert: OWASP Testing Guide 2003-2005 Lead.
V3 AUTHORS
Anurag Agarwal
Daniele Bellucci
Arian Coronel
Stefano Di Paola
Giorgio Fedon
Alan Goodman
Christian Heinrich
Kevin Horvath
Gianrico Ingrosso
Roberto Suggi Liverani
Alex Kuza
Pavol Luptak
Ferruh Mavituna
Marco Mella
Matteo Meuccl
Marco Morana
‘Antonio Parata
Cecil Su
Harish Skanda Sureddy
Mark Roxberry
‘Andrew Van der Stock
V3 REVIEWERS
‘+ Marco Cova Matteo Meucei
© Kevin Fuller Nam Nguyen
V2 AUTHORS
10
Vicente aguilera
Mauro Bregolin
Tom Brennan
Gary Burns
Luca Carettoni
Dan Cornell
Javier Ferndndez-Sanguino
GGiyn Geoghegan
Stan Guaik
Madhura Halasgikar
Eoin Keary
David Litchfield
‘Antonio Parata
Yiannis Pavlosogiou
Carlo Peliccioni
Harinath Pudipeddi
‘Alberto Revell
Mark Roxberry + Mark Curphey
© Daniel Cuthbert
‘+ Sebastien Deleersnyder
* Stephen DeVries
‘+ Stefano Di Paola
+ David Endler
+ Giorgio Fedon
‘Andrea Lombardini
Ralph M. Los
Claudio Merloni
Matteo Meucci
Marco Morana
Laura Nunez
Gunter Ollmann
(OWASP Testing Guide v3.0
Tom Ryan
‘Anush Shetty
Larry Shields
Dafydd Studdard
‘Andrew van der Stock
Ariel Waissbein
Jeff williams
V2 REVIEWERS
Vicente Aguilera
+ Marco Belott
+ Mauro Bregolin
* Marco Cova
+ Danie! cuthbert
‘+ Paul Davies
Stefano Di Paola
Matteo GP. Flora
Simona Forti
Darrell Groundy
Eoin Keary
James Kist
katie MeDowell
Marco Mella
Matteo Meucci
syed Mohamed A
Antonio Parata
Alberto Revell
Mark Roxberry
Dave Wichers.
TRADEMARKS
Java, Java Web Server, and JSP are registered trademarks of Sun Microsystems, Inc.
= Merriam-Webster is a trademark of Merriam-Webster, Inc.
= Microsoft is a registered trademark of Microsoft Corporation.
= Octave is a service mark of Carnegie Mellon University
"Verisign and Thawte are registered trademarks of Verisign, Inc.
"Visa isa registered trademark of VISA USA.
= OWASP is a registered trademark of the OWASP Foundation
All other products and company names may be trademarks of their respective owners. Use of a term in this document
should not be regarded as affecting the validity of any trademark or service mark
a @
ABOUT THE OPEN WEB APPLICATION SECURITY PROJECT
[overview
‘The Open Web Application Security Project (OWASP) is an open community dedicated to enabling organizations to develop,
purchase, and maintain applications that can be trusted. All of the OWASP tools, documents, forums, and chapters are free
and open to anyone interested in improving application security. We advocate approaching application security as a people,
process, and technology problem because the most effective approaches to application security includes improvernents in
all of these areas. We can be found at http://Avww.owaso org
(OWASP is a new kind of organization. Our freedom from commercial pressures allows us to provide unbiased, practical,
cost-effective information about application security. OWASP is not affiliated with any technology company, although we
support the informed use of commercial security technology. Similar to many open-source software projects, OWASP
produces many types of materials ina collaborative, open way. The OWASP Foundation is a not-for-profit entity that
ensures the project's longterm success. For more information, please see the pages listed below:
Contact for information about communicating with OWASP
* Contributions for details about how to make contributions
+ Advertising if you're interested in advertising on the OWASP site
= How OWASP Works for more information about projects and governance
= OWASP brand usage rules for information about using the OWASP brand
STRUCTURE
The OWASP Foundation is the not for profit (501c3) entity that provides the infrastructure for the OWASP community. The
Foundation provides our servers and bandwidth, facilitates projects and chapters, and manages the worldwide OWASP
Application Security Conferences.
LICENSING
All OWASP materials are available under an approved open source license. If you opt to become an OWASP member
organization, you can also use the commercial license that allows you to use, modify, and distribute all OWASP materials
within your organization under a single license,
For more information, please see the OWASP Licenses page.
PARTICIPATION AND MEMBERSHIP
Everyone is welcome to participate in our forums, projects, chapters, and conferences. OWASP isa fantastic place to learn
about application security, to network, and even to build your reputation as an expert,
Ifyou find the OWASP materials valuable, please consider supporting our cause by becoming an OWASP member. All
‘monies received by the OWASP Foundation go directly into supporting OWASP projects
2 (OWASP Testing Guide v3.0
For more information, please see the Membership page.
PROJECTS.
(OWASP's projects cover many aspects of application security. We build documents, tools, teaching environments,
Buidelines, checklists, and other materials to help organizations improve their capability to produce secure code.
For details on all the OWASP projects, please see the OWASP Project page
OWASP PRIVACY POLICY
Given OWASP's mission to help organizations with application security, you have the right to expect protection of any
personal information that we might collect about our members.
In general, we do not require authentication or ask visitors to reveal personal information when visiting our website. We
collect internet addresses, not the e-mail addresses, of visitors solely for use in calculating various website statistics,
We may ask for certain personal information, including name and email address from persons downloading OWASP
products. This information is not divulged to any third party and is used only for the purposes of:
= Communicating urgent fixes in the OWASP Materials
‘= Seeking advice and feedback about OWASP Materials
= Inviting participation in OWASP’s consensus process and AppSec conferences
(OWASP publishes a lst of member organizations and individual members, Listing is purely voluntary and “opt-in”. Usted
‘members can request not to be listed at any time.
All information about you o your organization that you send us by fax or mail is physically protected. If you have any
questions or concerns about our privacy policy, please contact us at owasp@owasp.org
B @
Pa
‘The OWASP Testing Project has been in development for many years. With this project, we wanted to help people
understand the what, why, when, where, and how of testing their web applications, and not just provide a simple checklist
or prescription of issues that should be addressed. The outcome of this project is a complete Testing Framework, from
which others can bulld their own testing programs or qualify other people's processes. The Testing Guide describes in
details both the general Testing Framework and the techniques required to implement the framework in practice.
Writing the Testing Guide has proven to be a difficult task. It has been a challenge to obtain consensus and develop the
content that allow people to apply the concepts described here, while enabling them to work in their own environment and
culture. It has also been a challenge to change the focus of web application testing from penetration testing to testing
integrated in the software development life cycle.
However, we are very satisfied with the results we have reached. Many industry experts and those responsible for software
security at some of the largest companies in the world are validating the Testing Framework. This framework helps
organizations test their web applications in order to build reliable and secure software, rather than simply highlighting areas
of weakness, although the latter is certainly a byproduct of many of OWASP’s guides and checklists. As such, we have made
some hard decisions about the appropriateness of certain testing techniques and technologies, which we fully understand
will not be agreed upon by everyone, However, OWASP is able to take the high ground and change culture over time
through awareness and education based on consensus and experience.The rest of this guide is organized as follows. This
introduction covers the pre-requisites of testing web applications: the scope of testing, the principles of successful testing,
and the testing techniques. Chapter 3 presents the OWASP Testing Framework and explains its techniques and tasks in
relation to the various phases of the software development lifecycle. Chapter 4 covers how to test for specific
winerabilties (e.g., SAL Injection) by code inspection and penetration testing,
Measuring (in)security: the Economics of Insecure Software
‘basic tenet of software engineering is that you can't control what you can't measure [1). Security testing is no different.
Unfortunately, measuring security is @ notoriously difficult process. We will not cover this topic in detail here, since it would
‘take a guide on its own (for an introduction, see [2])
(One aspect that we want to emphasize, however, is that security measurements are, by necessity, about both the specific,
technical issues (e.g,, how prevalent a certain vulnerability is) and how these affect the economics of software, We find that
‘most technical people understand at least the basic issues, or have a deeper understanding, of the vulnerabilities. Sadly,
few are able to translate that technical knowledge into monetary terms, and, thereby, quantify the potential cost of
wulnerabilties to the application owner's business. We believe that until this happens, ClOs will nt be able to develop an
accurate return on security investment and, subsequently, assign appropriate budgets for software security.
While estimating the cost of insecure software may appear a daunting task, recently, there has been a significant amount of
‘work in this direction. For example, in une 2002, the US National Insitute of Standards (NIST) published a survey on the
cost of insecure software to the US economy due to inadequate software testing [3]. Interestingly, they estimate that a
better testing infrastructure would save more than a third of these costs, or about $22 billion a year. More recentiy, the
links between economics and security have been studied by academic researchers. See [4] for more information about
some of these efforts.
‘The framework described in this document encourages people to measure security throughout their entire development
process, They can then relate the cost of insecure software to the impact it has on their business, and consequently develop
ery (OWASP Testing Guide v3.0
appropriate business decisions (resources) to manage the risk. Remember: measuring and testing web applications is even
‘more critical than for other software, since web applications are exposed to millions of users through the Internet.
What is Testing
What do we mean by testing? During the development life cycle of a web application, many things need to be tested. The
Merriam-Webster Dictionary describes testing as:
* Toput to test or proof.
© Toundergo a test
‘+ Tobe assigned a standing or evaluation based on tests
For the purposes of this document, testing is a process of comparing the state of a system/application against a set of
criteria. In the security industry, people frequently test against a set of mental criteria that are neither well defined nor
complete. For this reason and others, many outsiders regard security testing as a black art. This document's aim is to
change that perception and to make it easier for people without in-depth security knowledge to make a difference.
Why Testing
‘This document is designed to help organizations understand what comprises a testing program, and to help them identify
the steps that they need to undertake to build and operate that testing program on their web applications. It is intended to
give a broad view of the elements required to make a comprehensive web application security program. This guide can be
used as a reference and as a methodology to help determine the gap between your existing practices and industry best
practices. This guide allows organizations to compare themselves against industry peers, understand the magnitude of
resources required to test and maintain their software, or prepare for an audit. This chapter does not go into the technical
details of how to test an application, as the intent isto provide a typical security organizational framework. The technical
details about how to test an application, as part of a penetration test or code review, will be covered in the remaining parts
of this document,
When to Test
Most people today don't test the software until it has already been created and isin the deployment phase of its life cycle
(Le, code has been created and instantiated into a working web application). This is generally a very ineffective and cost-
prohibitive practice. One of the best methods to prevent security bugs from appearing in production applications is to
improve the Software Development Life Cycle (SDLC) by including security in each of its phases. An SDLC is a structure
imposed on the development of software artifacts. if an SDLC Is not currently being used in your environment, itis time to
pick onel The following figure shows a generic SDLC model as well as the (estimated) increasing cost of fixing security bugs
in such a model.
15 Figure 1: Generic SOLC Model
Companies should inspect their overall SDLC to ensure that security is an integral part of the development process. SDLCS
should include security tests to ensure security is adequately covered and controls are effective throughout the
development process.
What to Test
It can be helpful to think of software development as a combination of people, process, and technology. If these are the
factors that "create" software, then its logical that these are the factors that must be tested. Today most people generally
test the technology or the software itself
An effective testing program should have components that test People — to ensure that there is adequate education and
awareness; Process —to ensure that there are adequate policies and standards and that people know how to follow these
policies; Technology — to ensure that the process has been effective in its implementation. Unless a holistic approach is
adopted, testing just the technical implementation of an application will not uncover management or operational
vulnerabilities that could be present. By testing the people, policies, and processes, an organization can catch issues that
would later manifest themselves into defects in the technology, thus eradicating bugs early and identifying the root causes
of defects. Likewise, testing only some of the technical issues that can be present in a system will result in an incomplete
and inaccurate security posture assessment. Denis Verdon, Head of Information Security at Fidelity National Financial
presented an excellent analogy for this misconception at the OWASP AppSec 2004 Conference in New York [5]: "If cars were
built like applications [..] safety tests would assume frontal impact only. Cars would not be roll tested, or tested for stability
in emergency maneuvers, brake effectiveness, side impact, and resistance to theft.”
Feedback and Comments
{As with all OWASP projects, we welcome comments and feedback. We especially lke to know that our work is being used
and that its effective and accurate.
PRINCIPLES OF TESTING
‘There are some common misconceptions when developing a testing methodology to weed out security bugs in software.
‘This chapter covers some of the basic principles that should be taken into account by professionals when testing for security
bugs in software.
16 (OWASP Testing Guide v3.0
‘There is No Silver Bullet
While itis tempting to think that a security scanner or application firewall wil either provide a multitude of defenses or
identify a multitude of problems, in reality there are no silver bullets to the problem of insecure software. Application
security assessment software, while useful as a first pass to find low-hanging fruit, is generally immature and ineffective at
in-depth assessments and at providing adequate test coverage. Remember that security is a process, not a product.
‘Think Strategically, Not Tactically
Over the last few years, security professionals have come to realize the fallacy of the patch-and-penetrate model that was
pervasive in information security during the 1990's. The patch-and-penetrate model involves fixing a reported bug, but
without proper investigation of the root cause. This model is usually associated with the window of vulnerability shown in
the figure below. The evolution of vulnerabilities in common software used worldwide has shown the ineffectiveness of this
‘model. For more information about the window of vulnerability please refer to [6]. Vulnerability studies [7] have shown
that with the reaction time of attackers worldwide, the typical window of vulnerability does not provide enough time for
patch installation, since the time between a vulnerability being uncovered and an automated attack against it being
developed and released is decreasing every year. There are also several wrong assumptions in the patch-and-penetrate
‘model: patches interfere with the normal operations and might break existing applications, and not all the users might (in
the end) be aware of a patch’s availability. Consequently not all the product's users will apply patches, either because of
this issue or because they lack knowledge about the patch's existence.
‘Vunerabiny ie Thevendor Security foots are
vumerabity® Totes updated (08
pe clone sera noe
+ mo
si ate
ame pied
ie ese a
bony
Sse
: aes
! \eieee
Figure 2: Window of exposure
To prevent reoccurring security problems within an application, itis essential to build security into the Software
Development Life Cycle (SDLC) by developing standards, polices, and guidelines that fit and work within the development
‘methodology. Threat modeling and other techniques should be used to help assign appropriate resources to those parts of
a system that are most at risk
‘The SDLCis King
The SDLC isa process that is well-known to developers. By integrating security into each phase of the SOLC, it allows for a
holistic approach to application security that leverages the procedures already in place within the organization. Be aware
that while the names of the various phases may change depending on the SDLC model used by an organization, each
conceptual phase of the archetype SDLC will be used to develop the application (e., define, design, develop, deploy,
v7 @
maintain). Each phase has security considerations that should become part of the existing process, to ensure a cost-
effective and comprehensive security program.
Test Early and Test Often
When a bug is detected early within the SOLC, it can be addressed more quickly and at a lower cost. A security bug is no
different from a functional or performance-based bug in this regard. A key step in making this possible is to educate the
development and QA organizations about common security issues and the ways to detect and prevent them. Although new
libraries, tools, or languages might help design better programs (with fewer security bugs), new threats arise constantly and
developers must be aware of those that affect the software they are developing. Education in security testing also helps
developers acquire the appropriate mindset to test an application from an attacker's perspective. This allows each
organization to consider security issues as part oftheir existing responsibilities.
Understand the Scope of Security
Itis important to know how much security a given project will require. The information and assets that are to be protected
should be given a classification that states how they are to be handled (e.,, Confidential, Secret, Top Secret). Discussions
should occur with legal council to ensure that any specific security need will be met. In the USA they might come from
federal regulations, such as the Gramm-Leach-Biiley Act [8], or from state laws, such as the California $8-1386 [9]. For
organizations based in EU countries, both country-specific regulation and EU Directives might apply. For example, Directive
196/46/ECA [10] makes it mandatory to treat personal data in applications with due care, whatever the application,
Develop the Right Mindset
Successfully testing an application for security vulnerabilities requires thinking “outside of the box." Normal use cases will
test the normal behavior of the application when a user is using it in the manner that you expect. Good security testing
requires going beyond what is expected and thinking like an attacker who is trying to break the application. Creative
thinking can help to determine what unexpected data may cause an application to fail in an insecure manner. It can also
help find what assumptions made by web developers are not always true and how they can be subverted. This is one of the
reasons why automated tools are actually bad at automatically testing for vulnerabilities: this creative thinking must be
done on a case-by-case basis and most web applications are being developed in a unique way (even if using common
frameworks).
Understand the Subject
One of the first major initiatives in any good security program should be to require accurate documentation of the
application. The architecture, data-flow diagrams, use cases, and more should be written in formal documents and made
available for review. The technical specification and application documents should include information that lists not only
the desired use cases, but also any specifically disallowed use case. Finally, Its good to have at least a basic security
infrastructure that allows the monitoring and trending of attacks against an organization's applications and network (e..,
IDS systems)
Use the Right Tools
While we have already stated that there is no silver bullet tool, tools do play a critical role in the overall security program.
‘There isa range of open source and commercial tools that can automate many routine security tasks. These tools can
simplify and speed up the security process by assisting security personnel in their tasks. I is important to understand
exactly what these tools can and cannot do, however, so that they are not oversold or used incorrectly
‘The Devilis in the Details
Itis critical not to perform a superficial security review of an application and consider it complete. This will instill a false
sense of confidence that can be as dangerous as not having done a security review in the first place. Itis vital to carefully
review the findings and weed out any false positive that may remain in the report. Reporting an incorrect security finding
can often undermine the valid message of the rest of a security report. Care should be taken to verify that every possible
section of application logic has been tested, and that every use case scenario was explored for possible vulnerabilities.
Use Source Code When Available
While black box penetration test results can be impressive and useful to demonstrate how vulnerabilities are exposed in
18 (OWASP Testing Guide v3.0
production, they are not the most effective way to secure an application. ifthe source code for the application is available,
it should be given to the security staff to assist them while performing their review. Its possible to discover vulnerabilities
within the application source that would be missed during a black box engagement.
Develop Metrics
{An important part of a good security program is the ability to determine if things are getting better. tis important to track
the results of testing engagements, and develop metrics that will reveal the application security trends within the
organization. These metrics can show if more education and training are required, if there is a particular security
‘mechanism that is not clearly understood by development, and if the total number of security related problems being
found each month is going down. Consistent metrics that can be generated in an automated way from avallable source
code will also help the organization in assessing the effectiveness of mechanisms introduced to reduce security bugs in
software development. Metrics are not easily developed, so using standard metrics like those provided by the OWASP
Metrics project and other organizations might be a good head start
Document the Test Results
To conclude the testing process, itis important to produce a formal record of what testing actions were taken, by whom,
when they ware performed, and details ofthe test findings. It is wise to agree on an acceptable format for the report which
is useful to all concerned parties, which may include developers, project management, business owners, IT department,
audit, and compliance. The report must be clear to the business owner in identifying where material risks exist and
sufficient to get their backing for subsequent mitigation actions. The report must be clear to the developer in pin-pointing
the exact function that is affected by the vulnerability, with associated recommendations for resolution in a language that
the developer will understand (no pun intended). Last but not least, the report writing should not be overly burdensome on
the security tester themselves; security testers are not generally renowned for their creative writing skills, therefore
agreeing on a complex report can lead to instances where test results do not get properly documented.
TESTING TECHNIQUES EXPLAINED
This section presents a high-level overview of various testing techniques that can be employed when building a testing
program. It does not present specific methodologies for these techniques, although Chapter 3 will address this information,
This section is included to provide context for the framework presented in the next chapter and to highlight the advantages
and disadvantages of some of the techniques that should be considered. In particular, we will cover:
‘+ Manual inspections & Reviews
‘+ Threat Modeling
= Code Review
‘© Penetration Testing
MANUAL INSPECTIONS & REVIEWS
Overview
‘Manual inspections are human-
