1 Overview 5 Management
2 Life cycle 6 Tools
3 Static testing 7 Web testing
4 Dynamic test techniques 8 Software quality
Web testing versus traditional testing Web application components Testing practice Web testing tools
Web-based application characteristics
Application runs in two or more machines
A wide number of users accesses concurrently
Heterogeneous execution environments: hardware, network connections, operating systems, Web servers and Web browsers Fewer UI controls, fewer events
Limited keyboard event support
Slide 3
Client/Server and Web application
Client/Server application Web application
Requires installation on the clients machine
Application is a menu-driven
Do not require any installation on the client machine
URL-driven
Connected mode (connection Disconnected mode (state exists always until logout) less) Less number of network issues
Many issues like hardware compatibility, browser compatibility, version compatibility etc
Slide 4
Client/Server and Web application
Client/Server application Limited number of users Web application Unlimited number of users
Less securiry issues
Maximum securiry issues
Majorly used in Intranet areas Used in Internet areas Performance issues will not be Performance issues are very taken into consideration importance
The user interaction with the The user interaction is server is mainly through a user through a compatible web interface browser
Slide 5
Web testing versus traditional testing Web application components Testing practice Web testing tools
Web application components
Server-side components core application service components other application service components Client-side components web browsers add-on/plug-in components
Slide 7
Server-side components
Core application service components web servers
store Web pages or HTML files and their associated contents also serve advanced technology components such as Java servlets, ActiveX controls, and back-end database connectors act as data repositories for Web applications refer to a set of components that extend their services to other components or integrated application components
Slide 8
database servers
application servers
Server-side components (contd)
Other application service components search servers
catalog and index data that is published by Web servers act as intermediaries between networks and the Internet protect private networks from Internet intruders
proxy servers and firewalls
communication-related servers
e-commerce-related servers multimedia-related servers
Slide 9
Client-side components
Web browsers applications that retrieve, assemble, and display Web pages there are multiple browsers and browser versions available Add-on/Plug-in components support various forms of interactivity and animation within Web pages Macromedia Shockwave, Java applets and ActiveX controls
Slide 10
Web testing versus traditional testing Web application components Testing practice Web testing tools
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 12
User interface tests
User interface testing normally refers to a type of integration testing Often done in conjunction with other tests usability testing, functionality testing, exploratory testing, and task-oriented functional testing UI testing issues design of UI components implementation of UI components
Slide 13
UI design testing
Who is the applications target user? profiling the target user
computer experience web experience domain knowledge application-specific experience
How to evaluate design? design approach user interaction (data input) data presentation (data output)
Slide 14
UI design testing (contd)
Design approach point out as many design deficiencies early in the testing as possible, inconsistency in the design User interaction navigation methods: UI control, pages mouse/keyboard action matrices action commands: names of commands, default commands feedback and error messages
evaluated for clarity and consistency
Slide 15
Data presentation
UI implementation testing
Examines applications toward operation evaluates whether UI features work properly Do not be mistaken for functional tests
Slide 16
Usability testing
A metric that helps determining the users satisfaction when
interacting with a product or service through interfaces Factors
ease of learning efficiency of use
memorability
error frequency and severity
subjective satisfaction
Process identify any usability problems, collect quantitative data on participants' performance (e.g. time on task, error rates), and determine participant's satisfaction with the product
Slide 17
Accessibility testing
Web content must be available to and accessible by everyone navigate within a Web page and between Web pages navigate with keyboard only or other specialized input devices easily follow the contents and instructions provided Methods and tools for designing accessible products Web Content Accessibility Guidelines (W3C) several tools are available for automating: evaluation, repair, and transformation
Slide 18
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 19
Web application functional testing
Methods Functional acceptance simple tests (FASTs) Task-oriented functional tests (TOFTs) Forced-error tests (FETs) Boundary condition tests and equivalent class analysis Exploratory
Slide 20
Functional Acceptance Simple Tests
Exercises the lowest level of functionality for each command of a program Objective: check for the appropriate behaviors of UI controls based on the intended designs What to check? links (content links, image links,...) basic controls (backward and forward navigating, zoomin and zoom-out,...) action command checks (add, remove, update, create user accounts, data-entry tests...) other key features (log in/log out, e-mail notification, search, credit card validation)
Slide 21
Task-Oriented Functional Tests
Check whether the application can do useful tasks correctly How to do? create a features-to-be-tested list by dissecting product specification carefully, also consider competitive influences and market demands each entry on the list should be used to define a test case that checks whether the features requirements have been met
Slide 22
Forced-error tests
Contains negative test cases that are designed to force a program into error conditions Objective: to find any error conditions
Ways of compiling a list of error conditions (p.259) Testing process (p.260)
Slide 23
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 24
Common types of errors in database
Data integrity errors any bug that causes erroneous results to be stored missing or incorrect data in records, in tables, data might be outdated,... Output errors caused by bugs in the data retrieving and manipulating instructions, although the source data is correct See the list in ebook (p.330)
Slide 25
White-box methods
Code walk-through (peer-review process) point out any inefficiencies, redundancies, inconsistencies, or poor coding practices
executing the SQL statements or stored procedures one at a time
Testing triggers identify all the triggers analyze and catalog the conditions under which a trigger will be executed write and execute SQL statements or stored procedures
Slide 26
Inefficiency coding error example
SELECT * FROM staff SELECT id, state, salary, name FROM staff
This type of error only causes the database to do unnecessary extra work. The overall performance might be affected
Slide 27
Redundancy coding error example
A redundancy error in the ASP code can be caught by code walk-through
Send a query to the SQL database from an ASP Set RS = Conn.Execute (Select * from STAFF) Now, loop through the records If NOT RS.EOF Then Do while Not RS.EOF Notice that the If statement The code that manipulates, formats and displays records goes here is redundant because the ... condition is already Loop covered in the Do while End If
loop.
Slide 28
Black-box methods
Designing test cases executed on the browser-side
e.g. wrong data type, problematic characters: ', ,
drawbacks: the results do not sure that the data itself is
properly written; erroneous results do not necessarily mean database error
Testing for transaction logic check the database tables to ensure the transaction logic Testing for concurrency issues need to lock records to prevent concurrent updates and prevent data errors in the database
Slide 29
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 30
Configuration and compatibility testing-1
Configuration testing is designed to uncover errors related to various software and hardware combinations validate that a certain Web system installed on a dualprocessor computer operates properly Compatibility testing determines if an application, under supported configurations, performs as expected with various combinations of hardware and software releases determine which manufacturers and server brands, under the same configuration, are compatible with the Web system
Slide 31
Incompatibility issues example
Configuration and compatibility testing-2
When to run compatibility and configuration testing? should begin after the first round of functional tests has been completed and, ideally, after many functional errors have been discovered It is not practical to execute the entire set of tests on every environment testing a cross section of available hardware and software partition various operating environments Consider: server-side, client-side, network devices and connectivity How to test? (p.411)
Slide 33
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 34
Security testing
A combination of many protective measures taken to ensure the safety of the data and resources of both the owners and the users of computer systems Requires a combination of knowledge of security technologies, network technologies, programming, and real-world experience in penetrating the security of network systems Issues anatomy of an attack attacking intents common vulnerabilities and attacks testing for security
Slide 35
Anatomy of an attack
Information gathering viewing the source code
path names names and/or e-mail addresses of programmers, testers, and support staff
Network scanning scanning the system to determine which servers are alive and reachable from the Internet Attacking an active process of intrusion
Slide 36
Attacking intents
To steal
To disrupt activities
To embarrass To play a game
Slide 37
Common vulnerabilities and attacks-1
Software bugs, poor design, and programming practice buffer overflows backdoors javascript java ... Cookies
Malicious programs: virus and worm, trojan horses
Slide 38
Common vulnerabilities and attacks-2
Misuse access privilege attacks
Password cracking
Denial-of-service (DoS) attacks Information leaks social engineering keystroke capturing garbage rummaging scanning and probing
Slide 39
Testing for security
Testing the requirements and design
Testing the application code
Testing the deployment Testing with user protection via browser settings
Slide 40
Testing the requirements and design
Requirements are key understand how much the system is worth Access control need to consider access controls for system administrators, back-up utilities, the owners of the application, and users of the application servers access rights should be limited to the minimum rights required and only for the duration Client privacy issues: what information needs to be private? build a matrix of specific screens available to each class of user
Slide 41
Testing the application code
Backdoors check to make sure that all backdoors are closed in the production release ID and password testing check for the misuse of superuser accounts look for IDs and passwords in the clear when connecting to and accessing servers, directories, databases, and other resources
Slide 42
Testing the application code (contd)
Guidelines for testing information leaks test for comments in the HTML code test for error messages that reveal server name, root directory name,... turn off the directory browsing option on the Web server check to ensure that Access to Source configuration on the Web server is turned off check to ensure that the Write privilege to local path is blocked
Slide 43
Testing the application code (contd)
Testing the use of GET and POST check for the preferred use of POST instead of GET check the information in URLs try to access the file or directory shown in the URL Testing parameter-tampering e.g. www.mtesting.com/saving_acct001.pdf www.mtesting.com/saving_acct002.pdf SQL injection attacks:
SELECT * FROM bank WHERE ID=ADMIN AND PASSWORD=no OR 1 #
Slide 44
Testing the application code (contd)
Testing for buffer overflows requires testing everywhere a buffer is used send very large amounts of data to the buffer; test boundary conditions Testing for bad data, e.g. send an asterisk (*) instead of the persons last name entering </body></html> as a name Testing reliance on client-side scripting try to bypass authentication by going to the URL directly
Slide 45
Testing the deployment
Installation defaults check that the defaults provide the maximum amount of security Default passwords should never have passwords preset Program forensics using log files
Slide 46
Testing practice
User interface tests
Functional tests
Database tests Configuration and Compatibility testing
Security testing
Performance testing
Slide 47
Performance testing
Problem Solution web applications enable multiple users to access the application simultaneously evaluate the systems capability to perform critical functions during periods of normal and peak usage Often requires the aid of automated testing tools to simulate workload, collect measurement data, and present data that can be used for performance analysis Types of tests performance testing load testing stress testing
Slide 48
Types of tests 1
Performance testing an information-gathering and analysis process
metrics to collect: transactions per second, hits per second, concurrent connections, throughput,... to predict when load levels will exhaust system resources
Load testing evaluates system performance with a predefined load level minimum configuration and maximum activity levels be determined
Slide 49
Types of tests 2
Stress testing evaluates the behavior of systems that are pushed beyond their specified operational limits evaluates responses to bursts of peak activity that exceed system limitations
Slide 50
Three phases of performance testing
Planning phase
Testing phase
Analysis phase
Slide 51
Planning phase 1
Identifying baseline configuration and performance
requirements
consider hardware and software configurations, network
bandwidth, memory requirements, disk space, connectivity technologies... environment and resources, workload, acceptable response time
Determining the workload user-specific load
the number of concurrent users, the ratio of different types of users, and the behavior of different types of users TPS, ratios of specific pages requested by users...
Slide 52
application-specific load
Planning phase 2
Determining when to begin testing should be started as early in the product development process as possible Generating loads hardware-intensive approach
the use of multiple client workstations in the simulation of real-world activity
software-intensive approach
the use of a tool to simulate many users accessing the system simultaneously
Developing test cases
Slide 53
Testing phase
Generating test data by manually by programming by using existing data from the system under test Setting up a test bed of data Setting up the test suite parameters
Run tests Collect data
Slide 54
Analysis phase
Characterize the system organize the performance data gathered Analyze results and and look for the sources of performance
problems
examining the performance monitors on the servers receiving
the load hardware-related and software-related problems
Optimize the system under test add processing power? add RAM? add storage? add bandwidth?...
Slide 55
Web testing versus traditional testing Web application components Testing practice Web testing tools
Some web testing tools
Rule-based analyzers (HTML validators, link checkers)
Load/performance testing tools
GUI capture and playback tools Runtime error detectors
Web security testing tools
Java-specific testing tools Database testing tools
Slide 57
