UNIT 06 Testing tools and measurements

Manual Testing

Manual Testing
Manual testing is a software testing process in which test cases are executed manually without using any
automated tool. All test cases executed by the tester manually according to the end user's perspective. It
ensures whether the application is working, as mentioned in the requirement document or not. Test cases
are planned and implemented to complete almost 100 percent of the software application. Test case
reports are also generated manually.

Limitations of manual testing

 Manual testing requires more resources and time as the complete process is manual.
 It's prone to human error for the cases where intensive data calculation is involved.
 Testing is a repetitive process. For each release you have certain test cases which are executed just to make
sure that nothing is broken due to new features. These are called regression cases. To execute same cases
again and again is boring.
 It's not suitable for large scale and time bounded projects.
 Verifying a large amount of data is not possible.
 Performance testing is impractical manually.
 GUI difference, component size are difficult to verify manually.
 GUI objects size difference and color combination etc is not easy to find out in manual testing.
 Load testing and performance testing is not possible in manual testing.
 Running test manually is very time consuming job.
 Regression Test cases are time consuming if it is manual testing.

Automation Testing
Automation testing is the application of tools and technology to testing software with the goal of reducing
testing efforts, delivering capability faster and more affordably. It helps in building better quality software
with less effort.
Software Test automation makes use of specialized tools to control the execution of tests and compares the
actual results against the expected result. Usually, regression tests, which are repetitive actions, are
automated.
Testing Tools not only helps us to perform regression tests but also helps us to automate data set up
generation, product installation, GUI interaction, defect logging, etc. Automation tools are used for both
Functional and Non-Functional testing.

Need for automation testing

Why Automation Testing?
Test automation has many benefits for app testing cycles. This allows you to build better apps with less
effort. Also, it is less time-consuming.

1. Faster Feedback Cycle

Test automation helps you reduce the feedback cycle and bring faster validation for phases in the
development of your product.

2. Team Saves Time

By automating your testing procedure, your team has to spend less time validating newly developed
features. It also improves communication with other departments like marketing, design, or product owners
who rely on the results of these tests. These departments can easily check the logs of the automated tests
and see what’s happening.

3. Reduced Business Expenses

When using an automated test environment, your company will save money as fewer resources are spent
on testing your product. The idea is that you should not be doing any manual testing. Over the course of a
whole project, this can make a big difference.

4. Higher Test Coverage

Manual testing puts limits on how many tests you can verify. Automation allows you to spend time writing
new tests and adding them to your automated test suite. This increases the test coverage for your product,
so more features are properly tested resulting in a higher quality application.

5.Reusability: The scripts are reusable and you don’t need new scripts every time. Also, you can redo the
steps that are exactly as the previous ones.

6. Bugs: Automation helps you find bugs in the early stages of software development, reducing expenses
and working hours to fix these problems as well.

7. Faster Time to Market

This reduces the feedback and testing cycle and allows companies to bring their products to the market
faster.

8 Better Insights: Automated testing provides better insights than manual testing when some tests fail.
Automated software testing not only gives insights into the application but also shows you the memory
contents, data tables, file contents, and other internal program states. This helps developers determine
what’s gone wrong.

9 Improved Accuracy
Even the best testing engineer will make mistakes during manual testing. Especially when testing a complex
use case, faults can occur. On the other side, automated tests can execute tests with 100-percent accuracy
as they produce the same result every time you run them.

10 Automated Testing Provides More Features

An automated test suite can help you with more features—for example, simulating thousands of virtual
users interacting with your web application in order to see how the application behaves. It’s impossible to
simulate this kind of behavior by doing manual testing. Features like this save developers a lot of time!

11. Less Stress on QA Team

By implementing an automated testing strategy, you allow your QA team to spend time on tasks other than
manual testing.
For many QA engineers, testing automation creates the opportunity to build new tools to further optimize
the current testing suite or extend it with new features.

12 Fewer Human Resources: You just need a test automation engineer to write your scripts to automate
your tests, instead of a lot of people doing boring manual tests over and over again.

13. Eliminate Human Error

Manual testing opens up the opportunity for humans to make mistakes. Especially for complex scenarios, it
makes sense to use test automation to avoid mistakes. You can still make mistakes, even with test
automation. However, the rate of mistakes is significantly lower when using test automation for your test
suite
Automation Testing vs Manual Testing

Automation Testing Manual Testing

Automated testing is more reliable. It Manual testing is less reliable. Due to

performs same operation each time. It human error, manual testing is not
eliminates the risk of human errors. accurate all the time.

Initial investment of automation testing Initial investment of manual testing is

is higher. Investment is required for less than automation. Investment is
testing tools. In the long run it is less required for human resources. ROI is
expensive than manual. ROI is higher in lower in the long run compared to
the long run compared to Manual Automation testing.
testing.

Automation testing is a practical option Manual testing is a practical option

when we do regressions testing. where the test cases are not run
repeatedly and only needs to run once or
twice.

Execution is done through software tools, Execution of test cases is time consuming
so it is faster than manual testing and and needs more human resources
needs less human resources compared to
manual testing.

Exploratory testing is not possible Exploratory testing is possible

Performance Testing like Load Testing, Performance Testing is not a practical

Stress Testing etc. is a practical option in option in manual testing
automation testing.

It can be done in parallel and reduce test Its not an easy task to execute test cases
execution time. in parallel in manual testing. We need
more human resources to do this and
becomes more expensive.

Programming knowledge is a must in Programming knowledge is not required

automation testing to do manual testing.

Build verification testing (BVT) is highly Build verification testing (BVT) is not
recommended recommended

Human intervention is not much, so it is It involves human intervention, so it is

not effective to do User Interface testing. highly effective to do User Interface
testing.
Automation Testing tools
Katalon
Launched in 2015, Katalon is a free-licensed, cross-browser tool that enables running automation testing for
APIs, Web interfaces, and mobile (Android and iOS). Additionally, this tool provides analysis reports and test
recording.

Selenium
Selenium is a popular open-source (released under the Apache License 2.0) automation testing framework.
Originally developed in 2004 by Jason Hugging, Selenium remains a widely-known and used tool for
testing web applications. It operates across multiple browsers and platforms (macOS, Windows, and Linux)
and can write tests in various programming languages, such as Python, Java, C#, Scala, Groovy, Ruby, Perl,
and PHP.

Bugzilla?
Bugzilla is an open-source issue/bug tracking system that allows developers to keep track of outstanding
problems with their product. It is written in Perl and uses MYSQL database.
Bugzilla is a Defect tracking tool, however, it can be used as a test management tool as such it can be easily
linked with other Test Case management tools like Quality Center, Testlink etc.

QTP is an automated functional Testing tool that helps testers to execute automated tests in order to
identify any errors, defects or gaps in contrary to the expected results of the application under test. It was
designed by Mercury Interactive and later on acquired by HP and now MicroFocus. Full form of QTP is
QuickTest Professional while UFT means Unified Functional Testing.
Why QTP is the best testing tool?
It is an icon-based tool that automates the regression and Functional Testing of an application
Both technical, as well as a non-technical tester, can use Micro Focus QTP
It provides both features- Record as well as Playback
We can test Desktop as well as the Web-based applications
It allows Business Process Testing (BPT)
QTP Testing is based on scripting language VB script

Selenium
Selenium is often used for regression testing. It offers testers a playback tool that allows them to record and
playback regression tests. In fact, Selenium is not a single tool but rather a suite of software that includes
various tools (or components):
Selenium IDE (Integrated Development Environment)
Selenium WebDriver
Selenium client API
Selenium Remote Control
Selenium Grid

MANTIS is an open source bug tracking software that can be used to track software defects for various
software projects. You can easily download and install the Mantis for your use. Mantisbt now also provides
a hosted version of the software. You can easily customize Mantis to map your software development
workflow.
Some salient features of Mantis Bt are
Email notifications: It sends out emails of updates, comments, resolutions to the concerned stakeholders.
Access Control: You can control user access at a project level
Customize: You can easily customize Mantis as per your requirements.
Mobile Support: Mantis supports iPhone, Android, and Windows Phone Platforms.

LambdaTest
 LambdaTest is a cloud-based automation testing tool for desktop and mobile applications. This tool allows
for manual and automated cross-browser testing across more than 2000 operating systems, browsers, and
devices.
LambdaTest allows testers to record real-time browser compatibility testing. Plus, it enables screen
recording and automated screenshot testing on several combinations at a time.

Ranorex
Ranorex is a test automation tool for web, desktop, and mobile. This tool provides numerous benefits, such
as codeless test creation, recording and replaying testing phases, and reusable test scripts.

Appium
Appium is an open-source test automation framework. This framework supports multiple programming
languages (Python, Java, PHP, JavaScript, etc.) for writing tests and can integrate CI/CD tools (e.g., Jenkins).

Eggplant
Eggplant was developed by TestPlant to provide testers the possibility to execute different types of testing.
Similar to Selenium, Eggplant is not a single tool but rather a suite of tools for automation testing, and each
tool performs different types of testing.

7. Kobiton
Kobiton is a cloud-based platform that can perform both manual and automated mobile and web testing.
Its AI-driven scriptless approach can automate performance, visual and UX, functional, and compatibility
testing. In addition, Kobiton offers automated crash detection, which ensures comprehensive quality.

Features of test tools

Static Test Tools: These tools do not involve actual input and output. Rather, they take a symbolic approach
to testing, i.e. they do not test the actual execution of the software. These tools include the following: ,
1) Flow analyzers: They ensure consistency in data flow from input to output.
2) Path tests: They find unused code and code with contradictions.
3) Coverage analyzers: It ensures that all logic paths are tested.
4) Interface analyzers: It examines the effects of passing variables and data between modules.

Dynamic Test Tools: These tools test the software system with ‘live’ data. Dynamic test tools include the
following
1) Test driver: It inputs data into a module-under-test (MUT).
2) Test beds: It simultaneously displays source code along with the program under execution.
3) Emulators: The response facilities are used to emulate parts of the system not yet developed.
4) Mutation analyzers: The errors are deliberately ‘fed’ into the code in order to test fault tolerance of the
system.

Selecting a testing tools

To select the most suitable testing tool for the project, the Test Manager should follow the below tools
selection process

Step 1) Identify the Requirement for Tools

 How can you select a testing tool if you do not know what you are looking for?
You to precisely identify your test tool requirements. All the requirement must
be documented and reviewed by project teams and the management board.

Step 2) Evaluate the Tools and Vendors

 After baselining the requirement of the tool, the Test Manager should
 Analyze the commercial and open source tools that are available in the market, based on the project
requirement.
 Create a tool shortlist which best meets your criteria
 One factor you should consider is vendors. You should consider the vendor’s reputation, after sale support,
tool update frequency, etc. while taking your decision.
 Evaluate the quality of the tool by taking the trial usage & launching a pilot. Many vendors often make trial
versions of their software available for download

Step 3) Estimate Cost and Benefit

 To ensure the test tool is beneficial for business, the Test Manager have to balance the following factors:
 Cost
 Value
 benefit
However, after discussing with the software vendor, you found that the cost of this tool is too high compare
to the value and benefit that it can bring to the teamwork.
 In such a case, the balance between cost & benefit of the tool may affect the final decision

Step 4) Make the Final Decision

To make the final decision, the Test Manager must have:

 Have a strong awareness of the tool. It means you must understand which is the strong points and
the weak points of the tool
 Balance cost and benefit.
 Even with hours spent reading software manual and vendor information, you may still need to try the tool
in your actual working environment before buying the license.
 You should have the meeting with the project team, consultants to get the deeper knowledge of the tool

Testing Metrics are the quantitative measures used to estimate the progress, quality, productivity and
health of the software testing process. The goal of software testing metrics is to improve the efficiency and
effectiveness in the software testing process and to help make better decisions for further testing process
by providing reliable data about the testing process.

Why Test Metrics are Important?

"We cannot improve what we cannot measure" and Test Metrics helps us to do exactly the same.
 Take decision for next phase of activities
 Evidence of the claim or prediction
 Understand the type of improvement required
 Take decision or process or technology change

Types of Test Metrics

Process Metrics: It can be used to improve the process efficiency of the SDLC ( Software Development Life
Cycle)
Product Metrics: It deals with the quality of the software product
Project Metrics: It can be used to measure the efficiency of a project team or any testing tools being used
by the team members