BOOK 1

Unit 1: Software Quality Concepts and Culture

Session 1: Definitions of quality and Society's concerns for quality
Session 2: The costs and impacts of bad quality and Cost-of-Quality model
Session 3: Quality attributes of software
Session 4: The dimensions of quality engineering and the roles of people
Session 5: Quality Vocabulary

Unit 2: Software Varification and Validation

Session 6: Verification & Validation Terminology and Foundations
Session 7: Objectives and Constraints of Verification & Validation
Session 8: Risks Associated with Software Development
Session 9: Risks associated with Software Testing
Session 10: Risk Analysis
Session 11: Risk Management
Session 12: Planning the V&V effort
Session 13: Documenting V&V strategy, including tests and other artifacts
Session 14: V&V involvement at different phases of the life cycle

Unit 3: Requirements Validation and Reviews

Session 15: Desk checking, walk through, Inspections, Reviews
Session 16: Prototyping to validate requirements (Summative prototyping)
Session 17: Acceptance Test Design
Session 18: Validating product quality attributes
Session 19: Formal Requirement Analysis

BOOK 2

Unit 4: Testing and Problem Analysis

Session 20: Unit Testing
Session 21: Test Planning and Designing
Session 22: Coverage analysis and structure based testing
Session 23: Black-box functional testing techniques
Session 24: Integration Testing
Session 25: Developing Test Cases based on Use Cases / Customer Stories
Session 26: Operational profile-based testing, System and Acceptance Testing
Session 27: Regression Testing
Session 28: Deployment Process and Deployment testing
Session 29: Debugging, fault analysis, tracking and reporting
Session 30: Test Automation and Testing Tools

Unit 5: Quality Standards and Process

Session 31: Software life cycle processes, organizational implementation of standards
Session 32: IEEE software quality related standards
Session 33: Software quality models and metrics
Session 34: Quality-related aspects of software process models
Session 35: The ISO 9000 Quality Management Systems, ISO/IEEE Standard 12207
Session 36: Introduction/overview of ISO 15504 and SEI CMM
Session 37: Quality-related process areas of ISO 15504
Session 38: Quality-related process areas of the SW-CMM and CMMI
Session 39: The Baldrige Award criteria as applied to software engineering
Session 40: Quality aspects of other process models

Unit 6: Quality Engineering and Quantifiable Quality Improvement

Session 41: Quality Engineering and Quality Planning
1
Session 42: Quality Measurement and Improvement
Session 43: Analysis, Feedback and Follow-up Actions in Product and Process Assurance
Session 44: Tool Support for Quality Engineering
Session 45: Quality Models
Session 46: Defect Classification and Analysis
Session 47: Root Cause Analysis
Session 48: Risk Identification for Quantifiable Quality Improvement
Session 49: Statistical Process Control
Session 50: Continuous Quality Improvement

2
UNIT 1: SOFTWARE QUALITY CONCEPTS
AND CULTURE
Introduction
Software is usually error prone since it is entirely developed by humans. It is almost impossible to
completely eradicate the errors in software. Therefore strong emphasis is given to software
testing activities throughout the software development.

This unit is aimed at providing an entrance to software testing and QA. This unit describes what is
software quality looking more from angles of quality concepts and also aims at giving the student
an idea of the quality culture. The unit is named as “Software Quality Concepts and Culture”

During this unit initial emphasis is given to definitions of quality moving on to costs associated
within it. Then quality attributes are discussed and different dimension are considered from quality
engineering perspective. Finally vocabulary of the quality is provided for the students to get
familiar with the terminology related to entire course.

Thus, this unit is covered through five sessions as described below.

• Session 1: Definitions of quality and Society's concerns for quality
• Session 2: The costs and impacts of bad quality and Cost-of-Quality model
• Session 3: Quality attributes of software
• Session 4: The dimensions of quality engineering and the roles of people
• Session 5: Quality Vocabulary

Objectives
By the end of Unit 1, learners should be able to:
• To define what is “Quality” in different views such as transcendental view, user view,
manufacturing view, product view and value-based view in a systematic manner
• To describe how quality is perceived based on people’s roles and responsibilities broadly
grouped as “Consumers” and “Producers” and to compare “Quality Expectations” of
these two groups separately
• To explain what is “Cost-of-Quality” or what attributes to “Cost-of-Quality” and to analyze
the impact of bad quality at different stages of software development life cycle. Also, to
describe the methodologies of minimizing the Cost-of-Quality and to list cost of software
quality model categories
• To explain what are attributes of quality and to analyze how each of these attributes
affect the quality
• To describe overall context of “Quality Engineering” covering the framework and the
processes by looking at various dimensions
• To describe and compare different roles of people that contributes to “Quality
Engineering” (e.g. compare the person who does quality planning against the person who
does testing)
• To recognize the vocabulary used by software organizations in terms of quality and
testing

3
Session 1: Definitions of quality and Society's concerns for
quality

Definitions of Quality and Society’s Concerns

Definitions of Quality
Quality: Perspectives and Expectations
Transcendental view
User view
Manufacturing view
Product view
Value based View
Views of Quality (additional to the above – definitions may overlap)
Consumer or Customer view
Producer view
Provider View
Supplier view

Aim
To describe how to define software quality

Objectives
To define what is “Quality” in different views such as transcendental view, user view,
manufacturing view, product view and value-based view in a systematic manner
To describe how quality is perceived based on people’s roles and responsibilities broadly
grouped as “Consumers” and “Producers” and to compare “Quality Expectations” of
these two groups separately
To list numerous other definitions of “Quality”, taking into consideration historical
perspective of quality (evolving perceptions of quality) focusing on quality in software
engineering

Introduction
Software quality may include many different attributes and may be defined and perceived
differently based on people’s different roles and responsibilities.

Following are some definitions of quality cited by famous gurus of quality

“Meeting the needs of the customer, both present and future” (Deming, cited in Beckford
1998, p. 71)
“Fitness for use” (Juran, cited in Bessin 2004, p. 3)
“Conformance to requirement” (Crosby, cited in Beckford 1998, p. 52)

4
There are two views of quality named as producer view of quality and customer view of quality
respectively. Meeting requirements is a producer’s view of quality. Being fit for use is the
customer’s definition. In some text books customer is referred as consumer.

In addition to the producer and customer views of quality, the organizational infrastructure also
includes a provider and a supplier view. These views are as follows:
• Provider view – This is the perspective of the organization that delivers the products and
services to the customer.
• Supplier view – This is the perspective of the organization (that may be external to the
producer’s company, such as an independent vendor) that provides either the producer
and/or the provider with products and services needed to meet the requirements of the
customer.

The infrastructure for quality products and services is illustrated in Figure 1.1.

Figure 1.1: Infrastructure for quality products and services

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John Wiley &
Sons, New Jersey cover the following
• Objective 1 and 2 are covered in Chapter 2 (pp 15 – 17)

• Objective 3 is covered in Chapter 2 (pp 24 – 26)

The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006

Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd defines
Quality through Chapter 2, sections 2.2 (pp 27-28)

Additional general reading is preferred for objective 3 where some definitions of quality
are as given below. It is not necessary find the below cited literature. It is important for
students to have a wider general knowledge in quality definitions. The following link can

5
 give you reference that would be applicable for this session and some of the other
sessions to follow.
http://www.acm.org/crossroads/xrds6-4/software.html

SAQ
Q1) What is software Quality?

Q2) What is your view of software quality? What are other views not mentioned in the
given key reference book by Jeff Tian

Q3) The producer of a product views the quality of the product differently from the
consumer.

1. Define the producer’s view of quality, and the consumer’s view of quality.
2. Can there be a difference between the quality of the delivered product from the
quality as understood by the producer? If ‘yes’, list two reasons that can cause
this difference.
3. Similarly, can there be a difference between the quality of the delivered product
from the quality as understood by the consumer? If ‘yes’, list two reasons that
can cause this difference.

Session 2: The costs and impacts of bad quality and Cost-of-

Quality model

6
Cost of Quality
Quality Vs Productivity
Types of Costs in Quality
Prevention Costs
Appraisal Costs
Failure Costs

Aim
To identify what type of costs are associated with quality and implications of these costs

Objectives
To explain what is “Cost-of-Quality” or what attributes to “Cost-of-Quality”:

To analyze the impact of bad quality at different stages of software development life cycle

To describe the methodologies of minimizing the Cost-of-Quality

To list cost of software quality model categories

Introduction
Quality is an attribute of a product or service. Productivity is an attribute of a process. They have
frequently been called two sides of the same coin because one significantly impacts the other.
There are two ways that quality can drive productivity. The first, which is an undesirable method,
is to lower or not meet quality standards. For example, if testing and rework components of a
system development process were eliminated or reduced, productivity as measured in lines of
code per hours worked would increase. This is often done under the guise of completing projects
Cost-of-Quality can be defined as the money spent beyond the cost of building the same product
right the first time. This can also be interpreted as the costs that are associated with the non-
achievement of product or service quality as defined by the requirements.

Cost associated with the quality can be divided into following categories:

• Prevention - Money required preventing errors and to do the job right the first time is
considered prevention cost. Prevention money is all spent before the product is actually
built.

• Appraisal – Appraisal costs cover money spent to review completed products against
requirements. This money is spent after the product or subcomponents are built but
before it is shipped to the user.

• Failure – Failure costs are all costs associated with defective products.
Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John Wiley &
Sons, New Jersey provides some understanding on objective 1, 2 and 3 are covered in
Chapter 3 (pp 28 – 39)

7
 It is strongly recommended to use “Google Search (www.google.com)” or any other
search engine available through internet to get a thorough understanding about this
session. There are numerous comprehensive documents and articles to get a good
understanding on this through internet.

Krasner H. 1998, ‘Using Cost-of-Quality approach for software’, Crosstalk The Journal of
Defence Software Engineering, November 1998 cover all the objectives related to this
session. The following URLs can be accessed to find this reference. If these URLs may
not work due URLs being updated please use the Google Search to get to the related
reference
• Some pages of the article (pp 6 to 11) is covered by
http://www.compaid.com/caiInternet/casestudies/krasner-CoSQ-xtalk.pdf

• A slide presentation is prepared by the same author can be found in

http://www.saspin.org/saspin_Sep99_krasner2.pdf

SAQ
Q1) What contributed most to Cost of software Quality in a software project that you have
worked as per your view? And how can it be reduced?

Q2) Explain how software quality costs increases or decreases as you progress in the
software development life cycle using graphs or charts and provide appropriate
methodologies in reducing these costs. Please provide what methodology is used at
which stage of the software development life cycle.

Session 3: Quality attributes of software

Quality attributes of Software

Functionality attributes
Reliability and sub attributes
Usability and sub attributes
Efficiency and sub attributes
Maintainability and sub attributes
Portability and sub attributes
Flexibility
Reusability
Interoperability

Aim
To describe various types of quality attributes and how they affect software systems

Objectives

8
 To explain what are attributes of quality
To analyze how each of these attributes affect the quality

Introduction
Quality is a multifaceted concept driven by customer requirements. The level of quality can vary
significantly from project to project and between organizations. In IT, the attributes of quality are
examined in order to understand the components of quality, and as a basis for measuring quality.
Some of the commonly accepted quality attributes for an information system are described in
below table.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement Chapter 2 (pp 18-19) cover both the
objectives at high level.

It is strongly recommended to use “Google Search (www.google.com)” or any other

search engine available through internet to get a thorough understanding about this
session. There are numerous comprehensive documents and articles to get a good
understanding on this through internet. The following URLs can be helpful, but, if this
URLs may not work due URLs being updated please use the Google Search to get to the
related reference
http://www.cert.org/archive/pdf/tr021.95.pdf

SAQ
Q1) Explain how customer and product developer will define quality?

Q2) As specified above all these attributes are applied on QA and QC processes.
Suggest a method/s for tester or customer to find out quality of application or system
(Take an example/s and describe each).

9
Session 4: The dimensions of quality engineering and the roles
of people

Quality Engineering
Quality engineering: Activities and Process
Quality Planning
Quality Assessment and Improvement
Quality engineering in software processes
Quality Assurance (an overall idea is sufficient)
Quality Control (an overall idea is sufficient)
Roles of people mapped with different quality engineering activities and aspects

Aim
To describe quality engineering from a wider context by looking at it from different dimensions

Objectives

10
 To describe overall context of “Quality Engineering” covering the framework and the
processes by looking at various dimensions
To describe and compare different roles of people that contributes to “Quality
Engineering” (e.g. compare the person who does quality planning against the person who
does testing)

Introduction
Different customers and users have different quality engineering expectations under different
market environments. Therefore we need to move beyond QA activities towards quality
engineering. The quality engineering activities can be grouped as pre-QA, in-QA and post-QA
activities.

Quality Assurance is a set of support activities (including facilitation, training, measurement, and
analysis) needed to provide adequate confidence that processes are established and
continuously improved to produce products that meet specifications and are fit for use.

Quality Control is the process by which product quality is compared with applicable standards,
and the action taken when a nonconformance is detected. It focuses on defect detection and
removal. This is a line function - performance of these tasks is the responsibility of the people
working within the process. The software testing is a quality control activity.

A software tester would involve in quality control where as a QA manager may involve in quality
planning and a process team (a team dedicated to look at software process activities) may
involve in quality assurance.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John Wiley &
Sons, New Jersey cover objective 1 and 2 through Chapter 1 (pp 3 – 6) and Chapter 5
(pp 53 – 64)

This section is actually overall related to the entire quality engineering discipline covered
through the entire book, but the student is only required to understand quality engineering
as a whole at high level. While reading the chapters 1 and chapter 5 students may come
across other sections that are inter-related. If the student does not feel he has the proper
understanding student is request to read other relevant chapters. If students read
chapters 1 to 5, it should give a reasonable knowledge to get a good feel of this section.
Students are also encouraged to do additional reading through internet or other sources
that is available within reach.

The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006

Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd defines
Quality Assurance & Quality Control through Chapter 2, sections 2.2 (pp 27-29). It also
provides details regarding Test Planning through Chapter 15 (pp 352-366)

Objective 2 can be specifically covered by reading different activities that add to quality
engineering and by analysing who and what stage these activities are performed. As an
example role of the person who does the quality planning should be an experience
mature person in the organisation belonging to the management carder where as a tester

11
 who gets involved in defect detection can be a person at engineer level with less
maturity.

SAQ
Q1) What are the dimensions of Quality Engineering considering activities involved in
quality engineering? (hint : quality planning could be treated as one dimension)

Q2) Explain one of the key activities in quality engineering detailing out roles and
responsibilities of the person(s) who will be performing this activity

Q3) As per your view what aspect of quality engineering is most vital? How do you feel
organizations should address this aspect to achieve the best quality?

Session 5: Quality Vocabulary

Understanding widely used concepts and words in software engineering as given below
Defect
Policy
Procedure
Process
Productivity
Quality
Quality – Producer View
Quality – Customer View
Other concepts and words used in software quality engineering

Aim
To comprehend the vocabulary in Quality targeting software engineering

Objectives
To identify widely used terms such as quality, processes, defects and products
To identify other terms used in software quality engineering

Introduction
The quality language is the way quality professionals describe the principles, concepts, and
approaches used for improving quality. Until the vocabulary is learned and its use encouraged in
the organization quality becomes a difficult program to achieve. For example, when the words
“process” or “defect” are used, there must be a common understanding of what is meant by those
terms.

Illustrated below are majority of the key words that would be used throughout this entire course

Acceptance Testing

12
 Testing to ensure that the system meets the needs of the organization and the end user
or customer (i.e., validates that the right system was built).
Audit
An independent inspection or assessment activity that verifies compliance with plans,
policies, and procedures, and ensures that resources are conserved. Audit is a staff
function; it serves as the "eyes and ears" of management.
Baseline
A quantitative measure of the current level of performance.
Benchmark
An industry or best-of-class norm.

Black-Box Testing
Data driven testing that focuses on evaluating the function of a program against its
specifications.
Cost-of-Quality (COQ)
Money spent above and beyond expected production costs (labor, materials, or
equipment) to ensure that the product the customer receives is defect-free. This includes
the cost of repairing a defective product that was shipped to a customer and the
associated damage costs.
Defect
From the producer's viewpoint, a defect is a product requirement that has not been met,
or a product attribute possessed by a product or a function performed by a product that is
not in the statement of requirements that define the product. From the customer's
viewpoint, a defect is anything that causes customer dissatisfaction, whether in the
statement of requirements or not.
Dynamic Testing
Testing which involves executing the system’s code.
First Party Audit
An internal audit conducted by auditors who are employed by the organization being
audited, but who have no vested interest in the audit results.
Functional Tests
Tests of business requirements that address the overall behavior of the system.
Information Systems (IS)
This is a general term for the computer systems in an organization that provides
information about its business operations. It's also used to refer to the people who
manage these systems. Typically, in a large corporation, "IS" or the "IS department"
refers to a central or centrally-coordinated system of computer expertise and
management, often including the organization's entire network of computer resources.
Integration Testing
Testing performed on groups of modules to ensure that data and control are passed
properly between modules.
International Organization for Standardization (ISO)
ISO is a network of the national standards institutes of 156 countries, on the basis of one
member per country, with a Central Secretariat in Geneva, Switzerland, that coordinates
the system. ISO is a non-governmental organization: its members are not, as is the case
in the United Nations system, delegations of national governments.
Metric
13
 Two or more measures combined in a relationship to each other to produce information
about an entity.
Outputs
Products, services, or information that results from a process.
Policy
Managerial desires and intents concerning either processes (intended objectives) or
products (desired attributes).
Problem
Any deviation from defined standards. Same as a defect.
Problem Reporting/Tracking
The process of reporting outstanding problems; having them assigned for resolution, and
closing them out when the customer has been notified that the problems have been
solved.
Procedure
The step-by-step method followed to ensure that standards are met.
Process
(1) The work effort that produces a product. This includes efforts of people and
equipment guided by policies, standards, and procedures. (2) A statement of purpose
and an essential set of practices (activities) that address that purpose. A process or set of
processes is used by an organization or project to plan, manage, execute, monitor,
control, and improve its software related activities.
Process Definition
A description of the organization's current best practice approaches so that the process is
understood, consistently performed, and ready for improvement or redesign.
Process Improvement
The progression of steps taken to change a process to make it produce a given product
faster, more economically, or of higher quality. Such changes may require the product to
be changed. It involves improving process capability and reducing variation by analyzing
the process and product results, identifying root-cause problems, and changing the
process to eliminate root causes. The defect rate must be maintained or reduced.
Process Re-engineering
The fundamental rethinking and radical redesign of business processes.
Product
The output of a process: the work product. There are three useful classes of products:
Manufactured Products (standard and custom), Administrative or Information Products
(invoices, letters, etc.), and Service Products (physical, intellectual, physiological, and
psychological). Products are defined by a statement of requirements.
Production Costs
The cost of producing a product. Production costs, as currently reported, consist of (at
least) two parts; actual production or right-the-first time costs (RFT) plus the Cost-of-
Quality. RFT costs include labor, materials, and equipment needed to provide the product
RFT.
Productivity
The ratio of the output of a process to the input, usually measured in the same units. It is
frequently useful to compare the value added to a product by a process, to the value of
the input resources required (using fair market values for both input and output).
Productivity Metric
14
 The ratio of work product (i.e., function points) divided by work effort (i.e., staff days).
Quality
Operationally, the word quality refers to products. A product is a quality product if it is
defect-free. To the producer, a product is a quality product if it meets or conforms to the
statement of requirements that defines the product. This statement is usually shortened
to: quality means meets requirements. To the customer, a product is a quality product if it
meets the customer’s needs, regardless of whether the requirements were met. This is
referred to as fit for use.
Quality – Producer View
The producer’s view of quality has these four characteristics: Doing the right thing, Doing
it the right way, Doing it right the first time, and Doing it on time without exceeding cost.
Quality – Customer View
Meeting requirements is a producer’s view of quality. This is the view of the organization
responsible for the project and processes, and the products and services acquired,
developed, and maintained by those processes.
Quality Assurance (QA)
The set of support activities (including facilitation, training, measurement, and analysis)
needed to provide adequate confidence that processes are established and continuously
improved to produce products that meet specifications and are fit for use.
Quality Control (QC)
The process by which product quality is compared with applicable standards, and the
action taken when a nonconformance is detected. It focuses on defect detection and
removal. This is a line function - performance of these tasks is the responsibility of the
people working within the process.
Quality Improvement
The change to a production process so that the rate at which defective products (defects)
are produced is reduced. Some process changes may require the product to be changed.
Quality Management (QM)
(1) A philosophy consisting of continuous process improvement activities involving
everyone in the organization in an integrated effort to improve performance at every level.
It requires the commitment of executive management, and an empowerment of
employees at all levels that enables them to participate in the improvement of the
processes that create products and services. (2) Quality management integrates
fundamental management techniques, existing
improvement efforts, teamwork, and technical tools in a disciplined approach focused on
continuous process improvement. It is also called total customer focus, total quality
control, quality assurance, and a leadership program.
Quality Measure
A quantitative assessment of the extent that a product or service demonstrates
successful performance, conforms to its requirements, or possesses a given attribute.
Quality Metric
Any relationship between identified measures that indicates, to the metric users, a level
of quality.
Regression Testing
Testing after changes have been made to ensure that no unwanted changes were
introduced.
Standard

15
 A requirement of a product or process. For example: 100 percent of the functionality must
be tested.
Static Testing
Testing performed without executing the system’s code; can be manual (e.g.,reviews) or
automated (e.g., code or writing analyzers).

Unit Testing
Testing performed on a single, stand-alone module or unit of code.
Validation
Any activity that helps assure that the end product (e.g., system) under defined operating
conditions meets its currently approved requirements and expectations.
Verification
All QC activities throughout the life cycle that assure that interim product deliverables
process their inputs in accordance with specifications and standards.
White-Box Testing
Testing based on knowledge of internal code structure and logic.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, chapters 1 to 5 (pp 3 – 64) will cover
important terms where student is required to only understand what each term mean.

The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006

Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd cover
important terms through Chapter 2 to 8 (pp 25-205).

It is strongly recommended to use “Google Search (www.google.com)” or any other

search engine available through internet to get a thorough understanding about this
session. There are numerous comprehensive documents and articles to get a good
understanding on this through internet.

http://www.isixsigma.com/dictionary/glossary.asp

http://www.aptest.com/glossary.html

SAQ
Q1) What is meant by Quality Vocabulary?

Q2) According to your knowledge describe below terminologies which used in the quality
language and give possible example/s for each of the terminology.

1. Defect 4. Process

2. Validation 5. Quality Assurance

3. Verification 6. Quality Control

16
7. Quality Management

17
UNIT 2: SOFTWARE VERIFICATION AND
VALIDATION
Introduction
The development of sophisticated software products is increasing at an unprecedented rate.
In order to deal effectively with the increased complexity, functionality and time-to-market
pressures, organizations need practical techniques that can help improve software quality and
achieve aggressive project schedules. Therefore, understanding of the latest software
Verification and Validation (V&V) techniques will help improve the abilities in critical quality
role.

Verification ensures that the system (software, hardware, documentation, and personnel)
complies with an organization’s standards and processes, relying on review of non-
executable methods. Validation physically ensures that the system operates according to plan
by executing the system functions through a series of tests that can be observed and
evaluated.

Starting with definitions of V&V this unit further move on to risk analysis and management,
looking at risks from both the software development and software testing disciplines and
discussing further the other aspects of V&V.

Thus, this unit is covered through nine sessions starting with session 6 and ending with
session 14 as described below.
• Session 6 : V&V terminology and foundations

• Session 7 : Objectives and constraints of V&V

• Session 8 : Risks Associated with Software Development

• Session 9 : Risks Associated with Software Testing

• Session 10: Risk Analysis

• Session 11: Risk Management

• Session 12: Planning the V&V effort

• Session 13: Documenting V&V strategy, including tests and other artifacts

• Session 14: V&V involvement at different points in the lifecycle

Objectives
By the end of Unit 2, learners should be able to:
• define, describe and analyze software verification and validation and their constraints
• define a risk, how it would affect the quality, how risks arise and what factors
determine risks in software development and testing both
• identify what is ‘Risk Analysis’, recognize the intention of performing risk analysis,
how it is important for an organization and interpret the risk analysis process from a
testing perspective,
• identify what are the ‘Risk Management’, activities, disciplines associated with Risk
Management and benefits of risk management
• define risk reduction methods and contingency planning

18
 • discover the importance of considering the V&V effort, V&V effort organizing
methods, aspects of V&V planning and industry best practices of planning V&V
• identify verification and validation techniques, types of V&V documentation and
demonstrate documenting V&V strategy
• identify phases of the software development life cycle and discover V&V involvement
at different phases of the life cycle

Session 6: Verification & Validation Terminology and

Foundations

19
Verification and Validation terminology and foundations
Definitions of Verification and Validation
Distinction of Verification Vs Validation
Commonly used terminology in Verification and Validation

Aim
To define and distinguish software verification and validation

Objectives
To define software verification and validation and to describe the distinction between
them
To identify the terminology commonly utilized in the software verification and
validation

Introduction
The term verification and validation (V&V) are sometimes confused. They are in fact different
activities.

Verification is a quality process that is used to evaluate whether or not a product, service, or
system complies with a regulation, specification, or conditions imposed at the start of a
development phase. Verification can be in development, scale-up, or production. This is often
an internal process.

Validation is the process of establishing documented evidence that provides a high degree of
assurance that a product, service, or system accomplishes its intended requirements. This
often involves acceptance and suitability with external customers.

It is sometimes said that validation can be expressed by the query ‘Are you building the right
thing?’ and verification by ‘Are you building the thing right?’. 'Building the right thing' refers
back to the user's needs while 'building it right' checks that the documented development
process was followed. In some contexts, it is required to have written requirements for both as
well as formal procedures or protocols for determining compliance.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John
Wiley & Sons, New Jersey covers the above objectives in Chapter 4 - sections 4.3
and 4.4 (pp 46 – 51)

Additional reading is recommended from the books mentioned below if they can be
obtained.
• A book by James S. Collofello on Introduction to Software Verification and
Validation (SEI Curriculum Module: SEI-CM-13-1.1)
• A book by by Srinivasan Desikan & Gopalaswamy Ramesh on Software Testing
– Principles and Practices Chapter 2, Section 2.3 (pp 29 – 30)
• A book by Sommerville, I. 2001, Software Engineering 6th Edition Addison –
Wesley

20
 Also, it is recommended to use a search engine available through internet (e.g.
www.google.com) to get a thorough understanding about this section. Internet is a
good source of information readily available even if you do not get access to the
aforementioned books. The following link is such a link that would provide a guide to
software verification and validation

ftp://ftp.estec.esa.nl/pub/wm/wme/bssc/PSS0510.pdf

SAQ
Q1) Define software verification & validation (answer giving an example for each)
Q2) Describe the importance of Verification & Validation for a software product

Session 7: Objectives and Constraints of Verification &

Validation

Verification Objectives
Validation Objectives (Test Objectives)
V&V Limitations
Theoretical foundations
Impracticality of testing all data
Impracticality of testing all paths
No absolute proof of correctness

Aim

21
To comprehend verification & validation by identifying the objectives and constraints of
Verification & Validation

Objectives
To analyze the importance of verification and validation through the objectives of V&V
To identify constraints of verification & validation

Introduction
The overall objective of software V&V approaches is to ensure that the product is free from
failures and meets its user’s expectations. There are several theoretical and practical
limitations that make this objective being very hard to be obtained for many products.

Specific V&V objectives for each product must be determined on a project-by-project basis.
This determination will be influenced by the criticality of the product, its constraints, and its
complexity. Although approaches for determining whether a product satisfies its requirements
and specifications with respect to safety, portability, usability, maintainability, serviceability,
security, etc., are very important to many systems, they would not be addressed in this
session.

Once a set of V&V objectives has been identified, an overall integrated V&V approach must
be determined.

A test objective (goal) is a statement of what the test team or tester is expected to accomplish
during a specific testing activity. Test objectives, are usually defined during requirements
analysis, and guide the development of test scenarios, test cases, test scripts, and test data.

Test objectives are not simply a restatement of the system’s requirements, but the actual way
in which the system will be tested in order to assure that the system objective has been met.
If requirements are lacking or poorly written, then the test team must have a defined method
for uncovering and defining test objectives.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John
Wiley & Sons, New Jersey covers the above objectives in Chapter 4 - sections 4.3
4.4 and 4.5 (pp 46 – 52)

Additional reading is recommended from the books mentioned below if they can be
obtained
• Introduction to Software Verification and Validation (SEI Curriculum Module: SEI-
CM-13-1.1) by James S. Collofello
• Software Testing – Principles and Practices by Srinivasan Desikan &
Gopalaswamy Ramesh
• Software Engineering 6th Edition by Ian Sommerville, Addison – Wesley 2001

Also, recommended to use a search engine available through internet to get a

thorough understanding about this section where some of these links are given below
• venus.cs.depaul.edu/se682/Presentations/VandV.ppt
• http://books.google.co.in/books?id=4p0RQPMVuDsC&pg=PA24&lpg=PA24&dq=
Objectives+and+Constraints+in+V%26V&source=web&ots=JJ8jvwyoER&sig=Dn
uznoN5ARQZPcpGaFZPFJVOkkI&hl=en&sa=X&oi=book_result&resnum=10&ct=
result#PPA15,M1

22
 • http://www.cdf.toronto.edu/~csc340h/summer/lectures/w10/L10-part2-v&v.pdf

SAQ
Q1) Compare and contrast objectives of verification against objectives of validation
Q2) Explain constraints that can be faced during verification and validation

Session 8: Risks Associated with Software Development

Risks associated with software development

Definition of software risk
Identification of software risks
Classification of software development risks

Aim
To identify and describe risks in software development

Objectives
To define what a risk is and how it would affect the software development quality
To describe how risks arise in software development and what factors determine risks
in software development
23
Introduction
There are many definitions of risk that vary by specific application and situational context.
One is that risk is an issue, which can be avoided or mitigated (wherein an issue is a potential
problem that has to be fixed now.) Risk is described both qualitatively and quantitatively. In
some texts risk is described as a situation which would lead to negative consequences.

Qualitatively, risk is proportional to both the expected losses which may be caused by an
event and to the probability of this event. Greater loss and greater event likelihood result in a
greater overall risk.

In general terms risk can be defined as the following mathematical equation

Risk = (probability of an event occurring) X (impact of an event occurring)

Each software system has a unique set of risks. Some of those risks are associated with the
software functions, and other risks are associated with the process that develops the
software. The risks associated with development should be assessed for each software
system during development.

A few of the risks associated with software development are listed below.

• Improper use of technology

• Inability to translate user needs into technical requirements
• Uncontrolled system access
• Ineffective security and privacy practices for the application
• Program errors
• Operating system flaws

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John Wiley &
Sons, New Jersey covers the above objectives in section 3.3.3 (pp 36 – 37) and section
21.1 (pp 353 - 354)

The above recommended book focuses towards risks identification techniques, but what
is required to understand in this section specifically is the identification of risks associated
to software development. This can be targeted by the following book if can be obtained
• The book by Quality Assurance Institute (QAI) 2005b, CSTE Prep Guide (CBOK
for CSTE), Software Certifications, QAI. Refer the section 4.2 of skill category
four (pp 4-4 to 4-16)

It is strongly recommended to use a search engine available through internet to get a

good understanding about this section. Two of such web references that would help you
are given below
• http://www.stylusinc.com/Common/Concerns/software_development.php

• http://www.efst.hr/management/Vol8No2-2003/4-boban-pozgaj-sertic.doc

SAQ
Q1) Define what is a software risk
Q2) List and describe two of risk identification techniques

24
 Q3) List and describe four risks associated to software development

Session 9: Risks associated with Software Testing

Risks associated with software testing

Software development Vs software testing from a risk perspective
Classification of software testing risks

Aim
To identify and describe risks in software testing

Objectives
To identify software testing activities as opposed to software development activities
To define and distinguish software testing risks from software development risks

Introduction
Testing is inherently a risk-based activity. Most companies would not pay to add testing to the
cost of the project if there was not a cost associated with the risk of failure. Exhaustive testing
is impractical for most applications under development. Exceptions include applications that
25
support very high risk processes, such as air traffic control, nuclear power generation station
operations, defense systems, and so on. The project team must design a test strategy that
utilizes a balance of testing techniques to cover a representative sample of the system in
order to minimize risk while still delivering the application to production in a timely manner.

The test manager must determine the appropriate amount of testing to perform based upon
the risks associated with the application. These risks can arise from the newness and
reliability of the technology being used, the nature of the application, or from the priority of the
business functions under test. The amount of testing that should be performed is directly
related to the amount of risk involved.

Understanding the risk-based nature of testing is also the key to dealing with the chronic
problem of inadequate test resources. Risk must be used as the basis for allocating the test
time that is available, and for helping to make the selection of what to test and how to allocate
resources.

The test manager is also responsible for identification of potential risks that might impact
testing. A few of the primary testing risks include:

• Not Enough Training/Lack of Test Competency

• Us versus Them Mentality
• Lack of Test Tools
• Lack of Management Understanding and Support of Testing
• Lack of Customer and User Involvement
• Not Enough Schedule or Budget for Testing

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John Wiley &
Sons, New Jersey covers the above objectives in section 3.3.3 (pp 36 – 37) and section
21.1 (pp 353 - 354)

The above recommended book focuses towards risks identification techniques, but what
is required to understand in this section specifically is the identification of risks associated
to software testing. This can be targeted by the following book if can be obtained
• The book by Quality Assurance Institute (QAI) 2005b, CSTE Prep Guide (CBOK
for CSTE), Software Certifications, QAI. Refer the section 4.3 of skill category
four (pp 4-16 to 4-18)

It is strongly recommended to use a search engine available through internet to get a

good understanding about this section. Two of such web references that would help you
are given below
• http://web.mit.edu/swrt/testing.html

• http://www.compaid.com/caiinternet/ezine/cost_of_quality_3.pdf

SAQ
Q1) Compare and contrast software development risks and software testing risks
Q2) List and describe four risks associated to software testing
26
Session 10: Risk Analysis

Risk Analysis
The definition of risk analysis
The intension and importance of risk analysis
Risk analysis models
Risk analysis process

Aim
To identify importance and the methodology in performing risk analysis from a software
testing perspective

Objectives
To identify what is ‘Risk Analysis’ in a software process and a product
To recognize the intention of performing risk analysis and how it is important for an
organization
To interpret the risk analysis process from a testing perspective which is forming a
team, identification of risks, estimation of the magnitude of the risk and selection of
test priorities etc

27
Introduction
The risk analysis is done during the test strategy formulation and test planning. The objective
of performing risk analysis as part of test planning is to help allocate limited test resources to
those software components that pose the greatest risk to the organization. Testing is a
process designed to minimize software risks. To make software testing most effective it is
important to assure all the high risks associated with the software, will be tested first. The
decision the testers need to make is where to allocate test resources. Figure 2.5.1 provides
an illustration of where to allocate test resources.

Case A shows that the risk of loss is much less than the cost of testing for that risk. For
example, names on the master file may be misspelled but the impact of that defect is minimal,
while testing to check that every name on a master file is correct could be expensive. On the
other hand, case B shows that the cost of testing is much less than the potential loss. In that
case, testing should occur. When the risk of loss and the cost of testing are close to the same
dollar amount, a decision on how much testing should occur is a risk decision.

Figure 2.5.1: Illustrations of the Need for Software Testing

It’s always worth to carry out a risk analysis and do regular reviews which will be by means of
testing systems and planning appropriately.

Performing risk analysis during test planning is a four-step process as follows:

1. Form the risk analysis team

28
 2. Identify risks
3. Estimate the magnitude of the risk
4. Select testing priorities

Required Readings
The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006
Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd
explains risk analysis briefly in Risk Management through Chapter 15, section
15.2.11 (pp 362-366).

The recommended book does not specifically cover risk analysis. It is strongly
advised for students to obtain below mentioned references to get a good hold of this
section. Most references are web references and access to internet would be
sufficient to obtain enough reading on this section.

• The book by Quality Assurance Institute (QAI) 2005b, CSTE Prep Guide (CBOK
for CSTE), Software Certifications, QAI. Refer the section 4.4 of skill category
four (pp 4-19 to 4-24)

• The following link will provide an overview of risk analysis and will provide a risk
analysis tutorial to gain wider knowledge: http://www.solver.com/risk-analysis/

• The EAI community web site has launched a web page consisting of risk
dimensions and tasks to be followed to conduct a risk assessment and allocate
resources accordingly: http://it.toolbox.com/blogs/enterprise-solutions/testing-
building-a-test-plan-step-2-2945

• It is important to gain knowledge on how risks could be identified and the process
that should perform on Risk analysis to measure such risks. Following link states
what are the goals of a Risk Analysis which is more towards on the cost
associated with risks:
http://searchsecurity.techtarget.com/tip/1,289483,sid14_gci1178862,00.html

• The following link will provide an overview of risk analysis in Risk Management
for Information Technology Systems:
http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf

SAQ
Q1) Briefly explain what is quantitative risk analysis and qualitative risk analysis?

Q2) What are the steps / process for conducting risk analysis? Explain how you could
perform a risk analysis by taking a software project as an example.

29
Session 11: Risk Management

Risk Management
Risk management activities and disciplines
Benefits of Risk Management
Risk reduction methods
Contingency planning

Aim
To understand risk management and to link risk management with testing discipline

Objectives
To identify what is ‘Risk Management’, activities, disciplines associated with Risk
Management and benefits of risk management
To comprehend risk reduction methods and contingency planning

Introduction
Risk management guides project managers to focus on what can go wrong; rather thinking
only on everything goes as planned. This enables aggressive risk-taking. It occurs anytime an
investor or project manager analyzes and attempts to quantify the potential for losses in an
investment/project and then takes the appropriate action.

30
Risk management is a totality of activities that are used to minimize both the frequency and
the impact associated with risks. The first part of risk management is to understand, identify
and determine the magnitude of risks. The next component of risk management is
determining risk appetite. Risk appetite defines the amount of loss management is willing to
accept for a given risk. Risk appetite in a bank’s credit card processing activity is the amount
of losses they’re willing to accept associated with credit card processing. Let us assume that
the risk appetite is one percent of charges made to credit cards. To manage a risk appetite of
credit card losses of 1% the bank will first monitor to determine the actual losses. If the loss is
less than 1% the bank may decide that they are losing potential business and issue many
more credit cards. If the loss exceeds 1% the bank may reduce the credit limit on credit cards
and/or terminate credit cards with individuals whose accounts are not current. There are two
activities associated with risk management which are:
• Risk Reduction Methods
• Contingency Planning

Internal process auditors in a company should monitor and evaluate the effectiveness of the
organization’s risk management system.

Required Reading
The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006
Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd cover
Risk Management through Chapter 15, section 15.2.11 (pp 362-366).

It is strongly advised for students to obtain below mentioned references to get a good
hold of this section. Most references are web references and access to internet would
be sufficient to obtain enough reading on this section.

• The book by Quality Assurance Institute (QAI) 2005b, CSTE Prep Guide (CBOK
for CSTE), Software Certifications, QAI. Refer the section 4.4 of skill category
four (pp 4-24 to 4-26)

• The following elaborates what’s Risk management and benefits in general, under
the topic of; “Controlling Costs with Risk Management“
http://www.ibc.ca/en/Business_Insurance/Risk_Management/

• The following link contains pdf document on Risk Management Standards

provided by the IRM, AIRMIC and ALARM. :
http://www.theirm.org/publications/documents/Risk_Management_Standard_030
820.pdf

• A document published by the EnterpriseCM Inc. comprises 17 steps to success

on Risk Management in projects. The following link contains a pdf describing the
steps that should be performed. They are being categorizing under four areas;
such as Planning, Resourcing, Controlling and Monitoring:
http://hosteddocs.ittoolbox.com/JC021304B.pdf

SAQ
Q1) List down the high risks that are common to technical project. Explain how you
could perform risk management on those risks.
Q2) What are the Risk reduction methods and what is contingency planning?
Q3) Describe the procedure of the risk management process

31
Session 12: Planning the V&V effort

Planning the V&V effort

Importance of Planning V&V
Organizing V&V effort
Standard and Guidelines for Planning V&V
Industry best practices of planning V&V

Aim
To explain planning V&V effort during a project run

Objectives
To discover the importance of considering the V&V effort
To describe the V&V effort organizing methods
To describe the aspects needed when planning the V&V strategy
To discover industry best practices of planning V&V

Introduction
Software V&V effort is very important in all part of software engineering actions as it make
sure the quality of the product and the way it followed to produce the product though it is
small or larger software in all the phases in SDLC. (e.g.: Inventory systems to software built
for Space Craft machines)

The main objectives of the V&V effort are

- To find defects
32
 - Determine if required functions and attributes are built into the software system.

Typically verification and validation is applied in parallel in SDLC. Verification and validation
planning can be divided in to 5 steps.
1. Identify the V&V scope,
2. Establish specific objectives from the general project scope,
3. Analyze the project input prior to selecting V&V tools and techniques,
4. Select tools and techniques, and
5. Develop the Software Verification and Validation Plan (SVVP)).

Required Reading
The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006
Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd
provides details regarding Test planning through Chapter 15, section 15.2 (pp 352-
366).
The recommended book does not cover planning aspect of verification and validation
specifically. It is strongly advised students to obtain below mentioned references to
get a good hold of this section. Most references are web references and access to
internet would be sufficient to obtain enough reading on this section.

• Objective 1 is covered by the following reference

www.eecs.ucf.edu/~turgut/COURSES/EEL6883_SEII_Spr07/PaperPresentations
/Fujii-p33.ppt (Slide 5)

• Objective 2 & 3 are covered by

venus.cs.depaul.edu/se682/Presentations/VandV.ppt (Page 9-Page14) and
http://www.cs.txstate.edu/~js55/SVVPLAN.pdf

• Objective 4: www.info.ucl.ac.be/~pecheur/publi/FinalNASAReport2.pdf

SAQ
Q1) Why it is important to consider V&V effort in SDLC?
Q2) What are the key tasks that need to consider in SDLC for V&V?
Q3) What are the V&V effort organizing methods and their
advantages/disadvantages?
Q4) What are the industry level best practices that showed benefits of following V&V
efforts?

33
Session 13: Documenting V&V strategy, including tests and
other artifacts

Documenting V&V strategy, including tests and other artefacts

Verification Techniques
Validation Techniques
Types of V&V documentation
Standards and Guidelines for documentation
Documentation hierarchy

Aim
To describe and demonstrate the documentation in V&V

Objectives
To identify verification and validation techniques
To identify types of V&V documentation
To identify documentation standards and guidelines
To demonstrate documenting V&V strategy

Introduction
Verification and validation represents both static testing (verification) and dynamic testing
(validation). Together they comprise the test activities.
Verification techniques are used throughout the software development life cycle (SDLC).
Walkthroughs, reviews and inspections are three common techniques that have great impact
in identifying defects at each stage of the development and maintenance life cycle. The
Figure 2.8.1 represents these verification techniques as a whole picture and the Table 2.8.1
represents some examples of verification.

Review Type Review Stage Review Board

34
 • Review • Phase-End Review • Individual
• Walkthrough o Project Feasibility Review Review
• Inspection Review • Group Type
o Software Review +
Requirements
Review Review
o Software Stage
Architecture Review +
o Software Design
Review Review
o Software Code Board
Review
o Software Test
Review
• Checkpoint Review
• Post Implementation
Review
Figure 2.8.1 – Verification Techniques

Table 2.8.1 – Verification Examples

Validation assures that the end product (system) meets requirements and expectations under
defined operating conditions. Within an IT environment, the end product is typically
executable code. Validation ensures that the system operates according to plan by executing
the system functions through a series of tests that can be observed and evaluated for
compliance with expected results. Table 7-4 illustrates how various techniques can be used
throughout the standard test stages. Each technique is described below.

Table 2.8.1 – Validation techniques used in test stages

35
When performing verification techniques some of documentation that can be used is as
follows.
• Review logs – Review logs are maintained to record to finding when performing a
particular review. Refer Figure 2.8.2 for more details
• Check sheets (check lists)

Figure 2.8.2 – A review log

Figure 2.8.3 – A Check sheet

When performing validation techniques some of the key documentation that can be used is as
follows.
• Test Strategy and Test Plan: This document describes how the validation will be
performed. Test Plan can be the governing document which covers all the testing
activities. A test strategy document can be a high-level governing document that will
cover both verification and validation.
• Test cases and test scripts
• Test results log
• Check sheets (check lists) : Check lists can be used here to make sure right steps
were followed from a process perspective
• Defect logs.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John
Wiley & Sons, New Jersey covers the above objectives through section 6, 7 & 12 (pp
67 – 101 and pp 203 - 219) which targets validation techniques

The book by Quality Assurance Institute (QAI) 2006, CSQA Prep Guide (CBOK for
CSQA), Software Certifications, QAI, (pp 7-1 to 7-26) and (pp 7-29 to 7-33) also
cover these objectives. It is good if this book can be obtained

The access to following URLs are also recommended which specifically focuses on
test documentation.

• http://www.kaner.com/pdfs/Test_Documentation.pdf

• http://it.toolbox.com/blogs/malach/testing-documentation-569

• http://www.acutest.co.uk/acutest/ieee829

• http://www.onemanwrites.co.uk/2008/08/07/testingdocumentation/

36
SAQ
Q1) Identify and describe three verification and three validation techniques
Q2) Define how test planning takes place? (students may use examples when
answering this)
Q3) Propose some guidelines and best practices you have used in documentation in
your past

Session 14: V&V involvement at different phases of the life

cycle

V&V involvement at different phases of the life cycle

Phases in software development life cycle
V&V involvement at different phases of life cycle

Aim
To apply V&V to phases of software development life cycle

Objectives
To identify phases of the software development life cycle
To discover V&V involvement of different phases of the life cycle

Introduction
The way in which V&V get involved at different stages of the life cycle would differ and how
V&V is involved at different phases of the software development life cycle is described below
where more emphasis is provided towards validation techniques.
• Requirements Phase Activities
- Determine test strategy
- Determine adequacy of requirements
- Generate functional test conditions
• Design Phase Activities
- Determine consistency of design with requirements
- Determine adequacy of design
- Determine adequacy of the test plans
- Generate structural and functional test conditions
• Program (Build) Phase Activities
- Determine consistency with design
- Determine adequacy of implementation
- Generate structural and functional test conditions for modules and units
• Test Phase Activities
- Test application system
• Installation Phase Activities
- Place tested system into production
• Maintenance Phase Activities
- Modify and retest

37
Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John
Wiley & Sons, New Jersey covers this topic through Chapter 4 (pp 41 – 52)

The access to following URLs are also recommended that would help the student to
get a wider understanding on this area

• venus.cs.depaul.edu/se682/Presentations/VandV.ppt

• www.eecs.ucf.edu/~turgut/COURSES/EEL6883_SEII_Spr07/PaperPresentations
/Fujii-p33.ppt

• http://www.ics.uci.edu/~djr/classes/ics121/PresentationsPDF/Topic5.pdf

• http://www.enre.umd.edu/ctrs-lab/psam-paper2.PDF

• http://davidfrico.com/vvbrief.pdf

SAQ
Q1) Describe typical phases of the software development life cycle
Q2) Explain the way in which V&V gets involved by considering any of the two
phases of software development life cycle

38
UNIT 3: REQUIREMENT VALIDATION
AND REVIEW
Introduction
Requirement validation is concerned with showing that the requirements actually define the
system that customer wants. If this validation is inadequate, errors in the requirements will be
propagated to the system design and implementation .Expensive system modifications may
be required at a later stage to correct problems with the requirements .The cost of errors in
the requirements is particularly high if errors are not discovered until the system is
implemented.

Validation should not be seen as a process to be carried out after the requirements document
has been completed .Regular reviews involving both users and software engineers are
essential while requirement definition being formulated.

Requirement review is a manual process which involves multiple readers from both client and
contractor staff checking the requirement document for anomalies and omissions.
This unit covers five sessions as described below

• Session 15: Desk checking, Walk through, Inspections, Reviews

• Session 16: Prototyping to validate requirements (Summative prototyping)
• Session 17: Acceptance Test Design
• Session 18: Validating product quality attributes
• Session 19: Formal Requirement Analysis

Objectives

By the end of Unit 3, learners should be able:

• To demonstrate the concepts and processes of desk checking, Walkthrough,
inspection
• To describe the prototyping concept, types and formal prototyping methodologies
leading to demonstrating the benefits of prototypes
• To enable the recognition of the concepts of acceptance testing and to demonstrate
how to perform acceptance testing
• To describe, distinguish and demonstrate different types of quality attributes
• To enable recognition and application of formal requirement analysis techniques

39
Session 15: Desk checking, walk through, Inspections,
Reviews
Desk Checking
Evolution of desk checking
Walk through
Inspection

Aim
To enable the demonstration of the concepts and processes of desk checking, walkthroughs
and inspections

Objectives
By the end of Session 15, learners should be able to demonstrate the,
1. Concepts of Desk Checking
2. Concepts of Walk through
3. Concepts of Inspection
4. Different Inspection techniques
5. Difference between inspection and walk through

Introduction
Desk checking, or self-inspection, is an intense examination of a working product or
document to ensure its correctness, completeness, consistency, and clarity.

Walkthroughs and inspections are formal manual techniques that are a natural evolution of
desk checking. While both techniques share a common philosophy and similar organization,
they are quite distinct in execution.

An inspection is the most formal type of group review. Roles (producer, moderator, reader
and reviewer, and recorder) are well defined, and the inspection process is prescribed and
systematic.

Walkthroughs differ from inspections in that the programmer does not narrate a reading of the
product by the team, but provides test data and leads the team through a manual simulation
of the system.

Required Reading
The recommended book by Jeff Tian, 2005, Software Quality Engineering - Testing,
Quality Assurance, and Quantifiable Improvement, IEEE Computer Society, John
Wiley & Sons, New Jersey cover the following

• Objective 1 and 2 are covered in Chapter 14 (pp 244)

• Objective 3 and 4 are covered in Chapter 14 (pp 237 – 244)

Guide to the CSTE Common Body of Knowledge covers the objective 5

40
 • Section 1.10.2.7 (pp 1-84)

The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006

Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd cover
static testing concepts and processes through Chapter 3, section 3.2 (pp 48-56)

It is strongly recommended to use “Google Search (www.google.com)” or any other

search engine available through internet to get a thorough understanding about this
session.

Desk Checking, Self Inspection

http://vva.dmso.mil/mini_elabs/VVtech-informal.htm#inf1

Peer review, Inspection, Desk check, Walkthrough

http://www.construx.com/Page.aspx?hid=1776

SAQ
Q1) What is a desk checking?
Q2) Explain the stages of generic inspection process
Q3) Explain the process and roles of the participants of the Fagan inspection process
Q4) Inspection techniques can be varied based on the
• Size and scope of the inspection
• Formality of the inspection

Describe the different inspection techniques which can be used for above two
scenarios

Q5) Provide the limitations of inspection

Q6) Distinguish inspection from Walk through

Session 16: Prototyping to validate requirements (Summative

prototyping)

41
Prototyping process
Types of prototyping
Formal prototyping methodologies

Aim
Is to enable defining of the prototyping concept and to demonstrate how requirements can be
validated through prototyping

Objectives
By the end of Session 16, learners should be able to
• To describe the prototyping concept and types of prototypes
• To describe formal prototyping methodologies
• To demonstrate the benefits of prototypes

Introduction
The prototype methodology assumes that the user does not have a rigid definition of
requirements. Prototyping applies an “I’ll know it when I see it” environment and produces an
operational methodology of the user’s requirements currently defined. The user can then see
the system in operation and make changes for another generation or prototype of the system.

The development of prototype versions Continues until the user believes the system contains
the requirements desired by the user. Used alone, the prototype methodology lacks the
necessary controls to ensure correct and high-quality processing.

In rapid prototyping, there are usually two basic approaches to design systems being used.
The approach chosen can either be formative or summative. The formative approach is being
used for situations wherein the prototype is first built based on the current stage of the design.
It is then tested on a control group with the results being used to integrate into the next stage
of development in order to further enhance and improve on the usefulness of the current
design.

The summative approach on the other hand takes a different course in developing product
design. A single test exercise is performed at the end of the overall design enhancing
process. At most times the approach is a two step stage. Following the summative approach
can sometimes make it too costly to make design changes as the end stage nears. But this
approach is less time consuming and can be more cost effective initially if the projected
design is seen to require little refinements when a Prototype is done. But in the long run,
using the formative approach will definitely be more beneficial.

Required Reading
The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006
Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd cover
prototyping through Chapter 2, section 2.5.2 (pp 34-36)

42
 What is prototyping?

http://www.umsl.edu/~sauterv/analysis/prototyping/proto.html

• Objective 1,4 is covered in above URL

Software prototyping

http://en.wikipedia.org/wiki/Software_prototyping#Methods

• Objective 2 and 3 are covered in above URL

It is strongly recommended to use “Google Search (www.google.com)” or any other

search engine available through internet to get a thorough understanding about this
session.

Additional Reading
Ian Somerville 2000, Software Engineering, 6th edition
• Chapter 8

SAQ
Q1) Describe the prototyping process
Q2) Explain the following prototyping types
• Throwaway prototyping
• Evolutionary prototyping
• Incremental prototyping
• Extreme prototyping
Q3) List and describe the formal prototyping methodologies
Q4) Provide the advantages and disadvantages of summative and formative
prototyping technique

Session 17: Acceptance Test Design

43
Acceptance Testing
Acceptance Criteria
Product acceptance
Procedure acceptance
Service Level acceptance
Selecting Test cases for Acceptance Testing
Executing Acceptance Tests

Aim
To enable the recognition of the concepts of Acceptance testing and to demonstrate how to
perform acceptance testing

Objectives
By the end of Session 17, learners should be able to
1. To define what is “Acceptance testing”
2. To identify different acceptance criteria such as product acceptance,
procedure acceptance, service level agreements
3. To identify how to select the test cases for acceptance testing
4. To demonstrate how to perform the acceptance testing
5. To identify the primary and specific concerns during acceptance testing
phase

Introduction
Acceptance Testing is used to ensure that the system meets the needs of the organization
and user or customer (i.e., validates that the right system was built). Acceptance criteria can
be defined as Product acceptance, Procedure acceptance and service level agreements. Test
cases for acceptance testing should be selected from the existing set of test cases from
different phases of testing. The primary concern of the user during acceptance testing is that
the deliverables meet the user requirements.

Required Reading
The recommended book by Sirinivasan Desikan and Gopallawa Ramesh, 2006,
Software Testing - Principles and practices Chapter 6, section 6.6 (pp 158-162)
covers objective no 1,2,3,4
Guide to the CSQA Common Body of Knowledge, 2006 -Skill category 10 covers
objective no 5

What is User Acceptance Testing?

http://www.exforsys.com/tutorials/testing/what-is-user-acceptance-testing.html

Additional Reading
Guide to the 2006 CSTE, CBOOK –Skill Category 7 provides further details on
Acceptance testing.

44
SAQ
Q1) Explain the primary and specific acceptance testing concerns?
Q2) Describe how to select test cases for acceptance testing by providing examples

Session 18: Validating product quality attributes

Defining software product quality attributes

Types of quality attributes
45
Validation of quality attributes

Aim
To describe, distinguish and demonstrate different types of quality attributes

Objectives
By the end of Session 18, learners should be able to,
1. define and distinguish different quality attributes
2. demonstrate examples of product quality attributes and their validations

Introduction
The product quality attributes are attributes of the software that, if they are wanted and not
present, pose a risk to the success of the software, and thus constitute a business risk.

Some of the product quality attributes are described below.

Availability:-The ease with which the product can be accessed and used

Compatibility:-The extent to which the product works well with associated products or systems

Constructability or practicality:-The extent to which the design of the product enables

assembly or construction

Expandability:-The ease with which the product can be modified or extended to encompass
additional functionality

Flexibility: - How well the product can be used or applied in other environments, or modified to
do so

Functionality: - The set of actions or functions and their specified properties that satisfy stated
or implied needs

Maintainability: - The effort required to keep in good working order, and/or to make
appropriate modifications

Operability or usability: - An assessment of the effort required to use, or ease of use, of a

product as perceived by a stated or implied set of users

Portability: - The ability of the product to be transferred from one environment to another

Reliability: - The capability of a product to maintain its level of performance under stated
conditions, or for a stated period of time

Efficiency: The relationship between the level of performance and the amount of resources
required in the process I.e. the ratio of the energy expended to the corresponding measure of
useful work or output

Security: - The extent to which the product can be protected from unwanted intrusion
46
Required Reading
Product Quality Attributes

http://www.maxwideman.com/issacons1/iac1177a/index.htm

Guide to the CSTE Common Body of Knowledge

• Section 1.1.3 Software Quality Factors

SAQ
Q1) What is a software-quality attribute?
Q2) Give some examples of Product quality Attributes
Q3) Explain some techniques that are used in order to validate the Product Quality
Attributes

Session 19: Formal Requirement Analysis

47
Formal Requirement Analysis
Formal Requirement Analysis techniques
Stakeholder interviews
Contract-style requirement lists
Measurable goals
Prototypes
Use cases
Software requirements specification
Stakeholder identification

Aim
To enable recognition and application of formal requirement analysis techniques

Objectives
By the end of Session 19, learners should be able to,

1. describe what is meant by Formal Requirement Analysis

2. demonstrate the techniques of analyzing requirements

Introduction
Requirements analysis, also called requirements engineering, is the process of determining
user expectations for a new or modified product. These features, called requirements, must
be quantifiable, relevant and detailed. In software engineering, such requirements are often
called functional specifications .
Some formal techniques that can be incorporated in requirement analysis are described
below.

1. Stakeholder interviews
Stakeholder interviews are a common method used in requirement analysis. These interviews
may reveal requirements not previously envisaged as being within the scope of the project,
and requirements may be contradictory. However, each stakeholder will have an idea of their
expectation or will have visualized their requirements.

2. Contract-style requirement lists

One traditional way of documenting requirements has been contract style requirement lists. In
a complex system such requirements lists can run to hundreds of pages.

Measurable goals

Best practices take the composed list of requirements merely as clues and repeatedly ask
"why?" until the actual business purposes are discovered. Stakeholders and developers can
then devise tests to measure what level of each goal has been achieved thus far. Such goals
change more slowly than the long list of specific but unmeasured requirements. Once a small
set of critical, measured goals has been established, rapid prototyping and short iterative
development phases may proceed to deliver actual stakeholder value long before the project
is half over.

48
3. Prototypes
Prototypes are mock-ups of an application. Mock-ups allow users to visualize an application
that hasn't yet been constructed. Prototypes help users to get an idea of what the system will
look like, and make it easier for users to make design decisions without waiting for the system
to be built.

4. Use cases
A use case is a technique for documenting the potential requirements of a new system or
software change. Each use case provides one or more scenarios that convey how the system
should interact with the end user or another system to achieve a specific business goal. Use
cases typically avoid technical jargon, preferring instead the language of the end user or
domain expert. Use cases are often co-authored by requirements engineers and
stakeholders.

5. Software requirements specification

A software requirements specification (SRS) is a complete description of the behavior of the
system to be developed. It includes a set of use cases that describe all of the interactions that
the users will have with the software. Use cases are also known as functional requirements.
In addition to use cases, the SRS also contains nonfunctional (or supplementary)
requirements. Non-functional requirements are requirements which impose constraints on the
design or implementation (such as performance requirements, quality standards, or design
constraints).

6. Stakeholder identification
A major new emphasis in the 1990s was a focus on the identification of stakeholders. It is
increasingly recognized that stakeholders are not limited to the organization employing the
analyst. Other stakeholders will include:
¾ those organizations that integrate (or should integrate) horizontally with the
organization,
¾ the analyst who is designing the system for any back office systems or organizations
and
¾ Senior management

Required Reading
The recommended book by Srinivasan Desikan and Gopalaswarmy Ramesh, 2006
Software Testing – Principles and Practices, Dorling Kindersley (India) Pvt. Ltd cover
Use Case scenarios through Chapter 5, section 5.4.2 (pp 120-122)
Requirements analysis

http://searchsoftwarequality.techtarget.com/sDefinition/0,,sid92_gci1248686,00.html

Requirements analysis

http://en.wikipedia.org/wiki/Requirements_analysis

Additional readings

Software requirements

49
 http://www.swebok.org/ch3.html

SAQ
Q1) Explain what is meant by formal requirement analysis?

Q2) Demonstrate two techniques used for formal requirement analysis through
examples?

50