Software Quality Assurance

Software Testing Strategies

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

Software Testing Techniques

Testing Strategy

Testing Strategy outlines in broad terms how to use testing to assess the extent to which the goal for a product has been met.

Software Testing Techniques

Testing Strategy

Testing Strategy integrates software test case designing methods into a well planned series of steps that results in the successful development of a software.

Software Testing Techniques

Testing Strategy Characteristics

Testing begins at component level and works towards the integration of the entire system. Different testing techniques at different points of time. Testing is conducted by the developer and ITG. Testing and debugging are different, but debugging is accommodated in any testing strategy.

Software Testing Techniques

Why Testing Strategy?

Testing often takes more project effort than any other software engineering activity. If it is conducted haphazardly, time is wasted, unnecessary effort is expended, and even worse, errors sneak through undetected.

Software Testing Techniques

Why Testing Strategy?

Testing Cost Testing Cost

% of TotaL Project Cost

100 80 60 40 20 0 Common Medium Involving human life risk

Testing Cost

Software Criticality

Software Testing Techniques

V&V

Verification

Are we building the product right? A set of activities that ensure that software correctly implements a specific function. Are we building the right product? A set of activities that ensure that the software is traceable to customer requirements.

Validation

Software Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

Software Testing Techniques

The V-Model

10

Software Testing Techniques

The V-Model

11

Software Testing Techniques

The V-Model

12

Software Testing Techniques

The V-Model

13

Software Testing Techniques

The W-Model

14

Software Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

15

Software Testing Techniques

Unit Testing

Testing of the smallest unit of software design i.e.; a module or a component. It is white-box oriented as well as black-box. Unit Testing is normally done by the developer of the unit.

16

Software Testing Techniques

Unit Testing

module to be tested

software engineer

results test cases

17

Software Testing Techniques

Unit Testing: Driver and Stub

The developer cannot test the module, he/she is building independently. Module may interact with other modules that are not yet developed. So, programmer uses the dummy subprograms to test his module/ component and these are stubs or driver. Driver and stubs are overhead.

18

Software Testing Techniques

Driver

A driver is nothing more than a Main program that,

accepts test data, passes it to the component to be tested, and prints relevant results

19

Software Testing Techniques

Stub or Dummy Subprogram

It serves to replace modules that are subordinate (called by) the component to be tested.

It uses the subordinate modules interface, may do little data manipulation, prints verification of entry, and returns control to the module undergoing testing.

20

Software Testing Techniques

Driver and Stub

driver
interface local data structures

module

boundary conditions independent paths

error handling paths

stub

stub test cases

21

Software RESULTS Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

22

Software Testing Techniques

Integration

Building a system or sub-system from unit-tested components is called integration. There are two approaches to integrate a system:
1. 2.

Big-bang Approach Incremental Approach

23

Software Testing Techniques

Integration

Big-bang Approach
Put all the components together at once, at the same time.

Incremental Approach
Add components one by one. Two approaches are:
1. 2.

Top-down Integration Bottom-up Integration

24

Software Testing Techniques

Integration Testing

It is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. Integration testing should be black-box testing with tests derived from the specification.

25

Software Testing Techniques

Top-down Integration Testing

Start with high-level system and integrate from the top-down replacing individual components by stubs where appropriate.

As Modules/ Components subordinate to the main control module are incorporated into the structure in either a depth-first or breadth-first manner.

26

Software Testing Techniques

Top-down Integration Testing

27

Software Testing Techniques

Top-down Integration Process

1.

2.

3.

4.
5.

The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module. Depending on the integration approach selected (depth-first or depth-first), subordinate modules are replaced one at a time with actual components. Tests are conducted as each component is integrated. On completion of each set of tests , another stub is replaced with the real component. Regression testing may be conducted to ensure that new errors have not been introduced

28

Software Testing Techniques

Top-down Integration: Pros & Cons Pros

Major decision points are verified early in the test process. Using depth-first integration, a complete function of the software may be implemented. Stubs are required which are overhead. Problems occur when processing at low levels in the hierarchy is required to test upper levels. No significant data flow upward in the program structure until all the stubs are replaced by the actual components.

Cons

29

Software Testing Techniques

Bottom-up Integration Testing

Integrate individual components in levels until the complete system is created.

30

Software Testing Techniques

Bottom-up Integration Process

Low-level modules are combined into clusters that perform a specific software function. A driver is written to co-ordinate test case input and output. The cluster is tested. Drivers are removed and clusters are combined moving upwards in the program structure.

31

Software Testing Techniques

Bottom-up Integration

32

Software Testing Techniques

Bottom-up Integration: Pros & Cons

Pros

Processing required for components to a given level is always available. The need for stubs is eliminated. As integration moves upward, the need for separate test drivers lessens. The program as an entity does not exist until the last module is added.

Cons

33

Software Testing Techniques

Sandwich Testing

In practice, most integration involves a combination of bottom-up and top-down integration testing approaches, called Sandwich Testing. Top-down for upper levels. Bottom-up for lower levels.

34

Software Testing Techniques

A Final Picture

Bottom - Up Integration Time to get working program Drivers Stub Parallelism Test specification Product control seq. Early Late

Top - Down Early Early

Big Bang

Sandwich Early

Late

Early

Yes No Medium Easy

No Yes Low Hard

Yes Yes High Easy

Yes Yes Medium Medium

Easy

Hard

Easy

Hard

35

Software Testing Techniques

Regression Testing

Each time a new module is added as part of integration testing, or a bug is fixed in the software, the software changes. These changes may introduce some new defects. Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not generated unintended side effects.

36

Software Testing Techniques

Smoke Testing

An integration testing approach used for shrink-wrapped Softwares.

Software components that have been translated into code are integrated into a build. A series of tests is designed to expose errors that will keep the build for properly performing its function. The intent should be to uncover show stopper errors. The build is integrated with other builds and the entire product is smoke tested daily. The integration approach may be top-down or bottom-up.

37

Software Testing Techniques

Smoke Testing

An integration testing approach used for shrink-wrapped Softwares.

Software components that have been translated into code are integrated into a build. A series of tests is designed to expose errors that will keep the build for properly performing its function. The intent should be to uncover show stopper errors. The build is integrated with other builds and the entire

Synchronize and Stabilize product is smoke tested daily. approach The integration approach may be top-down or bottom-up.

38

Software Testing Techniques

Benefits of Smoke Testing

Integration risk is minimized. The quality of the end-product is improved. Error diagnosis and correction are simplified. Progress is easier to assess.

39

Software Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

40

Software Testing Techniques

System Testing

A series of tests to verify that all system elements have been properly integrated. A series of different tests whose primary purpose is to fully exercise the computer-based system.

41

Software Testing Techniques

System Testing Types

Recovery Testing Forces software to fail in a variety of ways and verifies that recovery is properly performed. Security Testing Attempts to verify the softwares protection mechanisms. The software designer tries to make penetration cost more than the value of information obtained by breaking in.

42

Software Testing Techniques

System Testing Types

Stress Testing

Executes the system in a manner that demands resources in abnormal quantity, frequency or volume. Exercises the system beyond its maximum design load. Attempts to uncover data combination within valid input classes that may cause instability or improper processing.

Sensitivity Testing

43

Software Testing Techniques

System Testing Types

Performance Testing

To test the run time performance of a system within the context of an integrated system. Occurs throughout all steps in testing process, means during unit testing and integration testing etc.

44

Software Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

45

Software Testing Techniques

Acceptance Testing

It provides final assurance that software meets all functional, behavioral, and performance requirements. Whether the software under test functions in a manner that can reasonably be expected by the customer. Does it conforms to the specification?

46

Software Testing Techniques

Acceptance Testing Types

Alpha Testing

At developers site by customer. At customers site in a live environment.

Beta Testing

47

Software Testing Techniques

References

Sommerville, Ian Software Engineering 6th edition. Pressman, Roger Software Engineering, A Practitioners Approach6th edition. Mayer The Art of Software Testing 2nd edition.

48

Software Testing Techniques

Outline

Testing Strategy V-Model and W-Model Unit testing Integration Testing System Testing Acceptance Testing

49

Software Testing Techniques

Software Quality Assurance

Feel free to ask!

Thanks!

Software Quality Assurance

Software Testing Strategies