Testing Metrics

Software Testing Metrics Document Detail: Description

Version No.

Test Metrics

1.0

Revision History: Revision Description

GlobalLogic Inc.

Action Taken (add/del/mod)

Effective Page No.

Creation Date 4-Dec-09

Date of Change

Author Ritesh Rohatgi,

Author

Table of Contents .......................................................................................................3 1.0 Objectives ........................................................................................3 2.0 Test Efficiency (TE)...........................................................................3 3.0 Defect Priority Index (DPI) ................................................................4 4.0 Defect Acceptance (DA)................................................................... 4 5.0 Defect Rejection (DR)....................................................................... 5 6.0 Bad Fix Defect (B)............................................................................ 5 7.0 Weighted Defect Density.................................................................. 6 8.0 Effort Variance (EV).......................................................................... 7 9.0 Schedule Variance (SV).................................................................... 7 10.0 Scope Change (SC).........................................................................7 11.0 Defect Cause Distribution Report.................................................. 7 12.0

Functional Test failure ..................................................................8

GlobalLogic Inc.

1.0

Objectives Test metrics that are commonly used for monitoring test preparation and execution. We'll focus especially on the use and interpretation of such test metrics for reporting, controlling and analyzing the test effort, including those based on defects and those based on test data. We'll also look at options for reporting test status using such metrics and other information. As you read, remember to watch for the glossary terms defect density, failure rate, test control, test coverage, test monitoring and test report.

2.0

Test Efficiency (TE) This metric determine the efficiency of the testing team in identifying the defects. It also indicated the defects missed out during testing phase which migrated to the next phase.

Test Efficiency =

DT * 100 % DT +DU

Where, DT = Number of valid defects identified during testing. DU = Number of valid defects identified by user after release of application. In other words, post-testing defect Test Efficiency Trend

GlobalLogic Inc.

3.0

Defect Priority Index (DPI) This metric determine the quality of the product under test and at the time of release, based on which one can take decision for releasing of the product i.e. it indicates the product quality. Defect Priority index= ∑(Priority Index*No of Valid Defect(s) for this Priority) Total Number of Valid Defects

One can divide the Defect Severity Index in two parts: 1.

2.

DPI for All Status defect(s): - This value gives the product quality under test.

DPI for Open Status defect(s): - This value gives the product quality at the time of release. For calculation of DPI for this, only open status defect(s) must be considered. DPI (Open) = ∑(Priority Index*No of open Valid Defect(s) for this Priority) Total Number of Valid Defects

4.0

Defect Acceptance (DA) This metric determine the number of valid defects that testing team has identified during execution.

Defect Acceptance =

Number of Valid Defects * 100 % Total Number of Defects

The value of this metric can be compared with previous release for getting better picture Defect Acceptance Trend

GlobalLogic Inc.

5.0

Defect Rejection (DR) This metric determine the number of defects rejected during execution.

Defect Rejection =

Number of Defect(s) Rejected * 100 % Total Number of Defects

This metric gives the percentage of the invalid defect the testing team has opened and one can control, if required, in future. Defect Rejection Trend

6.0

Bad Fix Defect (B) Defect whose resolution give rise to new defect(s) are bad fix defect. This metric determine the effectiveness of defect resolution process.

GlobalLogic Inc.

Bad Fix Defect =

Number of of Bad Fix Defect(s) * 100 % Total Number of Valid Defects

This metric gives the percentage of the bad defect resolution which needs to be controlled. Bad Fix Defect Trend

7.0

Weighted Defect Density This metrics can be used to check the defect density. In order to derive this metrics correctly, we will have to identify Defect categories and provide weights to each category.

GlobalLogic Inc.

8.0

Effort Variance (EV) This metric gives the variance in the estimated effort.

Effort Variance =

9.0

Actual Effort - Estimated Effort * 100 % Estimated Effort

Schedule Variance (SV) This metric gives the variance in the estimated schedule i.e. number of days.

Schedule Variance =

Actual No. of Days - Estimated No. of Days * 100 % Estimated No. of Days

10.0 Scope Change (SC) This metric indicates how stable the scope of testing is.

Scope Change=

Total Scope - Previous Scope * 100 % Previous Scope

Where, Total Scope = Previous Scope + New Scope, if Scope increases Total Scope = Previous Scope - New Scope, if Scope decreases Scope Change Trend for one release

11.0 Defect Cause Distribution Report Pareto Chart for defect causes - based on the first iteration or first release defect count for the various causes. The data is shown for the previous 3 iteration or previous 3 releases. The trend indicates whether the actions taken for the first iteration / release have been effective for the top cause

GlobalLogic Inc.

12.0

Functional Test failure

FTF = Number of Functional Test Cases Failed Number of Functional Test Cases Executed)

GlobalLogic Inc.