GSM Key Performance Indicator KPI Guidebook-ZTE
Short Description
GSM KPI description...
Description
GSM Key Performance Indicator （KPI）Guidebook
Version: V1.1 Security Category: Secret Serviceable range: Radio Planning Section
GSM Network Planning & Optimization Dept. of Engineering Service Division
ZTE Corporation
Secret▲ Secret▲
Work Guidebook for GSM P&O Dept.
Version Introduction Version V1.0
Date 2008-01-21
Writer Chen Chun
Assessor Zheng Hao
Amendment Records None 1、V3 KPI and EDGE KPI were added. 2、Amendment: mainly directed at V2.97
Zheng Hao, V1.1
2008-12-07
Yang Jiazhi
Chen Chun, Fei Aiping
and V6.0. 3、Some classic KPI cases and comments were added. 4、Principles for KPI risk analysis were added.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret▲ Secret▲
CONTENTS 1
Overview...........................................................................................................................1
2
CS Radio Service Statistical KPI....................................................................................4 2.1
Availability ........................................................................................................................4 2.1.1
SDCCH availability/SDCCH ....................................................................................4
2.1.2
TCH availability/TCH ...............................................................................................4
2.1.3
Cell utilization ...........................................................................................................4
2.2
Accessibility ......................................................................................................................5 2.2.1
Random Access Success rate.....................................................................................5
2.2.2
SDCCH blocking rate................................................................................................5
2.2.3
TCH blocking rate .....................................................................................................6
2.2.4
Radio connection ratio...............................................................................................6
2.2.5
SDCCH drop rate ......................................................................................................7
2.2.6
SDCCH assignment success rate...............................................................................7
2.2.7
TCH assignment success rate ....................................................................................7
2.2.8
Call setup success rate ...............................................................................................8
2.3
Mobility .............................................................................................................................9 2.3.1
2.4
Reliability ..........................................................................................................................9 2.4.1
TCH Call Drop Rate..................................................................................................9
2.4.2
Traffic Call Drop .....................................................................................................10
2.4.3
Call Completion rate................................................................................................10
2.4.4
UL RxQual 0-4 ........................................................................................................11
2.4.5
DL RxQual 0-4 ........................................................................................................11
2.5
3
Handover success rate ...............................................................................................9
Call Setup time ................................................................................................................12 2.5.1
Call setup time (GSM to PSTN)..............................................................................12
2.5.2
Call setup time (GSM to GSM)...............................................................................12 CS Radio Service Test KPI ...........................................................................................13
3.1
Call Setup Success Rate ..................................................................................................13
3.2
Handover Success Rate ...................................................................................................13
3.3
Call Drop Rate.................................................................................................................13
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret▲ Secret▲ 3.4
Coverage (Drive Test)......................................................................................................13 3.4.1
SQI...........................................................................................................................14
3.4.2
MOS ........................................................................................................................14
3.4.3
MOS & PSEQ .........................................................................................................15
4
PS Radio Service Statistical KPI ..................................................................................18 4.1
Accessibility ....................................................................................................................18 4.1.1
DL TBF establishment success rate .........................................................................18
4.1.2
UL TBF establishment success rate .........................................................................18
4.1.3
GPRS RLC ..............................................................................................................19
4.1.4
GPRS RLC ..............................................................................................................20
4.1.5
EGPRS RLC............................................................................................................20
4.1.6
EGPRS RLC............................................................................................................20
5
PS Radio Service Test KPI............................................................................................22 5.1
Accessibility ....................................................................................................................22 5.1.1
Attach success rate...................................................................................................22
5.1.2
Average GPRS attach time ......................................................................................22
5.1.3
PDP context activation success rate ........................................................................22
5.1.4
PDP context activation time ....................................................................................23
5.1.5
PING test success rate .............................................................................................23
5.1.6
Average PING test time ...........................................................................................23
5.2
Mobility ...........................................................................................................................24 5.2.1
5.3
Data Throughput..............................................................................................................24 5.3.1
Average FTP download rate ....................................................................................24
5.3.2
Average FTP upload rate .........................................................................................25
5.3.3
Throughput ..............................................................................................................25
5.4
Reliability ........................................................................................................................25 5.4.1
6
Routing Area Update Successful rate ......................................................................24
PDTCH Drop Rate ..................................................................................................25 Introduction to CN Statistical KPI...............................................................................27
6.1
Paging success rate ..........................................................................................................27
6.2
Location update success rate ...........................................................................................27
6.3
Attach success rate ..........................................................................................................27
6.4
GPRS average attach time ...............................................................................................28
6.5
PDP context activation success rate ................................................................................28
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret▲ Secret▲ 6.6 7
Routing Area Update Success rate ..................................................................................28 Comments on KPI Cases...............................................................................................30
7.1
Pengshui (Chongqing) PS Test Result.............................................................................30
7.2
Jishou (Hunan) China Mobile EDGE Test Result ...........................................................31
7.3
Acceptance requirements for EDGE KPI of Data Service of a network .........................33
7.4
Operation & Maintenance KPI of a Network ..................................................................34
7.5
Operation & Maintenance KPI of a network in Southeast Asia ......................................36
7.6
KPI requirements for Indonesia Telkomsel network .......................................................40
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret▲ Secret▲
1 Overview This guidebook collected the Key Performance Indicators (KPI) for CS and PS services, which are now commonly adopted in presales bidding projects; and the definitions of different KPIs and counter formulas for V2/V3 are explained here as 5
well. Meanwhile, with the aim to unify response on KPIs and avoid the possible market risks, after analyzing the performance indicators of current networks of both domestic and overseas key projects, the writer provides the reference values and risk values of different KPIs in this guide. If the KPIs listed in the bidding documents go beyond the reference values and risk values (suppose risk value of
10
the KPI is provided; otherwise, suggested values shall be applied.), we need to conduct risk analysis and evaluation, meanwhile, try to avoid the risk and send out pre-warning. The KPIs can be quite different due to the differences in network structure and dimension. To make sure that our KPI can meet requirements in bidding documents, we need not only support from theories, but accumulation of
15
work experience as well. The suggested values and risk values of KPI mentioned in this guide are reference suggestions concluded from the current conditions of GSM network. As GSM techniques and skills are developing and equipment performance is improving constantly, the reference standards for KPI will change accordingly. The key requirement for engineers engaged in presales network planning is to
20
thoroughly understand and master the related KPIs, which is also the basic but highest demand for radio network planning engineers. Why network operators set up KPIs and what are they aimed at? How to define KPI and what can be resulted from deviation of KPI? How much influence can the value of KPI impose on presales bidding?
25
How to clarify KPI and how to achieve favorable negotiation results during KPI clarification? How to avoid and pre-warn KPI risks? What is the development direction of KPI assessment? Each presales radio network planning engineer shall possess related techniques of
30
and up-to-date answers to the six questions above. For lack of space, the six This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 1，Total 45 pages
Secret▲ Secret▲ questions will not be further discussed in this guidebook. The engineers will be able to experience and get to know them in presales planning work in future. Generally, KPI can be classified into four types: statistical KPI, testing KPI, equipment performance KPI, customer perception KPI. Only statistical KPI and 5
testing KPI are elaborated here. As for equipment performance KPI and customer perception KPI, they are focused on in another guidebook. Nowadays, network operators are attaching more and more importance to KPI during negotiations. Besides the four types mentioned above, a lot more aspects are involved in KPI. KPI is also closely related to the basic network environment. Therefore, KPI is
10
affected by various factors. That is why operators are now regarding KPI as a very important leverage to shift risk and decrease network investment and maintenance cost. Equipment manufacturers must commit themselves to suitable, reasonable and timely KPI, so as to bring down contract risk and ensure profits. “Rationalize commitment, share risks” is an important principle in presales planning work.
15
The following are the factors that shall be taken into consideration when we work on KPI: 1. network dimension; 2. network type; 3. degree of subscriber(number) growth;
20
4. geographical distribution of network(urban, countryside, village) 5. equipment types; there is difference in KPI for V2 and V3. Functional principles for this guidebook (mainly directed at KPI): 1.
This guide is only available to the radio network planning engineers engaged in presales bidding. No one is entitled to spread it outside or extensively spread
25
it inside the company. The right to interpret the serviceable range of this guide remains with GSM Technical Research Section & Resources Section. 2.
It is suggested to release the guidebook in encrypted document. It can not be procured unless logging into document security software.
3. 30
The guide is a reference to radio network planning staff engaged in presales bidding. It can not replace the presales KPI project evaluation.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 2，Total 45 pages
Secret▲ Secret▲ 4.
The KPI values enclosed in this guide are valid within only half a year since the release of it. After half a year, it needs to be updated and reassessed.
5.
The guide is mostly applicable to V2.97 of V2 and V6.0 (ISMG) and V6.1 (Minos) of V3.
5
Notes: V6.0 (ISMG), V6.1 (Minos), sic passim
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 3，Total 45 pages
Secret▲ Secret▲
2 CS Radio Service Statistical KPI 2.1 Availability 2.1.1 SDCCH availability/SDCCH KPI KPI
SDCCH availability (SDCCH in service rate) reference
99%
value SDCCH usable total*100%/(SDCCH usable total +SDCCH unusable total）
Indicator definition
V2 Counter formula
V3
C11601*100%/(C11601+C11602) V6.0
C100030122*100%/( C100030122+ C100030123)
V6.1
C900060122*100%/(C900060122+C900060123)
The ratio of the SDCCH usable total to the total SDCCH allocated in network is mainly used to examine equipment performance, especially to inspect the utilization of SDCCH in the cells. It is directly related to the stability of equipment、transmission and power supply,
Notes
etc.. It reflects the utilization ratio of network equipment and the stability of initial network operation.
2.1.2 TCH availability/TCH KPI
TCH availability (TCH in service rate)
KPI
98%
reference value Indicator
TCH usable total*100%/(TCH usable total+ TCCH unusable total）
definition C11607*100%/(C11607+C11608)
V2 V6.0
Counter formula
V3
(C100030125+C100030134+C100030155)*100% / (C100030125 + C100030134 + C100030155 + 100030126+ C100030217+ C100030218) (C900060125+C900060134+C900060155)
V6.1
*100%/(C900060125+C900060134+ C900060155+C900060126+C900060217+C900060218)
The ratio of the SDCCH usable total to the total SDCCH allocated in network is mainly used to Notes
examine equipment performance, especially to inspect the utilization of SDCCH in the cells. It is directly related to the stability of equipment、transmission and power supply, etc..
5
2.1.3 Cell utilization KPI KPI
Cell utilization reference
80%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 4，Total 45 pages
Secret▲ Secret▲ value Indicator
Cells, in which ratio of actual traffic / desired traffic=18 95% sample >=25
TEMS test value
definition This KPI is for inspecting the sound distortion degree resulted from incapable link transmission. The formula is Ericsson’s patent. It takes into consideration many factors, including BER、FER、handover times、handover conditions、DTX activated (or not)、 Notes
and the encoder in use, etc., to finally get the SQI. There are different SQI top limits for different types of encoders. For half-rate encoder, the SQI top limit is 19; for full-rate encoder, 21; for enhanced full-rate encoder, 30. SQI ranges from 18 to 30. SQI value will increase when EFR is applied.
3.4.2 MOS KPI KPI
Mean Opinion Score ( MOS) reference
Refer to the form attached bellow.
value Indicator
Test value
definition Notes
The commonly used MOS assessment method consists of subjective and objective scoring parts. In the subjective part, engineers subjectively compare the original voice material and the deteriorated voice material according to ITU-T Suggestion book P.800 and P.830, and give their MOS scores respectively, then average the scores from different engineers. While in the objective part, PESQ method from ITU-T Suggestion book P.862 is employed, that is special equipment or software (e.g. the VQT test equipment of Agilent) is used to proceed
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 14，Total 45 pages
Secret▲ Secret▲ the assessment. In MOS, 0 means the worst quality, 5 stands for the highest. Standard PCM quality shall reach around 4.4. The MOS values for commonly used coding formats are as follows: G.711——MOS＝ 4.8 G.721、G.723、G.726——MOS＝ 4.2 G.728 ——MOS＝4.2 G.729 ——MOS＝4.2 G.723.1（5.3）——MOS＝3.5 G.723.1（6.3）——MOS＝3.98 MOS can be affected greatly by the performance of test equipment. The test results from different manufacturers are different. Currently, the test equipment from Actena performs the best.
Average MOS(Including FR &HR Voice version combination
MOS) Frequency
No
hopping
hopping
100% FR
3.04
3.29
100% EFR
3.63
3.57
100% AMR FR
3.56
3.54
TFO
frequency Frequency hopping
No frequency hopping
60% HR (+EFR) 30% HR (+EFR) 10% HR (+EFR) 60% AMR HR (+AMR FR) 30% AMR HR (+AMR FR) 10% AMR HR (+AMR FR)
3.4.3 MOS & PSEQ Subjective scoring method had been used in assessment on voice quality until 2001. Users were requested to assess the voice quality of incoming calls by scoring 1-5 5
(5- best, 1 - worst). This assessment method was regulated to be a standard by ITU in August, 1996. MOS assessment method was defined in its P.800 standard in details. But later it was found that this method not only took too much time, but also cost a lot. So there was an urgent need for a kind of economic test equipment, which can be used to simulate the voice quality received by users, and thus can
10
play its part in designing、testing and debugging new type of voice encoders. With the aim to meet the need, in February 2001 ITU issued the new test standard on voice transmission quality-P.862 (PESQ-Perceptual Evaluation of Speech Quality). It adopts the new algorithm developed by Psytechnics company. PESQ This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 15，Total 45 pages
Secret▲ Secret▲ can be used to compute voice sample’s MOS-LQO value (Mean Opinion ScoreListening Quality Objective). The computed result and the subjective scoring result are basically the same. PESQ compares the output signal (collected after the signal is transmitted through equipment) and the reference signal, then works out the 5
difference. In the ordinary course of event, the bigger the difference between the output signal and the reference signal, the smaller the reference value of MOS will be. PESQ is testified to be a reliable、economic and fast measuring tool for voice encoder. Compare PESQ’s predicted value and the result from subjective scoring method, we can see the correlation between the two values is 0.935(refer to
10
《P862E.doc》for correlation factors). For 22 kinds of ITU benchmark tests, the absolute error is within 0.5 MOS, 69.2% among which is within 0.25 MOS. It is worth our attention that PESQ value ranges from 0.5 to 4.5, while MOS value ranges from 1 to 5.
15 Fig 1 MOS/PESQ Correlation Diagram
At the present time, there are two types of QOS test method-subjective and objective methods. MOS subjective test method is defined in ITU-T P.800. The objective test methods includes PSQM、PSQM+ and PESQ etc.. ITU-T P.862 20
（ PESQ ） is the method currently recommended by ITU to test end-to-end
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 16，Total 45 pages
Secret▲ Secret▲ network voice quality. It would be ideal if MOS and PESQ are simultaneously supported in the QOS test. Form 1 Definition of MOS Values Level
MOS
Excellent
5
Good
4
Fair
3
Poor
2
Bad
1
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 17，Total 45 pages
Secret▲ Secret▲
4 PS Radio Service Statistical KPI 4.1 Accessibility 4.1.1 DL TBF establishment success rate KPI KPI
DL TBF establishment success rate Suggested value
95%
value
reference
Risk value
96%
Indicator type
Statistical value
Indicator definition
（GPRS DL signalling TBF link setup times+ GPRS DL data TBF link setup times +EGPRS DL signalling TBF link setup times +EGPRS DL data TBF link setup times）*100%/ DL packet channel request times（V3） (C70043+C70057+C70071)*100/C70041
V2
(C100010007+C100010015+C100010008+C100010016) *100%/(C100010141+C100010142+C100010143+
V6.0
C100010144+C100010145+C100010146+C100010147
Counter formula
+C100010148)
V3
(C900040007+C900040015+C900040008+C900040016) *100%/(C900040141+C900040142+C900040143+
V6.1
C900040144+C900040145+C900040146+C900040147 +C900040148)
This KPI reflects the request fulfilled status of PS downlink resources. Its TBF includes DL signaling TBF、GPRS DL data TBF and EGPRS DL data TBF. DL TBF link setup success rate can reflect the DL PS quality of a cell. Its major influencing factors are as follows: Notes
1.
Interference: internal & external interference;
2.
Parameters setting: eg. errors in Timer and functional parameters
3.
Stability of equipment and transmission: eg. instant breakdown(flash) resulted from high error rate or unstable link;
4.
Usability of data service functions.
4.1.2 UL TBF establishment success rate KPI
UL TBF establishment success rate
KPI
Suggested
reference
value
value
Risk value
Indicator type
92% 95%
Statistical value
Indicator
（GPRS UL signalling TBF link setup times + GPRS UL data TBF link setup times
definition
+EGPRS UL signalling TBF link setup times +EGPRS UL data TBF link setup times）
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 18，Total 45 pages
Secret▲ Secret▲ *100/ UL packet channel request times（V3） V2
(C70042+C70055+C70069)*100%/C70040 (C100010025+C100010033+C100010026+C100010034) V6.0
Counter formula
*100%
/(C100010159
+C100010160+C100010161+C100010168
+C100010163+C100010164+ C100010165+C100010166) V3
(C900040025+C900040033 +C900040026 +C900040034) V6.1
*100%/
(C900040159+
C900040160
+C900040161+C900040168+C900040163
+C900040164
+C900040165+C900040166) This KPI reflects the request fulfilled status of PS uplink resources. Its TBF includes UL signaling TBF、GPRS UL data TBF and EGPRS UL data TBF. UL TBF link setup success rate can reflect the UL PS quality of a cell. Its major influencing factors are as follows: Notes
1.
Interference: internal & external interference;
2.
Parameters setting: eg. errors in Timer and functional parameters
3.
Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link;
4.
Usability of data service functions.
4.1.3 GPRS RLC KPI
UL GPRS RLC retransmission rate
KPI
Suggested
reference
value
value
Risk value
10% 5%
Indicator type
Statistical value
Indicator
GPRS UL TBF NACK blocks *100% / valid data statistics received by GPRS UL TBF
definition V2 Counter formula
C70066*100%/ C70065 V6.0
V3 V6.1
C100010063*100%/C100010062 C900040063*100%/C900040062
This KPI reveals UL GPRS RLC retransmission rate, which is the most important indicator to reflect the sensibility of GPRS UL service and to show whether instability、 congestion and interference exist in the system. Its major influencing factors are as Notes
follows: 1.
Interference: internal & external interference;
2.
Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link;
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 19，Total 45 pages
Secret▲ Secret▲ 4.1.4 GPRS RLC KPI
DL GPRS RLC retransmission rate
KPI reference
Suggested
value
value
10%
Risk value Indicatory
5%
Statistical value
type Indicator
GPRS DL TBF NACK blocks *100% / valid data statistics received by GPRS DL TBF
definition V2 Counter formula
C70068*100%/ C70067 C100010065*100%/C100010064
V6.0 V3
C900040065*100%/C900040064
V6.1
This KPI reveals DL GPRS RLC retransmission rate, which is the most important indicator to reflect the sensibility of GPRS DL service and to show whether instability、 congestion and interference exist in the system. Its major influencing factors are as 解读 Notes
follows: 1.
Interference: internal & external interference;
2.
Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link;
4.1.5 EGPRS RLC KPI KPI
UL EGPRS RLC retransmission rate Suggested
reference
value
value
Risk value
Indicatory
Statistical value
10% 5%
type Indicator
EGPRS UL TBF NACK blocks *100% / valid data statistics received by EGPRS UL TBF
definition V2 Counter formula
C70080*100%/ C70079 V6.0
V3 V6.1
解读 Notes
C100010071*100%/C100010070 C900040071*100%/C900040070
Refer to UL GPRS RLC retransmission rate
4.1.6 EGPRS RLC KPI
DL EGPRS RLC retransmission rate
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 20，Total 45 pages
Secret▲ Secret▲ KPI reference value Indicatory
Suggested
10%
value Risk value
5%
Statistical value
type Indicator
EGPRS DL TBF NACK blocks *100% / valid data statistics received by EGPRS DL TBF
definition V2 Counter formula
C70082*100%/ C70081 V6.0
V3 V6.1
Notes
C100010073*100%/C100010072 C900040073*100%/C900040072
Refer to DL GPRS RLC retransmission rate
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 21，Total 45 pages
Secret▲ Secret▲
5 PS Radio Service Test KPI 5.1 Accessibility 5.1.1 Attach success rate KPI
Attach success rate
KPI reference value
Suggested
95%
value Risk value
98%
Attach success times/Attach attempt times×100％
Indicator definition Indicatory
test
type Counter
None
formula Notes
Normally, this KPI is the CQT value. Attach success means that the mobile phone receives the Attach Accept signaling within 15 seconds after it sends the Attach Request. It is required to perform the Attach test ten times at each test site.
5.1.2 Average GPRS attach time KPI KPI
Average GPRS attach time reference
value
Suggested value
2 sec
Risk value
1.5 sec
Sum of attach success duration/ Attach success times. It is the Indicator
duration from the mobile sends out the first Attach Request to it
definition
receives the Attach Accept signaling. It is required to perform the test ten times at each test site.
Indicatory type
Test value
Counter
None
formula Notes
Normally, this KPI is the same as the CQT value. Authentication included.
5
5.1.3 PDP context activation success rate KPI KPI value
PDP context activation success rate reference
Suggested
95%
value
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 22，Total 45 pages
Secret▲ Secret▲ Risk value
98%
Indicator
PDP activation success times/attempt times ×100％.
definition
PDP activation success means that the mobile phone receives the Activate PDP Context Accept signaling 15 seconds after it sends the Activate PDP Context Request.
Indicatory type
Test
Counter
None
formula Notes
Normally, this KPI is the same as the CQT value. It is required to perform the PDP activation test ten times at each test site. Test methods for GPRS and EDGE are the same.
5.1.4 PDP context activation time KPI
PDP context activation time
KPI
reference
Suggested value
1.5 sec
value
Risk value
1.2 sec
Indicator
Sum of PDP activation duration/ PDP activation success
definition
times. PDP activation time means the duration from the mobile phone sends out the first Activate PDP Context Request to it receives the Activate PDP Context Accept.
Indicatory type
Test value
Counter
None
formula Notes
Normally, this KPI is the same as the CQT value. Authentication included. GPRS and EDGE are not separated in this test.
5.1.5 PING test success rate KPI KPI
PING test success rate reference
Suggested value
95%
value
Risk value
98%
Indicator
Ping success times/Ping attempt times*100%
definition Counter
None
formula Notes
It is regarded as failure if the time exceeds 30 seconds.
5.1.6 Average PING test time KPI KPI value
PING test time reference
GPRS
Suggested value
1.5 sec
Risk value
1.4 sec
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 23，Total 45 pages
Secret▲ Secret▲
EDGE Indicator
Suggested value
1 sec
Risk value
0.6 sec
Sum of Ping success duration/ Ping success times
definition Indicatory type
Test value
Counter
None
formula Notes
Authentication is included in this KPI. The Ping package size will have influence on the test result. Generally, the smaller the Ping package is, the better the test result will be. It is sometimes referred as Round Trip Time.
5.2 Mobility 5.2.1 Routing Area Update Successful rate KPI
Routing Area Update Success rate
KPI reference Indicator
Suggested value
90%
Risk value
95%
Routing update attempt times*100% / Routing update success times
definition Indicatory type
test
Counter
None
formula Notes
Usually it can be statistical value. It also can be obtained from DT too.
5.3 Data Throughput 5.3.1 Average FTP download rate KPI KPI
FTP download rate GPRS
reference value
EDGE
Suggested value
20kbps
Risk value
30kbps
Suggested value
80kbps
Risk value
120kbps
Indicator definition
Actual download data quantity (byte)/actual download duration
Indicatory type
Test value
Counter formula
None
Notes
Normally, this KPI is the CQT value. Class 4 mobile. Size of the downloaded document will affect the test result. Usually, the smaller the document size is, the better the test result will be. It is related to the number of PS channels、terminal type、encoding method and test software, etc.. The test result can also be affected
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 24，Total 45 pages
Secret▲ Secret▲ by the adoption of encoding method for CS3 and CS4. 3static time slots are configured.
5.3.2 Average FTP upload rate KPI
FTP upload rate
KPI reference value
GPRS
EDGE
Suggested value
8kbps
Risk value
10kbps
Suggested value
25kbps
Risk value
35kbps
Indicator definition
Actual upload data quantity (byte)/actual upload duration
Indicatory type
Test value
Counter formula
None
Notes
Normally, this KPI is the same as the CQT value. Class 4 mobile. Size of the uploaded document will affect the test result. Usually, the smaller the document size is, the better the test result will be. It is related to the number of PS channels、terminal type、encoding method and test software, etc.. The test result can also be affected by the adoption of encoding method for CS3 and CS4. This KPI is the result of single time slot.
5.3.3 Throughput Mobile phone with multi-timeslot Level 8; single Preconditions
subscriber; 4 DL static signaling channels, 1 UL static signaling channel
Indicators (EN)
Indicators (CN)
Suggested value
GPRS: Max data throughput-DL
GPRS DL maximum transmission rate
9.8kbyte/s
GPRS: Max data throughput-UL
GPRS UL maximum transmission rate
2.4kbyte/s
GPRS: Average data throughput-DL
GPRS DL average transmission rate
5kbyte/s
GPRS: Average data throughput-UL
GPRS UL average transmission rate
1.3kbyte/s
EDGE: Max data throughput-DL
EDGE DL maximum transmission rate
26.5kbyte/s
EDGE: Max data throughput-UL
EDGE UL maximum transmission rate
7kbyte/s
EDGE: Average data throughput-DL
EDGE DL average transmission rate
14kbyte/s
EDGE: Average data throughput-UL
EDGE UL average transmission rate
3.5kbyte/s
5.4 Reliability 5.4.1 PDTCH Drop Rate KPI KPI
PDTCH Drop Rate reference
Suggested value
5%
value
Risk value
2%
Indicatory type
DT test value
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 25，Total 45 pages
Secret▲ Secret▲ Indicator
PDTCH drop times/ FTP download attempt times *100%
definition Counter
None
formula Notes
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 26，Total 45 pages
Secret▲ Secret▲
6 Introduction to CN Statistical KPI 6.1 Paging success rate KPI
Paging success rate
KPI
reference
Suggested value
90%
value
Risk value
93%
Indicator
Paging response times*100% / Paging request times
definition Counter
Paging request times refer to the sum of PAGING information sent out by all
formula
MSCs in the area, excluding the secondary paging information. Paging response times refer to the sum of the PAGING RES received by all MSCs in the area, including the second paging response. Statistical point is at MSC side.
Notes
BSC can indirectly provide this KPI, but its statistical point differs from that of MSC. The difference lies in call loss at Interface A.
6.2 Location update success rate KPI KPI
Location update success rate reference
Provided by the SMC side
value Indicator
Location update success times *100%/ location update request times
definition Counter
Provided by the SMC side
formula Notes
This KPI can be indirectly provided by BSC, but the value is different and its statistical point differs from that of MSC too.
6.3 Attach success rate KPI
Attach success rate
KPI reference
Suggested value
90%
value
Risk value
95%
Indicator
GPRS attach success times *100% / GPRS attach times
definition Indicatory
GPRS attach success times ： SGSN delivers （ATTACH ACCEPT，
type
3GPP TS 24.008），excluding redeliver
Counter
Statistics at SGSN side
formula Notes
excluding Unregistered subs
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 27，Total 45 pages
Secret▲ Secret▲
6.4 GPRS average attach time KPI KPI
GPRS average attach time Suggested value
To be confirmed
value
reference
Risk value
To be confirmed
Indicator
Customer attach establishment duration *100% / customer
definition
number of attach establishment success
Indicatory type
Statistics
Counter
Attach establishment time refers to the time duration from the
formula
receipt of GPRS ATTACH REQUEST by MS to the delivery of ATTACH ACCEPT by SGSN. If the attach time exceeds the timer, the attach duration is only computed within the attach accepted period.
Notes
6.5 PDP context activation success rate KPI KPI
PDP context activation success rate reference
value
Suggested value
80%
Risk value
85%
success times of PDP context activation started by MS*100%/ attempt times of PDP context activation started by MS Indicator
The attempt times of PDP context activation started by MS is the times
definition
GGSN received Create PDP Context Request from SGSN. The success times of PDP context activation started by MS is the statistic of Create PDP Context Response sent from GGSN to SGSN when Cause value is Request Accepted.
Indicatory type Counter
Test or statistics Statistics at GGSN side
formula Notes
The performance evaluation in China Mobile in 2003 (success times of PDP context activation started by GPRS terminal/ attempt times of PDP context activation started by GPRS) results are as follows: Excellency index: 94% Reach-the-mark index: 90% Lowest Threshold index: 85%
6.6 Routing Area Update Success rate KPI KPI
Routing Area Update Success rate reference
Suggested value
92%
value
Risk value
95%
Indicator
Routing update attempt times*100%/ routing update success times
definition
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 28，Total 45 pages
Secret▲ Secret▲ Indicatory type Counter formula Notes
From statistical value or DT. Statistics at SGSN side, provided by GPRS core network Update success rate at routing area crossing SGSN is lower than that at the SGSN routing area.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 29，Total 45 pages
Secret▲ Secret▲
7 Comments on KPI Cases 7.1 Pengshui (Chongqing) PS Test Result 1、 CQT scene，near cell 2、 EDGE Commissioned 5
3、 Multi-timeslot capability3+1 4、 Number of PDCH static channels: 3
Test
Test state
FTP type
serial
FTP
size
（Kbyte）
Download
Download
speed (Kbyte/s)
speed (kbps)
Average timeslot speed （kbps）
Pre-optimization
1
Download
102.381
8.99
71.93
23.98
Pre-optimization
2
Download
48.18
8.10
64.80
21.60
Pre-optimization
3
Download
204.8
2.97
23.78
7.93
Pre-optimization
4
Download
2188.817
7.21
57.72
19.24
Pre-optimization
5
Download
102.381
9.05
72.42
24.14
Pre-optimization
6
Download
102.381
8.66
69.29
23.10
Pre-optimization
7
Download
512.304
15.97
127.73
42.58
Pre-optimization
8
Download
512.304
13.34
106.69
35.56
Pre-optimization
9
Download
512.304
15.81
126.51
42.17
Pre-optimization
10
Download
512.304
15.74
125.88
41.96
Pre-optimization
11
Download
512.304
14.33
114.67
38.22
Ping Ping
package
interval
size
Ping timelag（ms） 1
2
3
839
634
612
4
32
61 2
5
32
816
590
581
2
100
571
650
697
2
500
839
955
5
500
862
760
77 9 75 6
107 5
82 1
857
75
Ping
timelag
success
（ms）
rate
8
9
10
566
765
561
761
655
100%
733
716
692
812
609
709
100%
753
651
711
729
707
688
691
100%
779
737
918
894
875
840
873
100%
672
811
690
811
709
825
775
100%
5
6
608
589
757
7
（byte） 0
Average
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 30，Total 45 pages
Secret▲ Secret▲ 5 2
800
2
1000
2
1200
959
910
115
115
7
880
8 107
8
869
840
897
86
845
9 90 7
102 8
95 7
101 7
823 918 959
781
883
105
881
4 119 1
846
108
103
2
102 0
860
7 101
8
104 2
865
100%
1010
100%
1002
100%
Comments on case: 1. When C/I is in good condition,
the EDGE download speed of current
equipment can reach above 40kbps. 5
2. The Ping success rate and Ping timelag can vary very much due to different Ping intervals、 different Ping package sizes and different Ping servers.
7.2 Jishou (Hunan) China Mobile EDGE Test Result Test Specification Test equipment：ANT Pilot for E-GPRS； 10
Test phone SEGAM OT490(one set) Test laptop: memory 1024M、harddisk 80G、CPU 1.60Ghz、Windows XPSP2 、 standard 28800bps modem 、 rate at 115200bps 、 modem error control and compression forbidden 、 no software irrelevant to the test.
15
Test method ： outdoor static FTP download test Test location ： 11 th
th
sites at Baoqing market in Fenghuang county Test time ： 19 —30 , June 2008 Test aim ： to test E-FPRS network performance under
static state
Test aim：to download 1000KB document 20
Timeslot ：4 static timeslots
Test Result This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 31，Total 45 pages
Secret▲ Secret▲ Test Branch
Test sites
office
Test
document
electrical
size
level
（KB）
（dBm）
RLC Download times
Test location
RLC
average
average
rate
rate
（Kbps）
（Kbps）
Fenghuang County Jishou
prefectural government office building
1000
-44
2
Gate of Prefectural government office
201.19
181.3
1000
-40
2
Lobby of MingDu Hotel
192.6
198.98
1000
-51
2
Opposite MingDu Hotel
197.09
206.26
197.74
201.79
207.29
193.91
198.83
196.38
Cell 1 Fenghuang County Jishou
prefectural government office building Cell 2 Fenghuang County
Jishou
prefectural government office building Cell 3
Basketball field
Jishou
Old district Cell 1
1000
-49
2
Jishou
Old district Cell 2
1000
-32
2
Jishou
Old district Cell 3
1000
-56
2
Jishou
Rainbow Bridge Cell 1
1000
-43
2
Opposite Wanmingta River
185.72
191.1
Jishou
Rainbow Bridge Cell 2
1000
-43
2
Opposite Wanmingta River
191.37
181.36
Jishou
TianXia Feng Huang Cell 1
1000
-34
2
197.46
186.55
Jishou
TianXia Feng Huang Cell 2
1000
-59
2
197.09
206.26
Jishou
BaoQing market Cell 1
1000
-38
2
Gate of Linlinlou Drug Store
207.85
198.02
Jishou
BaoQing market Cell 2
1000
-36
2
Gate of Linlinlou Drug Store
162.16
185.34
Jishou
BaoQing market Cell 3
1000
-46
2
Gate of Linlinlou Drug Store
180.51
180.09
Jishou
Xinhua Bookstore Cell 1
1000
-50
2
Lobby of Xinhua Hotel
195.81
191.5
Jishou
Xinhua Bookstore Cell 2
1000
-53
2
Lobby of Xinhua Hotel
198.02
188.38
Jishou
Xinhua Bookstore Cell 3
1000
-45
2
Parking lot of Xinhua Hotel
183.08
190.16
1000
-37
2
Lane, west to ChinaMobile business hall
178.96
157.44
1000
-45
2
Gate of XingGuang Hotel
185.46
187.98
1000
-60
2
Gate of XingGuang Hotel
184.07
184.45
1000
-50
2
Gate of XingGuang Hotel
193.98
184.51
1000
-62
2
Gate of XingGuang Hotel
176.22
185.21
Jishou
Jishou
Jishou
Jishou Jishou
Comprehensive Building Cell 1 Comprehensive Building Cell 2 Comprehensive Building Cell 2 Comprehensive Building Cell 1800M1 Comprehensive Building
of Jiandaoping Primary School Basketball field of Jiandaoping Primary School Kindergarten next to Jiandaoping Primary School
Square in front of TianXiaFengHuang Hotel Square in front of TianXiaFengHuang Hotel
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 32，Total 45 pages
Secret▲ Secret▲ Cell 1800M2 Jishou
Telecom Bureau Cell 1
1000
-51
2
Business Hall of Telecom Bureau
180.75
197.88
Jishou
Telecom Bureau Cell 2
1000
-64
2
Business Hall of Telecom Bureau
171.85
199.19
Jishou
Telecom Bureau Cell 3
1000
-57
2
Backdoor of Supermarket (next to Telecom Bureau)
182.7
179.49
Jishou
Telecom Bureau Cell 1800M1
1000
-59
2
Gate of furniture store (next to Telecom Bureau)
105.44
106.48
Jishou
Telecom Bureau Cell 1800M2
1000
-73
2
Business Hall of Telecom Bureau
181.85
185.85
Jishou
Telecom Bureau Cell 1800M3
1000
-84
2
Business Hall of Telecom Bureau
159.20
166.92
Comments on case: 1. When C/I is in good condition, the average EDGE download speed can reach above 180kbps with a peak value of above 200kbps; 2. This test is the CQT. Cell reselection and RAC renewal can be neglected, that 5
is they are not necessary to the test conditions.
7.3 Acceptance requirements for EDGE KPI of Data Service of a network Serial 1
Indicators
Explanation
GPRS/EGPRS
GPRS/EGPRS
Formula Attach
Target
success
GPRS/EGPRS attach success
attach
means
phone
rate ＝ GPRS Attach success
success rate
receives the Attach Accept signaling
times/attach attempt times ×
15 seconds after it sends the Attach
100％
that
the
mobile
>95%
Request. 2
Average
Attach
time
attach time
duration
from
sends
out
until
it
means the
the mobile
Attach
Request
receives
Average attach time ＝ sum of attach
success
95%
signaling 15 seconds after it sends the Activate PDP Context Request. 4
PDP
average
PDP average activation time
PDP average activation time
activation
means the duration from the
＝ sum
time
delivery
of
PDP
success
Context
Request
to
the
activation success times
receipt
of
Activate
PDP
Activate
of
PDP
95%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 33，Total 45 pages
Secret▲ Secret▲ log-in
means
that
the
homepage
rate ＝ WAP
website
log-in
success rate
fully and correctly displays
success times/ website log-
within 60seconds after the
in attempt times×100％。
mobile phone started the WAP website log-in request. 6
average
WAP
WAP homepage displaying time
average
WAP
homepage
refers to the duration from
displaying
time ＝ sum
displaying
the
log-in
successful
time
request is sent out until
displaying
the
homepage displaying success
WAP
website
homepage
fully
and
correctly displays. 7
WAP
webpage
means
success rate
webpage
that can
95%
success rate ＝ WAP webpage refresh
success
webpage
refresh
times
/
attempt
times×100％。 8
Ping
success
Ping success rate ＝ ping
rate
success
times/ping
>95%
attempt
times×100 9
Ping
average
Ping average timelag＝sum of
timelag
each
Ping
80kbps
quantity download
time(sec)×9 11
FTP
upload
FTP
speed
upload
speed ＝ actual
>20kbps
upload data quantity（Byte） /actual upload time(sec) ×9
Comments on case: 1. Requirements of operation and maintenance KPI are not as strict as those of acceptance check KPI. 2. Usually these KPI requirements can be satisfied.
5
7.4 Operation & Maintenance KPI of a Network SN
Indicator
Unit
Target Level
Permissible
Remark
Level 1
SDCCH Ratio
Drop
%
1
1.2
Base on system performance report of OMCR, Indication Definition: SDCCH Drop Ratio=SDCCH call drops/ Access
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 34，Total 45 pages
Secret▲ Secret▲ attempt Number 2
TCH Drop Ratio
%
2
2.2
Base on system performance report of OMCR, Indication Definition: TCH Drop Ratio = TCH call drops/ Seized TCHs (include handover )
3
Call
Success
%
98
97
Base on system performance report of
Ratio
OMCR,
Indication
Definition:
Success Rate = 1 － TCH
Call
assignment
failure rate 4
SDCCH
%
0.5
0.75
Base on system performance report of
Blocking Ratio
OMCR, Indication Definition: SDCCH Blocking
Ratio=SDCCH
overflows/
SDCCH call attempts 5
TCH
%
2
2.5
Base on system performance report of
Blocking Ratio
OMCR,
Indication
Definition:
TCH
Blocking Ratio=TCH overflows (include handover)/ TCH call attempts (include handover) 6
TCH
%
98
97
Base on system performance report of
Assignment
OMCR,
Indication
Definition:
TCH
Success Ratio
assignment Success Ratio=Number of TCH assignment successful / Number of TCH call attempts (include handover)
7
GPRS
average
Kbps
28
25
24-Hour measurement
Kbps
45
40
24-Hour measurement
%
99
97
24-Hour measurement
%
94
93
%
< 2 for 98%
< 2 for 90%
of all GSM
of all GSM
cells
cells
%
97
90
Depend on the DT route and method
%
2
3
Depend on the DT route and method
DL throughout per TBF 8
EDGE
average
DL throughout per TBF 9
Attach Success
Rate
(GPRS/ EDGE) 10
Handover Success Ratio
11
Call Blocking Ratio
12
Call
Setup
Success
Based on OMC-R statistics
Rate
(Drive Test) 13
Call Rate
Drop (Drive
Test) 14
Speech
MOS
>=
3
for
>=
3
for
Depend on the DT route and method
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 35，Total 45 pages
Secret▲ Secret▲ Quality
(Drive
95%
Test)
15
Coverage
dBm
(Drive Test)
16
of
of
measurement
measurement
samples
samples
>= -85 for 95%
UL
90%
of
>= -85 for 90%
Depend on the DT route and method
of
measurement
measurement
samples
samples
Kbps
>10
>10
Based on DT
Kbps
>30
>30
Based on DT
>99.999
>99.999
per BSC/RNC / week
Throughput 17
DL Throughput
18
BSC/RNC down
%
time
(< 6 sec)
(< 6 sec)
(unscheduled) 19
TRX/channel
%
element failure
< 1 of all active TRX
< 2 of all
per BSC/RNC / week
active TRX
(unscheduled) 20
SDCCH Mean
s
99
per BSC/RNC / BH of a week
Holding
Time 21
Radio Access Success Ratio
22
Worst
Cells
Less
GOS
than
Less
than
1.5% cells of
1.5% cells of
Network with
Network with
GOS 5% or
GOS 5% or
above
above
per BSC/RNC / BH of a week
23
RXQUAL 0 to 3
%
>97
>95
per BSC/RNC / BH of a week
24
AMR
%
90%
>97%
>99%
per city / monthly
pre-defined locations and Drive test route/NMS
95%
>98%
>99%
per city / BH of a week (drive test monthly)
3
Paging Success Rate
NMS
>90%
>95%
>97%
per city / BH of week. With condition that Core Network is achived the same value of Level 1 and Level 2
4
Call blocking rate
pre-defined locations and Drive test route/NMS
25(95%)
Pre Defined Drive Test Route
99%
>99%
>99.5%
per BSC/RNC / BH of a week
15
SDCCH Blocking Rate
Pre Defined Drive Test Route/NMS
97% : Inter Cell: >97%
20
RXQUAL 0 to 3
Pre Defined Drive Test Route/NMS
>80%
>85%
>90%
21
AMR voice FER
Pre Defined Drive Test Route/NMS
98%
>98%
per city / BH of a week
NMS
>98%
>98%
>98%
per city / BH of a week
Pre Defined Drive Test Route/NMS
>95%
>98%
>98%
per city / BH of a week
23
24
25
Successful PS Attachment Successful PDP context activation Data session Setup Success Rate (PDP context already exist)
20 Kbps
Pre-defined locations
GPRS (Class 4 MS): >30Kb
GPRS (Class 4 MS): > 9Kbps;
GPRS (Class 4 MS): > 40 Kbps
GPRS (Class 4 MS):> 30Kbps
GPRS (Class 4 MS): > 9Kbps;
per city / BH of a month
per city / BH of a month
GPRS (Class 4 MS): > 40 Kbps
per city / BH of a month
GPRS (Class 4 MS): > 20 Kbps;
GPRS (Class 4 MS): > 20 Kbps;
per city / BH of a month
N/A
> 150 Kbps
> 150 Kbps
per city / BH of a month
NMS
View more...
Comments
