TD-LTE KPI Optimization Guidelines for RL15TD_V1.1
Short Description
Download TD-LTE KPI Optimization Guidelines for RL15TD_V1.1...
Description
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Nokia Siemens Networks TD-LTE KPI Optimization Guidelines for RL15TD
Copyright © Nokia Siemens Networks. This material, including documentation and any related computer programs, is protected by copyright controlled by Nokia Siemens Networks. All rights are reserved. Copying, including reproducing, storing, adapting or translating, any or all of this material requires the prior written consent of Nokia Siemens Networks. This material also contains confidential information which may not be disclosed to others without the prior written consent of Nokia Siemens Networks.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 1 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Revision History Date
Revision
2011-12-1
V1.0
Revison History Draft version
Author Zhengwei Gao
Add Chapter 1 introducing main purpose of the document to avoid any ambiguity and misunderstanding. 2012-1-12
V1.1
Add hyperlinks of reference materials to Chapter 6 so that people can find them more easily.
Zhengwei Gao
Other minor changes.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 2 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Table of Contents TD-LTE KPI Optimization Guidelines for RL15TD ........................................................................................ 1 Revision History ............................................................................................................................................ 2 Table of Contents .......................................................................................................................................... 3 Glossary ........................................................................................................................................................ 5 List of Figures ................................................................................................................................................ 6 List of Tables ................................................................................................................................................. 7 1
Introduction ............................................................................................................................................ 8
2
KPI Overview ......................................................................................................................................... 8 2.1
3
KPI Categories .............................................................................................................................. 8
2.1.1
Accessibility ............................................................................................................................... 8
2.1.2
Retainability (Reliability) ............................................................................................................ 9
2.1.3
Throughput ................................................................................................................................ 9
2.1.4
Latency ...................................................................................................................................... 9
2.1.5
Mobility .................................................................................................................................... 10
2.2
Services Levels ........................................................................................................................... 10
2.3
Field and Network KPIs ............................................................................................................... 11
Field KPIs ............................................................................................................................................. 11 3.1
Field KPI Definitions .................................................................................................................... 11
3.1.1
(FTP) Service Accessibility Ratio ............................................................................................ 12
3.1.2
(FTP) Completed Session Ratio ............................................................................................. 13
3.1.3
(FTP) Single User Throughput ................................................................................................ 14
3.1.4
Attach Time ............................................................................................................................. 16
3.1.5
Attach Success Rate ............................................................................................................... 18
3.1.6
Service Request (EPS) Time, UE Initiated ............................................................................. 19
3.1.7
Service Request (EPS) Success Rate .................................................................................... 22
3.1.8
Service (EPS Bearer) Drop Rate ............................................................................................ 23
3.1.9
Handover Procedure Time ...................................................................................................... 24
3.1.10
Handover Success Rate ..................................................................................................... 25
3.1.11
(LTE) Round Trip Time ....................................................................................................... 26
3.1.12
(LTE) Single User Throughput ............................................................................................ 27
3.1.13
(LTE) Service Interrupt Time (HO) ...................................................................................... 28
3.1.14
(RB) Single User Throughput .............................................................................................. 29
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 3 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
3.1.15 3.2 4
5
Cell Throughput ................................................................................................................... 30
Test Conditions and Target Values ............................................................................................. 31
Network KPIs ....................................................................................................................................... 32 4.1
Network KPI Definitions .............................................................................................................. 32
4.2
Test Conditions and Target Values ............................................................................................. 33
Appendix I: Trial Network KPIs ............................................................................................................ 34 5.1
CMCC Large Scale Trial ............................................................................................................. 34
5.1.1
Test Environment and Baseline Configuration ........................................................................ 34
5.1.2
Field Results ............................................................................................................................ 35
5.2
Bharti Airtel Trial .......................................................................................................................... 39
5.2.1
Test Environment and Baseline Configuration ........................................................................ 39
5.2.2
Field Results ............................................................................................................................ 40
5.3
Maxis Aircel Trial ......................................................................................................................... 42
5.3.1
Test Environment and Baseline Configuration ........................................................................ 42
5.3.2
Field Results ............................................................................................................................ 44
5.4
Voentelecom Russia ................................................................................................................... 46
5.4.1
Test Environment and Baseline Configuration ........................................................................ 46
5.4.2
Field Results ............................................................................................................................ 47
5.5
6
Version: 1.1
Appendix II: Early Network KPIs ................................................................................................. 49
5.5.1
STC Saudi ............................................................................................................................... 49
5.5.2
Sky Brazil ................................................................................................................................ 49
References ........................................................................................................................................... 49
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 4 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Glossary IBLER
Copyright © Nokia Siemens Networks 2011
Initial BLock Error Rate
Company Confidential
Page 5 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
List of Figures Figure 1 LTE Bearer Service Architecture .................................................................................................. 10 Figure 2 Attach Procedure with Initial EPS Bearer Establishment (3GPP 23.401) .................................... 18 Figure 3 EPS Bearer Setup Procedure, Part I ............................................................................................ 21 Figure 4 EPS Bearer Setup Procedure, Part II ........................................................................................... 22 Figure 5 Test Environment of CMCC Large Scale Trial in Hangzhou ........................................................ 34 Figure 6 Test Environment of Bharti Airtel TD-LTE Trial ............................................................................ 39 Figure 7 Test Environment of Maxis Aircel TD-LTE Trial ........................................................................... 43
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 6 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
List of Tables Table 1 List of Field KPIs ............................................................................................................................ 12 Table 2 List of Network KPIs ....................................................................................................................... 33 Table 3 Baseline Configuration of CMCC Large Scale Trial in Hangzhou ................................................. 34 Table 4 Field Results of CMCC Large Scale Trial in Hangzhou ................................................................. 35 Table 5 Baseline Configuration of Bharti Airtel TD-LTE Trial ..................................................................... 39 Table 6 Baseline Configuration of Maxis Aircel TD-LTE Trial..................................................................... 43 Table 7 Baseline Configuration of Voentelecom TD-LTE Trial ................................................................... 46
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 7 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
1 Introduction This document is focused on what kind of KPI values can be achieved in TD-LTE with certain network configurations by collecting KPIs from trial networks and commercial networks. It can be seen as a complementary to [3], in which the KPI targets mainly come from lab tests and/or system-level simulations. KPIs from this document can be taken as the baseline KPIs which are useful for KPI negotiation with our customers and for KPI optimization. This document doesn‟t actually contain any approaches or methods which can be used in network optimization phase. Please refer to [5] for optimization guidelines for TDDFDD- LTE which is a counterpart of the FDD. Unfortunately, there is no official version for TDD-LTE version [6]yet. KPIs in this document are specified for RL15TD. This document is organized as follow: Chapter 1: This chapter gives an introduction to main purpose of this document. Chapter 2: This chapter gives an overview of KPIs in LTE, including KPI categories, KPI hierarchies and general information about field KPI and network KPI. Chapter 3: This chapter firstly introduces which kinds of field KPIs are interested and their definitions, and the second part summarizes field KPI baselines and conditions to achieve the expected KPIs. Chapter 4: This chapter firstly introduces which kinds of network KPIs are interested and their definitions, and the second part summarizes network KPI baselines and conditions to achieve the expected KPIs. Chapter 5: This chapter compiles achieved KPIs from global TD-LTE trial networks and early networks. Chapter 6: This chapter contains a list of useful reference materials.
2 KPI Overview Key performance indicators (KPIs) are a set of selected indicators used for measuring the network performance which determine the quality of a network. They are used during RFI/ RFQ phases and during the network/cluster acceptance process. Additionally, network KPIs can be used to monitor a network continuously and warn in time about potential problems.
2.1 KPI Categories There are several KPIs categories defined, each one looking at a different performance aspect. The names indicate the respective category according to 3GPP 32.410 and 32.450. The names in brackets indicate the terminology commonly used if different from 3GPP.
2.1.1 Accessibility Accessibility KPIs characterize the availability of a service, or service element to its users. They are expressed as a ratio of successful service requests to the total number of attempts. E.g. Call success ratio.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 8 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Service request attempts, which are rejected due to insufficient access rights, bad user authentication, or errors in request parameters are not considered and should be taken out of the samples used for calculating the metric.
2.1.2 Retainability (Reliability) Reliability KPIs can be referred to Service Drop Rates or to Transport Error Rates, that is, the ratio between erroneous or lost data units and the overall number of data units sent. The Service Drop Rates KPIs (i.e. drop rates) can be measured in the field with an FMT (Field Measurement Tool). They represent the ratio between unsuccessful („abnormal‟) terminations of a service request (e.g. network attach, register, service request) to the overall established services. For certain services, like bearer creation or VoIP call the service drop rate is also related to the duration of the session. This is motivated by the fact that the possibility of network errors, e.g. hand-over failures is higher if the service duration is longer. In this case, the drop ratio is expressed in [1/s] units instead of [%]. As always-on connectivity is the basic idea for LTE, traditional CDR KPIs may cause abnormal results. CDR per Data Volume has been proposed to solve this problem.
2.1.3 Throughput Throughput KPIs show the data rates that can be provided to the users of a cell under different radio conditions. The location of the UE in the cell and the number of active UEs in that cell will strongly influence the data rate that can be experienced by one user. Peak/Average cell throughputs and peak user throughputs are the metrics that characterize the capacity of an LTE network, and the average UL/DL user throughputs along a drive route (i.e. under different radio conditions) represent the end user experience. However, only cell throughput and peak user throughput metrics will be covered in this document.
2.1.4 Latency Latency KPIs in this document are referred to RTT (Round Trip Time) measurements and to signalling procedures (e.g. setup times) RTT for data applications is measured with the Ping application of the UEs operating system or with a comparable measurement tool. It records the time difference between sending an ICMP Echo Request to an IP host and the reception of the corresponding ICMP Echo Reply message. The measurement can be executed between a mobile terminal and an IP host, which is configured to respond to ICMP Echo Requests. The test is to be done for stationary and mobile end users. The measurement with stationary UE should be performed under average radio link conditions with normal SINR. As mentioned above, the latency category also refers to the time for signalling procedures on the network or on application level (e.g. RAB service setup, VoIP call setup flow). In case of measuring latencies over the radio link, the time needed to attach the mobile to the network and to setup the necessary radio resources is a significant component.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 9 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
2.1.5 Mobility Mobility KPIs build a group because all of them are related to hand-over procedures. They could, however, be discussed in the Latency and Accessibility groups as well, because HO-related KPIs are either service interrupt time or success ratio type KPIs. A successful handover is understood to be any successfully performed make before break procedure and it is marked by an entry in the log, which carries the identifier of the new cell (destination).
2.2 Services Levels In addition to the different categories, KPIs are also defined for different service levels as per 3GPP 36.300. These are:
Application Services
LTE E2E Network Service
Radio Bearer (RB) Service
IP based LTE Bearer Services: X2, S1, S5/S8 and external
Figure 1 shows the LTE Bearer Service Architecture where the KPIs categories above can be mapped into.
Figure 1 LTE Bearer Service Architecture
When it comes to field performance evaluation the most interesting KPIs are the “LTE E2E Network Service” KPIs as they characterize the overall LTE network performance. Another important group is the
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 10 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Application Service KPIs as they show the user‟s perceived quality of application services as impacted by the LTE network. The KPIs of this group are service specific. The services of common interest for mobile network operators and therefore, the ones that can be of interest during an acceptance process are:
Web Browsing (HTTP)
Data Upload / Download (FTP)
Email (POP/SMTP)
Conversational Audio Video (VoIP)
The current document concentrates on the Data Upload / Download application service.
2.3 Field and Network KPIs Field KPIs are the scope of drive testing campaigns either in a trial network or in a live network. They are also called drive test KPIs and can be obtained with a FMT, whereas network KPIs are based on counters collected via NetAct. Drive test KPIs measurements are recommended in all types of projects (trial, roll-out, swap and optimization). They reflect the network performance along the different drive routes and somehow, reflect the end user experience. They can also be collected in static or indoor locations (e.g. airports, shopping centres, office buildings). However, drive testing campaigns are expensive, time consuming and cannot cover the whole network as the NetAct measurements do. Self-optimization related features in later releases, part of the SON concept, may have an impact on the amount and need of drive testing. Network KPIs do have statistical significance only if the traffic levels are high enough. This is a reason why, especially during the cluster acceptance process when there is not enough load in the network, the network KPIs may not reflect the network performance and the acceptance process should be based on field KPIs. With the network KPIs it is also unknown if the users causing bad statistical data samples are well inside the coverage area or at the coverage borders. However, network KPIs can be used to monitor networks permanently and highlight potential problems. Network KPIs are valid and recommended in swap and optimization cases where networks are supposed to be mature and with enough traffic to provide reliable results. In future swap cases (too early at the moment), network KPIs shouldn‟t be used to compare with other vendors KPIs as counters and counter triggering conditions are likely to be different.
3 Field KPIs 3.1 Field KPI Definitions The defined field KPI lists do not present by any means a complete set of KPIs to be used in a customer case. It is the project team/CBT responsibility to define and agree with the customer the final list of KPIs to be used in each and every case. The measurement procedure of each KPI is just informative, actual test procedure depends on customer‟s requirements and may quite different from what has been stated here. The most interested field KPIs are listed below:
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 11 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Table 1 List of Field KPIs
KPI Name Application Services
PS Data Services (FTP, HTTP etc)
Control Plane LTE E2E Network Service
User Plane Radio Bearer Services
User Plane
KPI Category
Service Accessibility Ratio [%]
Accessibility
Completed Session Ratio [%] Single User Throughput [Mbps] Attach Time [ms] Attach Success Rate [%] Service Request (EPS) Time [ms] Service Request (EPS) Success Rate [%]
Reliability Throughput Latency Accessibility Latency Accessibility
Service (EPS) Drop Rate [%] Handover Procedure Time [ms] Handover Success Rate [%] (LTE) Round Trip Time (RTT) [ms] (LTE) Single User Throughput [Mbps] (LTE) Service Interrupt Time (HO) [ms]
Reliability Latency Accessibility Latency Throughput Latency
(RB) Single User Throughput [Mbps] Cell Throughput [Mbps]
Throughput Throughput
Note that: „Single User Throughput‟ KPI can be specified on each protocol layer, such as application layer, IP layer, L2 layer (i.e. PDCP/RLC/MAC layer) and L1 layer (i.e. physical layer).
3.1.1 (FTP) Service Accessibility Ratio
(FTP) Service Accessibility Ratio [%] Definition
It denotes the probability that the user can establish the necessary bearer (EPS) and access the FTP service successfully. Service access covers starting the FTP client on the UE, setting up mobile access and creating a TCP connection to the FTP server.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 12 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Measurement methodology
Version: 1.1
1. Set the FMT to generate calls to the FTP Server automatically with the following script/sequence: a. Connection attempt (network attach) b. Log into the FTP Server c. FTP Download 800MB file (or other size with transfer duration >1min) d. Wait 3 sec. after session finishes e. FTP Upload 300MB file (or other size with transfer duration >1min) f. Log out the FTP Server If UE gets into UE deregistered status between calls, this test also valid to check the Attach Success Rate g. Connection release (network detach) h. Leave 5 seconds before start next session i. Repeat all previous steps (from „a‟) 2. Continue drive testing till the drive route is completed. 3. Stop the FMT at the end of the route. 4. If the route does not generate more than 100 calls repeat the measurement on the same measurement route. 5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during the measurement. 6. Verify the PS Data Call success rate as per formula below
Formula (logical)
Assumptions, pre-conditions
The FTP transfer shall be executed in binary mode. Preconditions: UE EMM-DEREGISTERED
Message flow, trigger points
Trigger points: Start: ftp get / put command issued on the UE. Stop: first data byte sent / received by the UE, or ftp indicates network error (e.g. timeout expired).
Related E2E KPIs
(FTP) Completion Session Ratio.
3.1.2 (FTP) Completed Session Ratio (FTP) Completed Session Ratio [%] Definition
The Completion Session Ratio is the proportion of completed FTP sessions and sessions that were started successfully. The session is not completed if a predefined timer expires, or a network failure occurred that could not be repaired by automatic restarts.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 13 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Measurement methodology
Version: 1.1
1. Set the FMT to generate calls to the FTP Server automatically with the following script/sequence: a. Connection attempt (network attach) b. Log into the FTP Server c. FTP Download 800MB file (or other size with transfer duration >1min) d. Wait 3 sec. after session finishes e. FTP Upload 300MB file (or other size with transfer duration >1min) f. Log out the FTP Server If UE gets into UE deregistered status between calls, this test also valid to check the Attach Success Rate g. Connection release (network detach) h. Leave 5 seconds before start next session i. Repeat all previous steps (from „a‟) 2. Continue drive testing till the drive route is completed. 3. Stop the FMT at the end of the route. 4. If the route does not generate more than 100 calls repeat the measurement on the same measurement route. 5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during the measurement. 6. Verify the Completion Session Ratio as per formula below
Assumptions, pre-conditions The FTP transfer shall be executed in binary mode. FTP Application DL Throughput as measured by the DOS ftp application. If above measurement methodology is followed the UE needs to attach to the network between FTP UL/DL sessions if the Attach Success Rate wants to be measured. Formula (logical)
Message flow, trigger points
Trigger points:
Related E2E KPIs
Start: ftp get / put command issued on the UE Stop: after error free execution of the command, ftp returns with a prompt and indicates the number of bytes transmitted.
(FTP) Service Accessibility Ratio.
3.1.3 (FTP) Single User Throughput (FTP) Single User Throughput [Mbps]
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 14 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Definition
Version: 1.1
After the connection to the FTP server has been successfully established, the parameter describes the average data transfer rate measured over the data transfer phase. I.e. prerequisite to the data transfer is successful Service Access (availability of an EPS bearer and connection to the FTP service). The data transfer phase shall also be successfully terminated. The throughput is measured on application level in UL/DL direction.
Measurement methodology
1. Set the FMT to generate calls to the FTP Server automatically with the following script/sequence: a. Connection attempt (network attach) b. Log into the FTP Server c. FTP Download 800MB file (or other size with transfer duration >1min) d. Wait 3 sec. after session finishes e. FTP Upload 300MB file (or other size with transfer duration >1min) f. Log out the FTP Server If UE gets into UE deregistered status between calls, this test also valid to check the Attach Success Rate g. Connection release (network detach) h. Leave 5 seconds before start next session i. Repeat all previous steps (from „a‟) 2. Continue drive testing till the drive route is completed. 3. Stop the FMT at the end of the route. 4. If the route does not generate more than 100 calls repeat the measurement on the same measurement route). 5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during the measurement. 6. Verify the average DL/UL throughput of each session and determine the final average throughput of all sessions. 7. The measured mean throughputs can be displayed in diagrams as function of time to show the impact of FTP slow start.
Assumptions, pre-conditions The initial status of UE before starting the application: EMM-REGISTERED, ECM-CONNECTED. If above measurement methodology is followed the UE needs to attach to the network between FTP UL/DL sessions if the Attach Success Rate wants to be measured. The FTP application shall be executed in binary mode, and the file to be transferred shall be binary. Formula (logical)
Message flow, trigger points The average throughput is measured from opening the data connection to the end of the successful transfer of the content (file, e-mail or web page). Trigger points for ftp upload / download are defined in [ETSI102.250-2] Chapters 4.6.1.7 and 4.6.2.7.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 15 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Related E2E KPIs
Version: 1.1
(FTP) Service Access Time, (FTP) Session Time.
3.1.4 Attach Time Attach Time (Control Plane) [ms] Definition
With Attach, the mobile terminal registers at the LTE network. At the end of the procedure the UE is authenticated, and a default (nonGBR) bearer is established. The Attach Time is the interval between the RRC CONNECTION REQUEST (carrying the Attach Request) and the reception of a positive response by the UE (Attach Complete).
Measurement methodology
Assumptions, pre-conditions
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Check the UE DEGERISTERED between data calls Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the Attach Time
UE status before measurement: EMM-DEREGISTERED UE locked to LTE network.
Formula (logical) Message flow, trigger points
Trigger points on UE R interface (see Figure 2Figure 2, and refer to [3GPP23.401]):
Related E2E KPIs
Begin: 1. RRC CONNECTION REQUEST sent by UE carrying L3 NAS: Attach Request in its body. End: 21. RRC DIRECT TRANSFER message with L3 NAS: Attach Complete in its body sent by UE (refer to [3GPP24.008]).
a) Attach Success Rate. b) No 3GPP defined KPI for Attach Time found exists, or measurements defined in R8 documents.
Detailed attach procedure with initial EPS bearer establishment is shown below.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 16 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
UE BUE
eNodeB
new MME
1. Attach Request 2. Attach Request
Serving GW
Old MME/SGSN
Version: 1.1
PDN GW
HSS
PCRF
EIR 3. Identification Request 3. Identification Response
4. Identity Request 4. Identity Response 5a. Authentication / Security 5b. Identity Request/Response
5b. ME Identity Check
6. Ciphered Options Request 6. Ciphered Options Response 7. Delete Bearer Request
7. PCEF Initiated IP-CAN Session Termination
7. Delete Bearer Response
(A) 8. Update Location Request 9. Cancel Location 9. Cancel Location Ack 10. Delete Bearer Request
10. PCEF Initiated IP-CAN Session Termination
10. Delete Bearer Response
(B)
11. Update Location Ack 12. Create Default Bearer Request 13. Create Default Bearer Request 14. PCEF Initiated IP-CAN Session Establishment
(C) 15. Create Default Bearer Response First Downlink Data (if not handover) 16. Create Default Bearer Response 17. Initial Context Setup Request / Attach Accept 18. RRC Connection Reconfiguration
EUE
19. RRC Connection Reconfiguration Complete 20. Initial Context Setup Response 21. Direct Transfer 22. Attach Complete First Uplink Data 23. Update Bearer Request 23a. Update Bearer Request 23b. Update Bearer Response (D) 24. Update Bearer Response First Downlink Data 25. Notify Request 26. Notify Response
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 17 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Figure 2 Attach Procedure with Initial EPS Bearer Establishment (3GPP 23.401)
For initial Attach, the following specifics of the message flow in Figure 2Figure 2 have to be considered: A signaling bearer (SRB1) is to be established, before an Attach Request can be sent, i.e. the UE is expected execute a contention based Random Access (RA) procedure. Step 3. is not executed, since the IMSI is queried from the UE directly using Step 4. The optional Identity Request / Response messages in step 5.b do not have to be exchanged if the MEI is already returned by the UE using the Security Mode Complete message as part of step 5.a. Steps 7-11 are not executed, no EPS bearers are to be deleted.
3.1.5 Attach Success Rate Attach Success Rate (Control Plane) [%] Definition
The Attach Success Rate is defined as the ratio between the number of successful registrations (RRC DIRECT TRANSFER: Attach Complete) and the number of all requests (RRC CONNECTION REQUEST: Attach Request). It is the probability that a user can attach to the LTE network at any moment of time. The calculated success ratio figure excludes attach requests, which are rejected by authentication failures. On the other hand, network attach requests which are terminated by timer expiry (due to the unavailability of some LTE resource) are considered as unsuccessful registrations. If the success rate is calculated on the eNB by counting incoming RRC requests, RRC CONNECTION REQUEST retries are to be excluded, since they would increase the overall number of establishment attempts, and thus reduce the success ratio.
Measurement methodology
For defining the success ratio, the same series of “Attach Time” measurements can be used:
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the Attach Success Rate as per formula below
The KPI cannot be measured as end-user perceived ratio on certain mobiles, which do not indicate network attachment status to the end-user. Assumptions, pre-conditions
UE status before measurement: EMM-DEREGISTERED UE locked to LTE network.
Formula (logical)
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 18 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Message flow, trigger points
Version: 1.1
Trigger events on UE R interface (see Figure 2Figure 2):
Success: 21. RRC DIRECT TRANSFER message with L3 NAS: Attach Complete in its body sent by the UE Attempts: 1. RRC CONNECTION REQUEST messages (with cause Attach Request) sent by UE
Related E2E KPIs Attach Time, Service Request (EPS) Success Rate.
3.1.6 Service Request (EPS) Time, UE Initiated
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 19 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Service Request Time (EPS Bearer Setup), UE Initiated, Control Plane [ms] Definition
Time taken by the LTE network to setup an EPS bearer on request by the UE. The EPS bearer can be new (dedicated) or an existing one (e.g. the default EPS bearer). The latter is needed to re-assign Uu radio and S1 bearer resources to the existing EPS bearer of a previously idle UE. The EPS bearer has to be created before IP packets can be exchanged. The UE Initiated EPS Bearer Setup Time is the interval between the submission of the message RRC CONNECTION REQUEST: Service Request (refer to [3GPP36.331]) and the reception of the RRC CONNECTION RECONFIGURATION response on the UE. The Bearer Setup procedure is based on the Service Request procedure according to [3GPP23.401] and [3GPP36.300].
Measurement methodology
Assumptions, preconditions
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the EPS Bearer Setup Time Note below that UE pre-status is EMM-REGISTERED so this KPI excludes the Attach time
UE status before measurement: EMM-REGISTERED and idle initial state. UE locked to LTE network.
Formula (logical)
Message flow, trigger points
The message flow diagram is shown in Figure 3Figure 3 and Figure 4Figure 4. The user initiated EPS bearer setup time is measured between the trigger points BUE and EUE on the UE R interface:
Related E2E KPIs
Begin: RRC CONNECTION REQUEST(NAS: Service Request) sent by UE End: corresponding RRC CONNECTION RECONFIGURATION COMPLETE message or first PDU in UL sent by the UE (refer to [3GPP24.008]).
a) Attach Time. b) No 3GPP defined KPI for EPS bearer setup found (exists).
Detailed EPS bearer setup procedure is shown below:
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 20 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
LTE-Uu UE
S6a
S11
S1- MME eNodeB
MME
S-GW / P-GW
UE EMM-REGISTERED and ECM-IDLE
RRC CONNECTION
L2 SCTP: DL DATA NOTIFICATION
HSS
DL data
D
Paging Procedure BUE
Version: 1.1
BN
Begin measurement
REQUEST RRC CONNECTION SETUP
Setup SRB1
RRC CONNECTION SETUP COMPLETE (L3 NAS: SERVICE REQUEST)
S1-AP: INITIAL UE MESSAGE (L3 NAS: SERVICE REQUEST)
L2 SCTP: Create Bearer Request L2 SCTP: Create Bearer Response
If authentication vectors are not available in MME, authenticate UE S1-AP: UL NAS TRANSPORT RRC: DL INFORMATION TRANSFER
(L3: AUTHENTICATION AND CIPHERING REQUEST)
Activate EPS bearer setup in ePC
MAP: Send Authentication Info MAP: Send Authentication Info Ack
If authentication timer has expired, setup security association for user plane RB
(L3: AUTHENTICATION AND CIPHERING REQUEST) RRC: UL INFORMATION TRANSFER (L3: AUTHENTICATION AND CIPHERING RESPONSE)
S1-AP: DL NAS TRANSPORT (L3: AUTHENTICATION AND CIPHERING RESPONSE)
Figure 3 EPS Bearer Setup Procedure, Part I
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 21 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
LTE-Uu UE
S1- MME eNodeB
T1
S11 MME
S1-AP: INITIAL CONTEXT
Version: 1.1
S6a S-GW / P-GW
HSS
PM Counter: EPS_SETUP_ATT
SETUP REQUEST S1-AP: INITIAL CONTEXT SETUP FAILURE RRC SECURITY MODE COMMAND RRC CONNECTION RECONFIGURATION
Setup user plane RB
RRC SECURITY MODE COMMAND COMPLETE EUE
RRC CONNECTION RECONFIGURATION
End measurement
COMPLETE
Means „Delay DL Packet Notification Request“ to stop/ enable DL data transfer
First UL PDU
T2
S1-AP: INITIAL CONTEXT
PM Counter: EPS_SETUP_SUCC
SETUP COMPLETE L2 SCTP: Update Bearer Request L2 SCTP: Update Bearer Response
EN
First DL PDU
Figure 4 EPS Bearer Setup Procedure, Part II
The service request times for UE initiated and for network initiated EPS bearer setups are measured between the trigger points BUE and EUE, and between BN and EN, respectively. These trigger points are colored green in the diagram.
3.1.7 Service Request (EPS) Success Rate Service Request (EPS) Success Rate, Control Plane [%]
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 22 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Definition
Version: 1.1
Ratio between successfully established EPS bearers compared to the overall number of EPS bearer establishment attempts. It corresponds to the probability that a user or the LTE network can establish an EPS bearer at any moment in time. Requests which are terminated by timer expiry (due to the unavailability of some LTE resource) are considered as unsuccessful attempts. Authentication errors (requests rejected by the MME) are included in the total number of failures. Only the first RRC CONNECTION REQUEST is to be considered, since counting retries of the same message would increase the overall number of bearer establishment attempts, and thus reduce the success ratio.
Measurement methodology
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the EPS Bearer Setup Success Rate
Note below that UE pre-status is EMM-REGISTERED so this KPI excludes the Attach time Assumptions, preconditions
UE status before measurement: EMM-REGISTERED and idle initial state. UE locked to LTE network.
Formula (logical)
Message flow, trigger points
The message flow diagram is shown in Figure 3Figure 3 and Figure 4Figure 4. The trigger points for UE initiated EPS bearer setup success ratio on the R interface are:
Related E2E KPIs
Success: RRC CONNECTION RECONFIGURATION COMPLETE message sent by the UE, see EUE on Figure 4Figure 4. Total: an attempt is the sending of RRC CONNECTION REQUEST with appropriate establishment cause by the UE, see BUE on Figure 3Figure 3.
Service Request (EPS) Time, UE Initiated. For initial EPS bearers, refer to KPI “Attach Success Rate”.
3.1.8 Service (EPS Bearer) Drop Rate Service (EPS Bearer) Drop Rate, Control Plane [%] Definition
Ratio between abnormally released bearers, and the overall number of established EPS bearers. An abnormal release is defined as any EPS bearer
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 23 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
termination, which was not triggered by the mobile user (from UE side). Thus, it reflects the probability that an established bearer gets aborted due to insufficient network resources. Dropping the bearer gets visible to the end-user if an application service is actively using it. If the application automatically re-establishes the bearer, it remains unnoticed by the user.
Measurement methodology
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the EPS Bearer Drop Rate
Note below that UE pre-status is EMM-REGISTERED so this KPI excludes the Attach time. Assumptions, preconditions
UE status before measurement: EMM-REGISTERED and idle initial state. UE locked to LTE network.
Formula (logical) Trigger points on UE R interface: Message flow, trigger points
Success: submission of the message RRC CONNECTION RECONFIGURATION COMPLETE by the UE, refer to [3GPP36.331] Drop: successful establishments minus terminations by the user, i.e. the UE submitting a L3 DETACH REQUEST (refer to [3GPP24.008]) carried within the RRC message UL NAS TRANSPORT (refer to [3GPP36.331]).
For the calculation of a network initiated EPS bearer drop rate, EPC initiated L3 DETACH REQUEST-s are to be considered. Related E2E KPIs
Service Request (EPS) Success Rate.
3.1.9 Handover Procedure Time Handover Procedure Time, Control Plane [ms] Definition
It denotes the total time needed for the handover procedure as seen by the UE. It begins by receiving a Handover Command from the source eNB and ends by sending the Handover Confirm response to the target eNB by the UE. The value of the KPI also depends of the handover scenario. The following HO scenarios are distinguished (though not directly seen by the UE):
intra- and inter-frequency HO inter eNB HO via X2/S1 interface
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 24 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Measurement methodology
Version: 1.1
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify HO time for the different HOs
The handover time shall be measured on the UE R interface. Assumptions, preconditions
Since Inter-RAT HO is not part of RL15 the only HO scenarios considered are intra LTE. Therefore, the LTE UE must be locked to LTE network. UE status before measurement: registered and connected.
Formula (logical)
Average value from all measured samples.
Message flow, trigger points
The HO time is seen by the UE as the elapsed time between:
Related E2E KPIs
3.1.10
Start: UE receives Handover Command in the body of an RRC Connection Reconfiguration request from source eNB. Stop: UE sends Handover Confirm response to target eNB in an RRC Connection Reconfiguration Complete message.
a) (LTE) Service Interrupt Time (HO), Handover Success Rate b) No 3GPP defined KPI for HO procedure time found.
Handover Success Rate
Handover Success Rate, Control Plane [%] Definition
The Handover Success Rate is the ratio between successfully executed (committed) HO procedures and the number of all Handover attempts.
Measurement methodology
1 LTE UE/terminal with FMT Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the Handover Success Rate
The measurement is to be executed for different HO scenarios. Assumptions, preconditions
LTE terminal must be locked into the LTE network.
Formula (logical)
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 25 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Message flow, trigger points
Version: 1.1
Trigger points on eNB X2/S1 interfaces:
Total #: Handover Requests sent by Source eNB to Target eNB (HO via X2), or sent by MME to Target eNB(HO via S1). Success #: UE Context Release message received by Source eNB from Target eNB(HO via X2), or UE Context Release Command received by Source eNB from MME(HO via S1).
In case of intra eNB HO procedures, no Handover Request is sent. Both trigger points are counted by the Source eNB internally. Related E2E KPIs
3.1.11
Handover Procedure Time
(LTE) Round Trip Time
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 26 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
Round Trip Time (RTT), User Plane [ms] Definition
RTT in UL is the interval between sending a datagram by the UE & receiving the corresponding reply from an IP peer entity connected to the Gi interface of the P-GW. RTT in DL is the interval between sending a datagram to the UE & receiving the corresponding reply by the IP host (peer entity).
Measurement methodology
Stationary users in different cell positions, which are uniformly distributed across the cell. RTT is measured with the Ping application between the UE and an IP host (peer entity). As average RTT figure, the output of the Ping application is used (e.g. “MS-DOS >ping -n count -l size host name”). Size of ICMP packets: 32, …, MTU bytes. Ping shall be executed at least 100 times to reduce the impact of the first ICMP message, which triggers the setup of radio bearer establishments in UL and DL directions. Record the average RTT shown on the screen after the application has run. Repeat the test 5 times in each location under different SINR. Note: Ping tests can also be carried out as part of a drive test sequence in which case, it is not possible to guarantee certain SINR levels
Assumptions, preconditions
UE status before measurement: registered and connected. An IP address is assigned to the UE. The IP peer entity shall be located as close as possible to the SGi interface. Usually, 1 hop away the SAE-GW. Additional delays between the P-GW and the IP host shall be avoided or minimized.
Formula (logical) Message flow, trigger points
Trigger points both from UE, and from IP peer entity view:
Related E2E KPIs
No 3GPP defined KPI for LTE RTT found.
3.1.12
Start: ICMP ECHO REQUEST Stop: ICMP ECHO REPLY
(LTE) Single User Throughput
(LTE) Single User Throughput, User Plane [Mbps] Definition
The metric describes the data speed available to one user of the LTE network on UDP/IP level. It is given as the maximum (95%-ile) value that can be observed over a short period of time (e.g. of 1s) and as a mean value that
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 27 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
characterizes longer data transfer periods (minutes). Its value distribution over the radio cell is given as a function of the SINR. The maximum value is often referred to in the literature as instantaneous "Peak Throughput" that is achieved in optimal radio conditions. The user throughput can be given for a single user active in the cell (single user data rate), or to one of several concurrently active users. Measurement methodology
Stationary and mobile user covering all kinds of cell positions: near, medium, far, including different HO scenarios. The data rate is measured in UL/DL direction with UDP/IP traffic over a time period of several minutes.
Assumptions, preconditions
The IP host used as traffic generator (e.g. with Iperf) should be connected directly to the SGi interface. Additional delays and bandwidth restrictions between the S/P-GW and the server need to be excluded.
Formula (logical)
Message flow, trigger points
Trigger points on the UE R interface: • UL: UE sending UDP/IP packets to the IP peer entity. • DL: IP peer entity sending UDP/IP packets to the UE.
Related E2E KPIs
Cell Throughput, (FTP) Single User Throughput, (RB) Single User Throughput.
The Single User Throughput KPI has been defined on E2E network level to verify if the capacity of large non-GBR EPS bearers (QoS profile with large maximum bit rates) can in fact be exploited by IP based services. The same holds for the E-RAB and RB bearers being part of the EPS. Especially, the maximum achievable throughput of the RB bearer is interesting, since it is the critical resource of the LTE network. For this reason, the Peak Throughput is defined as a separate KPI on the RB level (see KPI: (RB) Single User Throughput), too. Because of this relationship (containment) between EPS, E-RAB and RB, user throughput KPIs can be verified with the same series of measurements. The KPIs are interesting as values calculated over the full duration of long data transfers (~minutes). Of equal interest are peak values observed for a short time period only (~seconds). Please note that throughput KPIs have also been defined for application services, e.g. Single User Throughput KPI of the Data Download / Upload service, but these do not characterize the capability of the LTE network alone, but also depend on the application service and its protocols.
3.1.13
(LTE) Service Interrupt Time (HO)
(LTE) Service Interrupt Time (HO), User Plane [ms] Definition
Service interrupt is the discontinuity (increased packet inter-arrival time) of an IP packet flow caused by hand-over. It is the interval between the last sent/received user IP packet of a continuous UL/DL data stream in the old cell,
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 28 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
and the first sent/received user IP packet in the new cell (user plane break). The value of the KPI also depends of the handover scenario. 1 LTE UE/terminal with FMT
Measurement methodology
Set FMT to generate data calls to the data server automatically Start drive test and continue driving until drive route is completed and enough samples are collected (re-drive route if necessary) Verify the Service Interrupt Time The measurement is to be executed for different HO scenarios: intra eNB, inter eNB via X2/S1 etc. Only mobile user scenarios are considered. FTP upload/download data transfer, or real-time media flow during the handover.
Assumptions, preconditions
It is assumed that the X2 interface is enabled between the source and target eNBs (for inter eNB HO scenarios). LTE terminal must be locked into the LTE network to measure the Service Interrupt Time.
Formula (logical) Message flow, trigger points
Related E2E KPIs
3.1.14
The trigger points for service interrupt time (from UE point of view):
Start: UE sends/receives last packet to/from source eNB
Stop: UE sends/receives first packet to/from target eNB.
Handover procedure time.
(RB) Single User Throughput
(RB) Single User Throughput, User Plane [Mbps] Definition
The metric describes the RB throughput achievable by one user. It can be given as single user throughput if only one user is active in the cell, or as multiuser throughput for a given number of concurrently active users. The user throughput distribution over the cell is given as function (CDF) of the radio conditions characterized by the SINR. The time variation of the KPI value in a given cell position is given with its maximum (95%-ile) (also referred to as instantaneous “Peak User Throughput”), and with its average (mean) value.
Measurement methodology
Stationary and mobile user covering all kinds of cell positions: near, medium, far, including different HO scenarios.
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 29 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
Version: 1.1
The eNB assigns all resources (resource blocks) to this user, except the resource blocks used for signaling and controlling. With the help of trace analysis, PDCP, RLC, MAC and PHY layer throughputs can be defined. With and w/o concurrency in UL / DL traffic. The mean user data rate is calculated over a few minutes, peak data rate values are averaged over short periods of time (e.g. 1s). Assumptions, preconditions
All transport bearers between the UE and IP peer entity should have higher capacity than the radio link in order to avoid bandwidth bottlenecks.
Formula (logical)
Message flow, trigger points
Trigger events on the eNB Uu interface or the UE R interface:
UL: eNB receiving PDUs, or UE sending PDUs DL: eNB sending PDUs, or UE receiving PDUs
Measured at the upper SAPs of the protocols PDCP, RLC, MAC or PHY. Related E2E KPIs
3.1.15
(LTE) Single User Throughput, Cell Throughput.
Cell Throughput
(RB) Single User Throughput, User Plane [Mbps] Definition
The metric shows the sustainable aggregate throughput of the cell (in UL/DL) available to “n” stationary users distributed uniformly in the cell and running a typical mix of applications. The “cell throughput” is the sum of all bits transported in all resource blocks carrying PDUs (i.e. bits in ULSCH/DL-SCH transport blocks) during one second. The cell capacity is also given as peak value (called peak cell capacity, or throughput), which is defined as the aggregate throughput of “n” users all located in best radio conditions. The cell throughput value is defined here on PHY level, but could be given for other protocol levels (UDP/IP, PDCP, RLC, MAC), too. When the (peak, average) cell throughput is expressed on UDP/IP level, it corresponds to the (peak, mean) user throughput value at comparable radio conditions.
Measurement methodology
Stationary users uniformly distributed in the cell. UE categories according to application mix. Peak value measured with one user in LoS cell position using UDP/IP load to approach the full buffer condition. Measured with UDP/IP traffic on PHY layer (UL-SCH/DL-SCH transport block bits) over several minutes. With the help of trace analysis PDCP, RLC and MAC layer values can also be given.
Assumptions, pre-
All IP transport bearers (S1, S5/S8, SGi) between the UE and IP peer entity
Copyright © Nokia Siemens Networks 2011
Company Confidential
Page 30 (50)
TD-LTE KPI Optimization Guidelines for RL15TD
conditions
Version: 1.1
should have higher bandwidth than the cell capacity to avoid bandwidth bottlenecks.
Formula (logical)
Message flow, trigger points
Trigger events on the eNB Uu interface or the UE R interface:
Related E2E KPIs
(RB) Single User Throughput.
UL: eNB receiving PDUs, or UE sending PDUs DL: eNB sending PDUs, or UE receiving PDUs
3.2 Test Conditions and Target Values
Field KPI Single User UL Peak Throughput
Single User DL Peak Throughput
Throughput Cell UL Average Throughput
Cell DL Average Throughput
Latency (Control Plane)
Attach Latency
Target Values
Comments
In case UL/DL conf=2:2(#1) and special subframe conf=10:2:2(#7): 15.4M In case UL/DL conf=3:1(#2) and special subframe conf=10:2:2(#7): 7.5M In case UL/DL conf=2:2(#1) and special subframe conf=10:2:2(#7): TM3: 59.6M TM7: 34M In case UL/DL conf=3:1(#2) and special subframe conf=10:2:2(#7): TM3: 80.7M TM7: 40.3M In case UL/DL conf=2:2(#1) and special subframe conf=10:2:2(#7): 0% load: 14.5M 50% load: 13.3M 70% load: 15.6M In case UL/DL conf=3:1(#2) and special subframe conf=10:2:2(#7): 0% load: 7.6M 50% load: 6.5M In case UL/DL conf=2:2(#1) and special subframe conf=10:2:2(#7): 0% load: 35.6M 50% load: 21.77M 70% load: 24.82M In case UL/DL conf=3:1(#2) and special subframe conf=10:2:2(#7): 0% load: 40M 50% load: 25.1M In case UL/DL conf=2:2(#1) and special subframe conf=10:2:2(#7): 0% load: 168ms
Copyright © Nokia Siemens Networks 2011
Company Confidential
CMCC Hangzhou TD-LTE trial: *Cat 3 UE is considered and MCS20 is the max MCS in UL for RL15. *PUCCH overhead is 4RB and PDCCH overhead is 3 OFDM symbols. *L3 TCP throughput is considered. *Network is unloaded. *System bandwidth is 20M.
CMCC Hangzhou TD-LTE trial: *Cat 3 UE is considered and MCS20 is the max MCS in UL for RL15. *PUCCH overhead is 4RB and PDCCH overhead is 3 OFDM symbols. *L3 TCP throughput is considered. *System bandwidth is 20M. *20 UEs distributed by 2:4:8:6. *DL MIMO is TM3/TM7 adaptive and UL is SIMO. *eNB SW version is different for each neighbor load: RL15_2_CD1.0 for 0% load, RL15_2_CD2.0 for 50% load and RL15_2_CD3.0 for 70% load.
Voentelecom Russia TD-LTE Trial: *Radio condition is classified as: Good: CRS SINR >25 Average: 10
View more...
Comments
