VoLTE Trouble Shooting in Ericsson LTE RTT

VoLTE Trouble Shooting in Ericsson LTE RTT.

Below are some samples of using RTT to troubleshoot VoLTE failures. IMSI = 311480125562721 on Date = 10/29/2014. 1) Example of how a Good VoLTE call will look in E/// RTT.

For a successful VoLTE call there is normally at Good Dedicated Bearer and a Good Context. The Bearer is SS=19 and is the length of the VoLTE call, which is 426.733 seconds in the screen shot above. The Good Context is from RRC Setup to Context Release and normally starts before the Bearer and ends after the Bearer. If the Good Context starts right before the VoLTE Bearer, you can use this to guestimate if the VoLTE call was a Termination or Origination. 2) This is an example of a VoLTE LC for a short VoLTE Call with UL RF Issues. The VoLTE Bearer is still listed as a VGood Call, because the MME/SGW tears down the call in a normal manner. It does not indicate to ECDS that the Bearer was torn down due to an RF Failure. That type of message is not provided and is why the Bearer is still listed as a VoLTE Good. The VLC occurs at roughly exactly the same time the VGood and the VLC is the RF record from ECDS. Hovering over the Context Release reasons provide this detail UE Release Cause = 11 = Radio Connection with UE Lost. Internal Cause = 14 = RRC Reconfiguration Time out. Both these simply indicate that the RF condition was bad. When I right-clicked and did a Handoff Popup, there was no handoff for this call. When you look at the next few Good Context records, they are all on 173.443.2 at 0.63/0.78 miles away.

If I look at the Right-Click UE Data Popup for some of the other Good Context after the VLC, I can see Avg CQI = 4.7 and UL Power Restricted at 100%. I also see that UL Nak% = 29.6% and UL BLER% = 31.0% and UL kbps = 27. This indicates to me that the mobile Uplink was in a bad RF condition.

The last thing to note that tends to indicate this was a bad VoLTE call is that for the VGood Dedicated Bearer records, the SGW Bytes Rxed in the UL is 0. I have seen in the past on some one-way audio that I had, that the QCI=1 Bearer record had either UL or DL SGW Bytes = 0. This is not a 100% indicator that a one way audio occurred, but it is a good sanity check.

4) A single VoLTE Call with multiple VLC in it.

In the above screen shot, the VoLTE Call starts at 16:54:01.6 and ends at 17:03:04.5 as seen in row 145 VGood Bearer record. We see one VLC at 16:59:36 (row 147) and another at 17:03:05.2 (row 141). The VLC at 17:03:05.3 is Release UE Cause = 11 = Radio Connection with UE lost and Internal Reason = 6 = RLC Failure on Signaling Bearer. To me this indicates that the VoLTE Call ended with a drop, since the VLC is at the same time of the Bearer End Time. The VLC that occurs at 16:59:36 is in the middle of the VoLTE call. This indicates to me that the RF Context was Released, but did not cause the VoLTE Bearer to be released. Since this VLC was in the middle of the VoLTE call, it likely caused some audio holes while the Radio Context was released and re-established. Note: The Subscriber Number is not populated in all the ECDS records. If you only have the Subscriber, search that first and then re-query with the IMSI to make sure you have all the records. 5) VoLTE Setup Fail (VSF) example.

RTT can only capture some of the VSF calls. In the case below, there is an ERAB Setup Fail. If we scroll to the right to the ERAB Setup Details, we will see that the setup failed due to Results = 2 = Failure in Radio Procedure. This indicates the ESF occurred due to Radio issues. It also reports a Dedicated Bearer Activation Failure (DBAF). I also see that a normal CD occurs shortly after the ESF.

6) VLC in the middle of a VGood call. This is an example of a VLC that occurs in the middle of a VGood Call. This means that the VoLTE Dedicated Bearer did not drop, but the RF Context did fail in the middle of the call which likely caused an audio hole for the end user. The VLC happened at 8:07:42 and the VGood happened from 8:07:25.3 to 8:07:54.1 which is only a 28 second VoLTE call. When I right-click on the VLC, I see that the HO results were Failed Tx2 Relocation Overall Expired. This simply means the HO timer expired after 5 seconds. Since this was a 5 second timer expire, I would think the user would have a large audio hole that would be noticeable. I can also see that the Serving RSRP was -117 for 473.544.22 at a distance of 4.71 miles. The Neighbor RSRP is -108 for 473.347.22 as seen in the Prep Out Handoff info. You must look at the timestamps on the HO information to make sure they are the same HO. In this example the Prep In/Execute In is from the previous HO. The next G Ctxt starts at 8:07:43.2 and when you hover over the VYes, you can see that there was QCI=1 in the Initial Context Setup event. So this is the 2 nd RF record for this VoLTE Call. That is why it had VYes in it. The VLC occurred on 473.544.22 at 4.71 miles and then the RF restarted on the same eNB 473.544.22 at 4.76 miles. Not sure why it did not restart on eNB = 473.347.22 since its RSRP was -108.

7) Creating a Distance Histogram for all the VLC and VSF and plotting them on a map. The Distance Histogram page can be used just to look at the VLC and VSF. This is done by only checking VSF and VLC when running the query. In the screen shot below, I queried the last 8 days of VLC and VSF and it only took 16 seconds to run for 7885 VLC and VSF that had Distance information. (VoLTE_Cat field is indexed). I then sorted by Total Count and found that 457.59.12 had the most VLC/VSF at 173. I then right-clicked on that row of data and chose “Show Call Locations on Map Popup”.

The Distance Histogram for the VSF/VLC are then plotted on the Map Pop. You can open multiple Map Popups at a time, but right-clicking on a new row of data.

8) Plotting where the VLC and VSF are occurring with Simple Query page. Since we are still in early stages of VoLTE, in most markets you can query the last 8 days of all VLC and VSF in the Simple Query page. Then you can right-click on the results table and choose “Show Call Location on Map Popup”. In LTE RTT, we only get the Distance measurement for one Mkt.eNB.Cell_ID and display this as an arc on the map. The count of records at that distance is displayed in text at the center azimuth of the sector. In the screen shot below, I zoomed into an area of interest on the map and changed the Arc Properties. I increased the Thickness to 50% and kept it as a Heat Map.

In contrast, I can turn off the Heat Map option and reduce the Thickness to 10%. This makes each arc drawn at 100% transparency.

Update: The Session Status=22 were removed from the VLC formula on 1/21/2015 to match the ELPT VLC formula.

LTE RTT Release V6.5 1/21/2015 at 8:00pm Eastern Control = NCC000000471189

Change

3) VLC with Session Status = 22 = ERAB Release with Data Lost. This is a new VLC introduced in E/// RTT October 2014 with the upgrade to ECDS 5.4.1. This was added to match the ELPT VLC formula this it: (pmErabRelAbnormalEnbQci + pmErabRelMmeActQci ) RTT already measured this with the VoLTE CD. pmErabRelAbnormalEnbQci = "The total number of E-RABs released abnormally per cell per QCI for requests initiated by eNB.” RTT had to add SS=22 in order to measure this peg. pmErabRelMmeActQci = "The total number of active E-RABs released per QCI for requests initiated by MME, except active E-RABs released due to successful HO. An E-RAB is considered to be active when there are data in the DL or UL queues for the E-RAB.” However there is some controversy in the markets that these are really customer impacting VoLTE Lost Call. Several Markets has used run IRIS traces on these IMSI and do not see any SIP errors or RTP errors. I have forwarded all this information to the Service Performance Team for their analysis. At this point, they still want to keep the pmErabRelMmeActQci in the ELPT VLC formula, so we will still be counting the SS=22 as a VLC. The good news is that since the SS=22 have been added to RTT, we can now look up these IMSI in other tools to further validate if these were customer impacting issues. Before we were not able to do that with just the peg, since it is simply a counter.

In the screen shot above, the SS=22 will occur at roughly the same end time as the G DBearer, which is 17:27:21 in the screen shot above. The SS=22 VLC indicates that the Dedicated Bearer release was triggered by the MME and that there was still Data in the queue that was not sent. The G Ctxt that occurs shortly after the VLC was the RF record for this VoLTE call. Bottom Line: The official corporate formula indicates this is a VLC. However, many of the markets have provided data to Service Performance Team that indicates these may not be customer impacting drops.