Alarm Handling Guide_Moshell Commands_V1

MTN

Alarm Handling Guideline Using Moshell Commands

Maria Riordan 7/21/2010

Contents INTRODUCTION:.........................................................................................................................3 Miscellaneous:.......................................................................................................... 66

INTRODUCTION: This alarm guide is to help assist in clearing alarms on the RNC and RBS using MOSHELL commands. Not all cases will be the same. A few things to note for all alarms are as follows:

1) Ensure that there are no existing REFS open for the site either with SMC or BSS.

2) Refer to the ALEX Documentation. This will give you a full and clear understanding of the alarm you are dealing with.

3) Acknowledge the alarm in the OSS Alarm List Viewer.

4) Check all associated alarms on RNC and RBS. Check the history of the alarm taking note if it is a re-occurring alarm. (lga | grep )

5) Use the guidelines below to clear the alarm.

6) Once alarm has been cleared confirm the site is carrying traffic as normal.

ANTENNA/ASC/RET: AuxPlugInUnit_CommunicationLostWithAsc: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> CommunicationLostWithAsc This alarm will generally require an RE to go to site. Communication with the ASC/RET is lost. The consequence of this alarm is that planning/optimisation will be unable to make a tilt change to the affected sector. Connection with the ASC will need to be restored on site or a replacement ASC will be needed. If the alarm is inter-failing or re-occurring, the best solution is to swap out the ASC. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lga | grep AuxPlugInUnit_CommunicationLostWithAsc *show history of current alarm get radio no *confirm if node is carrying traffic st plug *print state of all plug in units get xxx *do a get on the disabled proxy number acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the ASC/RET using the proxy number RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels

AuxPlugInUnit_CommunicationLostWithRet: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> CommunicationLostWithRet This alarm will generally require an RE to go to site. Communication with the ASC/RET is lost. The consequence of this alarm is that planning/optimisation will be unable to make a tilt change to the affected sector. Connection with the RET will need to be restored on site or a replacement ASC will be needed. If the alarm is inter-failing or re-occurring, the best solution is to swap out the RET. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lga | grep AuxPlugInUnit_CommunicationLostWithRet *show history of current alarm get radio no *confirm if node is carrying traffic st plug *print state of all plug in units get xxx *do a get on the disabled proxy number acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the ASC/RET using the proxy number RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels

TmaDevice_LnaDegraded: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> LnaDegraded

This alarm is issued by the TmaDevice Managed Object (MO) when one of the two transistors amplifying the RF signals in the Antenna System Controller (ASC) or the AISG/3GPP Tower Mounted Amplifier (ATMA) fails. It indicates that the LNA gain in one of the branches on the ASC has degraded. The UL signal will pass, but with decreased level. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected st plug *print state of all plug in units lpr SectorAntenna=x,AuxPlugInUnit=x *shows associated equipment with the ASC acl xxx *check what action you can perform on the ASC acc xxx restartAuxUnit *try restart the ASC/RET using the proxy number

If the alarm does not clear after restarting the ASC an RE will be required to visit the site and replace the ASC. TmaDevice_LnaFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> LnaFailure The fault is located in the Low Noise Amplifier (LNA), which amplifies the RF signals in the receiver (RX) chain. When the transistors in one branch have failed, the cell is able to carry traffic as long as the other branch works, but the RX chain performance is degraded. The LNA gain in one of the branches in the ASC is lost. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected st plug *print state of all plug in units lpr SectorAntenna=x,AuxPlugInUnit=x *shows associated equipment with the ASC get xxx *use this to show more information about the plug in unit acl xxx *check what action you can perform on the ASC acc xxx restartAuxUnit *try restart the ASC/RET using the proxy number

If the alarm does not clear after restarting the ASC an RE will be required to visit the site and replace the ASC. RetDeviceSet_RetFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> RetFailure This alarm is issued by the RetDevice or RetDeviceSet Managed Objects (MO) when the Remote Electrical Tilt Unit (RETU) malfunctions. The RETU is used to tilt the antenna according to the specified antenna tilt configuration parameters in the RET profile. The causes of the alarm are: • A faulty RETU • The RETU arm is unable to move for environmental reasons, for example, antenna icing • Antenna and RETU not installed RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lpr SectorAntenna=x,AuxPlugInUnit=2 get xxx *using the proxy number from the above command will show the state of the RetDeviceSet acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the RET using the proxy number

If a restart does not clear the alarm an RE will need to go to site and check/replace the RET unit.

RetDevice_RetFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> RetFailure This alarm is issued by the RetDevice or RetDeviceSet Managed Objects (MO) when the Remote Electrical Tilt Unit (RETU) malfunctions. The RETU is used to tilt the antenna according to the specified antenna tilt configuration parameters in the RET profile. The causes of the alarm are: • A faulty RETU • The RETU arm is unable to move for environmental reasons, for example, antenna icing • Antenna and RETU not installed RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lpr SectorAntenna=x,AuxPlugInUnit=2 get xxx *using the proxy number from the above command will show the state of the RetDeviceSet acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the RET using the proxy number

If a restart does not clear the alarm an RE will need to go to site and check/replace the RET unit. RetDevice_ElectricalAntennaTiltOutOfRange: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> ElectricalAntennaTiltOutOfRange This alarm is issued by the RetDevice Managed Object when the electrical antenna tilt is out of range. The electrical tilt on the site is set to a value that the RET and antenna does not support. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected get ret elec *shows the current electrical tilt settings on the sectors get retdevice tilt *shows the max and min tilt supported by the ret device lpr SectorAntenna=x,AuxPlugInUnit=2 get xxx *using the proxy number from the above command will show the state of the RetDeviceSet acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the RET using the proxy number

If restarting the RET does not clear the alarm confirm with Planning/Optimisation that the Antenna Type is correct: get sector antennatype *shows the antenna type for each sector

If the alarm still persists an RE will need to go to site and check/replace the RET Unit. AscDeviceGroup_GeneralHWError: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> GeneralHWError This alarm is issued by the ASC. It indicates that the ASC is faulty and needs to be restarted or replaced. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lpr SectorAntenna=x,AuxPlugInUnit=1 acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the ASC using the proxy number

If the alarm does not clear after restarting the ASC an RE will need to go to site and investigate/replace the ASC.

AntennaBranch_AntennaProblemInBranchA: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> AntennaProblemInBranchA This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – Antenna Jumper Cable RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_AntennaProblemInBranchA *shows history of alarm st plug *check the current status of the ASC/RET lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC/RET

NOTE: Equipment=1,SectorAntenna=x,AuxPlugInUnit=1 *ASC Equipment=1,SectorAntenna=x,AuxPlugInUnit=2 *RET cabx *get the port number of the ASC for the affected sector =============================================================================================== SMN APN PORT BOARD GREEN YELLOW RED PRODUCT NR =============================================================================================== 0 12 port_0_dev_4 RU21 steady 16hz off KRC 118 16/1 0 12 port_0_dev_4 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_1_dev_10 ASC steady 16hz off KRY 112 42/3 0 12 port_4_dev_5 RU21 steady 16hz off KRC 118 16/1 0 12 port_4_dev_5 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_5_dev_12 ASC steady 16hz off KRY 112 42/3 0 12 port_8_dev_6 RU21 steady 16hz off KRC 118 16/1 0 12 port_8_dev_6 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_9_dev_14 ASC steady 16hz off KRY 112 42/3 ----------------------------------------------------------------------------------------------lhsh 001200/port_1_dev_10 asc vswr *prints the VSWR value for the Antenna Branch lh asc asc vswr *command to show VSWR for all sectors

If the above commands don’t work try: lh asc dbg vswr 0 get -k

This command gives you the return loss value which can be converted to the desired VSWR value using the following link: http://www.soontai.com/cal_rtvswr.html Acceptable value for VSWR should be < 1.5. If the value is higher than this an RE will need to go to site and investigate/sweep the affected Antenna Branch. AntennaBranch_AntennaProblemInBranchB: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> AntennaProblemInBranchB This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – Antenna Jumper Cable RBS:

moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_AntennaProblemInBranchA *shows history of alarm st plug *check the current status of the ASC/RET lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC/RET

NOTE: Equipment=1,SectorAntenna=x,AuxPlugInUnit=1 *ASC Equipment=1,SectorAntenna=x,AuxPlugInUnit=2 *RET cabx *get the port number of the ASC for the affected sector =============================================================================================== SMN APN PORT BOARD GREEN YELLOW RED PRODUCT NR =============================================================================================== 0 12 port_0_dev_4 RU21 steady 16hz off KRC 118 16/1 0 12 port_0_dev_4 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_1_dev_10 ASC steady 16hz off KRY 112 42/3 0 12 port_4_dev_5 RU21 steady 16hz off KRC 118 16/1 0 12 port_4_dev_5 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_5_dev_12 ASC steady 16hz off KRY 112 42/3 0 12 port_8_dev_6 RU21 steady 16hz off KRC 118 16/1 0 12 port_8_dev_6 FU12 ON 16Hz OFF KRC 118 17/1 0 12 port_9_dev_14 ASC steady 16hz off KRY 112 42/3 ----------------------------------------------------------------------------------------------lhsh 001200/port_1_dev_10 asc vswr *prints the VSWR value for the Antenna Branch lh asc asc vswr *command to show VSWR for all sectors

If the above commands don’t work try: lh asc dbg vswr 0 get -k

This command gives you the return loss value which can be converted to the desired VSWR value using the following link: http://www.soontai.com/cal_rtvswr.html Acceptable value for VSWR should be < 1.5. If the value is higher than this an RE will need to go to site and investigate/sweep the affected Antenna Branch. AntennaBranch_FeederCurrentTooLowInBranchA RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> FeederCurrentTooLowInBranchA This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – RET – Antenna Jumper Cable RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_FeederCurrentTooLowInBranchA *shows history of alarm st plug *check the current status of the ASC lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET – if disabled RE will need to attend acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC

If no issues are seen with the ASC and the alarm still persists an RE will need to go to site and investigate any loose feeder connections. NOTE: Some other parameters to check

get antennabranch supervision *Value of 0 implies no Antenna Supervision. mom . antennaSupervisionThreshold *description of parameter

AntennaBranch_FeederCurrentTooLowInBranchB RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> FeederCurrentTooLowInBranchB This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – RET – Antenna Jumper Cable RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_FeederCurrentTooLowInBranchA *shows history of alarm st plug *check the current status of the ASC lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET – if disabled RE will need to attend acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC

If no issues are seen with the ASC and the alarm still persists an RE will need to go to site and investigate any loose feeder connections. NOTE: Some other parameters to check

get antennabranch supervision *Value of 0 implies no Antenna Supervision. mom . antennaSupervisionThreshold *description of parameter

AntennaBranch_FeederCurrentTooHighInBranchA RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> FeederCurrentTooHighInBranchA This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – RET – Antenna Jumper Cable RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_FeederCurrentTooLowInBranchA *shows history of alarm st plug *check the current status of the ASC lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET – if disabled RE will need to attend acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC

If no issues are seen with the ASC and the alarm still persists an RE will need to go to site and investigate any loose feeder connections. NOTE: Some other parameters to check

get antennabranch supervision *Value of 0 implies no Antenna Supervision. mom . antennaSupervisionThreshold *description of parameter

AntennaBranch_FeederCurrentTooHighInBranchB RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> FeederCurrentTooHighInBranchB This alarm is issued by the AntennaBranch Managed Object (MO) when one of the following units is faulty or a cable is faulty or disconnected: – ASC – RET – Antenna Jumper Cable RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected lga | grep AntennaBranch_FeederCurrentTooLowInBranchA *shows history of alarm st plug *check the current status of the ASC lst ret *check the current status of the ASC/RET lst asc *check the current status of the ASC/RET – if disabled RE will need to attend acl xxx *check what action you can perform on the ASC using the proxy number acc xxx restartAuxUnit *try restart the ASC

If no issues are seen with the ASC and the alarm still persists an RE will need to go to site and investigate any loose feeder connections. NOTE: Some other parameters to check

get antennabranch supervision *Value of 0 implies no Antenna Supervision. mom . antennaSupervisionThreshold *description of parameter

PLUG IN UNIT: FcuDeviceGroup_NumberOfHwEntitiesMismatch: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> NumberOfHwEntitiesMismatch This alarm is issued by the FcuDeciceGroup MO when there are too few fans in operation. It could be the cause of a faulty or disconnected fan unit cable or too few fans installed on site. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lpr AuxPlugInUnit=2,FcuDeviceGroup=1 get xxx *using proxy number from above command will show the number of Fans defined on the site get . numberoffangroups

An RE will need to go to site and confirm that the number of Fans installed on site matches what is configured. Fan unit cables will also need to be checked. FcuDeviceGroup_FanFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> FanFailure This alarm is issued by the FcuDevice Managed Object when there is a problem with one or more of the fans controlled by the Fan Control Unit RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms pr AuxPlugInUnit=2 acl xxx *shows what action you can perform on the Fan Control Unit (FCU) acc xxx restartAuxUnit *restarts the FCU alt *confirm the alarm has cleared If the alarm does not clear after restarting the FCU and RE will need to attend on site.

FanDeviceGroup_LeftFanSpeedTooLow: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> LeftFanSpeedTooLow The LeftFanSpeedTooLow alarm indicates that there are faults in one or more of following units: • BaseBand (BB) Subrack Fan Unit • Radio Frequency (RF) Subrack Fan Unit • Multi-Carrier Power Assembly (MCPA) Subrack Fan Unit • Power Subrack (PS) Fan Unit RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lga | grep FanDeviceGroup_LeftFanSpeedTooLow *show history of current alarm lpr Subrack=2,AuxPlugInUnit=1 st plug *print state of all plug in units get xxx *using proxy number from above command will show the status of the FAN cabx *shows the led status of the fan unit mom AuxPlugInUnit led *description of LED status acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the plug in unit

A red LED showing a steady light indicates a faulty FAN. If the alarm does not clear after restarting the unit an RE is required to attend the site and replace the Fan Unit

AuxPlugInUnit_PiuConnectionLost: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> PiuConnectionLost This alarm is issued by the AuxPlugInUnit Managed Object (MO) when communication between one of the Auxiliary Units (AU) and its device is lost. Use ALEX to check what actions to perform on the various types of Plug In Units. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lga | grep AuxPlugInUnit_PiuConnectionLost *show the history of the current alarm st plug *print state of all plug in units lpr AuxPlugInUnit=x *shows associated equipment with plug in unit get xxx *use this to show more information about the plug in unit acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the plug in unit

If the alarm does not clear an RE will be required to visit the site. TpaDevice_AmplificationError RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> AmplificationError This alarm is issued by the TpaDevice Managed Object (MO) when there is a fault affecting the Transmit Power Amplifier (TPA) in a Multicarrier Power Amplifier (MCPA) or a Radio Unit (RU). RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected cabx *will show you what MCPA is faulty if you check the LED status get cell maxDlPowerCapability *the sector with the faulty MCPA will have a lower value st plug *get the proxy number of the MCPA to be restarted lpr McpaSubrack=x,RbsSlot=x,AuxPlugInUnit=1 *also used to get the proxy number or the MCPA acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the MCPA get cell maxDlPowerCapability *confirm the value has changed – this value is calculated by the RBS

RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels get cell=xxxx maximumTransmissionPower *this is a set value on the RNC

This alarm is also associated with the following alarm on the RNC: UtranCell_NbapReconfigurationFailure

If the alarm does not clear the RE will need to go to site and change the faulty MCPA. RuDeviceGroup_GammaUplinkFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> GammaUplinkFailure This alarm is issued by the Managed Object RuDeviceGroup. The alarm indicates a failure in one or more of the units in the gamma uplink path. Cables carry the gamma link between the following: the Baseband Interface (BBIF) and Radio Frequency Interface (RFIF) boards; the Optical Baseband Interface (OBIF) board and the RRU; the Radio Unit Interface (RUIF) and the Radio Unit (RU). Between all other boards, the physical connections are made through the backplane of the boards. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects

alt *show current alarms lga | grep RbsSubrack=RU1,RbsSlot=x *see if the RU has been inter-failing cabx *LED status will indicate what board is faulty mom AuxPlugInUnit led *description of LED status st plug *get the proxy number of the faulty unit acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the RU

If the alarm is still present after restarting the unit lock the sector on the RNC until the RE can attend and replace the RU. NOTE: Identifying the affected RbsSubrack=RU1/FU1,RbsSlot=2 – RbsSubrack=RU1/FU1,RbsSlot=4 – RbsSubrack=RU1/FU1,RbsSlot=6 –

sector: A sector B sector C sector

If the Radio Unit is faulty the following alarm will be present on the RNC for the disabled sector UtranCell_NbapMessageFailure call_establishment_error UtranCell=xxxx RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels lbl cell=xxxxA1 *lock the sector affected by the faulty RU ldeb cell=xxxxA1 *once the RU has been replaced unlock the sector

Plug-In Unit General Problem RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Plug-In Unit General Problem The alarm is issued when all defined recovery attempts are performed for the PIU or contact with the PIU is lost for at least five minutes. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms lga | grep Subrack=1,Slot=8,PlugInUnit=1 *check the history of the alarm cab *confirm if there is contact or not with the PIU st plug *check the state of PIU acl xxx *check what action you can perform on the PIU using the proxy number acc xxx manualRestart *try restart the PIU

If there’s no contact with the board and the alarm does not clear an RE will be required to go to site and do a hard reset or replace the board. RaxDeviceGroup_GeneralSwError: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> GeneralSwError This alarm can be sent from all Device Groups, all Device Sets, and all Devices. The RBS EM identifies which Managed Object (MO) is issuing the alarm. The likely causes of this alarm are the following: • A software fault • An incorrect version of the software • A faulty configuration, depending on a software error (only when RruDeviceGroup issues the alarm) In this case it is the RAX board that has the problem. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms cab *confirm the RAX boards on the site st plug *check the state of the RAX boards acl xxx *check what action you can perform on the board using the proxy number acc xxx manualRestart *try restart the board

Check that the alarm has cleared. If the alarm is still present an RE will need to go to site and do a hard reset or change the RAX board. Confirm the site is carrying traffic.

TxDeviceGroup_GeneralSwError: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> GeneralSwError This alarm can be sent from all Device Groups, all Device Sets, and all Devices. The RBS EM identifies which Managed Object (MO) is issuing the alarm. The likely causes of this alarm are the following: • A software fault • An incorrect version of the software • A faulty configuration, depending on a software error (only when RruDeviceGroup issues the alarm) In this case it is the TX board that has the problem. As a consequence the capacity of the site is decreased. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms cab *confirm the TX boards on the site st plug *check the state of the TX boards acl xxx *check what action you can perform on the board using the proxy number acc xxx manualRestart *try restart the board

Check that the alarm has cleared. If the alarm is still present an RE will need to go to site and do a hard reset or change the TX board. Confirm the site is carrying traffic. RaxDeviceGroup_GeneralHwError: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> GeneralHwError This alarm can be sent from all Device Groups, all Device Sets, and all Devices. The RBS EM identifies which Managed Object (MO) is issuing the alarm. This alarm indicates that the component is faulty and my need to be replaced. The following alarm ‘UplinkBaseBandPool_UlHwLessThanUlCapacity’ will also be present on the RBS. This indicates that there is less hardware on the site than is licensed for as a result of the faulty RAX board. In this case it is the RAX board that has the problem. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms cab *confirm the RAX boards on the site st plug *check the state of the RAX boards acl xxx *check what action you can perform on the board using the proxy number acc xxx manualRestart *try restart the board

Check that the alarm has cleared. If the alarm is still present an RE will need to go to site and do a hard reset or change the RAX board. Confirm the site is carrying traffic.

LICENSE ISSUES: License Key File Fault: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> License Key file Fault This alarm is triggered when no valid License Key file (LKF) is found on the node. The LKF may be corrupt or have been deleted. When investigating this alarm please ensure that there are no open refs (with BSS) for the site. The site could be in integration and the license may not be loaded yet. CASE 1: BSS are working on the site and license has not been loaded yet. CASE 2: No contact with the Node. Check for alarms on transmission. If alarms are present escalate to either Transmission department or Telkom as necessary. CASE 3: Escalate to BSS to install License Key File. NOTE: For extra notes on installing a LKF see document License Issues on Node B. NodeBFunction_16QamLicenseNotValid: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> 16QamLicenseNotValid This alarm is triggered either when the 16QAM modulation license has expired or the 16QAM modulation license has been removed. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get . license *confirm the license installed on the node get . 16qam *shows the feature and license state on the node mom . featureState16Qam *gives an explanation of the feature st pp *checks the number of E1s on the site

NOTE: Check the MTN Configuration document to confirm what license should be installed on the site based on the number of E1s/Hardware. Also confirm with Optimisation what license should be installed on the Node. If the featurestate16Qam is ACTIVATED and the license state is DISABLED it will result in the alarm. To clear the alarm set the featurestate16Qam to DEACTIVATED. set NodeBFunction featureState16Qam 0 *Sets the feature to DISABLED since no license is installed on the site Password File Fault: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Password File Fault The alarm indicates that the password required at Security Level 1 and 2, for user access via Telnet, SSH, FTP, SFTP or a serial port, was never set or that the locally stored passwd file has been found to be corrupt. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects

alt *show current alarms passwd; rbs ; rbs *resets the password on the Node B alt *confirm the alarm has cleared

UpLinkBaseBandPool_ULHWLessThanULCapacity: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> ULHWLessThanULCapacity This alarm is triggered when the installed license key file defines a capacity that is greater than the available capacity for DCH given by the installed Random Access and Receiver (RAX) boards. It can also be triggered due to a HW failure on one of the RAX boards which decreases the available capacity below the capacity defined by the license key. A faulty RAX board (see alarm: RaxDeviceGroup_GeneralSwError/ RaxDeviceGroup_GeneralHwError) is the general cause of this alarm however if all RAX boards are operational and showing no errors this alarm will need to be escalated to Planning/Optimisation to consider increasing the UL capacity on the site. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get . license *shows the licensed capacity for Channel Elements in the Uplink get nodebfunction ava *shows the available Channel Elements in the Uplink based on the available RAX boards cab *confirm the RAX boards on the site st plug *check the state of the RAX boards

NOTE: Use the Product Number (from the ‘cab’ printout) to search the ALEX document (Product Overview -> Compatibilities for Hardware and Software) to confirm the number of Channel Elements supported by the RAX board. acl xxx *check what action you can perform on the board using the proxy number acc xxx manualRestart *try restart the board

If the RAX board restarts without any more alarms confirm that the ULHWLessThanULCapacity alarm has also cleared. alt *confirm alarm has cleared get nodebfunction ava *the available Channel Elements should now match the licensed capacity

If the fault on the RAX board does not clear after a restart an RE will need to go to site and do a hard reset or replace the RAX board. DownLinkBaseBandPool_DLHWLessThanDLCapacity: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> DLHWLessThanDLCapacity This alarm is triggered when the installed license key file defines a capacity that is greater than the available capacity for DCH given by the installed Transmitter (TX) boards. It can also be triggered due to a HW failure on one of the TX boards which decreases the available capacity below the capacity defined by the license key. If all TX boards are operational and showing no errors this will need to be escalated to Planning/Optimisation to consider increasing the DL capacity on the site. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get . license *shows the licensed capacity for Channel Elements in the Downlink get nodebfunction ava *shows the available Channel Elements in the downlink based on the available TX boards cab *confirm the TX boards on the site st plug *check the state of the TX boards

NOTE:

Use the Product Number (from the ‘cab’ printout) to search the ALEX document (Product Overview -> Compatibilities for Hardware and Software) to confirm the number of Channel Elements supported by the TX board. NodeBFunction_DlHwUsageExceedsDlLicenseLevel: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> DlHwUsageExceedsDlLicenseLevel This alarm is issued by the NodeBFunction Managed Object (MO) when the licensed limit has been reached in the RBS. The cause of the alarm is that the licensed Channel Element (CE) level is reached in the RBS. NOTE: When a site uses more Channel Element capacity than is licensed a grace period will be triggered along with this alarm. Once this grace period expires the site will be limited to the licensed capacity and the alarm will be cleared RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get . license *shows the licensed capacity for Channel Elements get nodeb ava *shows the available Channel Elements in the uplink/downlink based on the installed HW get . grace *shows the grace period left for UL and DL get . limited *shows if the site is limited to the licensed capacity

Example 1: In the example below there are 4 days grace period left. Once dlGraceTimeLeft reaches 0 the parameter dlLimitedByLicenseLevel will be set to True. A value of -1 for ulGraceTimeLeft indicates that the grace period has not yet been triggered for the UL. U2575-Ooseinde_VC> get . grace 100723-12:37:46 10.205.75.161 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/10008 ========================================================================================== MO Attribute Value ========================================================================================== NodeBFunction=1 dlGraceTimeLeft 4 NodeBFunction=1 ulGraceTimeLeft -1 ==========================================================================================

U2575-Ooseinde_VC> get . limited 100723-12:37:55 10.205.75.161 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/10008 ====================================================================================== MO Attribute Value ====================================================================================== NodeBFunction=1 dlLimitedByLicenseLevel false NodeBFunction=1 ulLimitedByLicenseLevel false ======================================================================================

Example 2: In this example the UL and DL grace time has expired (showing a value of 0) and the parameter ulLimitedByLicenseLevel and dlLimitedByLicenseLevel are set to True. U0595-Umgeni_Heights> get . grace 100723-10:50:26 10.209.84.1 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25336 ======================================================================================= MO Attribute Value ======================================================================================= NodeBFunction=1 dlGraceTimeLeft 0 NodeBFunction=1 ulGraceTimeLeft 0 =======================================================================================

U0595-Umgeni_Heights> get . limited 100723-10:50:34 10.209.84.1 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25336 ======================================================================================== MO Attribute Value ======================================================================================== NodeBFunction=1 dlLimitedByLicenseLevel true NodeBFunction=1 ulLimitedByLicenseLevel true ========================================================================================

In order for this alarm to clear you must wait for the grace period to expire. One other way of clearing the alarm is to upgrade the license file to match the available Channel Elements. However this will be up to the planner/optimisation engineer to decide. In most cases this alarm can be as a result of a spike in CE usage after a site is brought on air. All users will attempt to do a location update and use up all resources causing this spike.

NodeBFunction_UlHwUsageExceedsUlLicenseLevel: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UlHwUsageExceedsUlLicenseLevel This alarm is issued by the NodeBFunction Managed Object (MO) when the licensed limit has been reached in the RBS. The cause of the alarm is that the licensed Channel Element (CE) level is reached in the RBS. NOTE: When a site uses more Channel Element capacity than is licensed a grace period will be triggered along with this alarm. Once this grace period expires the site will be limited to the licensed capacity and the alarm will be cleared RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get . license *shows the licensed capacity for Channel Elements get nodeb ava *shows the available Channel Elements in the uplink/downlink based on the installed HW get . grace *shows the grace period left for UL and DL get . limited *shows if the site is limited to the licensed capacity

Example 1: In the example below there are 2 days grace period left. Once ulGraceTimeLeft reaches 0 the parameter ulLimitedByLicenseLevel will be set to True. A value of -1 for dlGraceTimeLeft indicates that the grace period has not yet been triggered for the DL. U4574-Elukhanyisweni_High_School> get . grace 100723-11:02:03 10.201.89.97 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25511 =============================================================================================== MO Attribute Value =============================================================================================== NodeBFunction=1 dlGraceTimeLeft -1 NodeBFunction=1 ulGraceTimeLeft 2 ===========================================================================================

U4574-Elukhanyisweni_High_School> get . limited 100723-11:02:42 10.201.89.97 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25511 ============================================================================================ MO Attribute Value ============================================================================================ NodeBFunction=1 dlLimitedByLicenseLevel false NodeBFunction=1 ulLimitedByLicenseLevel false

============================================================================================

Example 2: In this example the UL and DL grace time has expired (showing a value of 0) and the parameter ulLimitedByLicenseLevel and dlLimitedByLicenseLevel are set to True. U0595-Umgeni_Heights> get . grace 100723-10:50:26 10.209.84.1 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25336 ======================================================================================= MO Attribute Value ======================================================================================= NodeBFunction=1 dlGraceTimeLeft 0 NodeBFunction=1 ulGraceTimeLeft 0 ======================================================================================= U0595-Umgeni_Heights> get . limited 100723-10:50:34 10.209.84.1 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/25336 ======================================================================================== MO Attribute Value ======================================================================================== NodeBFunction=1 dlLimitedByLicenseLevel true NodeBFunction=1 ulLimitedByLicenseLevel true ========================================================================================

In order for this alarm to clear you must wait for the grace period to expire. One other way of clearing the alarm is to upgrade the license file to match the available Channel Elements. However this will be up to the planner/optimisation engineer to decide. In most cases this alarm can be as a result of a spike in CE usage after a site is brought on air. All users will attempt to do a location update and use up all resources causing this spike.

TRANSMISSION: Loss of Tracking: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Loss of Tracking The alarm is issued when the system clock enters "loss of tracking" mode. The value of the attribute, syncRefStatus is changed to LOSS_OF_TRACKING. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get 10 *prints the synchronization Managed Object acl 10 *shows what action you can perform on the MO acc 10 resetLossOfTracking *resets loss of tracking

When prompted to Enter mo LDN: Equipment=1,Subrack=1,Slot=2,PlugInUnit=1,ExchangeTerminal=1,E1PhysPathTerm=pp1

NOTE: To see the values for syncRefStatus and syncRefActivity use the below commands: mom refstate mom refactivity

Loss of Synch Reference Redundancy: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Loss of Synch Reference Redundancy This alarm can be triggered as a consequence of the primary alarm Loss of Tracking. As a consequence of the fault, the number of available synchronization references is reduced and there is only one left. Clearing the Loss of Tracking alarm should clear this alarm. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms get 10 *prints the synchronization Managed Object acl 10 *shows what action you can perform on the MO acc 10 resetLossOfTracking *resets loss of tracking When prompted to Enter mo LDN: Equipment=1,Subrack=1,Slot=2,PlugInUnit=1,ExchangeTerminal=1,E1PhysPathTerm=pp1

NOTE: To see the values for syncRefStatus and syncRefActivity use the below commands: mom refstate mom refactivity

PDH Loss of Signal: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> PDH Loss of Signal This alarm is issued by an E1 on the site when the Exchange Terminal (ET) board cannot detect any signal at the input port. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning

If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take. PDH Loss of Frame: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> PDH Loss of Frame This alarm is issued by an E1 on the site when the Exchange Terminal (ET) board cannot detect any signal at the input port. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take.

Loss of Cell Delineation on IMA Link: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Loss of Cell Delineation on IMA Link This alarm is triggered when the node cannot extract the ATM cells from the PDH frame. If there are many similar alarms it is likely that common hardware is at fault. However if there is just one alarm, the fault is probably with one IMA link. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take. Remote Defect Indication on IMA Link: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms ->Remote Defect Indication on IMA Link This alarm is triggered when the remote (far end) node has detected a transmission fault on the IMA link. This will also trigger the alarm ‘IMA Link Reception Unusable at Far End’ indicating that the link at the far end has become unusable. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take. However in most cases this will need to be escalated to Telkom.

IMA Link Reception Unusable at Far End: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms ->IMA Link Reception Unusable at Far End This alarm is triggered when the IMA link at the far end becomes unusable. It is likely that you will also see the alarm ‘Remote Defect Indication on IMA Link’ for the same link – indicating that the IMA link is faulty.

RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take. However in most cases this will need to be escalated to Telkom. IMA Link Reception Misconnected: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms ->IMA Link Reception Misconnected This alarm is triggered when there is a configuration mismatch between two nodes. The IMA link in the receive direction is not connected to the same remote (far end) IMA unit as the other reception links in the IMA group. It is likely that you will see one or both of the following alarms: ‘Remote Defect Indication on IMA Link’, ‘IMA Link Reception Unusable at Far End’. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1

E1s that are LOCKED and DISABLED they will most likely not be defined in the IMA Group and are waiting to be integrated. Check on the HS 3G Site planning page to confirm number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning If any E1/IMA Links are UNLOCKED and DISABLED use the DXX Web Reporter to check for any active alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

NOTE: To check for any errors on the E1 links use the following command: pmx e1 pm –m 1

Consult with Transmission department as for the next steps to take. However in most cases this will need to be escalated to Telkom.

IMA Group Insufficient Links: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms ->IMA Link Insufficient Links This alarm is triggered when one or more of the IMA links have failed. This reduces the number of IMA Links in the IMA group and the “required number of links” is less than defined. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1 get ima required *shows the required number of links needed in the IMA group

CASE 1: In the below example there are two UNLOCKED and ENABLED E1 links. However there is only 1 E1 link defined in the IMA group. U4784-Vredenburg_WPK_Silo> st pp 100730-12:08:56 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== Proxy Adm State Op. State MO =================================================================================== 110 1 (UNLOCKED) 1 (ENABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp1 111 0 (LOCKED) 0 (DISABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp2 114 1 (UNLOCKED) 1 (ENABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp3 ===================================================================================

U4784-Vredenburg_WPK_Silo> st ima 100730-12:09:02 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== Proxy Adm State Op. State MO =================================================================================== 17 0 (DISABLED) TransportNetwork=1,AtmPort=1-1-ima1 49 0 (DISABLED) TransportNetwork=1,ImaGroup=1-1-ima1 50 1 (ENABLED) TransportNetwork=1,ImaGroup=1-1-ima1,ImaLink=1 =================================================================================== U4784-Vredenburg_WPK_Silo> get ima required 100730-12:22:21 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== MO Attribute Value =================================================================================== ImaGroup=1-1-ima1 requiredNumberOfLinks 1 ===================================================================================

Check on the HS 3G Site planning page to confirm the number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning In the above example BSS will be required to do an IMA upgrade and add the 2nd working link to the IMA group. You might also need to contact Transmission and confirm what the correct configuration is on the DXX. IMA Group Insufficient Links at Far End: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms ->IMA Group Insufficient Links at Far End

This alarm is triggered when one or more of the IMA links have failed. This reduces the number of IMA Links in the IMA group and the “required number of links” is less than defined. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms – this will indicate what E1/IMA Link the alarm has been triggered for st pp *shows the status of the E1s on the site st ima *shows the status of the IMA links get ima *shows what IMA Link is associated with a particular E1 get ima required *shows the required number of links needed in the IMA group

CASE 1: In the below example there are two UNLOCKED and ENABLED E1 links. However there is only 1 E1 link defined in the IMA group. U4784-Vredenburg_WPK_Silo> st pp 100730-12:08:56 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== Proxy Adm State Op. State MO =================================================================================== 110 1 (UNLOCKED) 1 (ENABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp1 111 0 (LOCKED) 0 (DISABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp2 114 1 (UNLOCKED) 1 (ENABLED) Equipment=1,Subrack=1,Slot=1,PlugInUnit=1,Cbu=1,ExchangeTerminal=1,E1PhysPathTerm=pp3 ===================================================================================

U4784-Vredenburg_WPK_Silo> st ima 100730-12:09:02 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== Proxy Adm State Op. State MO =================================================================================== 17 0 (DISABLED) TransportNetwork=1,AtmPort=1-1-ima1 49 0 (DISABLED) TransportNetwork=1,ImaGroup=1-1-ima1 50 1 (ENABLED) TransportNetwork=1,ImaGroup=1-1-ima1,ImaLink=1 =================================================================================== U4784-Vredenburg_WPK_Silo> get ima required 100730-12:22:21 10.202.83.113 8.0c RBS_NODE_MODEL_M_1_8 stopfile=/tmp/29260 =================================================================================== MO Attribute Value =================================================================================== ImaGroup=1-1-ima1 requiredNumberOfLinks 1 ===================================================================================

Check on the HS 3G Site planning page to confirm the number of links that are actually completed on site. http://10.217.201.232/facts/report/Tx/DXX/8600/Planning/3G+HS+Site+Planning In the above example BSS will be required to do an IMA upgrade and add the 2nd working link to the IMA group. You might also need to contact Transmission and confirm what the correct configuration is on the DXX. Heartbeat Failure: OSS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarm Handling Operational Instruction -> Heartbeat Failure This alarm can be triggered when connectivity to the OSS is lost or the site has gone down due to transmission. RBS: moshell Uxxxx *log onto the node If you cannot MOSHELL to the node check for alarms on the RNC riorda_m@uas3g3> moshell u2652

__ __ ____ _____ | \/ |/ __ \ / ____| / \ | \ / | | | | (___ / /\ \ | |\/| | | | |\___ \ / ____ \| | | | |__| |____) | /_/ \_\_| |_|\____/|_____/ OSS Framework for MoShell-8.0c /\

Checking ip contact...Not OK Unable to connect to 10.209.89.241:23 Cannot connect to MO service, exiting... RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels pmx cell=xxxx pmTotNoRrcC|pmNoSystemRabRe|pmNoRabEstablish|pmNoNormalRabRel|pmNoFailedAfterAdm -m 1 *confirm if the site is carrying traffic

CASE 1 – Site is operational and carrying traffic but no connection to the node: If all channels are unlocked and enabled, the site is carrying traffic and no alarms are present on the RNC escalate this fault to Transmission/IP Routing. In this case VC32 and VC33 (used for O&M connectivity) will need to be checked by Transmission. Transmission will possibly need to recreate these VCs as they are hanging. Use the command ‘traceroute ’ to confirm where the problem is before escalating to Transmission. The traceroute command shows where the IP packet stops in the network, either at the DXX or IP router. riorda_m@uas3g3> traceroute 10.204.78.145 traceroute: Warning: Multiple interfaces found; using 10.217.12.16 @ ce1 traceroute to 10.204.78.145 (10.204.78.145), 30 hops max, 40 byte packets 1 10.217.12.252 (10.217.12.252) 0.614 ms 0.349 ms 0.279 ms 2 10.195.17.165 (10.195.17.165) 1.043 ms 0.878 ms 0.901 ms 3 10.195.17.13 (10.195.17.13) 0.952 ms 0.816 ms 0.844 ms 4 10.90.217.5 (10.90.217.5) 1.418 ms 0.991 ms 90.119 ms 5 10.90.250.14 (10.90.250.14) 1.840 ms 6.803 ms 1.097 ms 6 10.90.204.6 (10.90.204.6) 1.317 ms 1.131 ms 0.938 ms 7 10.204.47.252 (10.204.47.252) 0.855 ms 0.859 ms 0.729 ms 8 10.204.47.230 (10.204.47.230) 1.505 ms 1.504 ms 1.940 ms 9 10.204.78.145 (10.204.78.145) 7.887 ms 7.250 ms 8.814 ms riorda_m@uas3g3>

CASE 2 – Site is DISABLED and there are alarms present on the RNC: If the channels on the RNC are UNLOCKED and DISABLED and there are alarms present on the RNC it is more than likely a problem on the Transmission side and will need to be escalated to TX or Telkom based on the alarms present on the DXX. Possible RNC Alarms: NbapDedicated_RncRbsControlLinkDown UtranCell_ServiceUnavailable UtranCell_ServiceUnavailable UtranCell_ServiceUnavailable

transmission_error unavailable unavailable unavailable

IubLink=Iub_U2652-Bay_View_Estates,NbapDedicated=1 UtranCell=2652A1 UtranCell=2652B1 UtranCell=2652C1

CASE 3 – New Site Check in REMEDY if this is a new site. Generally a new site that is not yet fully integrated will display the Heartbeat Failure alarm as it is not yet connected on the OSS.

MISCELLANEOUS: Carrier_RejectSignalFromHardware RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> RejectSignalFromHardware This alarm is triggered by the Carrier Managed Object. This alarm can be issued for the AIU/sAIU, FU, MCPA, RFIF board, RRU, RU, TRX board or the TX board. If none of the boards are faulty it could be related to a frequency configuration issue. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms cabx *check the equipment on the site (particularly the RU and FU) get . fqband *check the highedge and lowedge of the Antenna Branch

Contact optimisation/planning to confirm that the UARFCNs and high and low edges are set correctly. If they are set correctly restart the RU relevant to the affected sector. st plug *shows the plug in units. acl xxx *check what action you can perform on the PIU using the proxy number acc xxx manualRestart *try restart the PIU

NOTE: Identifying RbsSubrack=RU/FU, RbsSubrack=RU/FU, RbsSubrack=RU/FU,

the affected sector: RbsSlot=2 – A sector RbsSlot=4 – B sector RbsSlot=6 – C sector

RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – some carriers may be disabled get cell=xxxx uarfcn *displays the DL and UL frequencies set for each carrier

The following alarm may be present on the RNC: UtranCell_NbapMessageFailure

NbapCommon_Layer3SetupFailure: RBS Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> Layer3SetupFailure This alarm is issued by the NbapCommon Managed Object. It is issued for the Node B Application Protocol (NBAP) Common. The likely cause is a communication failure between the RBS and RNC. The following alarm, ‘UtranCell_ExternalResourceUnavailable’ could also be present on the RNC for the affected cells. RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – all cells will be DISABLED and also NbapCommom on the IubLink ======================================================================================== 14622 1 (UNLOCKED) 1 (ENABLED) RncFunction=1,IubLink=Iub_U2141-Adgam_Place 14624 0 (DISABLED) RncFunction=1,IubLink=Iub_U2141-Adgam_Place,NodeSynch=1 14625 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,IubLink=Iub_U2141-Adgam_Place,NbapCommon=1 14626 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,IubLink=Iub_U2141-Adgam_Place,NbapDedicated=1

22656 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1 22669 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1,Hsdsch=1 22670 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1,Hsdsch=1,Eul=1 22684 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1,Rach=RACH 22685 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1,Pch=PCH 22695 1 (UNLOCKED) 0 (DISABLED) RncFunction=1,UtranCell=2141A1,Fach=1 ======================================================================================== lbl Iublink_Iub_Uxxxx *block the IubLink GERNC1> lbl Iublink=Iub_U2141 =================================================================================== 14622 RncFunction=1,IubLink=Iub_U2141-Adgam_Place 14625 RncFunction=1,IubLink=Iub_U2141-Adgam_Place,NbapCommon=1 14626 RncFunction=1,IubLink=Iub_U2141-Adgam_Place,NbapDedicated=1 =================================================================================== ldeb Iublink_Iub_Uxxxx *deblock the IubLink lst .xxxx confirm the channels are UNLOCKED and ENABLED

Confirm the alarm has cleared on the RBS. If the alarm is still present escalate to Transmission.

RNC: UtranCell_NbapReconfigurationFailure: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UtranCell_NbapReconfigurationFailure RBS does not provide the configured output power due to some internal hardware problems. Check the RBS for any HW faults, generally with the MCPA units. RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels get cell=xxxx maximumTransmissionPower *this is a set value on the RNC

RNC will internally set a reference value representing the previously configured maximumTransmissionPower in the cell. An alarm will be issued if the RBS reports a degradation of its maxPowerCapDl below 2dB. This alarm can be associated with the following alarm on the RBS: TpaDevice_AmplificationError RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected cabx *will show you what MCPA is faulty if you check the LED status get cell maxDlPowerCapability *the sector with the faulty MCPA will have a lower value st plug *get the proxy number of the MCPA to be restarted lpr McpaSubrack=x,RbsSlot=x,AuxPlugInUnit=1 *also used to get the proxy number or the MCPA acl xxx *check what action you can perform on the proxy number acc xxx restartAuxUnit *try restart the MCPA get cell maxDlPowerCapability *confirm the value has changed – this value is calculated by the RBS

If the alarm does not clear the RE will need to go to site and change the faulty MCPA. NbapDedicated_RncRbsControlLinkDown: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> NbapDedicated_RncRbsControlLinkDown This alarm is triggered when there is a temporary disturbance or interruption in the Iub link . It is also generally associated with the alarm UtranCell_ServiceUnavailable as all Utrancells will be DISABLED. RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – cells will be UNLOCKED and DISABLED

This alarms indicates a problem with Transmission. It will need to be either escalated to the Transmission department or Telkom. When this alarm is present you will be unable to Moshell onto the RBS. Check the DXX web reporter for any current alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

Once the Transmission issue has been cleared, confirm that all cells and channels are UNLOCKED and ENABLED and the site is carrying traffic.

UtranCell_ServiceUnavailable: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UtranCell_ServiceUnavailable This is a secondary alarm and is triggered when there is a problem with the NBAP signalling on the Transmission link. It is associated with the alarm NbapDedicated_RncRbsControlLinkDown RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – cells will be UNLOCKED and DISABLED

This alarm could indicate a problem with Transmission. It will need to be either escalated to the Transmission department or Telkom. Check the DXX web reporter for any current alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

UtranCell_ExternalResourceUnavailable: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UtranCell_ExternalResourceUnavailable This alarm is triggered by the Utrancell Managed Object. It is issued for the Node B Application Protocol (NBAP) Common. The likely cause is a communication failure between the RBS and RNC. The following alarm, ‘NbapCommon_ Layer3SetupFailure’ could also be present on the RBS. RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – all cells will be DISABLED

Check the RBS for any alarms.

RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms

This alarm could point to a problem with Transmission. It will need to be either escalated to the Transmission department or Telkom. Check the DXX web reporter for any current alarms. http://10.200.31.30/dll/w3rt.dll Login: ESA Password: ESA

UtranCell_RBSLocalCellNotAdded: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UtranCell_RBSLocalCellNotAdded This alarm is triggered when the RNC is trying to activate the UTRANCELL but is unable to as there is a mis-configuration with the RBS Local Cell on the RBS. The alarm UtranCell_NbapMessageFailure could also be present for the affected sectors.

RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – cells will be UNLOCKED and DISABLED get cell=xxxx cid *check the Cell ID on the RNC RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms and confirm what sector is affected get rbslocalcell cellid *check the Cell ID on the RBS and confirm the mismatch

Check the FACTs page to confirm the correct Cell ID to be used – “CI Allocation Lookup”

http://10.217.201.232/facts/report/CI_TRANSLATION/CI_TRANSLATION_LOG If the RBS that has the incorrect values use the following command to correct it: set rbslocalcell=SXCX localCellId=xxxxx *corrects the Cell ID on the RBS

NOTE: S1C1 – Sector 1,Carrier1 S1C2 – Sector1,Carrier2 etc..... If the RNC has the incorrect value use the following command to correct it: set cell=xxxxA1 cid=xxxxx *corrects the Cell ID on the RNC

Confirm alarm has cleared and the cells are UNLOCKED and ENABLED. Finally be sure to save a CV – see Node B Restart Procedure for help on creating a CV. cvls cvms [cvname_date_time][ursername][comment]

UtranCell_NbapMessageFailure: RNC Library in ALEX: Operation and Maintenance -> Fault Management -> Alarms -> UtranCell_NbapMessageFailure This alarm can be triggered when the RBS cannot fulfil the request to set up a UTRAN Cell issued by the RNC. The RBS will need to be checked for any alarms. RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – all cells will be DISABLED

Check the RBS for any alarms.

RBS: moshell Uxxxx *log onto the node lt all *load all Managed Objects alt *show current alarms

CASE 1 – Hardware Failure on RBS: A faulty TX board on the RBS can cause the untrancells on the RNC to be DISABLED. See alarm ‘Plug-In Unit General Problem’.

EXTRA NOTES: MTN Configuration Types RBS 3202

SMOKE Text Analysis

Node B Restart Procedure

License Issues on Node B

USEFUL Commands: Checking Performance: RNC: pmx cell=xxxx pmtotnoutranrejrrcconnreq|pmNoFailedAfterAdm -m 1 *shows rejections and failures for the last hour pmx cell=xx pmTotNoRrcC|pmNoSystemRabRe|pmNoSysRelSpeech|pmNoRabEstablish|pmNoNormalRabRel|pmNoFailedAfterAdm -m 1 pmx utrancell=xxxx pmCellDowntime -m 4 *shows downtime for last 4 hours pmr *displays all the predefined performance reports on the RNC pmom . *gives a description of a specific counter pmom . pmCellDowntimeAuto mom . *gives a description of a particular parameter pmx plug processorload -m 1 *shows processor load on individual boards

RBS: pmx e1 pm –m 1 *shows the errored seconds and severely errored seconds for the E1 links over the last hour pgets e1 *shows the current errors on the E1 links pmx aal2ap pmunsuc -m 2 *show all unsuccessful attempts across the aal2 paths for all classes pmx edch pmnoallowedeul pgets *shows all active counters on the RBS pmr *displays all the predefined performance reports on the RBS get radio no *shows the current number of active radio links

Locking and Unlocking site on RNC: RNC: moshell XXRNCX *log onto the RNC lt all *load all Managed Objects alt *show current alarms lst .xxxx *show status of channels – confirm if site has one or two carriers bls cell=xxxx *soft blocks the cells – takes 5 minutes lbl cell=xxxx *blocks all the channels lst .xxxx *confirm the channels are LOCKED and DISABLED ldeb cell=xxxx *unlock cells and channels one sector at a time lst .xxxx *confirm the channels are UNLOCKED and ENABLED

Miscellaneous: ala *shows current alarms WITH details altk *shows current alarms – UNACKNOWLEDGED and ACKNOWLEDGED get 0 *shows equipment type license key *shows licenses installed on the node cvls *prints configuration version on the node