Analysis of AT&T Wireless Failed CRM Upgrade Project via Project Management Institute (PMI) Accepted Standards

UNIVERSITY OF COLOMBO FACULTY OF GRADUATE STUDIES

Analysis of AT&T Wireless Failed CRM Upgrade Project In accordance with Accepted Standards of Project Management Institute (PMI)

Submitted for the fulfillment of requirements as Individual Assignment for the module Information Systems Project Management

By Evan Pathiratne

1

ABSTRACT This documentation is relating to a research and analysis project performed to identify the causes of a major project failure.

This does not include any in-depth information as to the processes carried out in the project but a high level overview of the activities involved in methodology used and what should have been the best steps/best practices that had to have been used to avoid this failure.

The project considered was a CRM upgrade for AT&T Wireless. The timeline was in the 2003-2004 season. AT&T Wireless Services Inc as of now is a defunct company which was acquired by Cingular Wireless LLC in 2004. AT&T Wireless was part of a much larger mother company AT&T Corp (American Telephone & Telegraph) before this acquisition.

This study revealed the less than perfect actions taken at the middle stages of the project which lead to its total demise. The documentation also contains a brief introduction to why the project was first launched, nature of CRM systems and best practices to be used in CRM upgrade projects (specific to Siebel CRM upgrades recommended by Oracle). Despite the advances in the IT industry over the last decade, there are still many failed IT projects. Many factors contribute to such failures. This research focuses on all aspects of failure related to the selected project. Supporting information was gathered from case study found on the web which contained process adopted the actual failure and some personal interviews conducted. After complete analysis the recommendations are put forward aligned with the best practices given by the Project Management Institute (PMI).

2

TABLE OF CONTENTS

3

1.0 INTRODUCTION 1.1 THE PROJECT AND REASONING BEHIND INITIATION This project (Siebel CRM system upgrade) was launched by AT&T Wireless in 2003. By this time the company which was a strong market leader in the 2000-2001 seasons has given away to competition from much more fast growing technology driven companies such as Verizon and Cingular.

In 2003 AT&T was still using an old fashioned phone network (supporting only TDMA) which did not support data transfers over mobile phones. Management set into action programs for conquering the mobile phone market segment which would eventually mean competitive advantage because of the increasing use of mobile at that time. As part of this initiative AT&T launched an expensive new GSM system that could handle not only data over cell phone but had the compatibility with other overseas providers. This change in technology meant that the existing CRM systems had to be upgraded to match the new needs. This new upgrade to the CRM will add new capabilities which will offer productivity advantages, increased business value, and lower operational costs to AT&T in the long run.

1.2 CHALLENGES FACED UPFRONT This was a challenge itself but it was made greater by the fact that AT&T had to convince its customer and new prospects to move to the new (GSM) systems. Less than half of AT&T Wireless new customers were choosing GSM, while its other competitors were signing up more than 75 percent of its new customers from GSM. This meant a further burden on AT&T to commence with the system upgrade and do the convincing both at once.

All this added with the fact at that time the Federal Communications Commission was on the process of approving a new rule on ‘number portability’ which will mean that mobile telephone users should be able to retain their mobile telephone numbers when changing from different mobile operators. AT&T had to manage all these concerns when managing the project.

4

Competitors Rise in Power

Deadlines Set by Government for number porting

Challenges faced by AT & T

Old Technology Used in the Network

Convincing Customers to move to a GSM Network

1.3 PROJECT OBJECTIVES  Provide AT&T with the existing CRM benefits even after the network is migrated to a new GSM network.  Support customer representatives in converting prospects into the new GSM network.  Provide competitive advantage against competitors after project implementation  Have an infrastructure complaint against new government set regulations on number porting.

1.4 THE NATURE OF CRM To focus on CRM (Customer Relationship Management) system upgrades its best at first to understand the concept of CRM and nature of such projects. In comparison with other projects (ex- Construction, Database, Network Infrastructure and Security projects) managing customer relationships is not a technology solution. It’s more similar to a business culture. What a CRM system does is it enables managing customer relationships through automating marketing, sales, back office functions such as client servicing and improved analytics to enable more accurate forecasting.

5

Automation is a component of CRM projects, but it should be the final component considered in such projects. A proper CRM solution must be built based on a company’s measurable business objectives, its reengineered business processes and dynamic customer requirements.

Therefore executive level management must be at the forefront of a CRM project. They have to be actively involved in decision making to ensure such scale projects are completed efficiently. CRM in way is a methodology for doing business.

2.0 THE PROJECT 2.1 TEAM AND METHODOLOGY TASKED The CRM suite which AT&T used was Oracle’s Siebel. Oracle provides a standard framework for company’s in determining the best possible upgrade agenda.

CRM upgrade project is similar to a CRM implementation; however, upgrade projects can be more efficient than implementations because of the use of previous implementation efforts and its outputs. Within this upgrade project AT&T had several key areas which included tasks that began with an Initial Upgrade Assessment then requirements definition. This is followed by a thorough testing phase until final, deployment. After which final users will be trained.

The project team consisted of project managers having team leaders report to them. The whole team was comprised of system developers as well as code testers. A Technical consultant for Siebel was also present to provide advisory functions. The methodology used for the project was standard SDLC (System development Life Cycle). Another addition to this life cycle was the use of iterative prototyping cycles. Pre-defined by the project managers this would enable the different systems states to be tested before actual final development and implementation.

Customer care executives were indirectly involved to the project since they were tasked with converting customers from the old network to a GSM network.

6

2.2 THE PROJECT FAILURE When the month of November 2003 approached AT&T was still lagging behind schedule .It was one of the last in the industry without number porting adopted into its telecommunication network. With the deadline looming most of the project tasks was hurried causing lapses in total quality of the CRM. The changes to code, requirements were done without a formal approval processes because it was thought it would be time consuming.

• The project team very optimistic on the fact that the CRM was completed successfully launched the system for the first time only to crash it completely. After that it never came The First System back up even the old CRM system had no functionality now. Crash

The Second System Crash

• The project team was at this time still struggling to get the CRM working when in late November customers when beginning to change their numbers between different providers (a feature at this time should have been with every telecommunications provider as imposed by the government law) caused another major system crashed. This time t was on the portion of the system where the number portability was handled.

• There were 50,000 [AT&T] customers per week that didn’t have service • AT&T was last among the top providers in terms of new customer signups for the last quarter that year. The Impact and • Meaning estimated $100 million in lost revenue Loss • Company sold to a previous competitor Cingular in February for $41 billion.

The effect this project had on AT&T

Similar to a CRM implementation a CRM upgrade also can be a project which can provide a company with a edge over its competitors, as well as if it is not done properly and proper measures are not taken lead to a company’s total demise. Next chapter examines the reasons for this failure in more detail

7

3.0 REASONS FOR FAILURE The failure was attributed to more than one reason. And indeed certain factors caused a greater impact than others. The weakness of the whole project life cycle is classified under the knowledge areas of project management as defined by PMI. (Each cause of failures is mentioned below in more detail)

1. Bad Communication management 2. Complacencies in Quality management 3. Ineffectively practiced Risk management 4. Time management needs to have been more effective and deadline centric 5. Initial Scope management inaccuracies 6. Lack of solid HR management throughout the project 7. Procurement management done without adequate research

3.1 BAD COMMUNICATION MANAGEMENT There was no communication between the lead integrators of the project and its teams. Deloitte and Touche was in charge of this operation. According to answers taken from former employees problems in communication can be clearly identified.

"Everything was siloed (separated) among the different groups, and we all worked independently of each other" “When they finished testing, code had changed elsewhere in the system, rendering the testing meaningless”

It should be the responsibility of the team leads to carefully communicate the direction of the project, what to do and when to do it. The extent of poor communication can be seen by the testing program that was run. The unit testing was done without essential interaction between other groups. This meant that whenever there is a change requirement (code/design changes) needed, it will most of the time be incompatible with other units. Ideally whole testing should take a holistic approach in the organization backed by a strong a communication plan.

8

Another communication flaw that lead to lack of employee morale was the constant rumors of layoffs and offshore outsourcing. "[The rumors] slowed things down,” "When stuff like that happens, people start looking for other work. I know I was looking for other work when I should have been testing."

For AT&T at a critical time in the project their long time CIO decided to resign. And to add to that the replacement was a former CIO who presided over the offshore outsourcing of portions of his previous company.

"We would see our people going into these long meetings with people from Indian companies. We‟d see whiteboards that had questions like, what opportunity do we have to outsource?" To add to all this, after the resignation and new appointment of the CIO AT&T CEO confirmed the layoff rumors. This was still when the project was at its critical stages. He told the project teams that the company would lay off 1,900 workers. But he did not say which departments the layoffs will occur and when it will be commenced.

An interview with a formal employee told "you had to wonder why you were working so hard if the whole thing was going to go away anyway" The confidence and morale of the employees at this time hit an all time low because most of them were not focused on the job at hand, and some of them were even seeking employment options elsewhere. 3.2 COMPLACENCIES IN QUALITY MANAGEMENT In major upgrades like this proper quality mechanisms are vital the overall success. As explained in previous chapters a CRM is not just a simple off the shelf software, it is a tailored system that reflects the total organizations objectives and goals. In this project however as expected the quality management mechanisms should have been set in place but the effective execution was done poorly.

9

A former employee (code tester) said, "In other projects I‟ve worked on, the project managers would freeze code while the teams did revisions to their pieces so you could test everything against the same code base, I didn‟t hear of that happening on this project." This meant that when certain unit testing was completed, code had changed elsewhere in the system without the other tester’s knowledge, making the testing meaningless. A properly set and executed quality management plan should not have caused this.

Due to a time constrain set by government the project managers were relaxing testing requirements for various pieces of the system as the project went forward. What this meant was on the launch date the system was bug-ridden to such an extent that it crashed. The upgrade project caused the total CRM system to be unstable; it was now unusable. "It went down and stayed down; they couldn‟t get it working again project staffers were brought in to work on the system for three straight days to try to rescue it, with no luck” said a former employee.

3.3 INEFFECTIVE TIME MANAGEMENT Due to failure of proper time management the quality of the product got affected. As the government posed deadlines for number porting closed-in industry sources say that at that time AT&T was counting on a further extension. Among companies affected by this law AT &T was one of the last to still hold out on number porting. With such pressures to complete a critical project on time adequate time management measures had to have been put in place. Apart from the CRM upgrade the customer conversion process (to a GSM network) also was inefficient. According to an employee “the customer service representatives needed about 20 minutes, on average, to work through five or six screens that fetched information from about 15 legacy systems”. Such slow access to customer information records was a reason why AT&T was lagging behind schedule.

10

3.4 INEFFECTIVELY PRACTICED RISK MANAGEMENT Risk management prepares companies for efficient operation in disaster situations. According to an employee "Two weeks before launch there was rollback talk but project managers hadn‟t preserved enough of the old system to make it feasible” It seems that after most of the project was completed the project managers identified that the earlier set estimates based on the complexity of the project was wrong. "[Integration was] far more technical and far more difficult to test in advance and also fix problems once they appeared" A risk management plan will set into motion contingency actions in such instances allowing options for recovery.

3.5 INITIAL SCOPE MANAGEMENT INACCURACIES Improper initial estimation of complexity and gravity of project was also a contributing factor to the project downfall. Michael Keith the then president of AT&T Wireless also acknowledged the fact that the integration was far more technical and far more difficult problems once they appeared. A former employee says “the back-end systems integration work was so complex that it wasn‟t unusual to see teams of 20 or more people assigned to write connections for a single system”. When you look at what the team had to do, that is the IT staff had to rip apart old relations from the different legacy systems to the new client/server-based Siebel system, and also rewrite them to work over the Web.

Creating effective change management is also part of scope management plan. If necessary change management process was set and practiced the critical testing of units would have not been compromised. As described earlier project managers relaxed the testing requirements as the deadline neared. The best practice is getting it formally approved by an appointed change approval board.

The other two factors, bad HR management and procurement management indirectly affected the project outcome.

11

The below diagram graphically represents the causes of failure

12

STEPS FOR FAILURE AVOIDANCE 4.0 PREVENTATIVE ACTIONS Experience has taught us that the best possible cure for defect is prevention rather than reaction. Below details the actions that could have been taken at project inception that would have eliminated the flaws of this unsuccessful project.

Quality Management

Communication Management

Other Factors

Preventive Actions

Scope Management

Risk Management

HR Management

Preventative Actions performed to avoid failure

4.1 GOOD QUALITY MANAGEMENT Quality Planning  Have a Proper plan-Freeze code on every software unit when testing  Design of Experiments (A Quality Planning Technique) – If such a technique is used the total team could collectively identify what is the most optimal quality needed for the project to succeed  Use Benchmarking – Adopt Oracle recommended Siebel best practices and look into other similar projects completed successfully. Use the standards and methodologies used as a baseline for this project.

13

Quality Assurance  Separate Team-According to resource availability its best to have a separate team for quality assurance, this will provide unbiased results.  Perform quality audit- Perform weekly service improvement checks and recommend changes Quality Control  Identify ways to eliminate causes of unsatisfactory performance  Use techniques such as cause and effect diagramming to pin point the flaws and eliminate that from its root case.  Testing Plan 1. Initially Unit testing should be done, it should be planned and commenced to ensure individual components are as error free as possible. 2. Integration should be done as a phased approach instead of the whole system at once. This will be a better method because a system such as a CRM is very vast and complex. 3. After the integration testing is completed Systems testing should be performed. This will test the entire system as one.

4.2 GOOD COMMUNICATION MANAGEMENT It is vital to keep the communication lines between the team and the project managers open and clear. Even if company wide layoffs due to offshore outsourcing are inevitable during the project, it is best to communicate that news clearly and finish up the changes in team structures as early in the project as possible. Communication takes most of a Project Manager’s time (at least 90%). Such time should be put use to the benefit of the project. Communications Planning  Find Needs -Clearly identify, analyze and determine the information and communication needs of the project stakeholders (who needs what information, when it’s needed, Way the information will be handed over)

14

Communications Planning (Cont’)  Information distribution and availability —Put stern measures for making needed information available to project stakeholders as fast as possible  Performance Reporting—Gather and analyze performance information. (Status reporting, progress measurement, and through those reports future forecasting). Performance reporting will be useful in human resource management as well. Status Reports will contain, -

start and completion dates which milestones passed ,percentage complete any threats /risks to the projected timeline description of problems encountered and resolved personnel/ equipment limitations

 Administrative closure—effectively broadcast relevant information from sources to formalize phase or project completion

4.3 ENSURE RISK MANAGEMENT The goal of this is to minimize negative impacts on the project from unexpected scenarios and prepare the project team for effective workarounds or mitigation plans to ensure project objectives are met on time.

Identify all Risks

Probability and Impact Assessment

Sort Top Ten Risk Items

Appoint Risk Owners

Track Progress of Top Ten Risks

Proposed Risk Management Process for AT&T

15

Ensure Risk Management (Cont’) 1. Initially company CEO, CIO, CFO and project managers should identify the risks that can be associated to this project in terms of Business(competitors, suppliers,

cash

flows),

Technology(software,

hardware,

network),Organizational(human resources, team work) and the Project Management( estimates, communications and resource allocation) 2. Then it should be the responsibility of the project managers to perform a probability and impact assessment of all the identified risks. 3. Based on the above, the Top Ten Risk Items should be identified and communicated to project team leads. 4. Each risk item should be a responsibility of an assigned risk owner. Such an appointed person will have the necessary experience and skill set to handle all areas associated with that risk. 5. Top Ten Risk Items should be tracked monthly in areas of planning, leadership, time and cost estimation and the monthly resolution progress associated to it. This progress should be made aware to all project team leads. 6. Apart from the main top ten risks it will be a good practice to also maintain a „Watch List‟ of other risk items. These risks will be identified as low risks but such risks may also have impact on the project

Risk Response Plan Develop a Contingency Plan (a Plan B) - what will happen if the system does crash? Do you have manual processes to revert back or portions of old system preserved? (This wasn’t the case for this project). AT&T Siebel system upgrade involved complex integrations and testing that pushed against a fixed deadline on number portability imposed by government. By identifying and preparing for the worst case scenario(s), even though it is unlikely will minimize risk of a massive business failure.

16

Response plan will include, 1. Storing backups of old system as and when necessary determined by project managers to ensure rollback in case of system crashes 2. The risk owner’s proactive involvement is resolving issues related to risk and updating progress to project managers monthly.

4.4 EFFECTIVE HUMAN RESOURCE MANAGEMENT Compared with most unsuccessful projects this project’s failure was not attributed to team conflicts. But it was mostly due to the lack of morale caused by rumors and eventual actualities of layoffs. Human resources should be managed aligned with effective communication strategy. (Described earlier)

Proposed Team structure and reporting hierarchy for AT&T Project

The project team will work under each of their respective team leads. QA team is separated to provide better results.

Effective HR management will allow individual and group skills to enhance the project performance. HR management will assign roles, responsibilities, and reporting relationships.

17

4.5 BETTER SCOPE MANAGEMENT As discussed under reason for failure it was identified that efficient change management was not set in place for the project. When a change( a deviation of normal planned objectives/scope) is identified it should pass through a formal process before it is accepted/ rejected. This way the relevant stakeholders of the project can adjust the project plans to accommodate the change with minimum effect to the projects overall goals. Planning Documents

Change Request

Is Change Acceptable?

No

Yes

Planning Scenarios

Implement Change

Update Planning Documents

Asses the Impact

Yes

Approve Chang?

No

Reject Change

Proposed integrated change control process for AT&T Project A change review team appointed will evaluate, accept/reject change requests based on provided information. This team will comprise of Project Manager and Team Leads.

4.6 OTHER FACTORS Procurement management could have been planed better. All of AT&T competitors selected another provider for number porting, AT&T should have carefully assessed this and identified the difficulties that would arise in choosing a different provider because it will increase the integrations need. AT&T choice of a competing vendor meant additional layers of integration which caused complexities. It’s best to reduce unnecessary integration whenever possible.

18

5.0 CORRECTIVE ACTIONS Even though the earlier mentioned processes and methodologies was not present in the project the project managers should have been able to carry out some remediation actions to correct the flaws. Below details such corrective actions that should have been performed that would have made the project a success

Since there was no roll back options, workarounds or plan B set in place the project managers should have relied more on the existing resources.

1. Persist with the Testing Plan even though the project ran late 2. Thorough Testing – Unit testing and integration testing is very important; this will reduce the implementation risk drastically. And all change requests should be going through the team leaders before it is accepted.

3. Eliminate the doubts team members have on the layoff rumors. A projects biggest asset is its human resources. When other planning areas have failed a cleverly set strategy for people motivation could have been the best way to get out of the mess that they have created.

4. Postpone layoffs and offshore outsourcing The problem here was that they have set in their minds even before project started that they would outsource/offshore.

19

6.0 CONCLUSION This example for a project failure can be used as a baseline for study of other project failures, since its contained elements that will urge a company’s top management into eagerly perusing its goals, as well as actions related to ineffective project management that led to a company being sold to its own competitors. This analysis project revealed even the smallest of mistakes is still being done in major projects. And also the human resource is still the most vital asset in any project.

7.0 ACKNOWLEDGEMENTS 1. The Lecturer in charge of this module Mr. Mangala Perera , who guided me in understanding the concepts and best practices used in modern day project management.

2. The following case study has been used in the compilation of this research and further information can be obtained from it. -

CXO Media Inc.( 2004). ‘Project Management: AT&T Wireless SelfDestructs’. [Online]. http://www.cio.com/article/32228/Project_Management_AT_T_Wireles s_Self_Destructs?page=1&taxonomyId=3198 [Accessed 5th April 2010]

20