What is Enough Planning

462

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 62, NO. 4, NOVEMBER 2015

What is Enough Planning? Results From a Global Quantitative Study Pedro Serrador and Rodney Turner

Abstract—Project planning is widely thought to be an important contributor to project success. However, there is a little research to affirm its impact and give guidance as to how much effort should be spent on planning to achieve best results. We aim to rectify this omission. Data was collected on 1386 projects from 859 respondents via a global survey. A significant relationship was found between the quality of the planning deliverables and success. Detailed analysis of the data collected revealed an inverted-U relationship between the percentage of effort spent on planning and project success. After correcting for key moderator effects, a significant relationship with an R2 of 0.15 was revealed. Further analysis showed that the fraction of planning effort that maximized the project success was 25% of project effort. This was substantially more than the 15% mean value reported by respondents. The greatest impact was found to be on the broad success measures with a lesser effect on project efficiency: time; budget; and scope. The inverted-U relationship between effort spent on planning and project success indicates that projects can spend too much time in planning, as well as too little. But we found that projects are spending less time in planning than the optimum to achieve best results. Index Terms—Efficiency, planning, project, success.

I. INTRODUCTION N THIS paper, we investigate the impact of project planning and project plans on project success. Does better project planning lead to more successful outcomes on projects? Traditional project management is based to a large extent on conjecture, with little empirical evidence in support of some of the memes [1]. Project planning is one such meme. Received wisdom is that planning is very important and the more effort that is put into the planning process, the better the project plans and the more successful will be the project [2], [3]. Time spent on planning activities will reduce risk and improve success. On the other hand, inadequate planning will lead to a failed project, [4], [5]. If poor planning has led to failed projects, then perhaps trillions of dollars have been needlessly lost [6]. Our survey of the literature suggests that there is a relationship between the amount of project planning and the quality of project plans, and between both of those and project success. But is there an optimum amount of planning and how much is too much? We believe this relationship needs to be clarified. This leads to our research question:

I

Manuscript received March 24, 2014; revised October 31, 2014, April 26, 2015, and May 26, 2015; accepted June 12, 2015. Date of publication July 23, 2015; date of current version October 16, 2015. Review of this manuscript was arranged by Department Editor P. ED Love. P. Serrador is with Serrador Project Management, Mississauga, ON L5E 3G3, Canada (e-mail: [email protected]). R. Turner is with SKEMA Business School, LSMRC Univ Lille Nord de France, Lille F59777, France (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TEM.2015.2448059

RQ: What is the impact of the amount of planning effort on project success? II. LITERATURE REVIEW A. Project Success Before we can discuss the impact of the project planning phase on success, we need to define what we mean by project success. Unfortunately, as Pinto and Slevin [7, p. 67] note “There are few topics in the field of project management that are so frequently discussed and yet so rarely agreed upon as the notion of project success.” Shenhar and Dvir [8] suggest five measures of project success: 1) project efficiency, 2) impact on the team, 3) impact on the customer, 4) business success, and 5) preparing for the future. In this paper, we refer to the following. 1) Project efficiency: completing the desired scope of work on time and within budget, while meeting scope goals. 2) Project success: meeting wider business, strategic and enterprise goals. We follow Cooke-Davies [12] who says that project management success is achieving the project efficiency goals and project success is achieving business and enterprise goals. Ultimately, whether or not, the latter achieved is a subjective judgment by key stakeholders [9]. Thomas et al. [5, p. 106] state that “Examples abound where the original objectives of the project are not met, but the client was highly satisfied,” as well as the reverse. Zwikael and Globerson [10] and Dvir et al. [2] suggest that project efficiency and project success are often correlated. Serrador and Turner have shown that this correlation is 0.60 [9]. While the measure of project success in the past has focused on tangibles [11], current thinking is that ultimately project success can best be judged by the primary sponsor [12] and will be based on how well they judge that the project meets the wider business and enterprise goals. B. Project Planning Mintzberg describes planning as the effort to formalizing decision making activities through decomposition, articulation, and rationalization [13]. In construction, preproject planning is defined as the phase after business planning where a deal is initiated and prior to project execution [14]. Another definition of planning is “what comes before action” [15]. For the purpose of this paper, we will use these definitions. 1) Planning phase: the phases and associated effort that comes before execution in a project. 2) Planning effort: the amount of effort in work hours expended in planning.

0018-9391 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

SERRADOR AND TURNER: WHAT IS ENOUGH PLANNING? RESULTS FROM A GLOBAL QUANTITATIVE STUDY

C. Planning in Construction Project management has a long history in the construction industry and there have been a number of studies on the relationship between planning and project success. Hamilton and Gibson [16] found that the top third of projects from a planning completeness perspective had an 82% chance of meeting their budget goals compared to only 66% of projects in the lower third. Similar results are seen for schedule and design goals. Shehu and Akintoye [17] found in a study of construction programs that effective planning had the highest criticality index of 0.870 of all the critical success factors studied. The project definition rating index (PDRI) is a widely adopted method for industrial projects to measure completeness of project planning [14]. By filling a questionnaire, the completeness of project planning can be assessed. No planning is indicated by a PDRI score of 1000 while a score of 200 or less is good planning [3]. Gibson, Wang, Cho, and Pappas show that effective preproject planning using PDRI leads to improved performance in terms of cost, schedule, and operational characteristics. They found that scores under 200 were associated with cost and schedule performance 3% below budget, whereas PDRI scores above 200 were associated with costs 13% over budget, 21% behind schedule, and twice as many change orders [18]. (Please note, PDRI is a measure of the completeness of project plans, not the amount of effort that has gone into the planning process which differs from our research question.) In addition, Gibson and Pappas note a marked difference in empirical measurements of project success based on the PDRI score [19]. In the construction industry, project success is closely linked to project efficiency so this can apply to efficiency and success [20]. D. Planning in the Information Technology industry The reports of high failure rates for software projects are well known [6], [21]. Some studies in this area have tried to quantify how much planning should be done for software projects. Poston [22] states that in software development projects, testing was 43% of overall project effort for the projects studied, whereas planning and requirements accounted for only 6% of effort. He also notes that the earlier defects are identified such as in the planning/design phase, the less they cost to fix. M¨uller and Turner reported a correlation between postcontract signing planning and project schedule variance [23]. Also, Tausworthe notes the importance of the work breakdown structure (WBS), a planning artifact, on software project success [24]. Deephouse et al. showed that project planning was consistently associated with success more than other practices [25, p. 198]. The dependence for successful planning was 0.791 for meeting targets and 0.228 for quality. However, they do qualify their findings by noting that respondents may have thought that if “the project was late, clearly the plan was not realistic.” E. Planning and Success in the General Project Management Literature Thomas et al. [5, p. 105] state, “the most effective team cannot overcome a poor project plan” and projects which started down

463

the wrong path can lead to the most spectacular project failures. Morris [4, p. 5] similarly argues that “The decisions made at the early definition stages set the strategic framework: Get it wrong here, and the project will be wrong for a long time.” Munns and Bjeirmi [26] state that for a project which is flawed from the start, successful execution may matter to only to the project team, while the wider organization will see the project as a failure. Thus, there is a recurring theme that planning is inherently important to project success or one could argue that without it project management would not exist. However, in these works it is just conjecture. Pinto and Prescott [27] found that a schedule or plan had a correlation of 0.47 with project success, while technical tasks had a correlation of 0.57 and mission definition a correlation of 0.70. Pinto and Prescott [27] again found that planning factors dominated throughout the project lifecycle. Planning was found to have the greatest impact on the following success criteria: perceived value of the project (R2 = 0.35); and client satisfaction (R2 = 0.39). Shenhar [28] notes that better planning is the norm in high- and superhigh-technology projects. This was found to apply consistently to the deliverables normally produced in the planning phase. Dvir and Lechler [29] found that the quality of planning had a +0.35 impact on R2 for efficiency and a +0.39 impact on R2 for customer satisfaction. Dvir et al. [2] noted the correlation between aspects of the planning phase and project success. The planning procedures effort was found to be less important to project success than defining functional and technical requirements of the project. The correlation was 0.297 for functional requirements and 0.256 for technical requirements. Zwikael and Globerson [10, p. 694] noted the following, “organizations, which scored the highest on project success, also obtained the highest score on quality of planning.” What appears to be clear is that activities we defined as a part of the planning phase: requirements definition, scope definition, and technical analyses are important to project success [30]. It is clear that activities occurring prior to execution and along with planning are important to project success [2]. Turner and M¨uller note that “There is growing evidence that competence in the traditional areas of the project management body of knowledge are essential entry tickets to the game of project management, but they do not lead to superior performance [31, p. 6]. They are hygiene factors, necessary conditions for project management performance.” F. Reasons not to Plan Andersen [32, p. 89] questions the assumption that project planning is beneficial from a conceptual standpoint. He asks, “How can it be that project planners are able to make a detailed project plan, when either activities cannot be foreseen or they depend on the outcomes of earlier activities?” Bart [33] makes the point that in research and development projects too much planning can limit creativity. Collyer et al. [20, p. 109] describe examples of failed projects such as the Australian submarine and the Iridium satellite projects. They say, “While useful as a guide, excessive detail in the early stages of a project may be problematic and misleading

464

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 62, NO. 4, NOVEMBER 2015

in a dynamic environment.” Collyer and Warren suggest that in dynamic environments, creating detailed long-term plans can waste time and resources and lead to false expectations [34]. Aubrey et al. [35] note that for one project management office (PMOs) they studied, overly rigorous planning processes resulted in an impediment to rapidity. Flyvbjerg et al. [36] highlight that senior management can choose not to use the estimates from the planning phase. Zwikael and Globerson [10] note that even though there is a high quality of planning in software and communications organizations, these projects still have low ratings on success. Chatzoglou and Macaulay [37] note that any extra planning will result in a chain reaction delay in the next phases of the project. Thomas et al. [5] write that for most projects there are pressures to reduce the time and effort spent on the planning phase. Also, Chatzoglou and Macaulay [37, p. 174] consider why planning is sometimes shortened or eliminated because managers think, “It is better to skip the planning and to start developing the requested system. However, experience shows that none of the above arguments are valid.” The literature does not support the conclusion that planning should not be done in projects though some caveats are highlighted. G. How Much to Plan? Surprisingly, little research has been done on how much planning should be done in projects. We have looked at planning quality and now we will look at the impact of the amount of effort spent planning. Daly stated, without presenting evidence, that schedule planning should be 2%, specifications 10%, and final design 40% of the total cost [38]. Now much of this design is done during execution. Similarly, Poston states that planning and requirements should be 6% of project cost, product design should be 16%, and detailed design should be 25% [22]. Empirical guidance on how much time should be spent in planning has become less common over time. Whether this is because this guidance was found not to be effective, the diversity of technology projects increased or it simply fell out of favor is not clear. Nobelius and Trygg [39] found front-end activities made up at least 20% of the project time. Similarly, Wideman [40] states that the typical effort spent in the planning phase in construction projects is approximately 20% of the total work hours. Chatzoglou and Macaulay [37, p. 183] outline a rule of thumb for planning effort for IS/IT projects, the three times programming rule and the lifecycle stage model: “one estimates how long it would take to program the system and then multiply by three” to get the total effort. Software testing is estimated to take roughly an equal amount of effort as development [41]. This leaves one third of total effort for the planning phase and other miscellaneous tasks. However, all of the above are just observations of how much time people spend on projects planning. There is no indication of what is the appropriate amount of planning. Choma and Bhat [42, p. 5, 7] found that “the projects with the worst results were those that were missing important planning components.” However, they also found that “the projects in this sample that took longer in planning had the worst results.” Their analysis

points to that either too much planning can be negative to project success or that a planning phase that lasts too long can be an indicator of a problem project. Similarly, Choo reported that there is a U-shaped relationship between problem definition time and project duration [43] in a study of 1558 projects in a global computer manufacturing firm. He reported a clear relationship between problem definition time, which shows similarity to the planning phase, and one measure of success, project duration. In this firm, it was correlated with project savings which he inferred was related to project success. In his final model, he reported a R2 of 14.8 between problem definition time and project duration and an optimal problem definition time between 0.20 and 0.30 of the overall project time for the cases studied, which were just IT projects in one company.

H. Conclusion Dvir et al. [2, p. 94] state that “With the advancement in computerized planning tools and the blooming in project management training, a certain level of planning is done in all projects, even in those that eventually turn out to be unsuccessful projects. Hence, when a certain level of planning is done in all types of projects, a significant statistical correlation cannot be found in the data.” This is an important point. The question of whether some planning versus no planning is correlated with project success may be a moot. The benefits of planning have been confirmed through the practice of project management. It has, thus, become an expected part of all projects. It has become, as suggested by Turner and M¨uller [31], and as a part of all project management books of knowledge, a hygiene factor for successful projects. The question now is how much planning leads to the greatest success. Table I summarizes our literature review above. From this table, we can see that the preponderance of the literature suggests that planning is important for project success. Some of this is based on empirical evidence, some just on conjecture. A smaller number of authors suggest that there is a negative correlation, but one of these is based on conjecture, and this paper suggests that you can do too much planning, but it is also beneficial up to a point. Table II summarizes the empirical results from the literature review. A metaanalysis using weighting was considered as described in Hwang et al. [44] but we did not consider this valid, given the varied nature of the source documents: different industries, different methodologies, and different types of cross-functional projects. A high-level metaanalysis reviewing the means was completed instead. These studies used different methodologies and even different definitions of planning and success. If we compare this to the approximately 20–33% effort spent on planning reported by Nobelius, Trygg, and Wideman, there appears to a clear return on this investment in terms of project success [39], [40]. Thus, from the literature review, we can get a preliminary answer to our research question: project planning effort has been found important for project success. However, what is

SERRADOR AND TURNER: WHAT IS ENOUGH PLANNING? RESULTS FROM A GLOBAL QUANTITATIVE STUDY

465

TABLE I SUMMARY OF THE REVIEWED LITERATURE ON RELATIONSHIP BETWEEN PLANNING AND SUCCESS Positive empirical relationship between planning and success Pinto and Prescott [27] Hamilton and Gibson [16] Deephouse et al. [25] M¨uller and Turner [23] Shenhar et al. [30] Dvir et al. [2] Gibson and Pappas [19] Dvir and Lechler [29] Gibson et al. [18] Zwikael and Globerson [10] Salomo et al. (2007) Wang and Gibson [3] Choma and Bhat [42]

Conceptual positive relationship between planning and success

No relationship between planning and success

Conceptual negative relationship between planning and success

Empirical negative relationship between planning and success

Tausworthe [24] Chatzoglou and Macaulay [37] Munns and Bjeirmi [26] Morris [4] Shenhar [47] Shenhar et al. [47] Ceschi [55] Zwikael and Globerson [10] Thomas et al. [5] Shehu and Akintoye [17] Blomquist et al. (2010) Collyer et al. [20]

Flyvbjerg et al. [36]

Bart [33] Andersen [32]

Choma and Bhat [42]

Zwikael and Globerson [10] Collyer et al. [20]

TABLE II SUMMARY OF EMPIRICAL RESULTS AFTER SERRADOR [46] Study

Empirical Relationship Aggregate

Pinto and Prescott [27]

Deephouse et al. [25]

Dvir et al. [2]

Dvir and Lechler [29]

Planning found to have the greatest impact on success factors perceived value of the project client satisfaction The dependence for successful planning was 0.791 for meeting targets and 0.228 for quality.

Meeting the planning goals is correlated 0.570 to overall project success measures. Quality of planning had a +0.35 impact on R2 for efficiency and a +0.39 impact on R2 for customer satisfaction.

2

Impact of planning on success, normalized to R2 Efficiency 2

Overall Success R2 = 0.39

R = 0.35

R = 0.35

R2 = 0.39 Average R2 = 0.37 R2 = 0.625

R2 = 0.34

R2 = 0.052 Average R2 = 0.34 R2 = 0.32

R2 = 0.32

R2 = 0.35

R2 = 0.35

R2 = 0.39

R2 = 0.28

R2 = 0.29

R2 = 0.39 Average R2 = 0.37 Zwikael and Globerson [10]

Gibson et al. [18]

Salomo et al. (2007)

Wang and Gibson [3]

Overall Average

Planning quality correlates as follows: R = 0.52 for cost R = 0.53 schedule R = 0.57 technical performance R = 0.51 customer satisfaction R2 = 0.42 Correlation between planning completeness and project success Project planning/risk planning and innovation success Goal clarity/process formality and innovation success PDRI score of a building construction project is related to project cost and schedule success (R = 0.475)

R2 = 0.27 R2 = 0.28 R2 = 0.32 R2 = 0.26 Average R2 = 0.28 R2 = 0.42

R2 = 0.42

R2 = 0.33

R2 = 0.30

R2 = 0.27 Average R2 = 0.30 R2 = 0.23

R2 = 0.23

R2 = 0.33

R2 = 0.33

R2 = 0.34

466

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 62, NO. 4, NOVEMBER 2015

the relationship? We, therefore, will investigate the following hypotheses. H1: There exists a relationship between planning quality and project success. H2: There exists a relationship between planning effort and project success. H3: There is an optimum amount of planning effort. III. RESEARCH METHODOLOGY We undertook a research to build on the existing literature and test our hypotheses. To do this, we took a post positivist view that a relationship can be found between the amount of project planning and project success. Postpositivism falls between positivism, where a completely objective solution can be found to a research question, and phenomenology, where all experiences are subjective [45]. Because perception and observation are at least partially based on subjective opinion, our results cannot be fully objective. Some concepts such as project success may not be fully quantifiable and are impacted by subjective judgment of the participants and sponsors. Postpositivism understands that though positivism cannot tell the whole truth in business research, its insights are nonetheless useful. We used inductive analysis to examine these relationships. We gathered data on quantities such as effort of planning phase, effort of overall project, and percentage of project effort which was dedicated to the planning phase. This information could be gathered using a quantitative approach employing techniques such as surveys, a qualitative technique such as interviews, or a mixed methods approach. The research question being examined here is well defined, so a quantitative approach was taken. A. Survey Data were collected from practitioners who are members of Project Management Institute (PMI) or members of LinkedIn project management groups. Invitations to fill out a questionnaire (an on-line questionnaire using surveymonkey.com) were posted on discussion boards of PMI communities of practice (CoPs) as well as a number of LinkedIn groups. A notice was also included in some groups’ mailings. We sought to gather a large dataset over as wide range as possible of different types of projects. Identifying the overall population sample pool was not possible. Though the membership numbers for the LinkedIn groups are available (typically in the 1000s) and membership numbers in the PMI CoPs are also available (membership up to the 10 000s), memberships in each of these groups are not mutually exclusive. There is also no way to know how many members read group postings. Respondents were asked to think of projects they had been involved with and select two: one “more successful” and one that they defined as “less successful.” The survey was targeted at project managers but was not restricted to people who managed the projects. The majority of respondents identified themselves as project managers or senior project managers. Participants were also asked about aspects of the project which we used as the 12 moderators in our analysis, see Table VIII for a full list and appendix for the survey questions.

It is the case that with most studies of project success that use questionnaires or interviews, the results rely on participants stating how successful a project was. This is subjective by nature. One could argue that there may be ways to measure success in an objective way; however, this likely only applies to project efficiency. Therefore, this paper will largely be concerned with perceived project success as reported by participants. To measure this factor, questions in the survey were largely based on a combination of the success dimensions defined by M¨uller and Turner [23] and Shenhar et al. [47]. Survey questions, in general, used a 5 or 7 point Likert-like numeric scale [48]. Pure Likert scales were not used as there were several questions where numerical responses were appropriate. The varying scale was used partially to follow the scales from the existing literature: using 7 point scales to allow optimum ordinal value for numeric ranges and 5 point scales for subjective ratings. Since a variety of scales was used, this ensured that item context effects as per Podsakoff et al. [49] were not an issue. Monosource bias and other response biases can occur in self-rated performance measures as discussed by Podsakoff et al. [49]. By targeting project managers, we intended to receive information from the individual who would have the best overall view of the project. The participants were asked to rate how other stakeholders viewed the success of the project. Some monosource bias was, therefore, inevitable. However, to reduce the impact and for privacy reasons, anonymity was allowed in the survey and company names were not captured. Projects can vary extensively and their need for planning can also be variable [34]. The goal of the research was to gather a large enough dataset to study the importance of planning in general over a wide range of projects. A total of 865 people started the survey with 859 completing at least the first portion of it which requested information on one successful project. Although each participant was asked to provide data on two projects, not all participants entered data for two projects; therefore, the total number of projects was 1539. After removal of outliers and bad data, the usable total available for study was 1386 projects. Projects which reported planning efforts over 2 × standard deviation (SD) were considered outliers or abandoned projects and fell outside the scope of this research. Projects which reported no planning time were removed as bad data. The remaining projects were reviewed for normality over the success factors and were found to have a normal distribution. People from over 60 countries answered the survey. The largest numbers came from the USA, 313 (36.5%), India, 59 (6.9%), Canada, 57 (6.6%), and Australia, 19 (2.2%). Some 183 (21.3%) chose not to answer the question. Although there was a preponderance of responses from North America, there was good representation from the whole world. B. Approach Inductive analysis was used to find the relationship between planning and success. In general, the simplest relationships were tested first and then testing continued using progressively more involved techniques. The typical progression is to use correlation analysis to understand if there is a relationship followed by linear regression to see if there is a dependent relationship.

SERRADOR AND TURNER: WHAT IS ENOUGH PLANNING? RESULTS FROM A GLOBAL QUANTITATIVE STUDY

467

TABLE III SUMMARY OF INDICES AND FACTORS Indices and factors Planning Effort Index Efficiency Factor Success Factor Overall success Factor

Description Ratio of planning phase effort (in hours) compared to overall project effort. Summated scale of project time, budget, and scope (1–7). Summated scale of the success of the project from the point of view of sponsors, clients, team, and end users (as reported by respondents) (1–5). Summated scale of project success including efficiency variables, success variables, and respondents’ overall assessment (1–5).

TABLE IV CRONBACH ALPHA ANALYSIS OF SUCCESS MEASURES Summary for scale: Mean = 30.776; SD = 8.45; Valid N: 1378; Cronbach alpha: 0.905; Standardized alpha: 0.922; Average interitem corr.: 0.632 Mean if deleted Var. if deleted StDv. if deleted Itm-Totl- Correl. Squared-Multp. R Alpha if deleted Project time goals Project budget goals Scope and requirements goals Project sponsors success rating Project team’s satisfaction Client’s satisfaction End users’ satisfaction Overall project success rating

26.496 26.045 25.831 27.398 27.437 27.366 27.411 27.446

51.906 55.008 54.238 55.604 57.063 55.901 57.228 55.770

7.205 7.417 7.365 7.457 7.554 7.477 7.565 7.468

0.640 0.539 0.637 0.821 0.791 0.827 0.767 0.814

0.516 0.416 0.421 0.840 0.725 0.851 0.744 0.783

0.903 0.912 0.900 0.884 0.888 0.884 0.889 0.885

TABLE V DESCRIPTIVES BY INDUSTRY WITH ANOVA RESULTS

Construction Financial services Utilities Government Education Other High technology Telecommunications Manufacturing Health care Professional services Retail All Groups p(F)

Planning effort index

Success Factor

Project success rating

Efficiency Factor

Overall Success Factor

Valid N

0.146 0.133 0.145 0.126 0.132 0.140 0.123 0.170 0.132 0.145 0.139 0.173 0.153 0.010

3.486 3.328 3.349 3.382 3.410 3.284 3.401 3.419 3.214 3.408 3.328 3.151 3.347 0.689

3.528 3.355 3.455 3.423 3.480 3.231 3.477 3.393 3.286 3.303 3.352 2.933 3.361 0.882

4.630 4.618 4.535 4.731 5.080 4.455 4.784 4.805 4.298 4.895 4.685 4.367 4.656 0.397

3.660 3.354 3.553 3.438 3.530 3.233 3.538 3.458 3.295 3.408 3.292 3.000 3.397 0.496

41 257 42 152 42 157 223 133 122 113 69 35 1386

This is followed by a nonlinear regression if a significant linear relationship is not discovered. Finally, moderated hierarchical regression analysis (MHRA) was used to understand how this relationship is impacted by moderating variables [50]. IV. RESULTS AND ANALYSIS Respondents were asked to provide project person hours spent on both planning and on the project as a whole. To facilitate the analysis, we created some indexes and factors (see Table III). Success factors were calculated using a summated scale, that is, each term was normalized, summed together, and then the sum was again normalized to a 1–5 or 1–7 range. After a confirmatory factor analysis using normalized varimax rotation was completed on the success factor, a Cronbach alpha analysis was performed (see Table IV). In general, an alpha value of 0.9 is required for practical decision making situations. while a value of 0.7 is considered to be sufficient for

research purposes [51]. The average was greater than 0.8 in all cases, and alpha would not greatly improve by deleting any of the survey questions (see appendix). The results of Cronbach’s alpha analysis supported the initial assumptions that the elements identified for measuring success were valid measures of success for this survey and accurately measured the judgments of respondents [2], [10], [47]. Projects came from a wide variety of industries (see Table V). The analysis of variance (ANOVA) results show a significant p value for planning effort index. This shows that planning varies with industry. Success does not vary significantly by industry; there are successful projects in all industries. A. Planning Quality Versus Success After performing a factor analysis on the 12 moderators collected (see Table VIII), it became clear that four of them were connected and described an underlying planning quality

468

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 62, NO. 4, NOVEMBER 2015

TABLE VI REGRESSION ANALYSIS FOR PLANNING QUALITY FACTOR VERSUS THE SUCCESS MEASURE Regression Summary for Dependent Variable: Success Factor R = 0.515; R2 = 0.265; Adjusted R2 = 0.265; p < 0.0001 Beta Intercept Planning Quality Factor

–0.515

B 5.259 –0.891

factor. We, therefore, named it the planning quality factor. A normalized varimax rotation was selected to achieve the highest loadings and best model fit. The other factors were not found to be significant. Planning Quality Factor = mean of the following four responses: 1) quality of the WBS; 2) quality of goals/vision; 3) stakeholder engagement level; and 4) experience level of team. Some components can be clearly seen to be the result of a thorough analysis and planning exercise: Quality of the WBS and quality of goals/vision. Another component can be seen as an important input to a good planning effort: Stakeholder engagement level. Experience level of the team does not immediately come to mind as a planning related variable. However, based on the factor analysis, it is related to planning quality. One can speculate that this may be because a better planning cycle may allow the selection of a more effective team or that more experienced teams complete more effective planning. Now that the meaning of the factor has been defined, we next completed a regression of success versus the planning quality factor (see Table VI). This shows a statistically significant relationship with a low p < 0.0001 between the planning quality factor and overall success. In addition, there is a strong R2 of 0.265. This result is in broad agreement with the average R2 reported in the literature of 0.34. The planning quality factor calculated here is a measure of planning quality similar to what was studied in the previous research. However, it is not as comprehensive, so a lower R2 is to be expected. This result is in keeping with previous research and validated the methodology of this research. Therefore, hypothesis H1 is supported. H1: There exists a relationship between planning quality and project success.

B. Planning Effort Versus Success To start the effort impact analysis, we examined the relationship between planning effort index and project success rating. Note that the rating was the single measure of overall success reported by participants. The rating was used rather than the success factors because it is easier to graph for illustration purposes. We found that in general the planning index increases within the success category. The exception is the failure category that showed the highest mean planning effort index of any group. The ANOVA analysis did not show a statistically significant relationship. By looking at these means, it appeared that a simple linear relationship did not exist. These data were now plotted to get a visual picture of the relationship (see Fig. 1). Looking at this graph, we can see the lowest amount of effort was typically spent on projects deemed

p-level 0.000 0.000

Fig. 1. Mean plot of planning effort index by project success rating with error bars (where 5 is a highly successful project).

not fully successful. In this case, one can hypothesize that inadequate planning impacted project success. Projects deemed outright failures reported the mean highest percentages of upfront project planning. This is an interesting finding in keeping with Choma and Bhat [42]. Based on Fig. 1, it was decided to review the data with an assumption that the relationship between the effort index and project success is not linear but could be polynomial in nature (see Fig. 2). There is clearly a quadratic relationship between the planning effort index and the overall success factor. This fits with position that if a project spends too much effort in the planning phase, too much of the overall budget and time will be spent before execution [37]. This would make the project less successful overall. Also, complex or challenging projects with a low probability of success may have very long planning phases. Conversely, a project that spends too little upfront time planning will also be less successful [2]. Therefore, an inverted-U curve fits with the proposition and the findings of the literature review. Table VII shows a more detailed analysis based on a nonlinear regression. The overall was p < 0.0059 which shows statistical significance of the polynomial model specification. The fit of this relationship is quite low with R2 less than 0.01. This suggests a small causal relationship indicating that less than 1% of project success can be attributable to the amount of effort spent planning. This is counterintuitive and deserved further analysis. The residuals were examined to confirm normality and homoscedasticity and results were acceptable. Twelve variables were examined to find their impact on the relationship between

SERRADOR AND TURNER: WHAT IS ENOUGH PLANNING? RESULTS FROM A GLOBAL QUANTITATIVE STUDY

469

TABLE VII NONLINEAR REGRESSION ANALYSIS OF PLANNING EFFORT INDEX VERSUS OVERALL SUCCESS FACTOR Regression Summary for Dependent Variable: Overall Success Factor R = 0.086; R2 = 0.007; Adjusted R2 = 0.006, p < 0.0059 Beta B p-level Intercept Planning effort index Planning effort index ∗∗2

0.255 −0.239

3.191 2.026 −4.063

0.000 0.001 0.003

TABLE IX MHRA ANALYSIS FOR SIGNIFICANT MODERATORS IN THE PLANNING EFFORT INDEX VERSUS OVERALL SUCCESS FACTOR RELATIONSHIP Variables entered Main Effects Planning effort index Planning effort index∗∗ 2 Moderators Internal vs vendor based Interaction Terms WBS∗ Planning effort index WBS∗ Planning effort index∗∗ 2 Experience∗ Planning effort index Experience∗ Planning effort index∗∗ 2 Internal∗ Planning effort index Internal∗ Planning effort index∗∗ 2 F for Regression R2

Fig. 2. Scatterplot and curve fitting for overall success factor versus planning effort index.

TABLE VIII SUMMARY OF MODERATOR FINDINGS FOR DEPENDENT VARIABLE SUCCESS AND INDEPENDENT VARIABLE PLANNING EFFORT INDEX Moderator Quality of WBS Quality of the goals/vision Stakeholder engagement level Experience level of team Internal versus Vendor based Methodology type (traditional versus agile) Novelty to organization Technology level of the project Project length Project complexity New product versus Maintenance Team size

Role Versus Project Success Independent variable and moderator Independent variable Independent variable Independent variable and moderator Moderator Independent variable and potential moderator Independent variable No relationship No relationship No relationship No relationship No relationship

planning and success, initially in the factor analysis and then as moderators (see Table VIII). When we completed an MHRA using these interaction relationships, we get the results shown inTable IX. For the MHRA analysis, we will be trying to discover the underlying relationship between dependent and independent variables and understand how it is impacted by moderating variables as per Sharma et al. [50]. MHRA analysis in SPSS enables us to explore these relationships in more detail. We can see that through the moderator analysis, a more significant relationship between planning

Step 1

Step 2

Step 3

1.972∗∗ −4.044∗∗

2.030∗∗ −4.103∗∗

13.007∗∗∗ −25.064∗∗∗

0.028

0.010

8.510∗∗∗ 0.016

−2.927∗∗∗ 4.662∗∗∗ −3.965∗∗∗ 8.944∗∗∗ 0.619+ −1.330+ 26.851∗∗∗ 0.145

5.404∗∗ 0.006

+ p < 0.10; ∗ p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001.

effort and project success has been uncovered, with R2 = 0.145, p < 0.001. In order to confirm the final model, we completed a general regression analysis with the interaction terms (see Table X). The result of this model is both a very good p value 60% over, 45–59% over, 30–44% over, 15–29% over, 1–14% over, on budget, under budget 7 point scale—see above

[2], [10], [47]

7 point scale—see above

[2]

5 point scale—see above

[23]

5 point scale—see above

[23]

5 point scale—see above

[23]

7 point scale

[53]

3 point scale—Low, Medium, High 3 point scale—< 1 year 1 to 3 years, > 3 years 4 point scale—Excellent, Good, Poor, Very Poor/Not used 4 point scale—see above

[54] [12]

[27], [53]

4 point scale—see above

[47]

6 point scale

[53]

3 point scale

[53]

4 point scale

[47]

3 point scale

[47]

3 point scale 4 point scale

[5] [9]

6 point scale—80–100%, 60–79%, 40–59%, 20–39%, 1–19%, 0%

[55]

[2], [10]

[24]

B. Summary of Recommendations Planning is important to project success as numerous authors have previously written [4], [5], [22]. It is clear from this research that the average project is not spending enough time on upfront planning to maximize success. This should not be surprising to researchers or practitioners; it appears that in industry, not enough planning is being done and that if longer planning phases were the norm, there would be higher overall project success. The inverted-U-shaped relationship between planning effort and success is significant and should be considered in future research.

SERRADOR AND TURNER: WHAT IS ENOUGH PLANNING? RESULTS FROM A GLOBAL QUANTITATIVE STUDY

The planning phase effort does not impact all aspects of success equally. The planning phase effort has the strongest relationship with overall project success. Reducing the effort spent on the planning phase may impact projects by reducing their final value to customers, stakeholders, and the company. This may be the case even though managers may still be able to deliver them on time and within budget. The phenomenon that projects may not be planning adequately could be a factor in the high project failure rates reported in the literature [6], [21]. It is recommended that projects consider doing more planning upfront both for traditional projects and for agile projects. However, projects with a too long planning phase were also found to have lower success ratings. Projects that schedule more than 25% effort on the upfront planning phase should be reviewed for progress and risk factors. Overplanning could be a symptom of a project that is too complex to deliver successfully, a lack of firm requirements or of a team that is not experienced enough in this project area: all of which could potentially lead to a failed project. C. Areas for Future Research This research does present some results that warrant further investigation. 1) Industry differences: Further research should consider focusing on specific industries, consolidating industry groups or collecting a larger volume of data. 2) Regional differences: This may be potential for research on regional differences from a variety of viewpoints: business environment, culture, infrastructure, and tradition. 3) Planning phase time: A research effort, perhaps qualitative, looking specifically at the factors that define the time spent on the planning phase could be considered. 4) Planning expertise: The impact of planning expertise, planning experience, or planning phase training on the required planning time and impact on project success is a factor that could be further examined. 5) Cost/Benefit of planning: The cost/benefit of additional planning for organizations that deliver projects is an area for potential future investigation. C. APPENDIX See Appendix Table in previous page. REFERENCES [1] J. R. Turner, M. Huemann, F. T. Anbari, and C. N. Bredillet, Perspectives on Projects. Evanston, IL, USA: Routledge, 2010. [2] D. Dvir, T. Raz, and A. Shenhar, “An empirical analysis of the relationship between project planning and project success,” Int. J. Project Manage., vol. 21, no. 2, pp. 89–95, 2003. [3] Y.-R. Wang and G. E. Gibson, “A study of preproject planning and project success using ANN and regression models,” in Proc. 25th Int. Symp. Autom. Robot. Constr., 2008, pp. 1–10. [4] P. W. G. Morris, “Key issues in project management,” in Project Management Institute Project Management Handbook, J. K. Pinto, Ed. Newtown Square, PA, USA: Project Management Institute, 1998. [5] M. Thomas, P. H. Jacques, J. R. Adams, and J. Kihneman-Woote, “Developing an effective project: Planning and team building combined,” Project Manage. J., vol. 39, no. 4, pp. 105–113, 2008.

473

[6] R. Sessions, “The IT complexity crisis: Danger and opportunity,” (2009). [Online]. Available: http://objectwatch.com/whitepapers, accessed Oct. 2013. [7] J. K. Pinto and D. P. Slevin, “Project success: Definitions and measurement techniques,” Project Manage. J., vol. 19, no. 1, pp. 67–72, 1988. [8] A. Shenhar and D. Dvir, Reinventing Project Management: The Diamond Approach to Successful Growth and Innovation. Watertown, MA, USA: Harvard Business, 2007. [9] P. Serrador and J. R. Turner, “The relationship between project success and project efficiency,” Project Manage. J., vol. 46, no. 1, pp. 30–39, 2015. [10] O. Zwikael and S. Globerson, “Benchmarking of project planning and success in selected industries,” Benchmarking, Int. J., vol. 13, no. 6, pp. 688–700, 2006. [11] K. Jugdev and R. M¨uller, “A retrospective look at our evolving understanding of project success,” Project Manage. J., vol. 36, no. 4, pp. 19–31, 2005. [12] T. J. Cooke-Davies, “The real success factors in projects,” Int. J. Project Manage., vol. 20, no. 3, pp. 185–190, Apr. 2002. [13] H. Mintzberg, The Rise and Fall of Strategic Planning: Reconceiving Roles for Planning, Plans, Planners. Englewood Cliffs, NJ, USA: Prentice-Hall, 1994. [14] E. Gibson and R. Gebken, “Design quality in pre-project planning: applications of the project definition rating index,” Building Res. Inform., vol. 31, no. 5, pp. 346–356, 2003. [15] A. Shenhar, IEEE Pers. Commun., 2011. [16] M. R. Hamilton, and J. G. E. Gibson, “Benchmarking preproject-planning effort,” J. Manage. Eng., vol. 12, no. 2, pp. 25–33, 1996. [17] Z. Shehu and A. Akintoye, “The critical success factors for effective programme management: a pragmatic approach,” Built Human Environ. Rev., vol. 2, pp. 1–24, 2009. [18] G. Gibson, Y. Wang, C. Cho, and M. Pappas, “What is pre-project planning, anyway?” J. Manage. Eng., vol. 22, no. 1, pp. 35–42, 2006. [19] E. Gibson and M. P. Pappas, Starting Smart: Key Practices for Developing Scopes of Work for Facility Projects. Washington, DC, USA: National Academy Press, 2003. [20] S. Collyer, C. Warren, B. Hemsley and C. Stevens, “Aim, fire, aim— Project planning styles in dynamic environments,” Project Manage. J., vol. 41, no. 4, pp. 108–121, 2010. [21] The Standish Group, “CHAOS Manifesto 2011,” 2011. [22] R. M. Posten, “Preventing software requirements specification errors with IEEE 830,” IEEE Softw., vol. 2, no. 1, pp. 83–86, Jan. 1985. [23] R. M¨uller and J. R. Turner, “The impact of performance in project management knowledge areas on earned value results in information technology projects,” Int. Project Manage. J., vol. 7, no. 1, pp. 44–51, 2001. [24] R. C. Tausworthe, “The work breakdown structure in software project management,” J. Syst. Softw., vol. 1, pp. 181–186, 1980. [25] C. Deephouse, T. Mukhopadhyay, D. R. Goldenson, and M. I. Kellner, “Software processes and project performance,” J. Manage. Inform. Syst., vol. 12, no. 3, pp. 187–205, Dec. 1995. [26] A. Munns and B. Bjeirmi, “The role of project management in achieving project success,” Int. J. Project Manage., vol. 14, no. 2, pp. 81–87, 1996. [27] J. K. Pinto and J. E. Prescott, “Variations in critical success factors over the stages in the project life cycle,” J. Manage., vol. 14, no. 1, pp. 5–18, Mar. 1988. [28] A. J. Shenhar, “One size does not fit all projects: Exploring classical contingency domains,” Manage. Sci., vol. 47, no. 3, pp. 394–414, 2001. [29] D. Dvir and T. Lechler, “Plans are nothing, changing plans is everything: The impact of changes on project success,” Res. Policy, vol. 33, no. 1, pp. 1–15, Jan. 2004. [30] A. J. Shenhar, A. Tishler, D. Dvir, S. Lipovetsky, and T. Lechler, “Refining the search for project success factors: A multivariate typological approach,” R&D Manage., vol. 32, no. 2, pp. 111–126, 2002. [31] J. R. Turner and R. M¨uller, “On the nature of the project as a temporary organization,” Int. J. Project Manage., vol. 21, no. 1, pp. 1–8, 2003. [32] E. S. Andersen, “Warning: Activity planning is hazardous to your project’s health,” Int. J. Project Manage., vol. 2, no. 14, pp. 89–94, 1996. [33] C. Bart, “Controlling new product R&D projects,” R&D Manage., vol. 23, no. 3, pp. 187–198, 1993. [34] S. Collyer and C. M. Warren, “Project management approaches for dynamic environments,” Int. J. Project Manage., vol. 27, no. 4, pp. 355–364, 2009.

474

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 62, NO. 4, NOVEMBER 2015

[35] M. Aubry, B. Hobbs, and D. Thuillier, “Organisational project management: An historical approach to the study of PMOs,” Int. J. Project Manage., vol. 26, no. 1, pp. 38–43, 2008. [36] B. Flyvbjerg, M. S. Holm, and S. Buhl, “Underestimating costs in public works projects: Error or lie?” J. Am. Planning Assoc., vol. 68, no. 3, pp. 279–295, 2002. [37] P. Chatzoglou and L. A. Macaulay, “Requirements capture and IS methodologies,” Inform. Syst. J., vol. 6, no. 3, pp. 209–225, 1996. [38] E. B. Daly, “Management of software development,” IEEE Trans. Softw. Eng., vol. 3, pp. 229–242, May 1977. [39] D. Nobelius and L. Trygg, “Stop chasing the front end process— Management of the early phases in product development projects,” Int. J. Project Manage., vol. 20, no. 5, pp. 331–340, 2002. [40] M. Wideman, Managing the Development of Building Projects for Better Results. (2000, Dec.) [Online]. Available: www.maxwideman.com [41] C. Kaner, J. Falk, and H. Q. Nguyen, Testing Computer Software, 2nd ed. Hoboken, NJ, USA: Wiley, 1993. [42] A. A. Choma and S. Bhat, “Success vs failure: What is the difference between the best and worst projects?” presented at the PMI Global Congress, Washington, DC, USA, 2010. [43] A. S. Choo, “Defining problems fast and slow: The u-shaped effect of problem definition time on project duration,” Prod. Oper. Manage., vol. 23, no. 8, pp. 1462–1479, 2014. [44] M. Hwang, J. Windsor, and A. Pryor, “Building a knowledge base for MIS research: A meta-analysis of a systems success model,” Inform. Resources Manage. J., vol. 13, no. 2, pp. 26–32, 2000. [45] W. Trochim, “Positivism and post-positivism,” Research Methods Knowledge Base. Cincinnati, OH, USA: Atomic Dog, 2006. [46] P. Serrador, “The impact of planning on project success: A literature review,” J. Mod. Project Manage., vol. 1, no. 2, pp. 28–39, 2013. [47] A. J. Shenhar, D. Dvir, O. Levy, and A. C. Maltz, “Project success: A multidimensional strategic concept,” Long Range Planning, vol. 34, no. 6,xbrk pp. 699–725, 2001. [48] D. Cooper, and P. Schindler, Business Research Methods. New York, NY, USA: McGraw-Hill, 2001. [49] P. Podsakoff, S. MacKenzie, J. Lee, and N. Podsakoff, “Common method biases in behavioral research: A critical review of the literature and recommended remedies,” J. Appl. Psychol., vol. 88, no. 5, pp. 879–903, 2003. [50] S. Sharma, R. Durand, and O. Gur-Arie, “Identification and analysis of moderator variables,” J. Marketing Res., vol. 18, pp. 291–300, 1981. [51] J. Nunnally, Psychometric Theory. New York, NY, USA: McGraw-Hill, 1978. [52] W. Beyer, CRC Standard Mathematical Tables. Boca Raton, FL, USA: CRC Press, 1987. [53] L. H. Crawford, J. B. Hobbs, and J. R. Turner, “Project categorization systems and their use in organisations: An empirical study,” presented at the PMI Research Conference, Jul. 2004, London, U.K., 2004. [54] V. Dulewicz and M. Higgs, “Assessing leadership styles and organisational context,” J. Manage. Psychol., vol. 20, no. 2, pp. 105–123, 2005. [55] M. Ceschi, A. Sillitti, G. Succi, and S. De Panfilis, “Project management in plan-based and agile companies,” IEEE Softw., vol. 22, no. 3, pp. 21–27, May/Jun. 2005.

Pedro Serrador received the B.Sc.(Hons.) degree in physics and computer science from the University of Waterloo, Waterloo, ON, Canada; the MBA degree from Heriot-Watt University, Edinburgh, Scotland; and the Ph.D. degree in strategy and programme and project management from SKEMA Business School (Ecole Suprieure de Commerce de Lille), Euralille, France. He is a Writer and a Researcher on project management topics and the Owner of Serrador Project Management, a consultancy in Toronto, Canada. He is also an Adjunct Professor at Humber College, Toronto, and the University of Toronto, Toronto. He specializes in technically complex and high risk projects, vendor management engagements, and tailoring and implementing project management methodologies; he has worked on projects in the financial, telecommunications, utility, medical imaging, and simulations sectors for some of Canada’s largest companies. His areas of research interest are project success, planning, and agile, and he has presented a number of peer-reviewed papers on these topics at academic conferences. He is an author of books and articles on project management, and is also a regular speaker at PMI global congresses. Dr. Serrador received the PMI 2012 James R. Snyder International Student Paper of the Year Award and the Major de Promotion Award for best Ph.D. Thesis 2012–2013 from SKEMA Business School.

Rodney Turner is a Professor of Project Management at SKEMA Business School, Lille, France, where he is the Scientific Director for the Ph.D. in Project and Programme Management, and is the SAIPEM Professor of Project Management at the Politecnico di Milano, Milano, Italy. He is also an Adjunct Professor at the University of Technology Sydney, Sydney, Australia. He is the author or editor of 18 books. His research areas cover project management in small to medium enterprises, the management of complex projects, the governance of project management including ethics and trust, project leadership, and human resource management in the project-oriented firm. Mr. Turner is an editor of the International Journal of Project Management. He is the Vice-President and an Honorary Fellow of the United Kingdom’s Association for Project Management, and an Honorary Fellow and former President and Chairman of the International Project Management Association.