Software Acquisition Management Guidelines
Short Description
The thesis provides guidelines from the viewpoint of the customer for managing an acquisition project concerning an outs...
Description
SOFTWARE ACQUISITION MANAGEMENT GUIDELINES by Daniel Svennberg A Thesis submitted for the Degree of Master of Science
Linköping, Sweden, 2001 The Department of Computer and Information Science Linköping University LiTH-IDA-Ex-01/32 Lars Ljungberg, Mats Ran Advisors, Q-Labs AB
Kristian Sandahl Examiner, Linköping University
Software Acquisition Management Guidelines Riktlinjer för ledning av programvaruupphandling LiTH-IDA-Ex-01/32
Linköpings Universitet, Institutionen för Datavetenskap Linköping University Q-Labs, Linköping and Maryland Q-Labs AB Fraunhofer, Maryland, Fraunhofer, Inc. Center for Experimental Software Engineering Copyright 2001 Daniel Svennberg All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means including, without limitation, photocopying, recording, or otherwise without the prior written permission of the author. Written permission is not needed if this publication is distributed for noncommercial purposes.
Abstract
The thesis provides guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer. Current frameworks for software acquisition management such as SA-CMM, IEEE 1062, ISO 12207, etc. as well as literature, articles, and interviews has been used as a basis for formulating the guidelines. The resulting guidelines include acquisition team roles descriptions, checklists for artifacts, customizable processes, and customization factors. The processes contain guidelines for the following issues: project steering, project management, planning and organizing, acquisition training, requirements management, risk management, contractor selection, contract management, product evaluation, transition to support, and post partum. The guidelines can be used as a basis for planning an acquisition project or be read as an overview over the many issues that concern software acquisitions.
i
Acknowledgements
Many thanks to (in alphabetical order)… Fraunhofer Institute, Jaak Urmi, John Marciniak, Kristian Sandahl, Lars Ljungberg, Linköping University, Mats Ran, Mikael Lindvall, Q-Labs, Victor R. Basili, Winifred Menezes, Östen Oskarsson, and everybody else … for your help making this thesis possible. D.S.
ii
Table of Contents
Abstract.................................................................................i Acknowledgements ............................................................ii 1
Introduction ...............................................................1
1.1 1.2 1.3 1.4 1.5 1.6 1.7
Background.............................................................................1 Target audience......................................................................2 Objective .................................................................................2 Problems .................................................................................2 Scope ........................................................................................3 Approach.................................................................................3 Outline.....................................................................................4
2
Software Acquisition Environment .........................6
2.1 2.2 2.3 2.4 2.5 2.6
The nature of software .........................................................6 Software acquisition.............................................................7 Software development .........................................................8 Outsourcing............................................................................9 Common problems .............................................................13 Frameworks ..........................................................................16
3
Customization .........................................................20
3.1 3.2 3.2.1 3.2.2 3.2.3 3.2.4
Customization levels ..........................................................20 Customization factors.........................................................21 Product characteristics............................................................22 Effort........................................................................................24 Contracting environment .......................................................26 Other factors ...........................................................................28 iii
Table of Contents
iv
4
Roles ........................................................................29
4.1 4.1.1 4.1.2 4.1.3 4.2 4.2.1 4.2.2 4.2.3 4.2.4 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.3.6 4.3.7 4.3.8 4.3.9 4.4 4.4.1 4.4.2 4.4.3 4.4.4
Sponsor roles........................................................................29 Sponsor....................................................................................29 Collaborating customer...........................................................30 Product manager.....................................................................30 Managing roles ....................................................................30 Acquisition manager...............................................................30 Technical manager ..................................................................32 Contract manager ...................................................................32 Administrator .........................................................................32 Assisting roles......................................................................33 Acquisition expert...................................................................33 Technical expert ......................................................................33 Domain expert.........................................................................33 Legal expert .............................................................................34 Translator................................................................................34 End user ..................................................................................34 Maintenance and support staff ...............................................34 Independent verification and validation .................................35 System engineering and technical assistance .........................35 Executing roles.....................................................................36 Contractor ...............................................................................36 Collaborating contractor .........................................................37 Subcontractor..........................................................................37 Supplier ...................................................................................37
5
Artifacts ...................................................................38
5.1 5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.2 5.2.1 5.2.2 5.2.3 5.2.4 5.3
Inception ...............................................................................39 Acquisition proposal ...............................................................39 Project specification ................................................................40 Acquisition plan......................................................................41 Risk list....................................................................................43 Requirements specification .....................................................44 Maintenance plan ...................................................................46 Solicitation............................................................................47 Request for proposal ................................................................47 Contractor evaluation criteria ................................................49 Proposal...................................................................................50 Contract...................................................................................51 Monitoring............................................................................53 Software Acquisition Management Guidelines
Table of Contents 5.3.1 5.3.2 5.3.3 5.3.4 5.3.5 5.4 5.4.1 5.4.2 5.4.3
Artifacts from the contractor ..................................................53 Product evaluation plan..........................................................54 Product evaluation report .......................................................55 Change request........................................................................55 Project status report................................................................56 Finalization...........................................................................57 Deliverables.............................................................................57 Post partum plan.....................................................................57 Post partum.............................................................................58
6
Processes................................................................60
6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.7.1 6.8 6.8.1 6.9 6.9.1 6.9.2 6.10 6.10.1 6.11 6.12
Overview...............................................................................60 Project steering ....................................................................66 Project management ...........................................................69 Planning and organizing ...................................................72 Acquisition training ...........................................................74 Requirements management ..............................................76 Risk management................................................................78 Risk identification ...................................................................80 Contractor selection ............................................................85 Contract types .........................................................................88 Contract management.........................................................96 Project status metrics............................................................100 Earned value..........................................................................102 Product evaluation ............................................................106 Product evaluation approaches .............................................109 Transition to support........................................................110 Post partum.........................................................................113
7
Results and summary ..........................................116
7.1 7.2
Guidelines summary ........................................................116 Outlook ...............................................................................122
8
Bibliography..........................................................124
A
Frameworks coverage..........................................129
A.1 A.2 A.3 A.4
SA-CMM .............................................................................129 FAA-iCMM.........................................................................130 IEEE 1062.............................................................................131 ISO 9000-3 ...........................................................................132
© Daniel Svennberg 2001
v
Table of Contents A.5 A.6 A.7
ISO 12207 ............................................................................133 ISO 15504 ............................................................................134 Euromethod ........................................................................135
Index.......................................................................137
vi
Software Acquisition Management Guidelines
1
Introduction
“Much of present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy.” – Frederick P. Brooks, Jr.
This chapter states the background of the thesis, its objective, the approach, and outlines the contents of the report.
1.1
Background Many organizations have a need to acquire customized software products that meet special demands that no commercial off-the-shelf product can meet. Many times the most efficient and economically viable solution to obtain the software is to outsource the development to a contractor instead of developing the software in-house. In order to manage such a software acquisition project successfully, many different non-trivial issues such as requirements elicitation and management, risk analysis and management, contractor selection, contract management, product evaluation, etc. need to be addressed. Several frameworks and standards have been developed for assisting and assessing such software acquisition efforts. 1
1 Introduction The purpose of this thesis is to gather practices from these frameworks and other sources and to present them as guidelines for software acquisition project managers. The thesis along with the diploma thesis (Assmann, 2000) of a German student, Danilo Assmann, are parts of an international project on software acquisition between Linköping University in Sweden, Q-Labs in Sweden and USA, and the Fraunhofer Institute in Germany and USA. The thesis by Assmann deals mainly with the evaluation and selection of a contractor, while this thesis deals mainly with the management of an acquisition project. It isn’t possible for a single master’s thesis to cover this subject in all its intricacies, therefore the guidelines should be considered as an introduction to software acquisition management, and hopefully as an inspiration on further studies on how to manage software acquisition projects.
1.2
Target audience The primary audience group for this master’s thesis consists of those who are assigned the task of managing a software acquisition project. Naturally, all groups that are involved in software acquisition projects may find this thesis interesting.
1.3
Objective The objective of this master’s thesis is to provide guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer.
1.4
Problems The objective stated above can be broken down into seeking answers to the following problems: • What problems are common?
2
Software Acquisition Management Guidelines
1 Introduction • What activities should be performed? • What roles are conceivable? • What artifacts are conceivable? • How can the activities, roles, and artifacts be customized for different kinds of projects?
1.5
Scope The management of custom-made software product acquisitions will be examined – not services. Fully or partially developed products are the main concern – not commercial off-theshelf products. The acquirer’s point of view is mainly addressed – not the contractor’s. Legal aspects will not be examined further than contract contents. The issues will be examined from a business perspective instead of a governmental or military perspective.
1.6
Approach An introductory study of the subject was performed by reading the thesis of the German student (Assmann, 2000), the different frameworks listed in section 2.6, Frameworks, on page 16, web pages, and articles on software acquisition, and performing interviews with Jaak Urmi, Östen Oskarsson, and Mats Ran. This gave an understanding of the concepts of software acquisition management and a number of ideas on how to formulate the guidelines and provide answers to the problems stated above. I found that the different sources gave quite a coherent picture of software acquisition management without many contradictions. The sources contained mainly different descriptions of the same issues. Like Assmann (2000) does in his thesis, I assume that the frameworks are useful in their own context and therefore are viable as input to the guidelines. Furthermore my goal was to extract and assemble the lion’s share of the advice
© Daniel Svennberg 2001
3
1 Introduction given in the main frameworks SW-CMM, SA-CMM, FAAiCMM, IEEE 1062, ISO 9000, ISO 12207, ISO 15504, and Euromethod, with as little filtering as possible, and then augment with advice from the other sources studied. Naturally, I had to integrate and arrange the advice into a suitable and coherent form for the purpose of the guidelines. The ideas on how to describe the guidelines were presented and discussed with several people, including Kristian Sandahl, Lars Ljungberg, Danilo Assmann, John Marciniak, Jon Valett, Winifred Menezes, Mikael Lindvall, and others. These discussions motivated me to write the guidelines in their present form. The guidelines described in this thesis are not a new model for software acquisition management, but an attempt to extract, assemble, and integrate advice from the existing models and frameworks and present them as management processes, checklists for artifacts, descriptions of roles, and customization factors. The option would have been to describe the guidelines in a more prosaic fashion, but that would have made the suggestions for customization less logical and to the point.
1.7
Outline The following is a description of the contents of each chapter in the report: Chapter 1, Introduction, on page 1 describes the background of the thesis, its objective, and approach. Chapter 2, Software Acquisition Environment, on page 6 investigates the nature of software, different types of software acquisitions, software development, outsourcing issues, problems with software projects, and software acquisition frameworks. Chapter 3, Customization, on page 20 describes a number of factors that can be used as a guideline on what level of formality is appropriate for the actual software acquisition project.
4
Software Acquisition Management Guidelines
1 Introduction Chapter 4, Roles, on page 29 describes the different roles that are conceivable for a software acquisition project. Chapter 5, Artifacts, on page 38 contains checklists for the different types of artifacts that are conceivable for a software acquisition project. Chapter 6, Processes, on page 60 contains suggestions on customized software acquisition management processes. Chapter 7, Results and summary, on page 116 summarizes the guidelines and contains an outlook on continued work.
© Daniel Svennberg 2001
5
2
Software Acquisition Environment
“A typical software project can present more opportunities to learn from mistakes than some people get in a lifetime.” – Steve McConnell
This chapter introduces the concept of software acquisition, its nature, environment, and problems. It also lists some frameworks that deal with software acquisition management.
2.1
The nature of software There are a number of inherent characteristics of software that affect the acquisition and development of customized software in a profound way. The most salient of these characteristics, in my opinion, are the following: Software is an almost infinitely malleable conceptual construct. As a consequence this makes it difficult to settle the requirements and makes the scope of the software prone to change. Software has no physical features and is therefore difficult to visualize. Brooks (1995) tells us that this makes it hard to design the software and to communicate the design of the software, which in turn makes it difficult to estimate how long it will take to develop the software. It also 6
2 Software Acquisition Environment makes it difficult to see progress in the development of the software. Software is extremely cheap to manufacture, as Reeves (1992) argues. All it takes is to transform the source code into executable code and clone as many copies as you wish. The actual costs for software lie in development and maintenance. Software is extremely complex. This is, according to Brooks (1995), due to the fact that software has a very large number of possible states, which increases exponentially with size. This makes it very hard to get an overview of the software, which in turn makes it hard to design the software to fully meet all quality requirements and to have full control over the possible errors.
2.2
Software acquisition There are several different ways to acquire software. According to IEEE 1062 (1998), software products can be classified according to the degree to which the acquirer can specify the features of the software. Commercial-off-the-shelf software, or COTS, is commercially available software. It is usually well defined and documented and a wide range of commercial users has demonstrated its fitness for use. The supplier isn’t normally willing to modify the software for the needs of a specific customer, and also controls the maintenance of the software. The cost to acquire the software is low to medium and the delivery time is immediate. The customer has no control of the quality characteristics of the software. Modified-off-the-shelf software, or MOTS, is similar to COTS software with the difference that the supplier is willing to make modifications to the software product according to the customer’s requirements. The software product’s fitness for use has been demonstrated in similar applications. The customer
© Daniel Svennberg 2001
7
2 Software Acquisition Environment has some control of the maintenance of the product and its quality characteristics, i.e. the customized part. Delivery time is short to long and the cost to acquire the software is medium to high. Fully developed software are unique, one-of-a-kind or lowvolume applications fully customized for the specific needs of the customer. The software product’s fitness for use is unprecedented, but the customer has full control over its maintenance and quality characteristics. The cost to acquire fully developed software is high and the delivery time is long. The characteristics of these product classes are summarized in the table below: COTS Scope Fitness-for-use
Fix Demonstrated
MOTS Partially customizable Demonstrated in similar applications
FD* Fully customizable Unprecedented
Maintenance
No control
Some control
Full control
Delivery time
Immediate
Short-Long
Long
Low-Medium
Medium-High
High
Acquisition cost (Not usage/maint. cost) Quality (ISO 9126)
Not controllable
Partially controllable Mostly controllable
*) Partially to fully outsourced
Table 1. Software acquisition types. FD stands for fully developed software.
This thesis applies to MOTS and especially fully developed software products.
2.3
Software development To be brief, software development essentially consists of four activities: analysis, design, coding, and testing. There are a number of methods that describe how and when these activities should be performed, ranging from code and fix, which is the software development version of trial and error, to perhaps the
8
Software Acquisition Management Guidelines
2 Software Acquisition Environment most well known method, the waterfall method, which in its purest form is to perform the activities in the following sequential manner: analyze, design, code, test, and never go back to a previous step. More recently, iterative, evolutionary, or incremental development methods such as eXtreme Programming (Beck, 1999) have become popular. Following such a method means that the program is developed in a series of iterations or increments with a subset of the requirements implemented in each iteration; this has the benefit that you can more easily change your mind about the requirements as the program evolves. See the figure below for a comparison between the waterfall model, the spiral model and extreme programming. Different software development methods suit different types of projects, but that is not for this report to examine. What is important is that the contractor can produce good results with the method they use. Waterfall
Spiral
XP
Analysis Design Coding Testing Scope Figure 1. The development cycles of the waterfall model, the spiral model and extreme programming (Beck, 1999).
2.4
Outsourcing There are many different reasons for outsourcing a software development project, such as the following examples:
© Daniel Svennberg 2001
9
2 Software Acquisition Environment • It’s difficult to hire developers. • It’s cheaper than hiring developers. • Somebody else can develop the program faster, cheaper, and with higher quality than you can. • You lack the necessary competence and resources. • You want to use your staff for other tasks. The following are examples of drawbacks with outsourcing: • You lose some control over the software development process, which implies that the risk is higher; possibly you risk putting the fate of your competitiveness in the hands of the contractor. • You don’t build up your own software development competence. • You don’t get any indirect labor hours since you don’t have your own development staff. Haimes, et al. (1997) states that there are three groups of participants in a software acquisition project, each with separate goals: • The end user wishes to have a product that function well in operation. Another possible goal for the end user is to get the product as quickly as possible. • The customer, that is the participant that finances the project, can have several goals: Similarly to the user, the customer wishes to receive a software product that function well in operation. The customer also wants to minimize cost and deviation from time schedule. Furthermore, the customer wants to maximize return on investment and to have a well-functioning relationship with the contractor.
10
Software Acquisition Management Guidelines
2 Software Acquisition Environment • The contractor on the other hand often wants to maximize profits and to maximize the potential for future earnings, such as additional contracts. This gives us the optimization problem that a software acquisition project consists of, with several of the goals overlapping and conflicting. For instance, the customer wants to maximize the technical performance of the product and on the same time minimize cost and deviation from time schedule. Furthermore, the cost minimization goal of the customer conflicts with the profits maximization goal of the contractor. Obviously tradeoffs are necessary to have an operational contract. Participant
Objective
End user
Maximize technical performance Minimize development time
Customer
Maximize technical performance Maximize customer-contractor relation Maximize return on investment Minimize cost Minimize deviation from time schedule
Contractor
Maximize profit Maximize future earnings
Table 2. Participants in a software acquisition project and their goals. Adapted from Haimes, et al. (1997).
The most common scenario when you think of an outsourcing project is that you have two parties involved – a single customer and a single contractor. Gallivan (1999) shows that this simple dyadic relationship is not always the case, as more complex outsourcing constellations are becoming more commonplace. Gallivan (1999) gives a taxonomy of outsourcing constellations: • The simple dyadic constellation consists of a single customer outsourcing to a single vendor. © Daniel Svennberg 2001
11
2 Software Acquisition Environment • The multi-vendor constellation consists of a group of vendors collaborating in a project outsourced by a single customer. • The co-sourcing constellation appears when a group of customers that have similar needs cooperate and jointly outsources a project to a single vendor. • The complex constellation is a combination of the multivendor and co-sourcing constellations, where you have a group of customers jointly outsourcing to a group of collaborating vendors.
One Vendor
Many Vendors V
C
One Client
V
V V
Simple Dyadic C
Many Clients
C
C
V
C
Co-Sourcing
Multi-Vendors C
V
C
V
C
V
Complex
Figure 2. A taxonomy of outsourcing relationships. (Gallivan, 1999)
To make the outsourcing scenario even more complex you can have one or more subcontractors involved and several supporting contractors and suppliers. Compared to the dyadic constellation, the multi-vendor relationship allows vendor specialization and reduces the risk of a vendor behaving opportunistically since the other vendors are ready and willing to substitute a non-performing vendor. 12
Software Acquisition Management Guidelines
2 Software Acquisition Environment Drawbacks are increased coordination costs and more complex contracts, etc. (Gallivan, 1999) Compared to the dyadic constellation, the co-sourcing relationship allows risk sharing and reduction, and cost and time savings due to buyer economies of scale. Drawbacks are increased coordination costs and that it is more difficult to control information and keep secrets because of the interdependent nature of the relationship between the customers, etc. (Gallivan, 1999)
2.5
Common problems Marciniak and Reifer (1990) states that the main problems in software acquisitions are: • Development costs exceeds budget. • Time schedule is overrun. • Outcome does not meet expectations. The Standish Group (1995) “Chaos” report investigated how common these problems are. 8 380 software development projects were surveyed with respondents representing small, medium, and large companies across major industry segments such as banking, health care, and federal organizations, etc. The report does not indicate the level of outsourcing in these software development projects, but the statistics will still give an idea of the extent to which software development projects suffer from these problems. The findings were: • 16% of the projects were completed on time and on budget, with all features and functions as initially specified. These projects were labeled as successful. • 53% of the projects were completed and operational but over budget, over the time estimate, and offered fewer
© Daniel Svennberg 2001
13
2 Software Acquisition Environment features and functions than originally specified. These projects were labeled as challenged. • 31% of the projects were canceled at some point during the development cycle. These projects were labeled as canceled.
Canceled 31%
Successful 16%
Challenged 53%
Figure 3. Findings from the Standish Group (1995).
In addition, the average project exceeded planned budget with 90% and schedule with 120%. The root cause to these problems is primarily ineffective management as stated in Brown, et al. (1998): In 1987, the Defense Science Board concluded that these software acquisition problems came not from technical difficulties, but from poor management. Since 1987, the situation has gotten worse, not better. Software has gotten bigger, more complex, and more expensive, and ineffective management is still the root cause of much that afflicts software acquisition.
Below is a list of some common problems in software acquisition projects. It is assembled and adapted from The Standish Group (1995), Ganssle (1998), Marciniak and Reifer (1990), Reifer (1997), and interviews with Urmi (2000), Oskarsson (2000), and Ran (2000): • Laissez-faire – The customer does not take an active part in the project once the contract is signed.
14
Software Acquisition Management Guidelines
2 Software Acquisition Environment • Administrative overload – Too much administratrivia to monitor contract compliance instead of focusing on technical matters. • Creeping scope – The customer keeps adding and changing scope and functionality to a job in progress on a tight schedule with limited resources. • Fragmentation – Members of the team, both customer and contractor, are pulled off to work on other projects at random times. • Goldplating – Having overkill requirements or making sophisticated and new rather than simple and proven technical solutions. • “I’m paid to engineer!” – The customer tells the contractor how to do the job, not what the job is. • Missing or bad indicators – Measurement of progress and overall performance is qualitative instead of quantitative. The performance levels of the indicators are poorly set. • Who’s in charge? – Too many bosses, decisions aren’t made in a timely manner. • No end user involvement – Not getting and incorporating the end user’s point of view on the functionality and usability of the product. • Poorly defined requirements – The customer gives an incomplete and unvalidated set of requirements to the contractor. • Acquisition incompetence – Failure to understand the particular needs of software acquisitions. • Overpromising – The marketing people of the contractor makes promises that the technical staff can’t keep. © Daniel Svennberg 2001
15
2 Software Acquisition Environment • Lack of discipline – Reverting to an ad hoc development process when deadlines are getting close. “Gotta crank out that code!” • Unrealistic expectations – Having an impossible schedule, and/or being unaware of the limitations of the technology being used. • Inadequate resources – Not having appropriate financial funding or lacking appropriate staff, tools, and equipment. • No executive support – No support or interest in the project from top management. • Lack of clear objectives – No clear statement of goal or vision causing project members to pull in different directions. • Ineffective communications – Lack of effective communication channels, information not reaching the right people in a timely manner. • Lack of competence – Not having the appropriate technical and leadership skills. • Friction – Cooperation is hindered by some of the involved parties not getting along for some reason. All of these problems can probably be avoided by better software acquisition management practices. Also see the risk taxonomy given on page 80.
2.6
Frameworks The following is a list of frameworks that in some way gives guidance on software acquisition management. Apart from standards, books and collections of guidelines are mentioned.
16
Software Acquisition Management Guidelines
2 Software Acquisition Environment There are of course more frameworks that cover the subject than those listed1. SW-CMM – The Capability Maturity Model for Software is one of the most well known frameworks for assessing the capabilities of software contractors. The purpose of the model is to help organizations determine the maturity of their current software development capabilities. The model characterizes the level of an organization’s maturity based on the extent to which measurable and repeatable software engineering and management practices are institutionalized. The software acquisition activities examined for this thesis are in the Software subcontract management key process area of the model. See Paulk, et al. (1993). SA-CMM – The Software Acquisition Capability Maturity Model is a framework developed to help organizations assess and improve their acquisition capabilities for software-intensive systems. The structure of the model is consistent with SW-CMM in that it is a staged model with key process areas grouped in five maturity levels: initial, repeatable, defined, managed, and optimizing. See Cooper, et al. (1999). FAA-iCMM – The Federal Aviation Administration Integrated Capability Maturity Model is an effort to combine the features of SW-CMM, SA-CMM, and SE-CMM2 into one consistent model. It is intended to be used as a tool for organizations to evaluate their acquisition, management, and engineering practices and to devise improvements on them. See FAA-iCMM (1997). CMMI – The Capability Maturity Model Integration framework is an effort to integrate existing maturity models. CMMI is under development and will eventually consist of a series of disciplines ranging from software engineering and systems engineering to
www.software.org/quagmire is a good starting point to find out more about available standards.
1
2
Systems Engineering Capability Maturity Model.
© Daniel Svennberg 2001
17
2 Software Acquisition Environment software acquisition, etc. A requirement for the CMMI framework is consistency with ISO 15504. This framework had not been extended to contain the software acquisition discipline when this thesis was written. See CMMI (2000). IEEE 1062 – The IEEE Recommended Practice for Software Acquisition is a standard that takes a comprehensive approach to giving recommended practice for managing software acquisition projects regarding COTS, MOTS, and fully developed software. It covers the whole acquisition process from planning organizational strategy to using the software and is compliant with ISO 12207. See IEEE 1062 (1998). ISO 9000 is a series of standards for quality management and quality assurance. It is mainly intended to provide guidance in a situation where a contract between two parties require the demonstration of a contractor’s capability to develop, supply and maintain products. In this series, the standards that are important for software development projects are ISO 9001 and ISO 9000-3. ISO 9001 – Quality systems – Model for quality assurance in design / development, production, installation and servicing deals with generic two-party contractual situations and ISO 9000-3 Guidelines for the application of ISO 9001 to the development, supply and maintenance of software gives guidelines for the application of ISO 9001 to software projects. ISO 9001 is roughly comparable to SW-CMM level 2. See SS-ISO 9000-3 (1992). ISO 12207 – The Standard for information technology – Software life cycle processes takes a holistic and comprehensive approach to the whole life cycle of a software product. It includes a welldefined terminology for software life cycles that is intended to standardize the terminology used by the industry. The standard can be used to establish software acquisition, supply, development, operation, maintenance, and supporting processes. It is also intended to foster an improved understanding between customers and vendors and among the parties involved in the life cycle of a software product. Worth mentioning is the goal of facilitating world trade in software. It replaces older U.S. military standards such as DOD-STD-2167A and DOD-STD-2168. See IEEE/EIA 12207.0 (1996). 18
Software Acquisition Management Guidelines
2 Software Acquisition Environment ISO 15504 – The Information Technology – Software Process Assessment standard is a framework for assessing software processes regarding acquisition, development, support, etc. It is influenced by and similar to SW-CMM in such that it has a capability scale. It is different from the SW-CMM in that it has not a discrete pass/fail mechanism and that it is aimed at assessing individual processes and not organizations. It is strongly aligned with ISO 12207. See ISO/IEC 15504 (1998). Euromethod is a framework with the intent to assist an acquisition at the contractual level. It aims to help the customer and contractor understand each other, improve risk management, and harmonize the terminology used in an acquisition effort. According to Breu (1995), Euromethod is particularly aimed at guiding public procurement processes in the context of the open European market, but can be used for all types of software acquisition projects. See Euromethod (1996). Other frameworks that provide important and useful advice for software acquisition management are the following: The Road to Successful ITS Software Acquisition (Salwin, 1998), Software Acquisition Management (Marciniak and Reifer, 1990), The Program Manager’s Guide to Software Acquisition Best Practices (Brown, et al., 1998), Guidelines for Successful Acquisition and Management of Software-Intensive Systems (Mosemann, 1996), A Guide to the Project Management Body of Knowledge (Duncan, 1996), and Defense Acquisition Deskbook (DAD, 2000).
© Daniel Svennberg 2001
19
3
Customization
“Depending on the circumstances, you should be as hard as diamond, flexible as willow, smooth-flowing like water or as empty as space.” – Morihei Ueshiba
This chapter investigates project categorization factors that indicate what level of formality is necessary for the actual software acquisition project.
3.1
Customization levels Software acquisition projects have different characteristics. Some projects are small, short-term, and easy to grasp, while others are the other way around. Cockburn (n.d.) states that … each organization must develop its own way of working, and each project must get its own, tailored methodology, dealing with culture, location, criticality, and technology.
To have effective acquisition processes that give good results, it is beneficial to employ an adequate level of formality for those processes according to the characteristics of the actual project. For the purpose of this thesis I give suggestions on three levels of formality for the acquisition management processes. These levels are:
20
3 Customization Minimal processes – The project is easily managed and the purpose of the acquisition processes is achieved with a low level of rigor. The processes contain just the activities without which the project would really be in trouble. This level is suitable for low-risk projects. Controlled processes – The project has a tangible complexity and a reasonable risk of going out of hands. To contain the complexity and risks, a higher level of formality is appropriate for the acquisition processes. The processes on this level contain the activities of the minimal processes and add some activities that are not absolutely necessary but make things more controlled and reliable. This level is suitable for medium-risk projects. Robust processes – The project is highly complex and has a strong tendency to get chaotic unless it is managed appropriately and performed with discipline. The processes contain activities that increase the formality and administrative effort but in return make things more robust and reliable. This level is suitable for high-risk projects. An acquisition process level below minimal would imply ad hoc, laissez-faire processes, which is not recommended.
3.2
Customization factors To determine what level of formality is appropriate for your project you can look at the customization factors in the following sections. The factors have been inspired by D.I.R. (2000), ISO 12207 (1995), and Euromethod (1996) and should be viewed as guidelines and not exact values on what level to choose. It is important to look at several factors and determine which are most important for your project before you decide upon a formality level.
© Daniel Svennberg 2001
21
3 Customization 3.2.1
Product characteristics The product characteristics factors obviously stem from the characteristics of the software product to be acquired. Requirements volatility Since software development is a creative process, and it is difficult to anticipate all technical problems beforehand, change of requirements occur more or less in all software development projects. However, due to the nature of the problem the software is to solve, the total set of requirements can be more or less change-prone. Minimal processes sufficient
Stable requirements. Highly unlikely that the overwhelming majority of the requirements will change.
Controlled A significant subset of the requirements is volaprocesses tile. recommended Robust processes recommended
Volatile requirements. High degree of uncertainty on a large or important subset of the requirements.
Program size The estimated program size indicates the effort that is needed to build the software. The values given in the table below are to be considered as suggestions since program size depends on what language is used and other factors. Minimal processes sufficient
1
22
Small. Less than 32 KLOC1.
1 KLOC = 1 thousand lines of source code. Software Acquisition Management Guidelines
3 Customization Controlled processes recommended
Medium. 32 – 128 KLOC.
Robust processes recommended
Large. More than 128 KLOC.
Criticality Criticality is the impact malfunctioning software will have on people’s safety and the magnitude of financial loss that can result, according to Oskarsson and Glass (1996). Minimal processes sufficient
People’s safety is not at risk, nor will any significant amount of money be lost if the software malfunctions.
Controlled People’s safety is not at risk, but there is a risk of processes moderate financial losses due to software malrecommended functioning. Robust processes recommended
The safety of people is at stake or huge amounts of money. A disaster occurs if the software doesn’t function properly.
Innovation Oskarsson and Glass (1996) define innovative projects as those in which the problem is new and different and hasn’t been solved with computer programs before. Less innovative projects use mature and proven technology. Minimal processes sufficient
Mature technology. Many similar applications exist.
Controlled The technology has been proven to work and is processes expanding but few similar applications exist. recommended © Daniel Svennberg 2001
23
3 Customization Robust processes recommended
Emerging unproven technology where the question is more on whether the problem can be solved. Unique application.
Transformational impact Transformational impact is the depth of change that the delivered software product will have on the user’s behavior and the user organization’s processes and products. Minimal processes sufficient
Minimal change for the user.
Controlled Moderate modifications or amendments to the processes user’s behavior or the internal work processes recommended and products of the user organization. Robust processes recommended 3.2.2
Fundamentally changes the behavior of the user and the work processes and products of the user organization.
Effort The effort customization factors delivery time, total cost, and development team size are implied by the estimated effort to build the software. The estimate is based on the product characteristics. The larger the effort is the higher formality is required to avoid unnecessary chaos. Delivery time Delivery time is the estimated calendar time for delivery of the finished software product. With a longer project there is a higher likelihood that new people will join the development team and others leave. Change in the requirements is also more likely. The values given in the table below are to be considered as suggestions.
24
Software Acquisition Management Guidelines
3 Customization Minimal processes sufficient
18 months or less.
Controlled 12 – 36 months. processes recommended Robust processes recommended
24 months or more.
Total cost Total cost is the estimated total cost to acquire the software in order of magnitude. The value ranges of low, medium, and high in the table below are difficult to specify since these values may change over time and are only given as suggestions. Minimal processes sufficient
Low. Less than $50,000.
Controlled Medium. $50,000 – $1,000,000. processes recommended Robust processes recommended
High. More than $1,000,000.
Contractor development team size The more people that are involved in the contractor’s development team the more communication is needed to coordinate the work, sort out misunderstandings, and resolve conflicts. The values in the table are not to be considered as absolute. Minimal processes sufficient © Daniel Svennberg 2001
Less than 10 developers.
25
3 Customization Controlled 10 – 30 developers. processes recommended Robust processes recommended 3.2.3
More than 30 developers.
Contracting environment The contracting environment customization factors are related to the contractor and the organizational environment that the acquisition takes place in. Organizations involved The more organizations, such as stakeholders, supporting contractors, consultants, independent testing organizations, etc. are involved, the more communication is needed to coordinate the work. Minimal processes sufficient
Two parties involved in a simple dyadic relationship; customer and contractor.
Controlled More than two parties are involved; either sevprocesses eral customers co-sourcing, or several contractors recommended collaborating, or subcontractors are used, and possibly supporting contractors. The organizational structure is not (too) complex. Robust processes recommended
26
Several organizations are involved in a complex organizational structure composed of some or all of the following: multiple customers, multiple contractors, multiple subcontractors, multiple supporting contractors, etc.
Software Acquisition Management Guidelines
3 Customization Contractor development team characteristics The contractor’s development team characteristics are the experience, constitution, and performance of the contractor’s development team. Minimal processes sufficient
Low staff turnover. Mostly experienced and welltrained personnel. Well functioning teamwork. Efficient and appropriate working methods. High performance.
Controlled Moderate staff turnover. Some inexperienced and processes untrained personnel. Teamwork could be funcrecommended tioning better. Working methods need some improvement. Medium performance. Robust processes recommended
High staff turnover. Lots of inexperienced and untrained personnel. Teamwork not functioning optimally. Inefficient and inappropriate working methods. Low performance.
Experience with contractor The type and level of experience with the contractor(s). Minimal processes sufficient
The customer and contractor have worked together on several previous occasions and have a good relationship built on trust. The customer knows how the contractor works and vice versa.
Controlled First time working with a contractor with good processes results working for other customers, or mixed recommended experiences with a previously used contractor. Robust processes recommended
© Daniel Svennberg 2001
First time working with a contractor that is not well known or has had bad results working for other customers, or bad experiences with a previously used contractor.
27
3 Customization Geographic dispersion Management becomes more difficult with customer and contractor on different geographical sites. Minimal processes sufficient
The customer, contractor, and other involved parties work at the same site or can visit and work on each other’s site on a daily basis.
Controlled The customer, contractor, and other involved processes parties work at different sites, with their respecrecommended tive teams located at the same site, and cannot meet on a daily basis. Robust processes recommended
3.2.4
The customer, contractor, and other involved parties have several teams involved in the project dispersed on several different sites and they cannot meet on a daily basis.
Other factors The factors stated above are only suggestions and you will probably have to look at more factors to be able to customize the project appropriately. Other factors include: standards required, laws and regulations, other projects, cultural differences, application domain, system complexity, etc.
28
Software Acquisition Management Guidelines
4
Roles
“No matter how much you want it to be a technical problem, it’s a people problem.” – Unknown
Having good people and good teamwork is key to the success of the project. This chapter presents the different types of roles that may be necessary for a successful software acquisition project. One person can be assigned several roles. The roles descriptions are assembled from and based upon the different frameworks stated in appendix A, Marciniak and Reifer (1990), Salwin (1998), and the interviews with Urmi (2000), Oskarsson (2000), and Ran (2000).
4.1
Sponsor roles The sponsor roles are played by persons representing the organizations that have the powers to initiate and cancel the acquisition project.
4.1.1
Sponsor The sponsor initiates and supports the software acquisition project enthusiastically but also has the power to cancel it if it will cost more to finish than will be gained from finishing the project. The responsibilities of the project sponsor are:
29
4 Roles • To define and communicate a goal and vision for the project. • To provide a stable and adequate funding and set the financial limits and other bounds for the project. • To appoint an acquisition manager that has the ultimate responsibility for the success of the project. The sponsor should be careful not to interfere too much with the way the acquisition manager manages the project. 4.1.2
Collaborating customer Should the acquisition project be a co-sourcing project with several customer organizations involved, the collaborating customer is another project sponsor whose collaboration needs to be coordinated with the other sponsors.
4.1.3
Product manager In some cases the software is part of a larger systems acquisition with the product manager for that system playing the sponsor role.
4.2
Managing roles The managing roles are played by the people on the customer side that are put in charge of managing, monitoring, and administering the proceedings of the acquisition project.
4.2.1
Acquisition manager The acquisition manager is appointed by the sponsor and is ultimately responsible for the success of the project. Therefore, the acquisition manager has the final word on how the project should be performed. The tasks and responsibilities of the acquisition manager involves the following:
30
Software Acquisition Management Guidelines
4 Roles • To adapt and customize the acquisition strategy according to the project characteristics. • To plan the project and execute it accordingly, re-plan as the project progresses, manage risks, and solve problems. • To recruit and organize people for the acquisition team, perform teambuilding and training activities, praise and encourage people, smooth the path for everyone, and be supportive. • To select contractor and supporting contractors. • To negotiate and sign agreements with the contractor and supporting contractors. • To manage the relationship with the contractor and other involved organizations built on moral, trust, communication, and collaboration. • To tell the contractor what the scope of the software is so that you and the contractor knows when you are done. • To manage the budget for the project within the limits set by the sponsor. • To coordinate the work for monitoring project progress and report to the project sponsor. • To perform reality checks periodically. Take necessary actions if the answers to the following questions aren’t satisfactory: o Are we acquiring the right software? o Are we acquiring the software right? o Is the contractor building the right software? o Is the contractor building the software right?
© Daniel Svennberg 2001
31
4 Roles • To make necessary trade-offs between delivery time, total cost, product scope, and product quality if deviation from the previous estimates and specifications of them should occur. • To coordinate work for evaluating and accepting the product. • To prepare for post-contract support and maintenance of the product. 4.2.2
Technical manager For large projects that are technically complex or with very high quality demands, such as life-critical software, the acquisition manager can delegate responsibilities for requirements and quality issues to a technical manager.
4.2.3
Contract manager For projects with many involved organizations or projects where the administrative burden of the contract is high, such as cost-reimbursement contracts, the acquisition manager can delegate responsibilities for contract management issues to a contract manager.
4.2.4
Administrator The administrative burden depends on the size and type of project. One should be careful not to over-administrate. Other members of the acquisition team perform the tasks of the administrator role, but for some projects a specific person can be assigned this role. The responsibilities of the administrator role include: • To maintain configuration and change management on project documents such as contract, requirements specification, project plan, risk list, etc.
32
Software Acquisition Management Guidelines
4 Roles • To document and file correspondence, meeting minutes, and decisions. • To administer payments to the contractor and supporting contractors. • To document and file measurements on progress, quality, requirements changes, etc.
4.3
Assisting roles The assisting roles are played by different experts and support contractors that can assist and give advise to the managing, steering, and executing roles.
4.3.1
Acquisition expert An acquisition expert can be used for giving advice on how to plan and perform the acquisition project, what contracting vehicle to select, and to train the acquisition staff in software acquisition management issues.
4.3.2
Technical expert A technical expert can be used to evaluate the requirements and to give independent cost and size estimates. Furthermore, the technical expert can be used to review technical documents and to audit the technical knowledge and capabilities of the contractor. The contractor can use technical experts for areas that they lack competence in.
4.3.3
Domain expert A domain expert is not necessarily a technical expert but knows the field in which the software is intended to be used very well. The domain expert can therefore be used to develop and validate the requirements for the software. Another task for the domain expert can be to develop user training for the software.
© Daniel Svennberg 2001
33
4 Roles 4.3.4
Legal expert A legal expert is essential for giving advise on how to write the contract and to inform on the actual laws that regulates the contract and other matters concerning the project, such as intellectual property rights, patents, licensing, warranties, copyright, etc. The legal expert can also help during disputes and schisms with the contractor. The use of a legal expert with softwarespecific knowledge is money well spent, especially in international projects, according to Salwin (1998).
4.3.5
Translator A translator is, according to Salwin (1998), a role that can be played by a person that is good at translating layman’s terms into technical terms and vice versa. This can be useful in specifying the requirements and transferring the requirements to the technical staff of the contractor. This person should be skilled at translating the needs of the customer into something that the developers can use to build the system.
4.3.6
End user The end user is the raison d’être of the software, unless of course the software doesn’t interact directly with a human user. Therefore, it is imperative to involve the end user on the elicitation of the requirements for the software, on evaluation of the graphical user interface, and on evaluation of the functionality of the software product.
4.3.7
Maintenance and support staff The maintenance and support staff has the primary tasks of keeping the software running, making upgrades, fixing bugs, and adding new features as well as giving technical support to the end users. Therefore, it is important to get suggestions from the maintenance and support staff on what to require from the software to make it easier to maintain and troubleshoot. They should also be involved in evaluating the maintenance and troubleshooting capabilities of the software and to review the
34
Software Acquisition Management Guidelines
4 Roles documentation to see if it conveys the necessary information required to do their jobs. The maintenance and support staff should be involved early in the project in order to be able to discuss how the maintenance and support organization should be organized and to facilitate a smooth transition of the software once it is delivered to the maintenance and support organization with maintained configuration management. This role can either belong to the customer organization or be outsourced. 4.3.8
Independent verification and validation An independent verification and validation contractor, IV&V, can be hired by the customer to make an in-depth technical assessment of the delivered software. This is mainly useful when the software affects peoples’ health or lives or when large amounts of money are at stake should the software malfunction. IV&V naturally increases the costs for the project significantly.
4.3.9
System engineering and technical assistance When the buyer or seller lacks adequate resources or specialized expertise, the use of a supplemental support or services contractor such as a system engineering and technical assistance contractor, SETA, can be necessary. The services provided by a SETA contractor can range from planning the project to testing, measuring, quality assurance, etc. The use of SETA contractors should be considered for the following areas, according to Marciniak and Reifer (1990): • Technical risk. • Independent testing. • Management support. • Integration.
© Daniel Svennberg 2001
35
4 Roles
4.4
Executing roles The executing roles are played by representatives from the contractors side, i.e. those developing the actual software product.
4.4.1
Contractor The contractor can be a sole contractor or a prime contractor in case several collaborating contractors are used. Choose contractor carefully – the contractor with the lowest bid or most optimistic schedule and budget estimates isn’t always the best choice. No management practices can make up for having a bad contractor. Once selected, the contractor should be seen as a team member and not an adversary. The responsibilities of the contractor are: • To develop and deliver the requested software. Should the project be cancelled, what has been developed up to that point should be delivered. • To demonstrate that the sufficient technical and managerial capabilities exists to be able to develop the software. Should subcontractors or supporting contractors be used, their competence and capabilities needs to be demonstrated as well. • To provide a feasible plan and work breakdown structure for the project as well as realistic estimates on total cost and delivery time. • To show the customer that progress is being made by holding regular demonstrations of the so far developed software. • To make sure that the requirements are understood correctly. • To foster an effective, functional, supporting, and collaborative culture in the relationship with the customer built on moral and trust. According to Ran (2000), both
36
Software Acquisition Management Guidelines
4 Roles parties must realize that they have a mutual responsibility for the success of the project and should refrain from taking advantage of each other. The involved parties should work as a team, jointly solve problems, and refrain from blaming each other. • To promptly and openly inform the customer of unanticipated problems and schedule changes. • To openly and jointly discuss, review, and mitigate risks. 4.4.2
Collaborating contractor Should more than one contractor be used, the collaboration between the contractors needs to be coordinated. This could be accomplished by assigning one of the contractors as a prime contractor with the responsibility to coordinate the work of the collaborating contractors.
4.4.3
Subcontractor The contractor can use subcontractors to deliver parts of the software. The competence and capabilities of the subcontractor needs to be evaluated, and you as a customer should have the right to decide which subcontractor should be used. If the contractor is using subcontractors, make sure that they are involved as soon as possible in the project.
4.4.4
Supplier It might be necessary to deal with a supplier of COTS and/or hardware during the course of the project.
© Daniel Svennberg 2001
37
5
Artifacts
“Everything has been said before, but since nobody listens we have to keep going back and beginning all over again.” – A. Gide
In the acquisition processes different documents, reports, deliverables, and other artifacts are used. This chapter contains suggestions for the contents of those artifacts. The artifacts have been organized into the phase that they are most likely to appear for the first time of the following four generic software acquisition project phases: inception, solicitation, monitoring, and finalization. The contents of the artifacts have been assembled from the different frameworks stated in appendix A unless otherwise stated. Many of the artifacts are documents, and there are many reasons to document something, such as the following: • To communicate vital information or to explain how you think. • To help remember something. • To help you think – the process of writing something down in a lucid and understandable way makes you do a lot of thinking you wouldn’t have done otherwise, regardless if the document is needed later or not.
38
5 Artifacts • To legally prove that a transaction has occurred or that a commitment exists. There is always a risk of documenting too much, writing things you won’t need later, or having too much bureaucracy. Therefore it is necessary for you to evaluate the need of each artifact and the applicability of each artifact checklist item in order to customize the artifact for the actual project. Some of the artifacts can be used as checklists during meetings while others require some written documentation. It is also important to remember that some of the artifacts, such as the acquisition plan, the requirements specification, and the risk list are likely to change during the course of the project. Therefore it is necessary to have some form of configuration and change management of these artifacts.
5.1
Inception During the inception phase the needs of the software are identified, the requirements are elicited, risks are analyzed, and the acquisition processes are planned.
5.1.1
Acquisition proposal The purpose of the acquisition proposal is to present the needs for and the feasibility of the acquisition, and also what benefits and risks exists so that an informed decision can be made by the sponsor on whether to initiate the acquisition project or not. In addition to the frameworks stated in appendix A, this artifact is based upon Pressman (1997). Contents checklist
Describe the organization’s needs for the software product and how they are related to the organization’s objectives. What is the background of the needs? What is the current situation? What justifies the needs for the software?
© Daniel Svennberg 2001
39
5 Artifacts
5.1.2
Conceptualize the software product and give operational scenarios. What will the software have done? What are the short-term and long-term consequences for acquiring the software? What are the advantages and drawbacks? Analyze the alternatives for acquiring the software: COTS, MOTS, internal development, outsourced development, joint venture development, enhancement of existing software, or combinations of these alternatives. What are the risks, costs, and benefits for each alternative? Also do a market study of the alternatives. Describe whether the benefits, short-term or long-term, outweigh the costs for acquiring the software? Analyze the technical feasibility of the software. Are there any developmental risks? Can the necessary technical resources such as skilled staff and equipment be made available? Does the technology exist to develop the software or is it better to wait? Determine any infringement, violation, or liability that could result from development of the system. Examine patents, copyrights, and other intellectual property rights. Describe any external constraints on the project, such as standards, regulations, other projects, interfacing systems, etc. Describe any public relations issues. Is there value in others knowing about this? How do we do that? Describe how the project will be funded and give a time frame for the project. State any priorities, assumptions, limitations, and tradeoffs considered.
List potential contractors.
Give a motivated acquisition decision recommendation.
Project specification The purpose of the project specification is to document the decisions made by the project sponsor regarding project goals,
40
Software Acquisition Management Guidelines
5 Artifacts bounds, expected outcome, etc. which serves as a decision basis for the acquisition manager. Contents checklist
Conceptualize the software product and give operational scenarios. What will the software have done? Define the purpose of the project. State the acquisition project’s goal and vision in specific, measurable, accepted, realistic, and time-based terms. Specify the expected outcome of the project in terms of total cost, delivery time, product scope, and product quality, etc. Specify the initially identified product requirements and their relative priorities.
Describe any identified risks.
Specify funding and time bounds.
5.1.3
Describe the organization’s needs.
Identify and give instructions regarding interfaces and collaborations with other projects. Identify legal issues, such as intellectual property rights, etc. Specify any external constraints, such as standards, regulations, interfacing systems, etc. Give instructions on security, access to information, and confidentiality issues. List potential contractors and/or capabilities that can be used for identifying potential contractors, such as required competence, etc. Describe routines for reporting project status. See the Project status report on page 56.
Acquisition plan The purpose of the acquisition plan is to describe how the acquisition project will be managed.
© Daniel Svennberg 2001
41
5 Artifacts Contents checklist
State the project funding and schedule bounds. Define the terminology that will be used in the acquisition project. Describe what tools, techniques, and methods will be used. Describe any standards, laws, practices, and conventions that are to be followed.
Determine problem resolution procedures.
Specify documentation requirements and procedures.
Specify configuration and change management procedures. Describe collaborations and interfaces with external organizations, support contractors, etc. and how the responsibilities are divided between the involved organizations. Specify communication and reporting channels. List important addresses, etc. Determine acquisition project team inter-group coordination and division of responsibilities. Outline a master budget for the different processes. Outline a master schedule describing the chosen acquisition life cycle and appropriate milestones. Specify security, access to information and confidentiality procedures.
Describe how public relations issues will be handled.
Describe routines for reporting project status
42
Describe the project’s background. Why is the project done? What is its purpose? State the project’s objectives.
Describe how measurements will be collected, analyzed and used.
Software Acquisition Management Guidelines
5 Artifacts
Plan each of the acquisition processes in chapter 6, Processes, on page 60: o Identify the process group members, their respective responsibilities, and the organization of the group. o Describe any collaborations and interfaces with other groups and organizations. o Specify the scope and objectives of the process activities. o Determine the working methods of the process group. o Specify and schedule the process activities. Determine who is responsible for each activity. o Specify the budget for the process activities. o Describe other resources allocated for the process activities in form of equipment, tools, materials, documents, facilities, etc. o Determine how the process activities will be coordinated and how progress, problems, etc. will be reported and reviewed.
5.1.4
Risk list The purpose of the risk list is to describe the identified risks, prioritize the risks and provide contingency and mitigation approaches for each risk. The risk list should be maintained current and used as a tool for communicating risk issues between the involved parties. Each item on the risk list can contain several of the following: Contents checklist
Risk identification.
Risk classification.
Date of last update.
Risk description, i.e. condition and consequences.
Probability of risk occurring.
© Daniel Svennberg 2001
43
5 Artifacts
Impact of risk occurrence.
Risk exposure, i.e. probability × impact.
Indicators of risk turning into a problem.
Name of person responsible for mitigation activities and date for last implementation of those activities.
Status of indicators and mitigation activities.
Contingency plan, what to do if the risk occurs.
5.1.5
Approaches to control, transfer, avoid, minimize, and mitigate risk.
Contingency triggers, what conditions are necessary for the contingency plan to be implemented.
Requirements specification The purpose of the requirements specification is to capture and describe the requirements of the software product and other deliverables. The requirements can be described several different ways such as shall-statements, problem statements, use cases, or user stories1. The contents checklist is partially based upon Kar and Bailey (1996). Contents checklist
State the project goals. Which goal is the most important one in case a trade-off decision needs to be made? Describe the software product in general terms; what is the background for acquiring the software, what is its usage domain, what type of product is it, etc.
More information about requirements can be found in numerous web sites and books. See for instance “Managing Software Requirements: A Unified Approach” by Dean Leffingwell, Don Widrig, and Edward Yourdon (ISBN 0201615932). Information on user stories can be found on: http://www.extremeprogramming.org/rules/userstories.html.
1
44
Software Acquisition Management Guidelines
5 Artifacts
Describe the different kinds of users the software will have. How do the different types of users use the software? What will the software have done for the user?
Are there any safety or security restrictions?
Describe the target hardware for the software.
Describe the external interfaces, to humans, hardware and other software.
Define the terminology used in the specification.
State any assumptions that have been made.
State the functional requirements. How shall the software handle the different types of input, and what output should it produce? How should errors be handled? State the non-functional requirements, in terms of performance, portability, efficiency, reliability, usability, availability, and maintainability. Verify and validate each requirement: o Has the requirement been designated an identification for traceability purposes? o What is the justification and motivation for the requirement? o How will the fulfillment of the requirement be verified? o What other requirements is the requirement related to? o Is the requirement necessary? Will a deficiency exist if the requirement is deleted? o Is the requirement minimal or can it be divided into separate requirements? o Does the requirement state what is required, and not how it should be met? o Is the requirement attainable? o Is the requirement complete or does it need further amplification? o Is the requirement consistent with the other requirements in such that it doesn’t contradict any
© Daniel Svennberg 2001
45
5 Artifacts other requirement, isn’t a duplicate of any other requirement and uses the same terminology as the other requirements? o Is the requirement unambiguous? o Is the requirement written without the use of vague words, such as “flexible” and “user friendly”? o Is the requirement verifiable by inspection, analysis, demonstration, or test and is a verification criterion stated for each requirement?
Prioritize the requirements.
Describe what risks are connected with each requirement.
Give implementation time estimates for each requirement.
Verify and validate the total set of requirements: o Is the total set of requirements complete, and doesn’t need any further amplification? o Is the total set of requirements consistent, in such that no duplicate or contradictory requirements exist, and the same terminology is used for all requirements?
5.1.6
State any installation requirements. State any requirements regarding manuals and documentation.
Maintenance plan The purpose of the maintenance plan is to describe how the support and maintenance organization will be working. Contents checklist
46
Identify the members of the support and maintenance organization, their respective responsibilities, and how they are organized. Describe any collaborations and interfaces with other groups and organizations.
Software Acquisition Management Guidelines
5 Artifacts
5.2
Describe the resources allocated for maintenance activities in form of equipment, tools, material, documents, facilities, etc. Specify allocated funding and budget for maintenance activities. Determine maintenance scope and how the maintenance activities will be coordinated. Describe procedures for problem and maintenance reporting. Describe how problem and modification analysis will be performed. Describe procedures for modification implementations. Describe how maintenance activities will be reviewed and accepted. State migration and software retirement procedures. State configuration management activities and procedures. State the continuation of requirements management activities and procedures. Specify what standards, methods, practices and conventions will be followed.
Solicitation During the solicitation phase requests for proposals are sent out to potential contractors, proposals are received and evaluated, and a contractor is selected with which a contract is negotiated and signed.
5.2.1
Request for proposal The purpose of the request for proposal, RFP, is to communicate the scope of the work and the contractual conditions to potential bidders.
© Daniel Svennberg 2001
47
5 Artifacts Contents checklist
Describe the customer’s company.
State the acquisition project’s goal and vision.
Describe the target domain of the software product.
List the deliverables of the project.
Include the requirements specification (see page 44).
Specify any support, training, installation, and maintenance requirements. Give instructions for bidders regarding the proposal such as: date for reply, number of copies, contents and structure of proposal, etc. See the Proposal on page 50. State any requirements regarding development practices, quality assurance procedures, standards compliance, configuration management, communication and reporting progress and problems, etc.
Specify the contract type (see page 88).
Cost and schedule estimates.
48
State any issues regarding security, nondisclosure, and confidentiality.
Describe any investment issues, such as facilities and equipment. Specify the customer’s rights to inspect the bidder’s facilities to investigate and evaluate financial position, technical capability, experience, quality practices, etc. State any legal issues regarding warranties, licenses, ownership rights, copyright, patent, usage rights, and other intellectual property issues. Determine use and control of subcontractors and suppliers. Describe identified risks and issues that need to be dealt with. State any other significant contractual terms and conditions, such as constraints. See the Contract on page 51.
Software Acquisition Management Guidelines
5 Artifacts 5.2.2
Contractor evaluation criteria The purpose of the contractor evaluation criteria is to give a set of criteria against which the bidding contractors’ proposals will be compared in order to be able to select the most appropriate contractor. Be explicit about the minimum standard for each criterion. The set of criteria can cover the following issues: Contents checklist
Technical approach, suggested solution, and understanding of the problem.
Proposed mitigation and management of certain risks.
Cost and schedule estimates.
Company and owner history. Any bankruptcies or litigations? How long has the company been in business? What previous customers has the company had? Contractor stability. Is the organization undergoing any major change, such as changing ownership or moving office, etc? Outcome of contractor’s previous projects regarding cost, schedule, performance, and quality as well as staffing levels, error rates etc. See Project status metrics on page 100. Comments from customers to previous projects. Contractor’s financial status. Eventual independent financial rating. The staff’s experience, competence, constitution, and performance.
Technical capabilities.
Management qualifications.
Resources: equipment, tools, facilities, etc.
Location.
Adequacy of software development processes and quality practices. Previous experience with contractor.
© Daniel Svennberg 2001
49
5 Artifacts
5.2.3
Legal issues regarding warranties, licenses, intellectual property rights, etc. Adherence to eventual standards.
Proposal The purpose of the proposal given by the bidding contractor(s) is to present their company, capabilities, suggested solution, estimates, advantages, etc. Contents checklist
Previous and current customers. Outcome and data on previous projects. Financial position. Understanding of problem, technical approach, solution suggestion, description on how the different requirements will be met.
Technical and managerial capabilities.
Working methods and development processes.
Quality practices.
Resources: equipment, tools, facilities, etc.
Compliance with standards (such as ISO 9000).
Technical and domain experience.
Cost and schedule estimates.
Mitigation and management of risks.
Organization and staffing. Key personnel. The staff’s experience and constitution.
Contact persons and addresses.
Charges, prices, fees and payment issues.
Use of subcontractors and suppliers.
Needed investments in facilities and tools, etc.
50
Company description. Its background and history.
Legal issues regarding warranties, licenses, intellectual property rights, etc. Software Acquisition Management Guidelines
5 Artifacts 5.2.4
Contract The purpose of the contract is to define the relationship obligations and promises that the customer and contractor make to each other. The contents checklist is based upon Oskarsson (n.d. a, b), Marciniak and Reifer (1990), and the frameworks in appendix A. Contents checklist
State the purpose of the contract.
Define the terminology used.
State the scope of work. Refer to the requirements specification (see page 44), which should be attached to the contract. Address software and documentation deliverables, support, installation, and training. Specify the acceptance procedures and criteria such as the following: acceptance test completed with a satisfactory result, installation completed, user training completed, all deliverables received, correction for errors found a determined period after delivery are corrected. Specify the quality measures by which the contractor’s work will be evaluated. Describe what constitutes a satisfactory performance of the contractor. Specify procedures and conditions for making modifications or amendments to the scope of work. Specify who is authorized to make changes in the contract and answer questions from the contractor. Also specify contract change control procedures. Describe payment conditions, considerations, and amounts. Determine allowable costs, expenses, charges and their variation. Determine when payments are to be made linked to deliverables and milestones. Specify concerns regarding taxes, late payments, and interests. Specify use of incentives for earlier delivery, reduced costs, significant results and achievements, or for use of certain “best practices,” etc. See page 88 for more on incentives.
© Daniel Svennberg 2001
51
5 Artifacts
52
Specify any penalties or payment reductions for delays or for not meeting certain requirements, etc. Ran (2000) says that you shouldn’t have penalties unless the risk is shared, i.e. both sides should lose something. Marciniak and Reifer (1990) state that penalties should only be used to correct a situation in the project or to prevent a similar situation from occurring in the future. Determine allowable royalty and license fees. State the timeframe of the contract and issues regarding a final delivery date if that is applicable. State conditions, provisions and obligations concerning contract termination. The contractor should deliver all material and deliverables that are associated with the contract if the contract is terminated and the customer should pay for the work performed so far. Describe responsibilities for maintenance undertakings. State ownership rights, copyright, patent, trade secrets, usage rights, licenses, and other intellectual property issues. Describe legal concerns on warranties, representation, infringement, limitation of liability, indemnification, nondisclosures, etc. Describe security policies, nondisclosure, access to information, and confidentiality issues. Specify how public relations issues will be handled. Specify the contractor’s rights and obligations to assign its duties, partition the work, and use subcontractors. Determine how and where disputes will be solved, and whether an arbitrator should be used. Specify the relationship between the parties in the contract such as joint venture, partnership, employer and employee relationship, etc. Describe the commitments by the involved parties and the division of responsibilities, obligations, and tasks. Determine the amount and conditions of customer involvement. Software Acquisition Management Guidelines
5 Artifacts
5.3
Specify relationships to other parties involved in the project, such as IV&V and SETA. Specify use and replacement of key personnel. Specify use of facilities and equipment. Also determine any investment issues regarding facilities and equipment. Establish means for monitoring progress, through WBS, short iterations, milestones, or other means. Specify the allowable requirements change rate (for fixed price contracts). Specify procedures for reporting problems and progress. Determine what status metrics should be reported by the contractor and how often. See the Project status metrics on page 100. Specify any requirements on the contractor’s software development, management and quality assurance processes. Specify any requirements on a software development plan for the project and a latest date to receive the plan from the contractor and what it should contain. Specify any requirements on the use of COTS or hardware or any restrictions on what suppliers to use. Determine the customer’s inspection rights and procedures for audits. Signatures.
Monitoring During the monitoring phase the progress and performance of the contractor is monitored and the software product is evaluated on a regular basis.
5.3.1
Artifacts from the contractor During the monitoring phase some of the communication from the contractor can consist of the following artifacts, depending on the actual project:
© Daniel Svennberg 2001
53
5 Artifacts • Project plans. • Test reports. • Problem reports. • Payment requests. • Progress and status reports. • Change requests. • Quality reports. 5.3.2
Product evaluation plan The purpose of the product evaluation plan is to specify the objectives, scope, and activities of the different product evaluations. See Product evaluation approaches on page 109 for a list of different kinds of evaluations that can be performed. Contents checklist
54
Specify who will participate in the evaluation and the responsibilities of each participant. Describe the objectives and scope of the evaluation. Give instructions for preparing for, carrying out, and evaluating the results of the product evaluation. Give a schedule for the evaluation. Specify what resources are required regarding equipment, tools, materials, documents, facilities, etc. Determine the test configuration, technical environment, and other prerequisite conditions. Describe each test case in the evaluation. What requirements are addressed during each test case? Specify input and expected results for each test case. Describe the coverage of the test cases in the evaluation. For instance, for an acceptance test it should be shown that all implemented requirements are covered by the test cases in the evaluation.
Software Acquisition Management Guidelines
5 Artifacts
5.3.3
State criteria for evaluating and reporting evaluation results. See the Product evaluation report on page 55.
Product evaluation report The purpose of the product evaluation report is to document the outcomes of a performed product evaluation. See Product evaluation approaches on page 109 for a list of different kinds of evaluations that can be performed. Contents checklist
State the time for the evaluation, the participants and their responsibilities. Identify the test configuration, technical environment, and other prerequisite conditions. Describe any special considerations, simplifications, and assumptions. List the outcomes of each test case. Determine whether the coverage of the test cases in the evaluation was sufficient or not.
Summarize all errors and problems encountered.
Analyze the most common errors and identify trends.
State how the errors and problems will be followed-up.
5.3.4
Describe the objectives and scope of the evaluation.
Give recommendations for decisions, such as changes to requirements, acceptance, etc.
Change request The purpose of the change request is to provide motivation and relevant information to enable an informed decision on whether to approve a change to requirements, contract or plans. Contents checklist
State what should be changed.
State who originated the request.
© Daniel Svennberg 2001
55
5 Artifacts
Give a justification for the proposed change.
State the priority of the change request.
5.3.5
Describe any assumptions and constraints that affect the change request. State the impact on cost, time, quality, scope, and risks.
Project status report The purpose of the project status report is to summarize the status of the project so that it can be used as a basis for a decision on whether to continue the project or cancel it. Contents checklist
56
List the status of the risks in the risk list. Any changes? Any problems that need to be dealt with? Report results from product evaluations and the contractor’s testing efforts. Analyze the technical feasibility of the contractor’s approach. State any technical problems or changes in the technical approach. State any changes to the requirements. Describe the available resources regarding equipment, tools, materials, documents, facilities, etc. Are the originally planned resources available? Are the resources appropriate? List measurements regarding schedule and budget, etc. See section 6.9.1, Project status metrics, on page 100. Identify and analyze trends regarding schedule and budget, etc. Is the project on schedule? Forecast delivery date. How much of the funding have been expended? Is the current funding appropriate? Will the project reach its budget goals? List the results achieved since the last report. What milestones have been accomplished? Describe ongoing activities. Software Acquisition Management Guidelines
5 Artifacts
5.4
What actions need to be taken within the project? What actions need to be taken by the sponsor or steering group? Does the benefits, short-term or long-term, outweigh the predicted remaining costs for acquiring the software? Give conclusions and recommendations and relate them to project objectives. Should the project continue or be canceled? Any changes to the project direction?
Finalization During the finalization phase the product and other deliverables are delivered to the support and maintenance organization and a post partum review is held.
5.4.1
Deliverables The deliverables received from the contractor could consist of some of the following artifacts, depending on the actual project: • Software executables. • Source code. • Technical documentation. • User training material. • Rest list – lists requirements not implemented and known errors. • User manual. • Support publications.
5.4.2
Post partum plan The purpose of the post partum plan is to describe how the post partum will be conducted. The plan is based upon Kerth (n.d.).
© Daniel Svennberg 2001
57
5 Artifacts Contents checklist
5.4.3
Determine who should attend the post partum. Preferably representatives from all involved organizations and roles should participate. Determine when to hold the post partum. Preferably within 7-14 days after project completion. Select an appropriate location for the post partum. Preferably away from the office. Determine the length of the post partum. 2-4 hours may be sufficient for minimal projects while 3 days could be needed for robust projects. Determine the rules for the post partum in order to create a non-threatening environment.
Post partum The purpose of the post partum is to document, analyze, and package experiences and wisdoms for reference in future acquisition projects. Contents checklist
58
Give the background for the project and the time of events in the project. Document the customization factors values (see the Customization factors on page 21) for the project. Compare project goals to actual outcome regarding the deliverables, time schedule, costs, etc. Indicate deviations and the reasons for them. Document what processes and methods were used and how the project was organized. Document how the project was customized. Document special difficulties and measures taken and the effect of these measures. Document the performance of the contractor and other involved organizations – their strengths and weaknesses. If earned value was used – document SPI, CPI, etc. Software Acquisition Management Guidelines
5 Artifacts
Document what development methodology and working methods were used by the contractor. Document what risks occurred and what measures were taken regarding risks. Document experiences and observations. Document what activities/techniques were considered key to the success of the project. Propose improvements on the acquisition processes and other things.
© Daniel Svennberg 2001
59
6
Processes
“I have it anecdotally that the most commonly used methodology in the world is chaos.” – Ron Jeffries
“A good process will tell you to do what a good manager and staff would do anyway.” – Tom DeMarco
This chapter describes the different acquisition processes. The contents of the processes have been assembled and adapted from the different frameworks stated in appendix A unless otherwise stated.
6.1
Overview The different aspects of managing a software acquisition project have been divided into the following eleven processes: • Project steering • Project management • Planning and organizing • Acquisition training • Requirements management • Risk management 60
6 Processes • Contractor selection • Contract management • Product evaluation • Transition to support • Post partum Each process contain suggestions on what activities to perform for the different project customization levels suggested in chapter 3, Customization, on page 20. The process template in the table below explains how to interpret the process descriptions. Keep in mind that the activities given in the different customization levels are only suggestions. When planning for an acquisition project you can use the suggestions as a basis but you have to remain flexible and adapt to the specific circumstances of the actual project. For instance, if you have a project where all of the factors described in chapter 3, Customization, on page 20, are suggesting minimal processes except for requirements volatility, which suggest robust processes, then you have to consider having most processes minimal, but a higher customization level for requirements management, and as a consequence a higher customization level on product evaluation, or other adaptations that fit your needs. And remember: following a process requires discipline. Process name Possible start condition for the process.
Possible end condition for the process.
Possible inputs to the process.
Possible outputs from the process.
Process group Roles that could be considered as participants in the process group. The roles that are in bold face are roles that should participate as a recommended minimum. © Daniel Svennberg 2001
61
6 Processes Minimal process activities
Activities suitable to perform in a project with minimal project characteristics.
Additional controlled process activities
Additional activities suitable to perform in a project with controlled project characteristics.
Additional robust process activities
Additional activities suitable to perform in a project with robust project characteristics.
Table 3. Process template explaining the elements of the process descriptions.
As an illustration, Figure 4 below depicts at what point in time of the acquisition lifecycle the processes occur.
Project steering Project management Planning and organizing Acquisition training Requirements management Risk management Contractor selection Contract management Product evaluation Transition to support Post partum Inception
Solicitation
Monitoring
Finalization
Figure 4. Timeline of the acquisition processes.
Figure 5 below shows how the processes are related to the different roles and major artifacts. The numbers in the figure refer to the following example of a sequence of major activities from all process customization levels:
62
Software Acquisition Management Guidelines
6 Processes Acquisition proposal Sponsor 1 Project steering
2
Project specification
Domain expert
8
18
Technical expert
7
Risk management
Requirements management
Risk list
Requirements specification
Project status report
15 End user Change request IV&V
3
Project management
12
Acquisition manager
Product evaluation
Translator Evaluation report
Reports and measurements 16
Administrator
14
13
Contract management
Evaluation plan
Reports, requests, etc.
Deliverables
Contractor
17 19 11 Contract
Proposal
RFP 9
10 Legal expert
Contractor evaluation criteria
Contractor selection
Transition to support Maintenance plan 5
6
Acquisition training
Planning and organizing
4 Acquisition expert
Acquisition plan
Post partum
Post partum plan
Post partum
Maintenance & support staff
20
Figure 5. Overview of the software acquisition processes, major roles and artifacts.
© Daniel Svennberg 2001
63
6 Processes 1. The sponsor initiates the project steering process. The needs of the organization, and other project concerns are elicited, analyzed, and documented in the acquisition proposal, which serves as a decision basis. 2. If the sponsor decides to initiate the project, the project goal, funding and time bounds, etc. are specified in the project specification. 3. An acquisition manager is appointed. 4. The acquisition manager customizes, plans, and organizes the acquisition project based upon the project specification, possibly with the assistance of an acquisition expert and others. The plan is documented in the acquisition plan. 5. The maintenance and support organization is identified and the maintenance plan is drafted. 6. The acquisition team performs the necessary acquisition training activities. 7. The requirements management process is initiated. The product requirements are elicited with the help from end users, maintenance and support staff, translator, domain experts, and technical experts, etc. The requirements are documented in the requirements specification. 8. The risk management process is initiated. Identified risks are documented in the risk list by the acquisition manager and others. 9. The contractor selection process is initiated with the contractor evaluation criteria being written followed by writing and sending out request for proposals to potential contractors. The acquisition manager is responsible for this with assistance from acquisition expert, legal expert, etc. 10. The proposals from the bidding contractors along with data gathered from audits and background checks are compared 64
Software Acquisition Management Guidelines
6 Processes with the contractor evaluation criteria and the most appropriate contractor is selected. 11. The contract is negotiated, reviewed and signed between the acquisition manager and the manager on the contractor’s side, with the legal expert advising the proceedings. 12. A joint review and discussion of the requirements specification and each of the parties risk lists is performed prior to commencing development. 13. During the development of the product the contractor delivers project plans, test reports, progress reports, payment requests etc. and the software acquisition team gives feedback and takes necessary action. 14. The contractor submits deliverables for evaluation by the software acquisition team on a regular basis. Feedback is given to the contractor through an evaluation report. Possibly the software is evaluated by and independent verification and validation organization. 15. Changes to requirements are handled and possibly the contract is renegotiated. 16. The acquisition team and the contractor collaborate to solve problems, and jointly reviews project status, plans, risk lists, etc. on a regular basis. 17. For lengthier projects it might be necessary to audit the contractor with the assistance of a technical expert, etc. 18. The acquisition manager regularly briefs the project sponsors on the status of the project through a project status report. The sponsors use this as a basis for a decision to continue or cancel the project. 19. After the final evaluation and acceptance of the software product it is transitioned to the maintenance and support © Daniel Svennberg 2001
65
6 Processes organization. The contract is settled, open items are resolved and final payment is made. 20. A post partum is planned and performed. Celebrate a project well done and document experiences and lessons learned. Remember not to make it a blame session. A more detailed description of the different activities that can be performed in a software acquisition project are given in the process descriptions that follow.
6.2
Project steering The purpose of the project steering process is to give the project sponsors control over the project. It is a two-phase process, where the first phase consists of eliciting and analyzing the needs of the organization, set a goal for the project and take a decision on whether to initiate the project or not. The second phase consists of evaluating the status of the project on a regular basis, at least for lengthier projects, in order to determine whether to continue the project or not. Project steering process Starts when the idea of the project is established.
Ends when the project is terminated.
Input: Organizational needs and objectives, Project status report
Output: Project specification
Project steering group The participants to consider for this process should be those who are authorized to make decisions about the project’s existence and those who can give advice on different project issues such as the following roles: sponsor, collaborating customer, product manager, administrator, acquisition expert, technical expert, domain expert, legal expert, translator, end user, main66
Software Acquisition Management Guidelines
6 Processes tenance and support staff, and system engineering and technical assistance. Make sure to involve the necessary organizations as early as possible in the project. Minimal process activities
Elicit the organizational needs. Review the organization’s objectives. Elicit, analyze, and conceptualize the needs and expectations of the organization. Determine the scope and qualities the software product must possess to achieve the organization’s objectives. Analyze and select acquisition type. Analyze the advantages, drawbacks, risks, costs, and benefits for the following alternatives: COTS, MOTS, internal development, outsourced development, joint venture development, or combinations thereof, and enhancement of existing software. Select the most suitable alternative. COTS should be the primary candidate, then MOTS, and only as a final solution fully developed software should be considered for most acquisition projects. Investigate intellectual property rights. Determine if there are any conflicts regarding patents, copyrights, trademarks, and registered names, etc. Decide to initiate project or not. Is this the right time to acquire the product? Is the product really needed or is it just cementing the old way of working? Are there any foreseeable risks that are very likely to terminate the project? What are the risks for not acquiring the software? Does the benefits, short-term or long-term, outweigh the costs for acquiring the software? Determine project parameters. Determine the scope of work of the project. Who will provide software support? Identify potential contractors or develop a list of capabilities that can be used for identifying potential contractors. Identify the extent of the responsibilities for the acquiring organization and contracting organization
© Daniel Svennberg 2001
67
6 Processes respectively. Identify interfaces and collaborations with other projects and organizations. Define the expected outcome; roughly estimate cost limits, time schedule, etc. See the Project specification on page 40.
Establish and communicate a project goal. Define the purpose of the project and determine what is considered a successful outcome of the project. Inform all involved parties. Appoint an acquisition manager. Establish project responsibility, authority, and accountability. Decide to cancel or continue the project based on the status of the project or external factors. For minimal, and thus short projects, this decision should be taken in the beginning of the development (about 15-20% into the project). For controlled and robust project this decision needs to be taken on regular intervals. Remember that termination can be success. Document and communicate the decision. The decision should be well motivated and given in a timely manner.
Additional controlled process activities
68
Investigate constraints and consequences. What short-term and long-term consequences (advantages and drawbacks) of the acquisition are there for different individuals and organizations? Identify external constraints on the project such as standards, regulations, interfacing systems, technical constraints, etc. Prepare an acquisition proposal. See the Acquisition proposal on page 39. Use it as a basis for the decision on whether to start the project or not. Document the project parameters. Write a Project specification (see page 40). Software Acquisition Management Guidelines
6 Processes
Evaluate the status of the project on a regular basis. Review the Project status report (see page 56). Evaluate the feasibility to achieve project goals with available resources and constraints. Estimate the tasks and resources necessary to complete the project. Has the business objectives changed? Has the market changed? Has other external constraints changed? Is the project progressing satisfactory? Are there any delays or budget overruns? Are appropriate resources allocated? Use the evaluation as a basis for decisions on whether to continue the project or not.
Additional robust process activities
6.3
Analyze the technical feasibility of the software. Identify technical and developmental risks. Investigate whether the necessary technical resources can be made available, etc.
Project management The purpose of the project management process is to perform the project according to plan, replan if necessary, and handle issues not planned for. Project management process Starts when the acquisition manager has been appointed.
Ends when the acquisition project is complete.
Input: Acquisition plan, Risk list, Measurements, Product evaluation report
Output: Project status report
Project management group The participants to consider for this process should be those with the responsibility and authority to manage and administer © Daniel Svennberg 2001
69
6 Processes the project such as the following roles: acquisition manager, technical manager, contract manager, and administrator. Minimal process activities
70
Customize, plan, and organize the acquisition project. Develop an acquisition plan. See the Planning and organizing process on page 72. Train the acquisition staff. Make sure that the people involved in the project are trained in how to perform the different process activities. See the Acquisition training process on page 74. Review project feasibility. Ensure the feasibility of the project regarding adequate and timely resources and that the goal is achievable with the given scope, funding, and schedule. Remember that you can have fixed scope, product quality, delivery date, or total cost but not all four. You may therefore need to make trade-offs to achieve the most important goal of the project. Discuss any feasibility issues with the project sponsors. Regularly review the work of the acquisition process groups. Regularly review what has been done, and decide on next actions for the different process groups. Follow up on previous decisions. Regularly review the work of the contractor(s). Determine how measurements are to be assembled, reported and analyzed. Gather measurements on progress, resource usage, cost, quality, and schedule, etc. and compare to plans and specifications. Identify trends. Evaluate the status of the project and take necessary action. See the Contract management process on page 96. Report project status. The acquisition manager should at agreed upon points in time assemble and report the status of the project to the Software Acquisition Management Guidelines
6 Processes steering group. See the Project status report on page 56.
Perform team-building and social activities. Perform suitable project kick-off and other social or teambuilding activities. Recognize the efforts of the staff members. Make the project fun and exciting. Don’t over-use overtime.
Additional controlled process activities
Identify and coordinate activities with other projects and sign contracts with supporting organizations. Establish problem resolution procedures. Any problems discovered during the project should be duly reported, investigated, and solved. The impact of any changes due to problems should be determined, controlled and monitored. Establish means for resolving the issues that will arise during the project, such as consensus building, negotiating, voting, brainstorming, etc. Inform the involved parties on the problems status. Document and track status of problems until resolved. Resolve both problem and cause. Evaluate the problem resolutions: Were they correctly implemented? Were new problems introduced? Establish quality assurance and configuration management procedures. Establish a configuration management methodology. Identify the items that should be placed under configuration management such as plans, agreements and specifications. Control and track changes to those items. Identify and standardize documents. Control the release and delivery of documents, correspondence, and other items in the project. Verify that the project documents are adequate, complete, and consistent by peer review or other means. Maintain plans and other documents current throughout the project as re-planning occurs, issues are resolved, requirements are changed, and risks are mitigated or discovered.
© Daniel Svennberg 2001
71
6 Processes Additional robust process activities
6.4
Perform in-depth problem analysis. Analyze, categorize, and prioritize problems and identify root causes. Detect trends. Develop a system for identifying, recording, and tracking problems that occur during the project. Update the risk list. Document lessons learned. For long-term projects it might be of value to prepare intermediate reports that documents observations, experiences and lessons learned and gives suggestions on improvements for the rest of the project and future projects.
Planning and organizing The purpose of the planning and organizing process is to customize the acquisition processes for the project and make plans for each process accordingly. Planning and organizing process Starts when the acquisition manager has been appointed.
Ends when the planning activities are finished.
Input: Project specification
Output: Acquisition plan
Planning and organizing group The participants to consider for this process should be those responsible for performing the project and those who can give input to and assistance on planning and organizing issues such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, system engineering technical assistance, contractor, collaborating contractor, and subcontractor. 72
Software Acquisition Management Guidelines
6 Processes Minimal process activities
Customize the acquisition processes. Review the Project specification (See page 40). Get input from involved organizations. Analyze the characteristics of the project and select an appropriate customization level, see Customization on page 20. Review stored information on how previous acquisition projects were managed. Select suitable working methods and make adaptations to suit the actual project. Form groups for all acquisition processes. Determine what roles are necessary for the project. Determine the necessary skills for those roles. Employ the best and most competent staff as possible. Form groups for the different acquisition processes. Assign responsibility, authority, and accountability for the different project activities. Develop an acquisition plan. Develop an appropriate Acquisition plan (see page 41). As a minimum the acquisition plan should cover the following: o Define the acquisition life cycle, activities, and milestones to be used in the project. Address all acquisition processes as appropriate: project steering, project management, planning and organizing, acquisition training, requirements management, risk management, contractor selection, contract management, product evaluation, transition to support, and post partum. o Schedule and budget the project activities. o Document how the project team is organized. o Determine resource allocation. Resources to consider are: tools, equipment, facilities, materials, documents, etc.
Review the acquisition plan. All affected parties should review the plan.
© Daniel Svennberg 2001
73
6 Processes Additional controlled process activities
Determine what type of help and services need to be obtained from external organizations besides the contractor. Organize and establish commitments for all involved external supporting organizations and individuals. Assign responsibility, authority, and accountability for the delegated project activities. Develop an acquisition plan. A more elaborate and detailed plan should be made for controlled and robust processes, i.e. most of the items on the Acquisition plan checklist on page 41 should be covered.
Additional robust process activities
6.5
None.
Acquisition training The purpose of the acquisition training process is to determine and develop the skills needed and the skills that the team members have in order to be able to perform their roles appropriately. Acquisition training process Starts when the acquisition team has been assembled.
Ends when the training needs have been met.
Input:
Output:
Acquisition training group The participants to consider for this process should be those who can determine training needs of the staff and provide or 74
Software Acquisition Management Guidelines
6 Processes acquire the required training and those in need of training. Roles to consider for the acquisition training group are: acquisition manager, technical manager, contract manager, administrator, acquisition expert, and all other roles in need of training. Minimal process activities
Determine skills and knowledge needs. Determine the skills and knowledge that is required to perform each role in the acquisition team. Determine training needs. Determine the current skills and knowledge of the persons that will be designated to the different roles. Determine and prioritize what training activities are essential for the project. Develop individual training plans. Determine who will provide the training and how the training will be performed. Develop or acquire training material. Develop or acquire the necessary training material from within the organization or from external sources. Review the quality of the training courses and material. Perform the training activities. Determine if the performed training was sufficient or if more training is required.
Additional controlled process activities
Maintain training records. Maintain training records such as who has completed which training.
Additional robust process activities
© Daniel Svennberg 2001
75
6 Processes
6.6
Spread knowledge. For long-term projects it might be of value to arrange seminars where different team members share their knowledge to the other team members.
Requirements management The purpose of the requirements management process is to elicit, review, and manage change of the product requirements. Apart from the frameworks in appendix A, this process has partially been based upon Christel and Kang (1992). Requirements management process Starts when the needs for the software have been identified.
Ends when the software has been transitioned to the support and maintenance organization.
Input: Project specification, Change requests
Output: Requirements specification
Requirements management group The participants to consider for this process should be those whose needs are to be fulfilled by the software and those who will maintain and support the software. Other participants should be roles that can give advise and valuable input on how to formulate the requirements. Thus the following roles should be considered: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independent verification and validation, system engineering and technical assistance, contractor, collaborating contractor, and subcontractor roles. 76
Software Acquisition Management Guidelines
6 Processes Minimal process activities
Elicit the requirements. Analyze the needs, conceptualizations, and requirements given in the project specification. Determine if the requirements should be elicited mainly before or after contract award. Define and analyze the operational and problem context. Analyze similar systems. Define operational modes, goals, and scenarios. Get requirements wish lists from suitable organizations, groups and individuals, especially end users and maintainers. Categorize and integrate the requirements. Consider possible future expansions of the system when eliciting the requirements. Document the requirements. Document the requirements in a suitable form: use cases, user stories, or requirements statements. See the Requirements specification on page 44. Prioritize the requirements. Identify the most critical requirements that have a strong influence on usability, functionality, efficiency, reliability, portability, maintainability, other quality factors, performance, cost, schedule, or risk and prioritize accordingly. Establish evaluation criteria for each requirement. Jointly review the requirements and evaluation criteria with the contractor prior to development start. Review and approve change requests. Review and appraise all requirements change requests on what impact they will have on schedule, scope, cost, quality, and risks. Avoid creeping scope, goldplating, and other common problems. Document and inform all parties about all approved requirements changes.
Additional controlled process activities
Formally validate and verify the requirements prior to the
© Daniel Svennberg 2001
77
6 Processes joint review with the contractor. Analyze the feasibility of the requirements and validate them against the needs of the organization. Verify that the requirements reflect end user needs, are verifiable, necessary, concise, implementation free, minimal, attainable, complete, consistent, and unambiguous. Verify especially that the requirements related to safety, security, and privacy protection are correct as shown by suitably rigorous methods.
Measure the requirements change rate. Baseline the requirements before development start and measure the rate of changes to the requirements. The target value for total requirements growth in function points or equivalent is ≤1% per month according to Brown (1998).
Additional robust process activities
6.7
Get external review on critical requirements. Let external experts review the requirements concerning safety, security, and privacy protection.
Risk management The purpose of the risk management process is to identify, manage, mitigate, monitor, and handle the risks of the acquisition project. Risk management process
78
Starts as soon as the need for acquiring the software is identified.
Ends when the acquisition project is complete.
Input: Project specification
Output: Risk list
Software Acquisition Management Guidelines
6 Processes Risk management group The participants to consider for this process should be those who are in charge of managing the project and those who can give input on possible risks such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, technical expert, domain expert, legal expert, end user, maintenance and support staff, independent verification and validation, system engineering and technical assistance, contractor, collaborating contractor, and subcontractor. Minimal process activities
Encourage risk identification and management. Encourage and reward project-wide participation in the identification and mitigation of risks and foster a nonthreatening environment in which the involved parties can discuss risks. Identify risks and develop a project risk list. Identify the project risks, see section 6.7.1, Risk identification, on page 80. Develop a risk list that contains all identified risks, their estimated probability and impact. Identify “show-stoppers.” See the Risk list on page 43. Analyze the developed risk list and decide which risks to take action on. Conduct joint risk reviews. Review risks jointly with all affected parties. Establish a common view of risk scenarios and a mutual understanding and expectation of what can go wrong in the project.
Additional controlled process activities
Develop a more elaborate risk list. Add mitigation and contingency plans and other information as deemed necessary for all risks in the risk list. Most items in the Risk list check list on page 43 should be covered
© Daniel Svennberg 2001
79
6 Processes for controlled and robust processes.
Continuously track risk status. Track risk status and update and maintain the risk list continuously throughout the project. Track mitigation activities and reassess risks on the risk list on a regular basis. Analyze, track, and control risks until mitigated. Establish means for reporting and communicating risk information throughout the project team.
Additional robust process activities
6.7.1
None.
Risk identification A risk can be defined as a potential problem. The risks for a project can be identified via brainstorming, reviewing previous projects, reviewing checklists and taxonomies, etc. The factors to look for are those that are likely to impact the project objectives of scope, quality, time, and cost according to Wideman (1992). There are several taxonomies and checklists to choose from. The table below has been assembled and adapted from Carr, et al. (1993), D.I.R. (2000), and Wideman (1992). It can be used as a basis for risk identification and classification. You can also have a look at the problems listed in section 2.5, Common problems, on page 13. It is recommended to build a company risk database. A. External risks Risk category A.1 Natural hazards and accidents A.2 Crime
A.3 Political unrest
80
Problem examples Storm, flood, earthquake, fire, etc. Vandalism, sabotage, theft, hacking, cracking, etc. Revolution, war, etc.
Possible impact Destruction of equipment and stored data. Destruction and theft of property and data. Cancellation of project.
Software Acquisition Management Guidelines
6 Processes Risk category A.4 Regulatory
A.5 Economical factors
A.6 Indirect effects
Problem examples Unanticipated government intervention in product requirements, pricing, export regulations, etc. Currency changes, inflation, taxation changes, etc. Unwanted social or environmental indirect effects from the use of the software.
Possible impact Increased total cost. Cancellation of project.
Increased total cost.
Lawsuits, loss of company credibility, etc.
B. Project and contract management risks Risk category B.1 Project management
B.2 Acquisition staff
B.3 Organizational needs
B.4 Estimation
B.5 Contractor selection
B.6 Contract
© Daniel Svennberg 2001
Problem examples Insufficient support from corporate management, poor communication, vague goals and direction.
Possible impact Increased total cost. Increased delivery time. Staff morale is lowered. Cancellation of project. Unavailability of suffi- Increased total cost. ciently competent staff. Increased delivery Fragmentation. time. Cancellation of project. Product gives no value The product concept to the end user. Benefit and needs of the organization has not to develop product is been sufficiently less than costs. investigated. Substantial Increased total cost. underestimation of Increased delivery required time and time. budget. Cancellation of project. Selection of a contracIncreased total cost. tor that is inexperiIncreased delivery enced, lowtime. performing, nonPoor quality. cooperative, or in risk Cancellation of project. of bankruptcy. Inappropriate contract Friction between par– not addressing neces- ties. sary issues, or with too Increased total cost. restricting or impeding Increased delivery clauses. time.
81
6 Processes Risk category B.7 Contract management
Problem examples Over-administration.
Possible impact Increased total cost. Increased delivery time. Staff morale is lowered. Friction, the parties do Increased total cost. B.8 Relationship between parties not communicate or Increased delivery work well together. time. Poor quality. Cancellation of project. Poor coordination beRework. B.9 Coordination between parties tween parties regardIncreased total cost. ing configuration Increased delivery changes, division of time. work, etc. Staff morale is lowered. Unstable or insufficient Cancellation of project. B.10 Funding funding. Time pressure – project Poor quality. B.11 Schedule is rushed. Product doesn’t meet end user’s needs. Large-scale developIncreased total cost. B.12 Project scale and complexity ment of a complex sys- Increased delivery tem with many intime. volved organizations. Conflicting political Poor quality. B.13 Internal politics agendas between inIncreased total cost. ternal groups and/or Increased delivery contracted groups. time. Cancellation of project. Suppliers not deliverIncreased delivery B.14 Suppliers of time. hardware and software ing as specified or not supporting critical system components in a timely manner. Lawsuits. Force MaIncreased total cost. B.15 Legal risks jeure. Increased delivery time. Cancellation of project.
C. Technical and product development risks Risk category C.1 Requirements
82
Problem examples The requirements are changing frequently due to incompleteness,
Possible impact Rework Product doesn’t meet end user’s needs.
Software Acquisition Management Guidelines
6 Processes Risk category
Problem examples unclarity, or invalidity. Scope creep. Goldplating.
C.2 Technical feasibility
A requirement is impossible to implement or a set of requirements are conflicting and cannot be implemented simultaneously. The design is more difficult to implement than anticipated. The delivered product is buggy and crashes frequently. Lack of indicators. Inadequate indicators. Reporting erroneous data. Reports are not given on a regular basis. Inexperienced managers. Inefficient organization. Ineffective communications. No control over schedule and budget. Poor cooperation between development teams and with external organizations such as other contractors. Inexperienced staff. Low moral, enthusiasm and productivity. No team spirit. High turnover. Difficult to recruit staff members. Bad attitude towards doing quality work and following standards. Fragmentation.
C.3 Product quality
C.4 Project status indicators
C.5 Development management
C.6 Development staff
© Daniel Svennberg 2001
Possible impact Increased total cost. Increased delivery time. Underestimation of total effort required. Product doesn’t meet end user’s needs. Increased total cost. Increased delivery time. Cancellation of project.
Damage to persons or expensive equipment. Lawsuits. Not knowing where project is heading. Diminished ability to correct the situation early on. Friction between parties. Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.
Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.
83
6 Processes Risk category C.7 Development system
C.8 Development process
C.9 Design and coding
C.10 Integration
C.11 Testing and evaluation
C.12 Maintenance
84
Problem examples Loss of key personnel. Inadequate development system. New system – training required. Development system doesn’t match the user’s or the maintainer’s system. Inefficient or unsuitable process. New and unfamiliar process. Too complicated or administrative process. Process is not followed. Lack of control and coordination of process. Poor quality assurance and lack of configuration management. Large parts of the product are unspecified – developers have to guess. Harder to implement design than anticipated. Goldplating. Integration of the software with COTS and hardware is harder than anticipated. Not sufficiently tested by developers. Not tested in a realistic environment prior to deployment. Not adequately tested. Not evaluated by end user. Maintenance of product impaired by poorly designed product, lack of documentation, etc. Maintainer might go out of business.
Possible impact Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time. Rework. Poor quality. Increased total cost. Increased delivery time.
Product doesn’t meet end user’s needs. Rework. Poor quality. Increased total cost. Increased delivery time. Rework. Increased total cost. Increased delivery time. Rework. Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time. Expensive to maintain. Increased total cost of ownership.
Software Acquisition Management Guidelines
6 Processes
6.8
Contractor selection The purpose of the contractor selection process is to evaluate and select a suitable contractor for the acquisition project. For a more in-depth study of contractor selection, see Assmann (2000). Contractor selection process Starts when the initial set of Ends when the contract has requirements has been elicited. been awarded. Input: Project specification, Requirements specification, Risk list, Proposals
Output: Contract
Contractor selection group The participants to consider for this process should be those who have the authority to sign a contract and those who can give advice on the selection of a suitable contractor and contractual issues such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, technical expert, domain expert, legal expert, and translator. Minimal process activities
Certify that the solicitation activities are conducted in accordance with relevant laws, policies, and guidance. Identify potential contractors. List contractors to send requests for proposals to. Well-known contractors should be primary candidates to minimize risk of getting unserious or over-optimistic bids according to Oskarsson (2000). Make cost and time estimates. Use estimates by analogy or parametric models for minimal processes as they are the least time-consuming to do accord-
© Daniel Svennberg 2001
85
6 Processes ing to Marciniak and Reifer (1990). Have the estimates independently reviewed.
86
Develop and send out a request for proposal to the potential contractors. See the Request for proposal on page 47. Select contractor. Select the winning contractor based upon all available data on the contractors such as the proposals (see page 50), eventual audits, previous customer comments, previous data on contractor performance, etc. Remember that the lowest bid doesn’t necessarily mean the lowest total cost. Develop and negotiate a contract. Select contract type (see page 88) and develop a suitable contract. Get legal assistance for this. Standard software contracts can be achieved from legal firms. Have legal counsel review all contracts. Determine how payment is to be made. Establish acquirer and contractor obligations. Incorporate the requirements and evaluation criteria in the contract. Establish acceptance test procedures. See the Contract on page 51. Jointly review the requirements. Review the requirements with the contractor prior to signing the contract to ensure a mutual understanding of the requirements. Jointly review the contract. Review the contract prior to signing it to ensure that both parties have a common understanding of it. Especially acceptance criteria and procedures, handling of changes to requirements, handling of problems after delivery, the obligations of the acquirer, and legal issues regarding intellectual property, warranty, confidentiality, etc. should be reviewed. Sign the contract.
Software Acquisition Management Guidelines
6 Processes Additional controlled process activities
Develop contractor evaluation criteria and evaluate the proposals against them. Develop Contractor evaluation criteria (see page 49) and evaluate the proposals against them and not against each other, according to Salwin (1998). Audit the candidate contractors. Conduct audits at the candidate contractors’ sites; Evaluate their financial position, technical capability, experience, and quality practices. Review the experience of the contractor’s staff. Consider interviewing the staff members. Assess the stability of the contractor, is the organization undergoing change, such as changing ownership or moving office? Interview previous customers. Check the background of the contractor and their staff. Evaluate the adequacy of the contractor’s development process. Evaluate the quality of previous work; see code samples for instance. Evaluate the contractor’s overall capabilities and identify any limitations and liabilities in meeting the organization’s objectives. Make cost and time estimates. Make bottom-up estimates based upon the work breakdown structure (WBS). Use the Delphi method, i.e. let various experts reach a consensus on the most plausible estimates. Have the estimates independently reviewed.
Additional robust process activities
Distribute proposal guidelines for review to potential bidders prior to issuing the formal request for proposal. Consider using two-phase acquisitions. Consider diving the project into two separate phases with separate contracts. The initial phase is used to define the requirements, develop alternate design approaches, and prototype implementations. The second phase consists of the fullscale development of the product. (Reifer, 1997)
© Daniel Svennberg 2001
87
6 Processes
6.8.1
Make cost and time estimates. Use several different estimation techniques, such as the Delphi method, Monte Carlo-simulations, parametric estimates, etc. Have the estimates independently reviewed.
Contract types Selecting contract type is a fundamental decision in the acquisition process. There are two main types of contracts, fixed-price and cost-reimbursable, and a number of variants of them. In fixed-price contracts, the contractor assumes the main part of the cost risk. Because of this, the scope of the work needs to be well defined and attainable, and the work and any changes to the scope require strict management. In contrast, the customer assumes the main part of the cost risk for cost-reimbursable contracts. This requires an administrative overhead of keeping detailed and accurate cost and progress records. A benefit of cost-reimbursable contracts is that you can have a more flexible management of the work and changes to the scope, suitable for innovative projects or projects where the scope of the work is volatile. A drawback is that there is a lack of incentives to control the cost and schedule, according to Urmi (2000). According to Marciniak and Reifer (1990), there are two types of incentives: cost incentives and award incentives. Cost incentives can be used to motivate the contractor to shorten schedule or reduce cost. The fee is based upon how well the contractor meets cost or schedule objectives. However, a caveat should be given that if the target cost of the incentive fee structure is severely off target to the disadvantage of the contractor, the contractor might consider continuing to accrue labor hours in order to increase revenue and make up for the loss of fee. This possibility could be negated by having negative fees or penalties. The customer should also consider stopping the work of the contractor to protect the budget. Award incentives can be used to motivate the contractor to meet certain success criteria regarding end user satisfaction and
88
Software Acquisition Management Guidelines
6 Processes software quality, or for using certain software development “best practices” such as peer reviews, configuration management, earned value, etc. Marciniak and Reifer (1990) argue that award fees need to be connected with the completion of a welldefined event. Brown, et al. (1998) presents some questions that should be of concern when using award incentives: How can the incentive targets be objectively evaluated? How will cases where factors beyond the control of the contractor impact the contractor’s achievement of the incentives criteria be dealt with? Are the additional administrative costs that will incur in managing the contract outweighed by the benefits achieved? In the tables below the different variations on these two contract types and the time and materials contract type are presented. Each table contains a brief description on how the contract works, when it is applicable, who takes the main cost risk – contractor or customer, and also rates its administration level on a subjective scale from very low to very high. The contract types presented are the most common, although there are other contracting arrangements, such as basic ordering agreements, etc. When choosing contract type you could consider dividing the project into phases, such as a requirements elicitation phase followed by a software development phase, and have different contracting vehicles for each phase. The contents of the contract types presented below are based upon Brown et al. (1998), Marciniak and Reifer (1990), Reifer (1997), and Wideman (1992). FFP – Firm fixed price Description
The customer pays a fixed price for the contractors work.
Applicability
FFP contracts apply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attainability of the project is very likely.
© Daniel Svennberg 2001
89
6 Processes Cost risk
Customer
Administration
Very low – if a lump sum is paid at final delivery. Low – if partial payments are used (e.g. monthly). High – if partial progress payments are used (e.g. milestone driven).
Contractor
FPEPA – Fixed price with economic price adjustment Description
A fixed price contract with price adjustments for certain costs for material, labor, travel expenses, etc.
Applicability
Similarly to FFP contracts, FPEPA contracts apply for projects where the scope of the work is well defined and non-volatile and the attainability of the project is very likely. The FPEPA contract is specifically applicable for lengthier contracts where both parties need to be protected from certain cost fluctuations, such as airfares, etc. This contract can be combined with FPR.
Cost risk
Customer
Administration
Moderate – with lump sum payment. High – with partial payments.
Contractor
FPR – Fixed price redetermination Description
90
An initial fixed price is negotiated. Certain factors are agreed upon à priori for adjusting the price at specific times such as requireSoftware Acquisition Management Guidelines
6 Processes ments growth, changes in function points, etc. Applicability
The FPR contract applies to situations where the scope of the work is partially well defined and non-volatile and partially volatile.
Cost risk
Customer
Administration
Moderate
Contractor
FPIF – Fixed price plus incentive fee Description
In addition to paying a fixed price, a variable fee is determined by negotiating the following parameters: • Target and ceiling cost • Minimum, maximum, and target fee • Adjustment formula (or share ratio) The adjustment formula specifies how target cost underruns and overruns – without going over the ceiling cost – will be shared between the customer and contractor.
Applicability
Similarly to FFP contracts, FPIF contracts apply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attainability of the project is very likely. FPIF contracts can specifically be used as an instrument to motivate the contractor to increase efficiency, shorten schedule, and reduce cost. This contract can be combined with FPR.
Cost risk
Customer
© Daniel Svennberg 2001
Contractor
91
6 Processes Administration
Moderate – with lump sum payment. High – with partial payments.
FPAF – Fixed price plus award fee Description
The contractor is paid a fixed price and a base fee. In addition, the contractor receives award fees when certain criteria are met. The base fee should be set low, around 3% of the target cost according to Marciniak and Reifer (1990), and the maximum fee could go up to 20%.
Applicability
Similarly to FFP contracts, FPAF contracts apply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attainability of the project is very likely. FPAF contracts can specifically be used as an instrument to stimulate the contractor’s performance in certain areas that are difficult to measure objectively such as end user satisfaction, software quality, use of “best practices,” etc.
Cost risk
Customer
Administration
High
Contractor
CS – Cost and cost share Description
92
In a cost share contract the customer and contractor agree on the ratio by which they will share costs. In a cost contract the customer pays for all allowable costs. The contractor receives no fee in either contract type. Software Acquisition Management Guidelines
6 Processes Applicability
Cost share contracts apply for joint development efforts where both parties share risk in order to share the future rewards from the product. Cost contracts apply for projects with nonprofit organizations.
Cost risk
100% by the customer for cost contracts. Shared, according to the agreed ratio, for cost share contracts.
Administration
High
CPFF – Cost plus fixed fee Description
The contractor's allowable costs are reimbursed and a fixed fee is paid upon delivery.
Applicability
CPFF contracts apply for projects where the costs cannot be reasonably estimated such as innovative projects or projects where the scope of the work is volatile or difficult to define.
Cost risk
Customer
Administration
High
Contractor
CPPF – Cost plus percentage fee Description
The contractor’s allowable costs are reimbursed and a percentage of the costs is received as a fee.
Applicability
CPPF contracts apply for projects where the costs cannot be reasonably estimated such as
© Daniel Svennberg 2001
93
6 Processes innovative projects or projects where the scope of the work is difficult to define. Cost risk
Customer
Administration
High
Contractor
CPIF – Cost plus incentive fee Description
In addition to reimbursing the contractor’s allowable costs, a variable fee is determined by negotiating the following parameters: • Target and ceiling cost • Minimum, maximum, and target fee • Adjustment formula (or share ratio) The adjustment formula specifies how target cost underruns and overruns – without going over the ceiling cost – will be shared between the customer and contractor.
94
Applicability
Similarly to CPFF contracts, CPIF contracts apply for projects where the costs cannot be reasonably estimated such as innovative projects or projects where the scope of the work is difficult to define. CPIF contracts can specifically be used as an instrument to motivate the contractor to increase efficiency, shorten schedule, and reduce cost.
Cost risk
Customer
Administration
Very high
Contractor
Software Acquisition Management Guidelines
6 Processes CPAF – Cost plus award fee Description
The contractor is reimbursed for allowable costs and receives a base fee. In addition, the contractor receives award fees when certain criteria are met. The base fee should be set low, around 3% of the target cost according to Marciniak and Reifer (1990), and the maximum fee could go up to 20%.
Applicability
Similarly to CPFF contracts, CPAF contracts apply for projects where the costs cannot be reasonably estimated such as innovative projects or projects where the scope of the work is difficult to define. CPAF contracts can specifically be used as an instrument to stimulate the contractor’s performance in certain areas that are difficult to measure objectively such as end user satisfaction, software quality, use of “best practices,” etc.
Cost risk
Customer
Administration
Very high
Contractor
TM/L – Time and materials / Labor hours Description
The contractor is paid an hourly rate for the time to perform the agreed upon work and is reimbursed for any necessary purchases.
Applicability
TM/L contracts apply for projects where the extent and duration of the work cannot be reasonably estimated. The contract type suits projects where you want to be able to quickly change the direction of the work and can monitor the work closely. A ceiling price
© Daniel Svennberg 2001
95
6 Processes should be determined. Cost risk
Customer
Administration
Low
Contractor
The table below gives suggestions for what contract types to consider for the different customization levels:
6.9
Minimal
FFP, FPR, FPIF, TM/L
Controlled
FPEPA, FPR, FPIF, CS, CPFF, CPPF, CPIF
Robust
FPEPA, FPR, FPIF, FPAF, CS, CPFF, CPPF, CPIF, CPAF
Contract management The purpose of the contract management process is to monitor the work of the contractor, gather data on project status, resolve problems, administer payments, and other contractual issues. Contract management process Starts when the contract has been signed.
Ends at the conclusion of the contract’s period of performance.
Input: Contract
Output: Measurements
Contract management group The participants to consider for this process should be those with the ability and authority to administer the contract, resolve contractual problems and issues, and those who can give assis96
Software Acquisition Management Guidelines
6 Processes tance in monitoring and assessing the progress of the contractor such as the following roles: acquisition manager, contract manager, administrator, acquisition expert, technical expert, domain expert, translator, and legal expert. Minimal process activities
Perform the acquirer’s contractual obligations. Perform the acquirer’s obligations as stated in the contract such as delivering software, hardware, data, or other deliverables essential for the development of the software product in a timely manner and with the agreed specifications. Gather project status data. Determine what to monitor, for instance measurements on budget, schedule, progress, risk, scope, quality, etc. Establish routines for gathering the data. See Project status metrics on page 100. Require frequent status reports etc. from the contractor (see Artifacts from the contractor on page 53). Review budget and schedule status. Track budget and schedule and identify trends. Compare to plan and what was accomplished. See Earned value on page 102. Watch out for delays, small delays accumulate. Force replanning if schedule slippage is greater than 10% according to Brown (1998). (Oskarsson, 2000) Evaluate the software product regularly. See the Product evaluation process on page 106. Report measurements and problems. Report measurements on the process and product, etc. on a regular basis to the acquisition manager for inclusion in the next Project status report (see page 56) to the steering group. Also give feedback to the contractor. Eventual problems should be reported immediately. Communicate regularly and promote a supporting relationship with the contractor. Create an effective, functional, supporting and collaborative
© Daniel Svennberg 2001
97
6 Processes culture in the relationship with the contractor built on trust. Both parties should realize that they have a mutual responsibility for the success of the project. The involved parties should work as a team, jointly solve problems, and refrain from blaming each other. Conduct regular project meetings with all involved parties to give feedback and discuss progress, requirements, budget, schedule, risks, problems, etc.
Manage contractual change. Coordinate all contractual changes with all affected groups and individuals. Ensure that all proposed changes to the contract are analyzed, documented, agreed upon or denied and communicated to all affected parties. Administer partial payments, incentives, etc. Settle the contract. On final delivery and acceptance of the product, the contract should be completed and settled, any open issues should be resolved. Final payment should be withheld until all is delivered. Check that deliveries have been completed successfully, ownership has been transferred, etc. See also the Transition to support process on page 110.
Additional controlled process activities
98
Review the contractor’s plans. Review the contractor’s plans regarding project management, risk management, quality control, configuration management, integration, and testing and use them as a basis for overseeing the contractor’s development effort. The contractor’s use of subcontractors, software/hardware suppliers, and tools should be reviewed and approved. Reestimate total cost and delivery time when appropriate. Base the estimates on WBS and extrapolate from actuals. Audit the contractor on a regular basis. Hold audits to determine project and product status. Audits can be performed both on predetermined occasions and on Software Acquisition Management Guidelines
6 Processes an ad hoc basis. Audit procedures should be agreed upon by both parties. Analyze the outcome of the audit, identify trends, and adjust the acquisition approach to mitigate risks if necessary. The outcome regarding risks, problems and eventual change of project direction should be discussed with the contractor. The outcome of the audit and responsibilities for resolving found problems should be agreed upon by both parties and documented. Technical experts or acquisition experts can be of assistance during audits. Examples on what to review: o That the working methods of the contractor regarding methodologies, processes, procedures, practices, quality assurance, etc. are adhered to and conducted in an effective fashion. o That the contractor has planned the project adequately and timely. o That the project is staffed appropriately, that the staff is working on the actual project, and that the personnel is trained as required by the contract. o That the allocated resources are appropriate. o That documents and deliverables meet specifications and follows coding and documentation standards, etc. o That the technical approach and working methods are appropriate. o The configuration management practices. o Risk mitigation activities. o Interview managers and technicians on technical and managerial difficulties.
© Daniel Svennberg 2001
99
6 Processes Additional robust process activities
6.9.1
Use external auditors. For longer projects you could have an external auditor visit the contractor and review the status of the project, interview the managers and especially technicians about the project and eventual problems. The auditor can also look at random error reports and track the error as a test of the contractor’s configuration management routines, etc. (Oskarsson, 2000)
Project status metrics It is important to choose metrics that motivates the right behavior, or in the words of Jim Coplien: What gets measured gets done.
The following are examples of metrics that can be used to measure the status of the project: • Requirements change rate. The target value for total requirements growth is ≤ 1% per month. (Brown, 1998) • Contractor staff turnover rate. The target value for voluntary staff turnover per year is 1-3%according to Brown (1998). High staff turnover indicates low staff moral. • Contractor staff overtime rate. (Brown, 1998) • Contractor staffing rate. Actual staffing compared to needs. (Marciniak and Reifer, 1990) • Test progress. Executed test cases compared to planned, successful tests, tests with errors, etc. (Oskarsson, 2000) • Software maturity index, SMI. Indicates the stability of the product. The product is stabilizing when SMI approaches 1.0. (IEEE 982.1-1988) MT = number of modules in current version. Fc = number of changed modules in current version. 100
Software Acquisition Management Guidelines
6 Processes Fa = number of added modules in current version. Fd = number of deleted modules in current version. SMI=
MT – (Fa + Fc + Fd) MT
• Developmental progress. Function points completed, requirements completed, incremental releases, modules completed, etc. (Marciniak and Reifer, 1990) • Milestones completed. Actual vs. planned, see the figure below. (Oskarsson, 2000) Time
Replan 2 Replan 1
1
2
3
4
Time
Milestones Figure 6. Milestone tracking and trend diagram, Oskarsson (2000).
• Problem reports. Open vs. closed. Marciniak and Reifer (1990) define problems as anomalies discovered in requirements, design, or software. • Size growth. How the size of each software item grows as a function of time. Size is directly related to cost according to Marciniak and Reifer (1990). • Error rate. The amount off discrepancies found during testing and evaluation. High error rates indicate product immaturity and low quality according to Marciniak and Reifer (1990). • Earned value metrics. See Earned value, on page 102. © Daniel Svennberg 2001
101
6 Processes
6.9.2
Earned value According to Fleming and Koppelman (1996) the concept of earned value has been around for at least a hundred years, first used on the factory floors in the late 1800s. The usage of earned value enables the prediction of final costs and final schedule results, within a certain range, for the actual project. Since 1967, every new major systems project implemented by the United States Department of Defense (DoD) has been required to use the Cost/Schedule Control Systems Criteria, or C/SCSC for short, which is a set of 35 requirements embedding the earned value concept. Over 700 major DoD contracts since 1977, where C/SCSC (and thus earned value) has been used has shown that you can, as early as 15-20% into the project, predict the final costs and time requirements within a predictable range of values. Fleming and Koppelman (1996) suggests a simplified version of earned value to be used for virtually any project who relates planned costs to actual expenses and don’t have a clue about what they got for what they spent. I agree with them and suggest that you can use the concept for any size of project and with any type of development model, even iterative. Earned value project requirements The earned value concept requires the following: 1. An appropriate defined scope of the project (whole project or iteration), or work to be done, which enables you to tell what’s in the scope and what’s not. This is preferably made with a work breakdown structure, WBS, down to the necessary level of granularity. For cost-reimbursable contracts, the buyer defines the top two or three levels of the WBS. For fixed-price contracts the contractor defines all of the WBS. The work should be defined down to discrete work tasks that can be managed and easily determined complete not complete.
102
Software Acquisition Management Guidelines
6 Processes 2. The work segments defined in the previous step must be planned and scheduled. Typically, CPM1 is appropriate to accomplish this. If several contractors and subcontractors are used, their schedules must of course be in accordance with a master schedule. 3. Responsibility for each work segment should be assigned and resources will need to be estimated and budgeted for all work segments within the defined project scope. When estimating, refrain from adding reserve time. Do this later when all of the work segments have been estimated. Thus, the so called Cost Account Plan should contain the following: a. Statement of work (scope). b. Schedule (start/stop dates for each task). c. Budget (in dollars or hours or units). d. Responsible person. e. Responsible department. f. Type of effort (recurring or non-recurring). g. Division into discrete work packages. h. Method used to measure earned value performance (milestone, formula, percent complete, standards, apportioned). 4. Establish a baseline of the estimated, scheduled and budgeted work segments in the previous step, not including any reserves, from which project or iteration performance can be measured. The baseline should be under con-
1
Critical Path Method
© Daniel Svennberg 2001
103
6 Processes figuration management to enable traceability back to the original baseline. All changes to the baseline have to be reviewed, controlled, and approved or rejected. The baseline is needed to be able to tell how much of the project is finished, to be able to compare completed work against actual work and actual costs. 5. Monitor iteration/project performance against the baseline and forecast final results. Also manage baseline changes to maintain the baseline current. An example Assume you have a one-year $100,000 project scheduled with an expenditure rate of $25,000 per quarter. At the end of the first quarter the actual expenditures are $22,000 and thus $3,000 under the planned effort. From these values you can’t really tell whether the project is behind schedule or underrunning costs. By adding the earned value dimension, which measures the value of the actual work accomplished, we can tell much more about the actual status of the project. Let’s say that the work segments completed at the end of the first quarter had an earned value of $20,000. Then we can immediately tell that we are $5,000 behind the planned 25 000 Planned work of $25,000. Since the $ actual expenditures to accomplish the work are $22,000 we 22 000 Spent can also tell that we have a cost 20 000 EV overrun of 10%. So, with earned value we are able to tell that the actual project is running late and 1Q overspending. See the figure to the left. Figure 7. Earned value example.
104
Software Acquisition Management Guidelines
6 Processes Testable requirements Peter B. Wilson (2000) suggests the use of testable requirements with earned value. A testable requirement is defined as a requirement that someone is able to write one or more test cases for that would validate whether the requirement has or has not been implemented correctly. As an example you plan to implement 80 testable requirements in 500 labor hours during the first month of the project. When the month has passed 475 labor hours has been expended and 72 of the requirements have been implemented correctly. This gives you an earned value of (72/80) * 500 = 450 labor hours. So in this case you overspent 25 labor hours. Schedule and cost performance indices Schedule performance index, or SPI, is determined by the following formula: Earned Value SPI = Planned Value This value tells you how much of the work planned to be accomplished actually was accomplished. A SPI < 1.0 is a warning signal. The SPI can be a valuable tool for use in conjunction with critical path analysis to forecast the expected completion date for the project. Cost performance index, or CPI, is determined by the following formula: Earned Value CPI = Actual Costs This value tells you how much work was accomplished for every project dollar spent. A CPI < 1.0 is a warning signal. The CPI can be used to forecast a statistical range of estimated final costs to complete the project. © Daniel Svennberg 2001
105
6 Processes Forecasting final cost and schedule A realistic bottoms-up estimate of the remaining tasks is the most desirable forecasting method, but CPI and SPI can be used to produce statistical forecasts of final required funds to complete the project. Statistical range of cost-to-complete forecasts: Low end forecast = (Best case)
High end forecast = (Most likely)
Total budgeted funds CPI
Total budgeted funds CPI * SPI
The To-Complete Performance Index, or TCPI, can be used to determine what performance level it will take to accomplish all remaining effort to achieve some specified management objective. Earned Value of work remaining TCPI = Funds remaining
6.10 Product evaluation The purpose of the product evaluation process is to evaluate the deliverables from the customer to see if they meet the specified requirements and quality objectives. This includes evaluation of partial or incremental deliveries and the final evaluation, i.e. acceptance test. 106
Software Acquisition Management Guidelines
6 Processes Product evaluation process Starts when the requirements are being developed.
Ends when the product and other deliverables have been finally accepted and delivered as agreed.
Input: Requirements specifica- Output: Product evaluation tion reports, Change requests Product evaluation group The participants to consider for this process should be those whose needs are to be fulfilled by the product, those who can give assistance and advice on the evaluation of the deliverables, and those who have the authority to accept the deliverables such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, administrator, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independent verification and validation, and system engineering and technical assistance. Minimal process activities
Determine what testing approaches to use. Determine the appropriate testing approaches and the testing effort required. See Product evaluation approaches on page 109. Identify the artifacts to be evaluated. Write test cases up-front. Make sure that evaluation criteria have been established for all the requirements with instructions on how to perform the evaluation for each requirement. Establish evaluation procedures and criteria for all chosen testing approaches. Conduct periodical evaluations. Conduct periodical evaluations throughout the acquisition
© Daniel Svennberg 2001
107
6 Processes project to minimize the risk of getting a product that doesn’t live up to the expectations of the end user. Write plans for the different evaluations if that is deemed necessary (see the Product evaluation plan on page 54).
Analyze and take action on evaluation results. Certify that the coverage of the tests is sufficient. Analyze the test results regarding requirements and quality fulfillment and determine what action needs to be taken for the found deficiencies. Determine if any changes need to be made to the requirements. Make appropriate documentation of the test results (see the Product evaluation report on page 55). Review the evaluation results with the contractor. Discuss the found deficiencies, non-conformances, remedies, and eventual requirements changes with the contractor. Don’t forget to give some positive feedback too to maintain good relations with the contractor. Conduct acceptance testing. Conduct the procedures for final evaluation and acceptance of the product as agreed upon in the contract. Use all evaluation results as a basis for accepting the product.
Additional controlled process activities
Document requirements change requests. See the Change request on page 55. Track found deficiencies. Document and follow-up on the deficiencies that were found during the evaluations. Overview the contractor’s testing efforts. Review the results of the contractor’s testing activities.
Additional robust process activities
108
Identify common errors. Software Acquisition Management Guidelines
6 Processes Collect statistics on common errors. Suggest improvements and fault preventions.
Consider using IV&V for products that can affect people’s health or huge amounts of money should they malfunction.
6.10.1 Product evaluation approaches The following is an overview of some different approaches for evaluating a software product. Several of these tests should be conducted throughout the project and during the final acceptance testing. It is important to remember that tests cannot show that a product is 100% error-free. I’ve not included tests that are primarily performed by the contractor such as unit and integration tests, but limited the list to tests in which the customer takes part: Alpha tests are conducted on pre-release versions of the software product by the customer in a controlled environment at the contractor’s site with developers present. (Pressman, 1997) Beta tests are conducted on pre-release versions of the software product by end users at one or more customer sites. The contractor is normally not present during beta tests. (Ibid.) Functional tests demonstrate that the functional requirements have been met. These tests can be conducted by working through use cases or performing several tests defined for each requirement or other means. Not only correct but also erroneous inputs should be given to the system. Installation tests should be conducted to ensure that the product can be installed correctly. Support tests should be conducted to ensure that the needs of the support and maintenance organization are met.
© Daniel Svennberg 2001
109
6 Processes Recovery tests forces the software to fail and verifies that the system recovers properly. (Ibid.) Security tests should be performed to verify that the protection mechanisms built into a system will in fact protect it from improper penetration. (Ibid.) Stress tests executes a system in a manner that demands resources in an abnormal quantity, frequency, or volume in an attempt to break the program. (Ibid.) Performance tests are important for real-time and embedded systems to see if they fulfill the performance requirements. (Ibid.) User interface tests are important tests performed by the end users to evaluate the user interface’s appropriateness.
6.11 Transition to support The purpose of the Transition to support process is to prepare for a smooth transition of the deliverables to the support and maintenance organization and to verify that all deliverables and other services, such as user training and installation, has been received as agreed upon. Transition to support process Starts when the software requirements are being developed.
Ends when the software and other deliverables have been turned over to the support organization.
Input: Deliverables
Output: Maintenance plan
Transition to support group The participants to consider for this process should be those in charge of approving the delivery of the product and those who 110
Software Acquisition Management Guidelines
6 Processes are to install, maintain and support the product such as the following roles: acquisition manager, technical manager, contract manager, administrator, technical expert, domain expert, legal expert, maintenance and support staff, and contractor, collaborating contractor, and subcontractor. Minimal process activities
Identify the support and maintenance organization. Identify and provide a budget for the support and maintenance organization early in the project, before the request for proposal is issued. Will maintenance be performed by the acquirer organization or will it be outsourced? Identify the deliverables to be transferred. Make a list of the deliverables that are to be transferred to the support organization, such as maintenance and support documentation, software, training materials, configuration management systems, etc. Plan the installation of the software product. Establish installation procedures regarding schedule, access to facilities, personnel, availability and access to systems and equipment, approval of installation, etc. Review the results of the user training activities. Make sure that the agreed upon user training has been performed. Verify that all deliverables has been received as specified.
Additional controlled process activities
Prepare the support and maintenance organization. Transfer and customize necessary processes from the development organization to the support organization. A maintenance plan should be developed. See the Maintenance plan on page 46. Review the deliverables and preparations prior to transfer.
© Daniel Svennberg 2001
111
6 Processes Review the usability of the software, its readiness for installation, status of user training, user manuals, status of installation preparations and activities. Review the readiness of the software for transition, the maintenance plans and manuals, the status of transition preparation activities. Review copyright and licensing concerns addressed and agreed to. Review concerns regarding back-up copies of the software. Additional robust process activities
Test the support capabilities of the support and maintenance organization. Before transferring responsibility for the software, the support organization needs to demonstrate its capability to support and maintain the software: o Does it have the appropriate hardware, software, physical, fiscal, and personnel resources? o Does it have a maintenance plan and other documentation, processes and procedures? o Does it have a configuration management system capable of supporting the software? o Is the personnel trained?
112
Maintain configuration and change management throughout the transition.
Software Acquisition Management Guidelines
6 Processes
6.12 Post partum The purpose of the post partum1 process is not about passing judgment, but to learn from experience. What can be done better on future acquisition projects? It also includes documenting data on the project effort and on the contractor’s performance for future reference. Apart from the frameworks mentioned in appendix A, this process is based upon Kerth (n.d.). Post partum process Starts when the project has been finalized.
Ends when the project has been sufficiently reviewed and documented for future reference.
Input: Project data
Output: Post partum
Post partum group Preferably representatives from all involved organizations and roles should participate in the post partum. Minimal process activities
Form a post partum group. Also assign responsibility, authority, and accountability for the post partum activities. Make sure that the people involved in the process are trained in how to perform the process activities. Allocate adequate resources for post partum activities. Resources to consider are: funding, people, tools, equipment, facilities, etc.
Post partum means “after birth” in Latin. I find this to be a more appropriate name for the process instead of the more common term post mortem, which means “after death.”
1
© Daniel Svennberg 2001
113
6 Processes
Set a goal for the post partum. There can be many different purposes for holding a post partum: o To capture effort data in order to understand the existing process, or to improve the existing process, or to provide data for future acquisitions, or to identify training needs, etc. o To document what happened and why. To capture the collective wisdom. o To repair damage to the team. o To enjoy the accomplishment.
114
Plan the post partum. See the Post partum plan on page 57. Prepare for the post partum. Inform the participants that the post partum is a learning experience and not a blame session. Establish rules that create a non-threatening atmosphere for the post partum. Gather the data that will be analyzed during the post partum such as data on efforts, resources, product quality, etc. Each participant should refresh their memory and individually review what happened. Conduct the post partum review. Review what happened during the acquisition project and identify good and bad acquisition practices. What went wrong and what went right? Review the efforts expended regarding budget, staff, schedule, etc. Identify deviations and reasons to them. What was accomplished? What was the quality of the delivered product? Review any special difficulties and how they were handled. Review the contractor’s performance and compliance with requirements. If the contractor doesn’t participate in the post partum review, a post partum from the contractor should be reviewed. Give improvement suggestions. What should be done different the Software Acquisition Management Guidelines
6 Processes next time?
Document data and lessons learned. Identify and document the software acquisition lessons learned, project data, customization data, and data on the contractor’s performance. See the Post partum on page 58.
Additional controlled process activities
None.
Additional robust process activities
Get a skilled facilitator. A suitable facilitator is preferably an outsider to the organizations involved in the project, though knowledgeable about software projects and with good people interaction skills.
© Daniel Svennberg 2001
115
7
Results and summary
“Caution: Cape does not enable user to fly.” – Batman costume warning label
This chapter summarizes the guidelines and gives an outlook on possible future research on software acquisition management.
7.1
Guidelines summary The objective of this master’s thesis is to provide guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer. The objective is further broken down into seeking answers to the following problems: • What problems are common? • What activities should be performed? • What roles are conceivable? • What artifacts are conceivable? • How can the activities, roles, and artifacts be customized for different kinds of projects? 116
7 Results and summary The guidelines presented in this thesis can be used as a basis for planning an acquisition project or be read as an overview over the many issues that concern software acquisitions. The following summarizes the answers to each of the above stated questions briefly: What problems are common? The most common problems with software acquisition projects are the following: • Development costs exceeds budget. • Time schedule is overrun. • Outcome does not meet expectations. According to the Standish Group (1995) Chaos report, which surveyed over 8 000 software development projects, only 16% of the examined projects were completed on time, within budget, and with all features and functions as initially specified. What activities should be performed? There are many activities that can and should be performed for software acquisition projects. In this thesis they have been collected into the following eleven processes: • Project steering • Project management • Planning and organizing • Acquisition training • Requirements management • Risk management • Contractor selection • Contract management • Product evaluation © Daniel Svennberg 2001
117
7 Results and summary • Transition to support • Post partum The activities and guidelines presented in the processes can be summarized into the following high-level activities and guidelines:
118
1.
Define and communicate a goal and vision for the project.
2.
Appoint an acquisition manager that has the ultimate responsibility for the success of the project. The role of the project sponsor is to set the outer limits for the project, provide stable and adequate funding and to support the project enthusiastically or cancel it.
3.
Adapt and customize the acquisition approach and strategy according to the characteristics of the project.
4.
Avoid having too much administration, but as a minimum: have written requirements and agreements, document important decisions and why they were taken, also measure progress, software quality, and requirements changes quantitatively.
5.
Know what you want. Have as stable, complete, feasible, and validated requirements as possible. It is important to have a defined scope so that you and the contractor know when you are done and when the contract needs to be changed.
6.
Define how the fulfillment of each requirement will be evaluated.
7.
Ensure end user involvement in the definition of the product requirements and in the evaluation of the product.
8.
Tell the contractor what you want and not how to do it. Jointly review the requirements before the development starts to sort out misunderstandings and ambiguities.
9.
Choose contractor carefully. The contractor with the lowest bid or most optimistic schedule and budget estimates Software Acquisition Management Guidelines
7 Results and summary isn’t always the best choice. No management practices can make up for having a bad contractor. 10. Create an effective, functional, supporting, and collaborative culture in the relationship with the contractor based on trust and mutual benefit. Both parties should realize that they have a mutual responsibility for the success of the project. The involved parties should work as a team, jointly solve problems, and refrain from blaming each other. 11. Establish effective communication channels and break down barriers between people and departments in the organizations involved in the project. Ensure that you understand each other. 12. Don’t take advantage of the contractor. Create a win-win situation. Have a mutually beneficial contract that you would be comfortable signing as either party. 13. Expect the best, but be proactive and prepare for all eventualities. Jointly discuss possible risks and how they should be mitigated with the contractor. 14. Employ the best and most competent people that is possible, train them, plan and organize their work. Provide necessary assistance and resources. 15. Plan for maintenance and identify the maintenance and support organization before issuing request for proposals. 16. Evaluate the evolving software product early and often prior to final acceptance evaluation to reduce risk of getting a software product that won’t meet the needs of the end user. 17. Perform reality checks periodically. Take necessary actions if the answers to the following questions aren’t satisfactory: o Are we acquiring the right software? o Are we acquiring the software right? o Is the contractor building the right software? o Is the contractor building the software right? © Daniel Svennberg 2001
119
7 Results and summary 18. Be realistic and reassess your expectations as you learn more about the problem domain and the technical feasibility of the program as the project progresses. The initial estimates and specifications of the following four parameters are likely to change to some degree during the course of the project: o Delivery time o Development costs o Product scope o Product quality You may therefore need to make trade-offs to achieve the most important goal of the project. What roles are conceivable? The roles that are conceivable for a software acquisition project can be divided into four categories: The sponsor roles provide the funding for the project and have the power to cancel the project. The sponsor roles presented in this thesis are the following: sponsor, collaborating customer, and product manager. The managing roles plans, manages, monitors, and administers the project. The managing roles presented in this thesis are the following: acquisition manager, technical manager, contract manager, and administrator. The assisting roles are played by different experts and support contractors that can assist and give advise to the managing, steering and executing roles. The assisting roles presented in this thesis are the following: acquisition expert, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independent verification and validation, and system engineering and technical assistance. The executing roles are played by representatives from the contractors side, i.e. those developing the actual software product.
120
Software Acquisition Management Guidelines
7 Results and summary The executing roles presented in this thesis are the following: contractor, collaborating contractor, and subcontractor. What artifacts are conceivable? There are many artifacts that are conceivable for a software acquisition project. The artifacts in this thesis are presented according to which phase of the project they are most likely to appear for the first time: During the inception phase the needs of the software are identified, the requirements are elicited, risks are analyzed, and the acquisition processes are planned. The artifacts presented in this thesis for the inception phase are the following: acquisition proposal, project specification, acquisition plan, risk list, requirements specification, and maintenance plan. During the solicitation phase requests for proposals are sent out to potential contractors, proposals are received and evaluated, and a contractor is selected with which a contract is negotiated and signed. The artifacts presented in this thesis for the solicitation phase are the following: request for proposal, contractor evaluation criteria, proposal, and contract. During the monitoring phase the progress and performance of the contractor is monitored and the software product is evaluated on a regular basis. The artifacts presented in this thesis for the monitoring phase are the following: project management plans, test reports, problem reports, payment requests, progress reports, change requests, quality reports, software demonstrations and incremental releases, product evaluation plan, product evaluation report, change request, and project status report. During the finalization phase the product and other deliverables are delivered to the support and maintenance organization and a post partum review is held. The artifacts presented in this thesis for the finalization phase are the following: software executables, source code, technical documentation, user training material, rest list, user manual, support publications, post partum plan, and post partum. © Daniel Svennberg 2001
121
7 Results and summary How can the activities, roles, and artifacts be customized for different kinds of projects? To answer that question we must first find a way to differentiate projects. This thesis examines a number of customization factors that are used to suggest an appropriate formality level of the acquisition processes based upon the project’s complexity and risk. By examining these factors you can determine which customization level is appropriate for your project. Three different customization levels are given in the thesis: Minimal processes: suitable for small projects with stable, welldefined and attainable requirements, developed by a small, efficient team from a well-known contractor, etc. Controlled processes: suitable for somewhat lengthier medium-sized projects with somewhat volatile requirements, developed by an average contractor, etc. Robust processes: suitable for large, long-term projects, with innovative, critical, and volatile requirements, and with a complex set of contractors and subcontractors involved in the development effort, etc. The thesis suggests what activities are appropriate for the different customization levels, and also gives some suggestions for customization of roles and artifacts.
7.2
Outlook Software acquisition management involves a wide variety of topics ranging from general management theory to special issues regarding software testing, contract types, risk management, etc. It isn’t possible for a single document to cover any of these topics completely. Therefore, the reader probably needs to study texts that cover these topics in depth to gain a better understanding on how to carry out a successful software acquisition project.
122
Software Acquisition Management Guidelines
7 Results and summary However, the following are some suggestions for future research on software acquisition management that could be based upon this thesis: • It could be further investigated how to customize and adapt the different processes and artifacts. Perhaps two customization levels are sufficient? • Are there more customization factors? Which factors are important for which process? • Similar guidelines for the acquisition of COTS software could be made. • Can an expert system or other tools be made for assisting software acquisition management? • How can the guidelines be adapted to match common software development methodologies?
© Daniel Svennberg 2001
123
8
Bibliography
Assmann, Danilo (2000), MASS – a Method for Assessing Software Subcontractors, Kaiserslautern, Fraunhofer IESE Beck, Kent (1999), Embracing Change with eXtreme Programming, Computer, Vol. 32, No. 10, Oct 1999, pp.70-77, IEEE 0018-9126/99 Breu, Michael (1995), A Concise Guide to Euromethod, European Software Institute, ESI-1995-CG004.01 Brooks, Frederick P., Jr (1995), The Mythical Man-Month: essays on software engineering, Anniversary ed., Reading, Addison-Wesley, ISBN 0-201-83595-9 Brown, Norm et al. (1998), The Program Manager’s Guide to Software Acquisition Best Practices version 2.2, Arlington, Software Program Managers Network Brown, Norm et al. (1999), The Guidebook of Software Acquisition Questions version 1.0, Arlington, Software Program Managers Network Carr, Marvin J. et al. (1993), Taxonomy-Based Risk Identification, Pittsburgh, Software Engineering Institute, CMU/SEI-93TR-6 Christel, Michael G. and Kang, Kyo C. (1992), Issues in Requirements Elicitation, Pittsburgh, Software Engineering Institute, CMU/SEI-92-TR-12 124
8 Bibliography CMMI (2000), Capability Maturity Model Integration, http://www.sei.cmu.edu/cmmi/ (15 September 2000) Cockburn, Alistair (n.d.), Balancing Lightness with Sufficiency, http://members.aol.com/acockburn/papers/barelysuffici ent.htm (2 August 2000) DAD (2000), Defense Acquisition Deskbook, http://web2.deskbook.osd.mil/default.asp (1 September, 2000) D.I.R. (2000), Tailoring the Guidelines, http://www.dir.texas.gov/eod/qa/tailor.htm (1 August 2000) Duncan, William R. (1996), A Guide to the Project Management Body of Knowledge, Newtown Square, Project Management Institute, ISBN 1-880410-12-5 Euromethod (1996), Euromethod version 1, Euromethod Project FAA-iCMM (1997), The Federal Aviation Administration Integrated Capability Maturity Model (FAA-iCMM) Version 1.0, U.S. Federal Aviation Administration Fleming, Quentin W. and Koppelman, Joel M. (1996), Earned value project management, Newtown Square, Project Management Institute, ISBN 1-880410-38-9 Gallivan, Michael J. (1999), Analyzing IT Outsourcing Relationships as Alliances among Multiple Clients and Vendors, Proceedings of the 32nd Hawaii International Conference on System Sciences Ganssle, Jack G. (1998), The Seven Habits of Highly Defective Programmers, Embedded Systems Programming – July 1998
© Daniel Svennberg 2001
125
8 Bibliography Haimes, Yacov Y. et al. (1997), A Holistic Management Framework for Software Acquisition, Acquisition Review Quarterly – Winter 1997, Defense Systems Management College, pp. 55-86 IEEE 982.1 (1988), IEEE Standard Dictionary of Measures to Produce Reliable Software, New York, The Institute of Electrical and Electronics Engineers, Inc. IEEE 1062 (1998), IEEE Recommended Practice for Software Acquisition, New York, The Institute of Electrical and Electronics Engineers, Inc., ISBN 0-7381-1514-2 IEEE/EIA 12207.0 (1996), Standard for information technology – Software life cycle processes, New York, The Institute of Electrical and Electronics Engineers, Inc., ISBN 1-55037-077-4 ISO/IEC 9126 (1991), Information technology – Software product evaluation – Quality characteristics and guidelines for their use, ISO/IEC ISO/IEC 15504 (1998), Information technology – Software process assessment, ISO/IEC Kar, Pradip and Bailey, Michelle (1996), Characteristics of Good Requirements, 1996 INCOSE Symposium Kerth, Norman L. (n.d.), An Approach to Post Morta, Post Parta, and Post Project Reviews, Beaverton, Elite Systems Marciniak, John J. and Reifer, Donald J. (1990), Software Acquisition Management, New York, John Wiley & Sons, Inc., ISBN 0-471-50643-5 Mosemann, II, Lloyd K. (1996), Guidelines for Successful Acquisition and Management of Software-Intensive Systems, U.S. Department of the Air Force, Software Technology Support Center
126
Software Acquisition Management Guidelines
8 Bibliography Oskarsson, Östen (2000), interviewed by author at Östen Oskarsson’s home, Linköping, 21 April 2000 Oskarsson, Östen (n.d. a), Tutorial – Acquisition and Supply of Software Development, http://www.oskarsson.se/useful_info/acqtutorial.html (4 April 2000) Oskarsson, Östen (n.d. b), Example of Software Contract Requirements, http://www.oskarsson.se/useful_info/Contract.html (3 May 2000) Oskarsson, Östen and Glass, Robert L. (1996), An ISO 9000 Approach to Building Quality Software, Upper Saddle River, Prentice-Hall, Inc., ISBN 0-13-228925-3 Paulk, Mark C. et al. (1993), Capability Maturity Model for Software, Version 1.1, Pittsburgh, Software Engineering Institute, CMU/SEI-93-TR-24 Ran, Mats (2000), interviewed by author at Q-Labs AB, Linköping, 3 May 2000 Reeves, Jack W. (1992), What is Software Design?, C++ Journal Vol. 2, Nr. 2 1992 Reifer, Donald J. (1997), Software management, New York, The Institute of Electrical and Electronics Engineers, Inc., ISBN 0-8186-8001-6 Salwin, Arthur E. (1998), The Road to Successful ITS Software Acquisition, Washington, U.S. Department of Transportation, Report No. FHWA-JPO-98-035 SS-ISO 9000-3 (1992), Kvalitetssystemstandarder – Del 3: Riktlinjer för tillämpning av SS-9001 vid utveckling, leverans och underhåll av programvara, SIS – Standardiseringskommissionen i Sverige
© Daniel Svennberg 2001
127
8 Bibliography The Standish Group (1995), The Standish Group Report Chaos, http://www.scs.carleton.ca/~beau/PM/StandishReport.html (28 August 2000) Urmi, Jaak (2000), interviewed by author at Ericsson AB, Kista, 14 April 2000 Wideman, R. Max (1992), Project and program risk management: a guide to managing project risks and opportunities, Project Management Institute, ISBN 1-880410-06-0 Wilson, Peter B. (2000), Sizing Software with Testable Requirements, Systems Development Management, Auerbach Publications
128
Software Acquisition Management Guidelines
A
Frameworks coverage
This chapter shows that the processes in the guidelines covers most issues in the main frameworks that have comparable processes or sections.
A.1
SA-CMM Guidelines
SA-CMM
Project steering
Implicitly covered in all key process areas by the “verifying implementation” section.
Requirements management
Requirements development and management
Risk management
Acquisition risk management
Project management
Project management, Project performance management
Planning and organizing
Software acquisition planning, Project performance management
Acquisition training
Training program
Contractor selection
Solicitation 129
A Frameworks coverage
A.2
130
Contract management
Contract tracking and oversight, Project performance management, Contract performance management
Product evaluation
Evaluation, Contract performance management
Transition to support
Transition to support
Post partum
Project performance management
Not covered
Process definition and maintenance, Quantitative process management, Quantitative acquisition management, Continuous process improvement, Acquisition innovation management
FAA-iCMM Guidelines
FAA-iCMM
Project steering
Needs, Coordination
Requirements management
Needs, Requirements
Risk management
Risk management
Project management
Project management, Configuration management, Coordination
Planning and organizing
Project management
Acquisition training
Training Software Acquisition Management Guidelines
A Frameworks coverage
A.3
Contractor selection
Outsourcing
Contract management
Contract management, Project management, Measurement
Product evaluation
System test and evaluation
Transition to support
Transition
Post partum
Not covered
Not covered
Integration, Software development and maintenance, Architecture, Alternatives, Quality assurance and management, Peer review, Prevention, Organization process definition, Organization process improvement, Innovation, Product evolution
IEEE 1062 Guidelines
IEEE 1062
Project steering
Planning organizational strategy
Requirements management
Defining the software requirements
Risk management
Defining the software requirements
Project management
Implementing organization’s process
Planning and organizing
Planning organizational strategy, Implementing organiza-
© Daniel Svennberg 2001
131
A Frameworks coverage tion’s process
A.4
132
Acquisition training
Implementing organization’s process
Contractor selection
Implementing organization’s process, Identifying potential suppliers, Preparing contract requirements, Evaluating proposals and selecting supplier
Contract management
Managing for supplier performance
Product evaluation
Accepting the software
Transition to support
Not covered
Post partum
Using the software
ISO 9000-3 Guidelines
ISO 9000-3
Project steering
Management responsibility
Requirements management
Purchaser’s requirements specification
Risk management
Not covered
Project management
Not covered
Planning and organizing
Not covered
Acquisition training
Not covered
Contractor selection
Contract review Software Acquisition Management Guidelines
A Frameworks coverage
A.5
Contract management
Measurement, Purchasing, Included software product, Management responsibility
Product evaluation
Testing and validation
Transition to support
Replication, delivery and installation, Maintenance
Post partum
Not covered
Not covered
Quality System, Internal quality system audits, Corrective action, Development planning, Quality planning, Design and implementation, Configuration management, Document control, Quality records, Rules, practices and conventions, Tools and techniques, Training
ISO 12207 Guidelines
ISO 12207
Project steering
Acquisition, Joint review, Management
Requirements management
Acquisition, Development, Verification
Risk management
Management
Project management
Management, Configuration management
Planning and organizing
Management, Tailoring, Verification
© Daniel Svennberg 2001
133
A Frameworks coverage
A.6
134
Acquisition training
Training
Contractor selection
Acquisition, Verification
Contract management
Acquisition, Quality assurance, Joint review, Audit, Problem resolution, Verification, Validation
Product evaluation
Acquisition, Verification, Validation
Transition to support
Not covered
Post partum
Not covered
Not covered
Supply, Operation, Maintenance, Documentation, Infrastructure, Improvement
ISO 15504 Guidelines
ISO 15504
Project steering
Acquisition preparation, Management, Project management
Requirements management
Acquisition preparation, Requirements elicitation
Risk management
Risk management
Project management
Configuration management, Documentation, Problem resolution, Management, Project management, Measurement
Planning and organizing
Management Software Acquisition Management Guidelines
A Frameworks coverage
A.7
Acquisition training
Human resource management
Contractor selection
Supplier selection
Contract management
Supplier monitoring, Verification, Validation, Joint review, Audit, Measurement
Product evaluation
Customer acceptance, Verification, Validation
Transition to support
Not covered
Post partum
Not covered
Not covered
Operation, Customer support, Development, System and software maintenance, Quality assurance, Quality management, Organizational alignment, Improvement, Infrastructure, Reuse
Euromethod Guidelines
Euromethod
Project steering
Acquisition goal definition, Acquisition planning
Requirements management
Acquisition goal definition, Tendering
Risk management
Acquisition planning, Tendering,
Project management
Acquisition planning, Tendering, Contract monitoring, Con-
© Daniel Svennberg 2001
135
A Frameworks coverage tract completion
136
Planning and organizing
Acquisition planning
Acquisition training
Not covered
Contractor selection
Tendering
Contract management
Contract monitoring, Contract completion
Product evaluation
Contract monitoring
Transition to support
Contract completion
Post partum
Contract completion
Software Acquisition Management Guidelines
Index
CMMI ...........................................17 code and fix .....................................8 Collaborating contractor................37 Collaborating customer .................30 Commercial-off-the-shelf..............7 Common problems ........................13 communications ...........................16 configuration management.........71 Contract .........................................51 Contract management....................96 Contract manager ..........................32 Contract types................................88 Contracting environment...............26 contractor ... i, 2, 3, 9, 10, 11, 18, 33, 34, 35, 36, 47, 50, 56, 57, 58, 59, 64, 65, 73, 74, 81, 94, 97, 109 Contractor development team characteristics ............................27 Contractor development team size 25 Contractor evaluation criteria........49 Contractor selection.......................85 Controlled processes ...................21 co-sourcing....................................12 Cost and cost share.......................92 cost incentives ...............................88 Cost performance index ..............105 Cost plus award fee.......................95 Cost plus fixed fee.........................93 Cost plus incentive fee..................94 Cost plus percentage fee...............93 cost-reimbursable ..........................88
A Acquisition expert......................... 33 Acquisition manager..................... 30 Acquisition plan............................ 41 Acquisition proposal..................... 39 Acquisition training ...................... 74 Administrative overload............. 15 Administrator................................ 32 Alpha tests ................................. 109 Approach......................................... 3 Artifacts ........................................ 38 Assisting roles............................... 33 Assmann ........................... 2, 3, 4, 85 Audit....................................... 87, 98 auditor ......................................... 100 award incentives ........................... 88 B baseline ....................................... 103 best practices................................. 89 Beta tests .................................... 109 C C/SCSC....................................... 102 Capability Maturity Model ........... 17 Capability Maturity Model Integration ................................ 17 Change request ............................. 55 Chaos” report ................................ 13 characteristics of software .............. 6 137
Index COTS ................ 7, 18, 37, 40, 53, 84 CPAF............................................ 95 CPFF ............................................ 93 CPI .............................................. 105 CPIF ............................................. 94 CPPF ............................................ 93 Creeping scope ............................ 15 Criticality ...................................... 23 CS ................................................. 92 customer i, 2, 7, 8, 10, 11, 12, 34, 35, 109 Customization factors ................... 21 Customization levels..................... 20 Customize .............................. 70, 73 D D.I.R. ............................................ 21 Deliverables .................................. 57 Delivery time ................................ 24 discipline ...................................... 16 DOD-STD-2167A ........................ 18 DOD-STD-2168 ........................... 18 Domain expert .............................. 33 dyadic............................................ 11 E Earned value ............................... 102 Effort............................................. 24 end user........... 10, 15, 81, 82, 83, 84 End user ........................................ 34 Error rate .................................. 101 estimates................................. 85, 87 Euromethod................................. 19 executive support ........................ 16 Experience with contractor........... 27 experts........................................... 33 External risks .............................. 80 eXtreme Programming ................... 9 F FAA-iCMM ................................. 17 feasibility...................................... 69 Federal Aviation Administration.. 17 138
FFP................................................89 Finalization....................................57 Firm fixed price ............................89 Fixed price plus award fee ...........92 Fixed price plus incentive fee ......91 Fixed price redetermination.........90 Fixed price with economic price adjustment.................................90 fixed-price .....................................88 forecast ........................................106 FPAF.............................................92 FPEPA ..........................................90 FPIF ..............................................91 FPR ...............................................90 Fragmentation .............................15 Frameworks ...................................16 Fraunhofer Institute ........................ ii Frederick P. Brooks.........................1 Friction .........................................16 Fully developed software ..............8 Functional tests..........................109 G Geographic dispersion...................28 Goldplating...................................15 groups ...........................................73 I IEEE 1062 ......................... i, 4, 7, 18 IEEE/EIA 12207............................18 incentives.......................................88 Inception........................................39 Independent verification and validation ...................................35 Innovation......................................23 Installation tests.........................109 intellectual property rights.........67 ISO 12207 .....................................18 ISO 15504 .....................................19 ISO 9000 .......................................18 ISO 9000-3 ....................................18 IV&V.....................................35, 108 Software Acquisition Management Guidelines
Index L Labor hours.................................. 95 Laissez-faire................................. 14 Legal expert .................................. 34 Linköping University.................. 1, ii M Maintenance and support staff...... 34 Maintenance plan.......................... 46 Milestones .................................. 101 Minimal processes....................... 21 Modified-off-the-shelf................... 7 Monitoring .................................... 53 MOTS ............................. 7, 8, 18, 40 multi-vendor.................................. 12 N nature of software ........................... 6 O Objective......................................... 2 objectives ..................................... 16 Organizations involved................. 26 Outline ............................................ 4 Outsourcing..................................... 9 Overpromising ............................ 15 overtime ..................................... 100 P participants.................................... 10 Performance tests ..................... 110 phase ............................................. 38 Planning and organizing ............... 72 Post partum ........................... 58, 113 Post partum plan ........................... 57 problem analysis ......................... 72 Problem reports ........................ 101 problem resolution...................... 71 Process group .............................. 61 Processes....................................... 60 Product evaluation ...................... 106 Product evaluation approaches ... 109 © Daniel Svennberg 2001
Product evaluation plan.................54 Product evaluation report ..............55 Program size ..................................22 Project and contract management risks...........................................81 Project management ......................69 project parameters ......................67 Project specification ......................40 Project status metrics...................100 Project status report.......................56 Project steering ..............................66 Proposal .........................................50 Q Q-Labs ........................................ 1, ii quality assurance .........................71 R reality checks.................................31 Recovery tests ............................110 Request for proposal......................47 requirements change rate ...........77 Requirements change rate ........100 Requirements management ...........76 Requirements specification ...........44 Requirements volatility .................22 resources.......................................16 RFP ................................................47 Risk identification .........................80 Risk list..........................................43 Risk management ..........................78 Robust processes..........................21 Roles..............................................29 root cause.......................................14 S SA-CMM ......................................17 Schedule performance index .......105 Scope ...............................................3 SE-CMM .......................................17 Security tests ..............................110 SETA .............................................35 sites ................................................28 139
Index SMI............................................. 100 Software development .................... 8 Software maturity index .......... 100 Solicitation.................................... 47 SPI............................................... 105 Sponsor ......................................... 29 staffing ....................................... 100 Standard........................................ 18 Stress tests.................................. 110 Subcontractor................................ 37 Supplier......................................... 37 support and maintenance organization........................... 111 support contractors ....................... 33 Support tests.............................. 109 SW-CMM .................................... 17 System engineering and technical assistance .................................. 35 T Target audience............................... 2 taxonomy .......................... 11, 12, 16 TCPI............................................ 106 team-building .............................. 70 Technical and product development risks ................... 82
140
Technical expert ............................33 Technical manager ........................32 test cases .....................................107 Testable requirements .................105 The Standish Group.......................13 Time and materials.......................95 Timeline.........................................62 TM/L.............................................95 To-Complete Performance Index 106 Total cost .......................................25 Transformational impact ...............24 Transition to support ...................110 Translator.......................................34 turnover .........................................27 turnover rate ..............................100 U use cases ........................................44 User interface tests ....................110 user stories.....................................44 W waterfall...........................................9 WBS ............................................102 work breakdown structure...........102
Software Acquisition Management Guidelines
View more...
Comments
