Articles

Page 1 of 15

Project Planning for Successful Software Development

Date: Apr 19, 2002 By Tyson Gill. Sample Chapter is provided courtesy of Prentice Hall. The software industry may never mature into the lucrative and diverse industry that we would all like it to become. But that doesn't have to be the future. Tyson Gill tells you why we should be focusing on the production of a simple but effective software blueprint during the project-planning phase. We've got big trouble in Software City. In the famous musical The Music Man, a fasttalking salesman comes to a small town named River City, Iowa. He tells the gullible folks that they "got big trouble." He warns them that the new pool hall will induce their children to fall prey to bad habits. The only thing that will save them, he declares, is to provide a wholesome alternative, such as a boys' marching band. Of course, it just so happens that the salesman deals in a fine line of quality band uniforms and instruments. Like The Music Man, I'm here to tell you that we got big trouble in Software City. But unlike The Music Man, I'm not saying that just to peddle a textbook equivalent of 76 trombones and 110 cornets. While some authors applaud the state of the industry, I have serious concerns. While there are particular teams that do experience success on certain projects, such successes are not the norm. Sources such as objective statistics, self-grading, client reaction, and anecdotal experience all confirm that the rate of failed software development projects is embarrassingly high. Failure rates from 80 to 90 percent are commonly cited and generally accepted. Of course, the accuracy of these figures depends on how success or failure is measured, but by any measure, the situation is not good. The need to improve our software development processes is very real. To make the picture worse, the cited failure rates reflect only projects that overtly failed to meet objectives. They do not include "successful" projects that cost far more than they might have, or took far longer to complete than was necessary. I disagree with authors who claim that failure rates are artificially high because they are based on unrealistic expectations (DeMarco, 1995). On the contrary, the expectations for minimal success across the industry are far too low. Furthermore, success is most often measured by short-term deliverables, not by long-term maintainability. If failure to maintain were also considered, the picture would look even more dismal. Poor performance is a pervasive and recurring problem that affects the credibility, effectiveness, and long-term survival of the software industry. Despite an undeniable need, fewer potential clients are willing to risk precious capital on custom development projects. They have either heard of, or they have experienced first hand, the development nightmares that all too commonly occur in these projects. As of this writing, the market for custom application development continues to diminish. I don't believe this is due to a lack of need, but rather to a lack of faith in our industry to deliver. Clients are not the only ones who are disenchanted. Developers are perhaps even more frustrated. Talented developers don't want to waste their talent over and over again on doomed projects. New developers get disillusioned with the business and find other work. Managers burn out from having to fight the same client relationship fires all day long. Owners of development shops sometimes just give up application development and change their business to provide more profitable and less problematic software services. We face the real possibility that the fledgling software industry will collapse into a much smaller number of players producing mass-market solutions. It may never mature into the lucrative and diverse industry that we would all like it to become.

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 2 of 15

This should not, and need not, be the future of our industry.1 Custom software development can and should be a thriving, rewarding, and cost-effective industry for both the consumer and for the provider. How is it that an industry that specializes in organizing and streamlining other business practices can itself be so abysmally unorganized and inefficient? When a problem is identified, our first reaction is to look around for someone or something to blame. Where does the blame lie here? Do we blame the people involved for creating this situation? Have they simply failed to implement the excellent processes and practices that are well-documented in the literature? Many would say yes. However, intelligent, talented, and experienced professionals staff many failed projects. We cannot simply dismiss them all as incompetent. We should search for root causes of the problems, evolve effective countermeasures, and apply them. We should create a continuous process improvement ethic. Are the "best practices" that we turn to for guidance perhaps not as good as proponents would like us to believe? I am not willing to say that. As I said before, the practices and approaches recommended in the literature are mostly sound, practical, and based on proven experience. We can't minimize their value and effectiveness. So what is it then? What is out of kilter in the software development industry? In my opinion, the source of the problem is not individuals or any specific practices, but overcomplication. In our conscientious efforts to minimize risk, adopt best practices, meet budget constraints, prevent wasted effort and so on, we introduce overly restrictive and complicated schemes which do not meet the fundamental requirements for productive planning. Our solutions, in part, make worse the very problems we are trying to avoid. We expend far too much time and effort in project planning and management with too little payback. The key to improved success is to simplify the process. Eliminate unnecessary effort. Expose and eradicate the sources of those problems that force the application of overly complex and counterproductive practices. This is best accomplished by focusing on the production of a simple but effective software blueprint during the project-planning phase. 2.1 The Pathology of Poor Planning In the human body, certain materials called allergens stimulate allergic responses. The immune system responds by creating antibodies to combat the allergens. As a side effect, the victim can suffer histamine responses like itchiness and a runny nose, or even more severe life-threatening allergic reactions. In a similar fashion, relatively minor problems identified in a software project often stimulate the production of management countermeasures. Too often, these responses cause even worse, and sometimes fatal, reactions in the project. The allergens in software projects are almost always buggy code and missed deadlines. In effect, they have lower than acceptable quality and productivity. These symptoms may in turn have been caused or aggravated by poor planning. They stimulate typical responses including more elaborate planning and a tightening of restrictions on the developer's ability to improvise. When this becomes a cyclical pattern, projects suffer. Either they become too congested to even breathe, or at the very least they suffer diminished ability to perform at peak capacity. Software development is uniquely challenging, but it is not rocket science.2 By that I mean that what we do is not that complicated, or at least it should not be. We analyze requirements and translate them into a relatively simple coding language. We're not putting a space shuttle into orbit or producing a blockbuster motion picture. So how can we have so much difficulty? How can we fall prey to the same few common modes of failure over and over again in our industry? In the human body, the different systems interact intimately. Similarly, planning, development, management, and the client together form the systems of a software project. These systems interact so intimately that any cure must be holistic in nature, considering the response of all systems to stresses or remedies. Each of these areas has a very different perspective on the typical recurring problems we face in software projects.

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 3 of 15

Management Perspective
From its perspective, management may sometimes recognize that the marketing department overcommitted deliverables. More often, it simply sees that the developers failed to deliver a satisfactory product. Its dissatisfaction may be based upon profitability, budget, timing, functionality, maintainability, usability, or any number of metrics. When this happens, management has limited courses of action available. It can replace or retrain developers if it suspects that the developers lack competence. It can insist upon more planning if it assesses that the planning was lacking. And it can tighten up management controls to ensure that developers follow the plan more closely. Most often, management applies a combination of these remedies in shotgun fashion, hoping to eliminate any problems next time around. Figure 2-133 Industry insiders are usually the last to recognize and correct their systemic problems.

Planners' Perspective
From the planners' perspective, we see a slight variation. When projects run into trouble, planners also see a fairly limited number of causes. Planners often feel that marketing doomed planning efforts right from the start by prematurely promising features, approaches, and budget. They feel that the planning process was rushed, inadequately budgeted, and impatiently supported by management. Planners feel that the development group did not fully utilize their planning documentation and that management allowed them to deviate too far from the planned specification. The project planner's remedy for these problems almost always involves more planning next time with greater management control over development.

Developers' Perspective
Developers feel they were handed unrealistic deadlines, constraints, and requirements. They feel they were given a poor project plan that did not meet their needs. They feel neglected and ignored by management when they ask for support to augment or adjust the plan, and then they feel hammered upon when deadlines are missed. Often they feel that management undermined development efforts by understaffing or moving critical staff in midproject. Developers typically want software planning, but they want better, not necessarily more, software planning. They want management, but they seek more responsive and supportive management, not more restrictive, counterproductive management.

Clients' Perspective
From the clients' viewpoint, they just want the software done well, and of course completed as quickly and inexpensively as possible. Poor planning is the first thing they blame if there are problems in the project. They usually feel that the planners did not understand their needs thoroughly enough and did not document adequately. They often complain that although they got what they specified, they did not get what was needed. Sounds kind of like the interaction between parents and their kids, doesn't it? The parents just want their kids to get an "A" in school. The kids may want that also, but when the kids get only a "C," the parents blame themselves for not taking a more active role. They grimly tighten up and create more restrictions for their kids and impose penalties if the kids deviate. They try to gather more measurable data by requesting more frequent grade reports from the teacher. When increased parental management conflicts with other demands upon them, the kids divert energy into working around those constraints and end up getting a "D" next time. Both sides have taken what they consider to be reasonable action at each step, but the interaction still degenerates into an increasingly counterproductive and hostile relationship. This same sort of situation occurs over and over again in software development. As a consultant, I was sometimes called upon to save failed projects. Management invariably insisted that the reasons for failure lie within planning and development. If they blame themselves, it is because they didn't manage enough. They vow to be

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 4 of 15

more diligent and proactive about increased project management next time. The developers want both more and less management at the same time. What they really mean, even if they haven't realized it, is that they want more supportive management. When you talk to the planners, they lament that they didn't have time to do more software planning and puzzle over why the developers have such problems after all the "good planning" they did do. When I came into these situations, it was usually only after management had already tried replacing developers several times and had tightened up management practices drastically. It still wasn't working, and probably had gotten worse. In some of these cases, management had become so desperate that it was willing to give me a free hand. In those situations, I was able to succeed dramatically. In other cases, management insisted that I fix the project while adhering to all of the management controls and requirements it had put in place. I declined those projects. I knew there was no way I could succeed where other fine developers had failed if I was locked into the same underlying causes of failure. Accordingly, for the software planning techniques in this book to succeed, management may need to "loosen up." That does not necessarily mean managing less, but managing differently, managing smarter. This book is called Planning Smarter, not "Managing Smarter," so it is not specifically about project management. However, smarter planning cannot succeed without smarter management, so it will touch on some management issues and practices. "Poor management can increase software costs more rapidly than any other factor." (Robertson, 1999)4 Planning smarter can indirectly encourage smarter management. When managers and planners cooperate well, successes are transmitted and reinforced throughout the system. Is the developer the blameless victim in all this? Certainly not! There are times when developers fail to deliver. Yes, there are hack programmers out there. Between teaching, hiring, and consulting, I have seen some amazingly bad ones. The bad practices that programmers can adopt are truly astounding. However, better planning and tighter project management generally do not even begin to fix coding problems. The only things that can mitigate these problems are training and talented, experienced developers with good programming standards and coding reviews. These are the types of problems and solutions that my first book addressed (Gill, 2000). Quite often, however, the developers are not the problem, or at least not the only problem. No developer, however talented and conscientious, can develop a project efficiently from a bad plan. The rule of "garbage in, garbage out" is universal. Sometimes, through heroic efforts, developers manage to succeed despite working from a nearly useless project plan. However, in many cases a bad plan all but guarantees development problems. A mediocre programmer may be able to succeed if given a good plan, but he has no chance at all working from a bad plan. "Attracting the best people is vital, but it is also essential to support them with an effectively managed software process." (Humphrey, 1989)5 Certainly, a bad programmer could mess up the best-laid plan, but it is not common to have a great plan and a poor developer. More frequently, you have the worst case of bad planning followed by bad implementation. In this case, the planners usually don't see the true nature of their complicity. Since the programmer obviously failed to achieve his or her goals, management fails to appreciate that the plan might have made success quite difficult. Managers conclude instead that they simply failed to exert enough control over the developers. They resolve to spend more time and money next time to maintain greater oversight and constraints upon the developers, limiting the ability of good developers to succeed despite the bad plan.

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 5 of 15

"A lot of software projects fail, but we software developers are not such dummies that our sheer incompetence can account for them all." (DeMarco, 1998)6 It is worth emphasizing again that the one thing that management, planning, and development all normally agree on is that more planning is better. More planning is the solution to all problems. It is the panacea of the software industry. When projects run into trouble, management almost invariably concludes that more planning is needed and tighter controls must be put in place to adhere to it. Managers reason that if they had invested more in planning, the good developers could be more efficient and the less experienced developers would encounter fewer problems. The flaw in this is that it assumes that planning is always good. This mistaken assumption leads to the erroneous conclusion that more planning must be better. The reality is that planning is often quite bad. It can be as appallingly bad as the worst coding samples. When you try to fix the problems caused by bad planning with even more bad planning, the problems compound. When a project has been poorly planned, it is usually no secret. Everyone, at least all the developers, knows it very early in the project and becomes quickly resigned to yet another Death March even before the ink has dried on the functional specification. "A Death March is a project that is doomed from the start. Everyone knows it and yet everyone just resigns to march along dutifully to certain doom." (Yourdon, 1999)7 It can quickly reach the point at which no planning may be better than all that bad planning. No matter how skilled the developers may be, the quality of the plan they receive is critical to their success. Typical software planning often fails to provide the developers with what they need to meet expectations. That may seem like a harsh assessment. After all, the industry has almost universally agreed upon the merits of current planning tools and techniques. However, it is also commonly accepted that the majority of software projects fail to meet their goals. To explain this contradiction, most software professionals would conclude that their peers simply fail to apply well-known and proven planning principles. I am not willing to accept that explanation. My follow-up question is, why do they fail to apply them? Walk into any software shop and you hear almost everyone there talk about project planning. Everyone appreciates the importance of doing it effectively. Many attempt to apply standard planning methodologies to their projects. They usually fail. Some talk about it incessantly and never do it. Some read all the literature and continually bemoan the fact that their shop doesn't implement this or that practice. Some produce incredibly complex planning documents that don't seem to provide much benefit. In my mind, if it isn't getting done well, you must simplify the process rather than make it larger and more complicated. Threatening, educating, and licensing people into compliance isn't going to have much real benefit if the practices are simply too complex or cumbersome to apply effectively. To eliminate the pathology of poor planning, we must simplify project requirements to the essential elements required to specify the desired software. Then we must put processes in place to obtain that information in the most efficient and direct way possible. The first example of a wellknown document that is typically overcomplicated and overinflated is the famous Vision Document. 2.2 The Vision Document A Vision Document is the starting point for most software projects. It is the primary deliverable produced in the Envisioning Phase8 and is therefore the first document produced in the planning process. The main purpose of this document is to move the project forward into detailed project planning and ultimately into development. The Vision Document is designed to make sure that key decision makers on both

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 6 of 15

sides have a clear, shared vision of the objectives and scope of the project. It identifies alternatives and risks associated with the project. Finally, it presents a budget for the detailed planning phase for the stakeholders to approve. A substantial amount of work is normally invested in the Vision Document. It typically takes several weeks or more to do the interviewing and high-level analysis, to agree upon the content, and to prepare the document. This is an essential first step. Certainly Vision Documents do have value at the time they are produced. They move the process along into subsequent phases. But do Vision Documents have value after that? We always like to think that the Vision Document will have enduring value after it is approved. We speak of it as a living document, with the expectation that it has legitimately useful content after approval and is worth maintaining over the life of the project. This is seldom the case. Despite the large amount of time and effort that is often put into these documents, they are rarely ever even looked at by the project team after they are approved. This is frustrating for both the vision authors and the client because they feel they have put a lot of effort into something with very limited return. While it may take a lot of effort to produce the Vision Document, it often presents only a very general outline of the project boundaries. At best, the Vision Document allows the detailed planners to get a vague idea of what they are going to work on before they interview the client experts. At worst, it has wasted a lot of unnecessary effort and creates the illusion that the detailed planners ought to be much farther along than they are when they join the project. This happens mainly because there is no overall planning plan and no customer orientation. The needs of the detailed planners were probably not considered in producing the Vision Document. Vision Documents that are not developed as an integral part of the larger software planning process are not likely to provide a helpful foundation for subsequent planning efforts. The Vision Document is designed to satisfy the expectations of internal and external decision makers more than the needs of the planners who will inherit it. The authors probably never asked the planners, "What would help you most during the envisioning phase and how would you like it presented?" Then they are surprised when the planners find little value in a document that was prepared without consulting them or truly understanding their needs. The fact is that the Vision Document does serve different audiences with very different needs. When managers read a Vision Document, they see the culmination of a lot of time and effort that helps them reach a decision regarding approval of the project. When planners and developers read a Vision Document, what they mostly see are boilerplate, broad generalizations, irrelevant detail, or premature overspecifications that must be corrected. They see very little information in the Vision Document that will benefit them and save them effort. Certainly the Vision Document is necessary to obtain high-level approval to proceed with design, but once the approval is obtained, what can and should the document contain that is of use to the designers? What material can it feed into the design phase to improve the efficiency of the development process? If the answer is that the Vision Document is intended to be only a throw-away document which moves the project along into the detailed phases, then the main thrust should be to improve process efficiency by eliminating all information that isn't essential to the goal of project approval. If the goal is to also improve overall process efficiency by laying the groundwork for subsequent phases, then the emphasis should focus on meeting the very different needs of the project planners. That can happen only by listening to the planners and by putting a software planning process in place that spans these project phases. To improve process efficiency in either case, what not to include may be a more critical decision than what to include. Only by understanding exactly what information our audience requires, and by limiting our effort appropriately, can we optimize the process. And only that way can we produce a Vision Document that retains its value past project approval.

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 7 of 15

Later in Chapter 7, we discuss the Vision Document further, identifying what not to include as well as what should be included. It provides some insights on how to seamlessly integrate your envisioning phase into the overall project planning process so that your Vision Document can become a basis of subsequent efforts. 2.3 The Functional Specification After the Vision Document is approved, effort typically shifts to production of the Functional Specification. Some companies create separate marketing and technical requirements documents, but however it is packaged we will call it a Functional Specification here to avoid getting lost in terminology. FlashbackVB Database Class I have taught a class on Visual Basic database programming for many years. As part of that class, my students complete a personal class project. Their first assignment is to produce a simple one-page Vision Document/Functional Specification of their project. I tell them to simply explain in plain English what their planned programs will do. You'd think it would be easy points. Despite the apparent simplicity of the assignment, I get a tremendous variety in content, most of which is not at all useful. Their planning documents are littered with sparse, overly general, unnecessarily detailed, or meaningless boilerplate. Although I expressly tell them not to dwell on implementation aspects such as database structure because that is documented in the next lesson, most do so anyway. Most of the students do not communicate effectively and do not seem to consider the needs of the reader. At one point I became so frustrated that I considered discontinuing this assignment. Then I decided that the fact that students have so many problems with it is the best reason to continue to assign it. Usually out of a class of 24, there will be two or three reasonably clear specifications that actually do give the reader some useful insight into what their project will do. I typically read one or two of those to illustrate an effective specification and point out what the student did right. The Functional Specification is the cornerstone of software planning. It embodies all of the business requirements, software requirements, and product specifications. The intent is to provide a complete, self-contained master plan of what will be produced. Normally, it is hoped that by the time this document is complete, all questions have been answered and the solution has been fully designed on paper. With the Functional Specification in hand, many managers and planners feel that they should be able to hand it to any development team and lock them in isolation until the system is ready for deployment. We often spend as much time preparing this document as we do developing the application it represents. This entails a tremendous expenditure of time, effort, and resources. The amazing thing is that in the end, no working code is actually produced. That is not to say it has no value, but it will not process one transaction, send one byte of data to a hardware controller, or accept one character of user input. Certainly one would argue that the Functional Specification pays for itself ten times over by preventing errors and inefficiency during development. This is the basic assumption implicit in most planning literature. That may be true when the Functional Specification is appropriate for the project it represents. However, all of the reported benefits of a Functional Specification do not normally consider its quality. They do not consider the cost and risk of a poor Functional Specification. The reality is that many Functional Specifications are quite poor and fail to produce a fraction of their expected benefits. In practice, we are fortunate when developers can find anything of value in a Functional Specification. Often it is highly debatable as to whether these documents are worth the effort that was put into them. At that point in their studies, my students should not be expected to know how to write an effective planning document. However, the same degree of variation in content and quality exists in most commercial Functional Specifications. These documents come from professionals who should be skilled in this work. Even though

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 8 of 15

the real-world specifications get bigger and take longer to produce, they don't necessarily get better. If anything, the authors of commercial specifications have even less ability to produce a useful specification than my students, relative to the scope of their tasks. Most commercial-quality specifications suffer from the same fundamental problems as the student specifications, except on a larger scale. They are verbose and complex, they lack cohesiveness, and they include a great deal of extraneous information. If anything, the problems are magnified because of the greater level of scope, detail, and interdependency required in a commercial application. Consequently, as with the Vision Document, the Functional Specification does not get utilized as effectively as the planners might imagine or hope. In far too many projects, the developers barely look at the specification. When they do consult it, they typically absorb only the few discrete sections that are useful and then ignore the rest. Usually the only parts of the specification that have much value are the screen mock-ups or prototypes. Even these mock-ups are often created without considering the needs of the developers. The rest of the information is frequently too sparse, disorganized, inconsistent, and incomplete to be of much value. Developers typically take what they can from the specification, and then go back to the client experts for additional information. From the developer's perspective, this is far more efficient and accurate then trying to glean more information from the specification. From the management and client perspectives, this is frustrating and redundant. Even worse, it opens the possibility of scope creep. The term scope creep describes the phenomenon in which features are added or changed after the specification is finalized so that they increase the time required to deliver the product. Planners probably spent many months working on the Functional Specification in an effort to anticipate and provide the developers with all the information they need to produce the product. They are understandably frustrated and bewildered when the developers never seem to use it effectively. Management concludes that greater control is needed to force those developers to adhere more rigidly to the specification. As with the Vision Document, the reason that these documents are so out of touch with the needs of development is that the planners probably never actually asked their developers in advance what they need and how to present it. Developers were handed a document that did not seem to be prepared specifically for them. And in truth it probably was not. It was probably prepared first and foremost to satisfy the very different expectations of management and client. What we end up with is a highly inefficient planning process that devours resources and doesn't meet the needs of the developers in the end. The best way to produce an effective specification that serves developers is to go direct to blueprint9 and use the Software Blueprinter application10 or something like it to produce a developeroriented specification as the planning process progresses. 2.4 Recognizing a Bad Plan We have been discussing the reality that the cornerstone documents of project planningthe Vision Document and Functional Specificationoften fail to produce the benefits that one might hope to gain, relative to the cost incurred. By recognizing the typical planning problems that plague projects, you can better redirect those projects and more readily avoid those mistakes in your own planning. And there is no lack of bad plans out there to be found. It is easy to recognize bad plans by looking for some telltale indicators.

Underplanning
First, look for signs of underplanning. If the plan is sparse, it may mean that the plan suffers from a lack of good planning. But that is not necessarily so. The best plans should be the most concise. You can't confirm underplanning based on small size

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 9 of 15

alone. You must look for the absence of critical components such as a Data Dictionary11 and a blueprint.

Overplanning
Overplanning is more reliably linked to large size. If the plan is bulky relative to the scope of the project, then it may suffer from too much bad planning. Bad plans tend to make up for their lack of quality with sheer volume. The key concern should be quality, not size, and conciseness is a key metric of quality. Don't be fooled by size!

Lack of a Data Dictionary

Perhaps the most glaring sign of a bad plan is the lack of a Data Dictionary. Without an unambiguous vocabulary, the language of the plan is almost certainly imprecise and inconsistent. This ambiguity will inevitably lead to programming questions or errors due to imprecision in what was felt to be perfectly clear and detailed narratives.

Metadata
Another symptom of overplanning is a predominance of metadata.12 Metadata is information obtained during the analysis of the client business processes. It includes things like Use Cases, Test Cases, Sequence Diagrams, Process Flow Diagrams, narratives, and other analytical graphics. Metadata may be useful in understanding the business requirements, but not necessarily useful in communicating requirements to the development team. Planners that call it a day after producing metadata alone seldom deliver a useful blueprint. Unless someone sifts through metadata to find its bearing on software requirements, it remains a nebulous fog, which can obstruct the vision of the developer. Information in blueprint form should not need to be further analyzed by the developer in order to establish its relationship to other elements or to the functionality to be implemented.

Extraneous Detail
Another telltale sign of a bad plan is the presence of a lot of unrelated, unorganized details. These plans contain extraneous information simply collected and tossed in, assuming that someone, somewhere, sometime, may need it. It was probably acquired in an uncontrolled information dump13 from the client, never really understood or analyzed, but included mainly to protect the planner from being blamed for omissions.

Premature Planning
Yet another sign of a bad plan is a premature or presumptuous planning schedule that violates phase integrity.14 For example, the analyst should not necessarily be the one to specify things like database structures, object models, and architectures. The purpose of the analysis phase is to determine the business need. The planner should leave structural decisions to the software architect, who will produce an implementation plan based upon the technical requirements. If you intercept a plan prior to completion, it is very easy to see if it contains premature planning. If the plan is complete, it may be more difficult to determine whether plans were made before all the necessary data was acquired.

Appearance
Appearance can indicate poor planning. A shoddily presented plan may reflect a lack of attention to detail and quality. If there is an apparent lack of pride in the

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 10 of 15

presentation, look closely for signs that it also lacks cohesiveness and sufficient attention to detail in the content. On the other end of the spectrum, some of the worst plans are the most impressivelooking ones. They contain full-color diagrams to illustrate virtually every imaginable aspect of the project. These are the ones likely to suffer from overplanning. The planner was probably more concerned about showing off his wide array of presentation graphics then actually providing anything useful. Look for indicators of insight into development needs, not for attention grabbing frills.

Boilerplate
You can also skim the plan for boilerplate to judge quality. Boilerplate will jump out at you after you learn to recognize it. It takes the form of endless tables of cut-andcopied information, and is reflected in meaningless platitudes such as "the software shall be user friendly and easy to use..." This kind of material makes great filler and feels warm and fuzzy, but conveys no meaningful information. Would anyone ever specify software that "shall be confusing to the user and difficult to use?" This stuff is there because the plan is targeting client and internal management fluff, not substantial developer requirements.

Rigidity
Look for signs of rigidity. These plans typically suffer from a lot of detailed mock-up screens and statements such as "any deviation from the plan will require a change order submitted in triplicate to be submitted to the steering committee for approval at their monthly status review meeting after initial screening by all department managers and the client representative pending feasibility and impact analysis..." This plan is likely to produce exactly what it specifies but is unlikely to produce satisfactory results. Overly rigid management is the most common knee-jerk response to previous bad planning, but it typically exacerbates the problems it is intended to control.

Lack of Process
Another way to recognize a bad plan is to simply ask to see what formal process produced this plan and where the format of the blueprint is specified. If these things aren't available, then the specification was almost certainly produced by an ad hoc planning process. This should make you very skeptical and suspicious.

Paperpushing
Always ask what automated system was used to organize and ensure the integrity of the specification. If the team used paper and pencil, email, word processors, and even spreadsheets, chances are the specification is full of inconsistencies. I know because I have tried to organize and maintain an effective software blueprint using a word processor and a spreadsheet, and I failed miserably. It is extremely difficult to maintain a living, internally consistent software blueprint without an automated tool that integrates all aspects of data (not mere document) management. You would be skeptical of an accounting firm that managed all your financial records with a pencil and paper, would you not? How could one imagine that a software blueprint of any complexity could be managed without software designed specifically for that specialized purpose?

Developer Driven
Ask the developers if they had helped specify, or at least had signed off on, the structure and content of the blueprint. If they did not have input, chances are that the specification does not serve their needs.

Retrospect

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 11 of 15

Unfortunately, the best way to recognize a bad plan is after it is too late. A bad plan drowns the team members in an ocean of irrelevant or inconsistent detail. It ties their hands so that they must implement inefficient or inelegant user interfaces and architectures. It forces them to make guesses or reverse engineer to try to gain the precision it lacks. It leaves the development team frustrated and confused about why they can't seem to find answers to a hundred implementation questions despite the impressive and voluminous specifications they were provided.

The Litmus Test

Here is a practical test. Take a small, well-defined section from your specification and hand it to a developer. Ask him to review it and then ask if he has any questions or issues that require clarification. Tell him that this is his last chance to ask questions before he is put to work on it. Does the developer question anything that you thought was perfectly clear from the document? Did he ask for definition of terms or clarification of logic? Did he feel confident that the document provided all the information needed to proceed? Did he have any concerns about the specified strategy for implementing the specification? Certainly, since we don't expect any plan to be perfect, there will be questions. But did these questions expose any issues that you thought were clearly stated or had been fully specified already in the document? Are the questions expected drill-down, or are they questions that you thought should have been clearly answered in the document? In many cases, when developers are given the chance to provide feedback, the plan is exposed as terribly lacking. If this is the case, then perhaps your planning effort should be improved. And let's be very clear: Improving may not mean more planning. It may mean less planning but smarter planning. After you finish this book, you will hopefully have a more critical eye when you review a project specification. Telltale signs of bad planning will jump out at you. Practice this skill. Whether you are a manager, an analyst, a planner, a developer, or a client, the ability to critically appraise specifications for true substance will be invaluable. When you recognize a bad plan, tell the authors specifically what they need to do to make it more useful. They will hopefully be grateful for specific, constructive suggestions. 2.5 Excellence Is Attainable After the grim portrait of software performance that has been painted so far, you might be tempted to resign yourself to the status quo. You might rationalize that although you have problems, so does everyone else. You might conclude that you can't do any better. You might further conclude that the quality and productivity of the software industry are as good as they can get and it is the expectations that are out of line. "The work of software development is largely incompressible." (DeMarco, 1995)15 However, there are projects that do succeed significantly better than others. They prove that you can do substantially better than the norm. Perfection may never be attainable, but excellence certainly is. "Large information systems don't have to take so long, they don't have to cost so much, and they don't have to fail." (Highsmith, 2000)
16

In After the Gold Rush, Steve McConnell (McConnell, 1999) reports that there is as much as a 600-to-1 ratio of performance within the industry. That is an astounding statistic. Some teams are doing extremely well (and we don't know how much better even they could be doing) while others are performing quite poorly. Even the

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 12 of 15

average level of performance is quite poor relative to the best performers. This wide gap in performance may be taken as grounds for pessimism, but it also offers hope and encouragement. It proves that there is tremendous room for improvement and that those improvements are achievable. Does this mean that there is a 600-to-1 difference in ability and talent among the individuals involved in these projects? Not likely. Even more unlikely is the fact that there would have to be a 600-to-1 difference in the average talent of the various teams, making the actual individual range of individual differences far greater. Nor are the successes merely a matter of luck. Yes, you can get stuck with a nightmare project with little hope of success. And there really are those rare dream projects that we all fantasize about. But no matter how the cards are dealt, successes are still made. There are companies that can reproduce success. They can win almost every hand. They overcome undesirable factors and capitalize on the favorable ones. These companies are not the Wizard of Oz. There are real programmers behind that curtain, like you, Tin Man, and you Scarecrow, and even you, Cowardly Lion. But they do have something you probably don't havean effective planning process. If you don't yet have your planning process down to a finely honed craft, you may want to consider the value of a structured software blueprint as the critical deliverable of your planning process. There are many other important components also. Configuration management, testing, code reviews, and coding standards are just a few. But the blueprint is the most critical and the most commonly overlooked. 2.6 A Software Blueprint The goal of any software planning effort should be to analyze the business requirements and translate that understanding into a specification that facilitates the development process as efficiently as possible. If you don't know where you're going, it is difficult to get there and impossible to know when you have arrived. Most planning efforts proceed by fanning out and mapping everything as they go, hoping that the end result will include a route to the ultimate destination. They craft their planning deliverables in ad hoc fashion as they go. The functional specification ends up becoming an apparently random maze designed to bewilder and befuddle the developers. This problem is inevitable when you don't have a clear planning goal defined from the onset. The software blueprint defines a tangible goal for the planning process in the same way that a Form 1040 defines a tangible goal for the tax preparation process. It should provide developers with exactly the type of information they need to construct a program, in the same way that a house blueprint provides the construction crew with exactly the information they need to build a house. While analogies and comparisons are never perfect, they do help make ideas more familiar and meaningful. House building is probably the most commonly cited analogy for software development. There certainly are a lot of similarities, but there are a lot of differences as well. Software development is a very customized process. However, house building can also be highly customized. The elements that make up a house are fairly simple. Floors, walls, doors, and windows make up every house. Likewise, there is a fairly simple set of pieces that make up every software project. If we can arrange these pieces into a standard blueprint that organizes them, then that blueprint should be all we need to describe most software projects effectively for the builder's purposes. A software blueprint is a single, simple, and standardized format to specify software requirements. Builders couldn't be expected to construct a new house without a blueprint, yet software developers are routinely asked to build software without an equivalent blueprint to work from. Most software specifications do not even begin to serve as an adequate blueprint. They are designed to serve many purposes, but not specifically to meet the needs of the developers. They do not provide developers with the exact information they

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 13 of 15

require, at the necessary level of detail and consistency, and in an easily accessible format. If software were a house, here is how the usual planning and construction process might proceed: 1. The salesperson initiates contact with the buyer and tells the project manager what the buyers are prepared to spend. The project manager has several meetings with the buyers. He gets a general idea of what they are looking for and prepares a Vision Document documenting the problems with their current house and specifying what size house the clients need. The Vision Document cites the projected cost (coincidently just a bit more than the buyers said they were willing to spend) and describes how much they will gain compared to their current home. The homeowners think that sounds pretty good, so they meet with the house designer every day for the better part of a year discussing what kind of colors they like, what kind of dimmer switches annoy them, and the hours of the day when they use the bathroom and kitchen. The house designer collects all the information, including tax returns, magazine pictures from Better Homes and Gardens, and poetry composed by the homeowners to describe their vision of the new house. The house designer prepares some usage scenarios which summarize the times that the homeowners are likely to use the garage door opener as well as some transition diagrams illustrating in detail their expected traffic patterns in the new house. He bundles these up with the magazine pictures provided by the prospective owners, and hands all this to the construction supervisor. The construction supervisor, already told what the house will consist of, what the buyers will pay, and when they need it completed, goes about the task of fabricating a construction plan and cost breakdown which reflect the timeline and total cost already predefined. The detailed estimate, tailored to meet the predetermined cost, reassures the steering team that their initial estimates were right on. The construction supervisor goes about gathering a construction team whose members laugh or cry, according to their particular nature, when they are told what the house will include, what it will cost, and when they are expected to complete construction. The construction supervisor hands off the house plan to the construction crew. In due course, the crew sets about to somehow build a "bedroom with two doors" and a kitchen that "radiates warmth" as they understand it from the documents they were given. The management team, distracted with planning future houses, doesn't pay much attention to the builders for a few months. The homeowners are too busy at their current jobs to offer much information. After all, they already took too much time off for all that earlier planning. The builders aren't sure where those two doors in the bedroom should be. They ask the planner and he tells them that the information is right there in the owner's childhood diary. The builders give up and put one into the hall and one into the bath. Finally when the house is nearly complete, the homeowners make a walkthrough. They list about 179 first pass changes, citing the fact that it clearly says in the wife's diary that she always wanted a door to the patio from her bedroom. Oh, by the way, where IS the patio that was inferred by that passage? The construction supervisor is sacked for not including the patio, and the

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 14 of 15

crew, already far over budget and months late, works far into the night to make the changes. That is pretty much the typical scenario in the software business. Needless to say, no home construction company could survive very long if it operated this way. In reality, the life expectancy for software firms is not high. So how can this scenario occur so frequently in the software industry? In fairness, creating software is not as straightforward and clean-cut as building a house. There are many more variables and far fewer standard materials and methods. Yet, there are many basic lessons from home construction that seem to escape the notice of software builders. First, you can't promise the customers exactly what they want for exactly what they would like to pay. Sure that may be a strategy for getting a sale, but not necessarily a formula for running a successful business. Second, you can estimate the cost of a house pretty well just by knowing the square feet enclosed. Based on only that one metric, you can tell the owners within a narrow range what their house will cost, only adjusted for any high-quality extras they might want. There is no such square footage metric for software, so the estimation process is infinitely more difficult. Third, you can't build a house based on traffic flow diagrams and room modeling charts that describe the house indirectly at best, with required details buried inconsistently among all the superfluous information. The builders of houses require a clear blueprint designed specifically to meet their needs. Yet in software development, more complex in many ways, we set about our jobs without an analogous blueprint. Yet the level of complexity is really quite comparable. Software blueprints can be far more simple, standardized, clear, and efficient than the documents we normally produce in their place. What elements would a comparable software blueprint contain? A Data Dictionary to establish a clear, unambiguous vocabulary Mockups or prototypes to lay out the floor plan of the screens Pseudocode to unambiguously define operational logic Precise definitions of data elements so that the forms and databases can be constructed Logic to clearly define the rules for data translations Narratives to describe relevant background With such a blueprint in hand, developers normally have all the information they need to produce software quickly and accurately the first time, with minimal input from the business experts. Builders of a house don't need to understand a lot about the day-to-day activities of the future owners. Likewise, software developers don't necessarily need to understand a lot about the client's business. The reason that many experts feel that greater understanding by developers is a requirement is because the developers usually do their own research or use their own judgment to compensate for incomplete project plans. Obviously, you can't take disorganized output generated during the planning process and call that a blueprint. A blueprint has very specific characteristics. It distills other information and presents it in a logically complete and consistent format for use by the software builders. Blueprint format has to be the expressed goal of all your efforts right from day one. The specific format of that blueprint must be clearly known from the start of the process, and all efforts should be dedicated to getting to that blueprint as efficiently as possible. You can't start with all the things you have been generating in the past

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004

Articles

Page 15 of 15

and create a blueprint from it. Also, you can't get to a blueprint by accident. Nor can you create a good blueprint as an afterthought. To produce a good software blueprint, you must engineer a planning process that begins to develop that blueprint from day one. The curious thing is that if you do so, that process can be considerably more simple and efficient than your current strategies that achieve far less useful results.

2004 Pearson Education, Inc. InformIT. All rights reserved. 800 East 96th Street Indianapolis, Indiana 46240

http://www.informit.com/articles/printerfriendly.asp?p=26397

12/10/2004