Good requirements do more than just define the “what” of the thing to be produced, they also define how it will be tested and verified to insure that it really is the desired end product. But most importantly, requirements are an explicit agreement between the producer of a product and the end consumer, or customer, of that product. Without requirements things can, and often do, devolve into a mess of misunderstandings, misconceptions, shoddy workmanship and unhappy people all around. This is not necessary and it is easily avoidable if both the software developers and the customers can reach agreement on what is to be produced, document that agreement with requirements, and then manage the process of changes and revisions in a way that doesn’t fundamentally derail the project.
Requirements and Software Project Failures
There is an often stated and occasionally debated statistic that between 30 and 50% of all software projects fail for one reason or another, with failure being defined as anything from the termination of a project to a product that is over-budget, late or lacking originally planned features. Irregardless of how one defines a failure, the fact that 30% or more of all software projects fail is rather dismal. Some of the reasons for these failures include poor planning, inadequate resources, lack of management support, inadequate testing and poor or lacking requirements. In 2002 the NIST published a report on the cost impact of inadequate software testing. While testing is an area of study unto itself, and not the focus of this article, it is worth noting that without requirements it is not possible to test something. In other words, if it is unknown what, precisely, something is supposed to do, then how can one test to see if it does it correctly, or at all? The report found that the costs of defective or failed software is on the order of tens of billions of dollars annually. Apparently this has not gone unnoticed in the software-intensive industries, as the report also noted that the amount of effort expended in the initial requirements phase of software development rose from around 10% in the 1960’s and 1970’s to 40% in the 1990’s. 
Poorly defined requirements are often cited as one of the root causes of software project failures.  Far too often the up-front requirements process is either ignored or truncated because management doesn’t perceive it as a productive activity (“The software people aren’t writing code, they’re just writing documents!”). This, coupled with the misperception that software is easy to change (how many times has one heard “no problem, it can be fixed in the software”?), can lead to hundreds, or even thousands, of wasted hours spent rewriting and retesting (inadequately) a software product that could have been done correctly at the outset at a fraction of the cost in a fraction of the time.
But all is not doom and gloom, and the situation is improving, albeit slowly. There has been a steady shift away from the so-called “waterfall” model of software development, in which all requirements are defined up-front as an initial step, towards what are referred to as “iterative” models, wherein the detailed requirements and the implemented code progress along in parallel towards an integrated whole under the guidance of an overall set of objectives in the form of high-level functional requirements.
The Producer-Consumer Relationship
In the world of software engineering and programming there are always producers and consumers, even if both are the same person. The producers, i.e. the software development personnel consisting of software engineers, developers, programmers and testers, are typically attempting to produce something that a consumer will want to use. The consumers, in this context, are also known as corporate management, sales, marketing, external entities such as government agencies or other companies, other software developers, or even youself. In this article I well refer to the consumers as customers, and I will refer to the output of the software development personnel as the product.
It is essential to recognize and establish this type of producer-customer relationship from the outset, because it is typically the customer who is footing the bill for whatever is being produced. The logic here is simple and inescapable: Without the approval, and subsequent funding, from customers there wouldn’t be any software development positions in the organization. Since it is obvious that a happy customer is more willing to pay for a product than a dissatisfied one, it follows that is in the best interests of the software developer to pay close attention to the needs of their customer.
What The Customer Wants
Now that we’ve defined the producer-consumer relationship in terms of customers, let us take a closer look at the customers. When a customer requests a product they typically want that product to have some specific set of functionality or features in order to meet some specific needs. In other words, they have requirements for the product they desire. This applies to any situation where something is sought to fill a perceived need, be it deciding on what to purchases at the grocery store (requirements: tasty, cheap, low calorie and low fat, for instance), shopping for clothes (requirements: good fit, 100% cotton, fashionable, suitable color, etc.), or ordering gravel for a driveway (requirements: each stone in a particular size range, a specific type of rock, a typical color and so on). In summary then, we can say that a requirement is a statement of how something is supposed to behave, or some set of characteristics it is supposed to have. A requirement is something that comes solely from the customer, not the producer (unless the producer happens to be clairvoyant, of course). If the customer can’t get something that meets their requirements then they may go elsewhere before they settle for something that is less than optimal if they have the choice, and that is how it should be. If they don’t have a choice they may have to settle for the non-optimal product, but the odds are that they won’t be too happy about it.
In order to produce something that a customer wants the developer needs to know what it is, specifically, that the customer desires. Too often software developers don’t have good requirements to follow, or worse, they do have them but they elect to ignore them. It has been my experience that these situations occur mainly for two reasons: (1) the customer doesn’t understand what a requirement is and why it is such a valuable piece of information, and (2) the software developer doesn’t understand why requirements are essential and why they need them. In those cases where both 1 and 2 hold true then it is little more than a recipe for confusion, wasted time, wasted resources and a lot of acrimonious finger pointing.
Allow me to present a couple of examples to illustrate the points I’ve raised:
I once interviewed a person for a job position who had an extensive amount of experience with requirements-driven software development, including commercial aviation and military standards. During the course of the interview he made the comment that, while he understood the value of requirements, he didn’t think that formal requirements were applicable to a commercial scenario. His reasoning was that the folks in marketing and corporate management didn’t understand the value of requirements and were also prone to changing their minds far too often to make them usable.
Well, I was rather surprised at this. What it told me, more than anything else, was that he himself didn’t understand the ramifications of not having a formal requirements process well enough to present a compelling case to the internal customers of the software his team was developing. I would expect someone with no experience with a formal requirements process to hold this view, but not from someone who had supposedly seen the benefits. What it also told me was that, at best, the management and marketing at his previous place of employment was rather inept and it was likely that they themselves had some disgruntled customers using poorly written and inadequately tested software. At worst it implied a situation were the company really didn’t give a flip about what the customer thought of their product so long as they paid for it.
In another instance there was a programmer working for me on a digital camera system. His job was to create a module that controlled some motors in the camera. After several months of effort it came time for the first formal review of our work, and a big part of that review was requirements compliance. Just one day before the review he wandered into my office and announced that he was amazed at the level of detail in our software requirements document that he’d never seen before. Needless to say I was rather stunned, as I had assumed from what I’d been hearing and from the results of our internal software reviews that he was following the requirements all along. I was wrong.
In reality he had apparently been making educated guesses as to what was required by skimming the diagrams in the software design description document, and in all fairness he managed to get most of it right in terms of functionality. Unfortunately he got enough of it wrong that it took some significant effort to get it all back on track, and all because he couldn’t be bothered to read the requirements document (his claim was that it was really boring and he didn’t think it applied to him). For me, what was really boring were the really long late-night hours I had to spend correcting his burst of “creativity” after he left the project shortly after his announcement about the requirements.
Over the years alternative forms of requirements have been proposed in an attempt to make them more readable for those who are not accustomed to seeing statements written in the stiff style of a “shall” requirement. But before taking a quick tour of the various ways one can capture a requirement, we should take a moment and nail down a solid definition of what a requirement actually is.
Requirements come in several forms depending on the level of detail being addressed. They may range from high-level functional requirements that define behavior in general terms (i.e. a list of features in a marketing brochure can be considered a very high level set of functional requirements) to detailed low-level implementation requirements that describe specific aspects of the software near the level of the actual software code.
Functional requirements may originate from marketing to define the features of a new product. In other situations the functional requirements may be derived from higher-level mission or science objectives. Functional requirements never specify how something is to be implemented, only what it is supposed to do from the customer or end-user’s perspective. One form of functional requirement that has gained popularity in recent years is the so-called “use case”, wherein the functional, or behavioral characteristics of software are captured in the form of an “if this, then that” scenario (sometimes involving an “actor”). Another way to put it is: “Use cases, stated simply, allow description of sequences of events that, taken together, lead to a system doing something useful.” A good functional requirement is also a verifiable requirement, and the functional requirements are the fundamental criteria for functional verification testing and customer acceptance of a product.
The software developers cannot be expected to write code for any but the simplest systems from the functional requirements alone; there just isn’t enough detail available to do the job correctly. The functional requirements provide the framework for successively more detailed levels of requirements. In requirements engineering parlance, low-level requirements are said to “flow” from higher-level requirements, and for each low-level requirements there is “traceability” back to a higher-level requirement.
A good (i.e. well-written) traditional detailed requirement is a statement that defines a specific characteristic that something will have or exhibit in clear, concise, consistent and verifiable terms. The wording is intentionally unambiguous and will include the word “shall” somewhere in the requirement. For example: “The software shall have the ability to periodically save its current execution state at user-specified time intervals.”
It is the detailed requirements that form the basis for the low-level testing that is used to detect and remove defects prior to the integration of the various software components into a complete product. Tests are used to verify that the low-level requirements have been implemented correctly, with at least one low-level test traceable back to each low-level requirement.
Embracing Requirements for Success
In order for requirements to be used effectively they must be accepted and embraced by both the producer and the customer. Or, to put it in business-speak, there must be buy-in from both management and the software developers on the producer side, and acceptance by the customer on the consumer side. A requirement is an essential part of the contract between the producer and the customer wherein the customer agrees to pay the producer for a specific product. Without this type of contract between producer and consumer there cannot be a workable business relationship, and make no mistake about it, being a producer of software in a commercial enterprise is a business relationship.
By the same token, being able to read, and in some cases write, the requirements that developers will work from is definitely in the best self-interests of the customer. For example, if marketing decides that a product needs a new feature, then they should at least be willing to work with a software engineer to define the functional requirements necessary to describe the feature. To expect the software folks to sit through a PowerPoint dog-and-pony show and then go off and create exactly what is in the head of a marketing person is completely unrealistic. It also indicates that the person advocating the new feature doesn’t really have a good grasp on what it’s really going to cost the company to implement that fancy new feature, and what the ramifications might be if that new feature doesn’t work out because of some little detail that would only become apparent when attempting to define the requirements for it.
Requirements Myths and Fallacies
During my career I’ve heard various reasons why requirements are something to be ignored or avoided, ranging from “they might do that for space missions, but we’re not flying our product to Pluto” to “we haven’t got the time or resources to do that sort of thing”. The possibly pithy responses below are based on years of experience, and as obviously wrong-headed as some of the responses might seem, one can still hear these myths stated today.
1. Requirements are hard to write.
This may be true for people who either can’t or won’t write things down, but in reality writing a good requirement is simply a matter of following the basic rules we’ve already seen earlier in this article. If someone is going to call themselves a software engineer, then they’d darn well better be able to do the basic software engineering task of writing a requirement. Otherwise they’re just a pretentious programmer with the wrong title.
2. Requirements make testing difficult and costly.
Untrue, as effective testing cannot be accomplished without good requirements to test against. What a set of good and sufficient requirements will do is result in more time spent testing, but the payoff for that is a product with fewer defects and happier customers. In a highly competitive market this can mean the difference between being in business with a loyal customer base, or being in bankruptcy court because of a consistently buggy product.
3. Detailed requirements stifle creativity.
This might apply to those “free spirits” who believe that software development is a creative artistic endeavor and not an exercise in applied engineering. But for most developers a good set of requirements actually makes their job easier by eliminating much of the guesswork and letting them focus on what needs to be done to meet the customer’s needs, instead of spending time pondering various possibilities and engaging in trial-and-error exercises that may, or may not, result in something like what the customer really wanted.
4. Requirements tie the hands of the developers.
As with the previous myth this really only applies to those who perhaps have authority issues and insist on doing things “their way”. Requirements, by intent, define the necessary characteristics of a product, and so long as the customer is the ultimate source of the requirements then following the requirements helps insure that the customer will get what they expect, not what the developer might have thought they wanted (or thought they needed, in some cases).
5. Requirements are a waste of time and money.
While it is true that creating requirements does take time (and therefore money), they are definitely not a waste of either. What is a waste of time and money is when developers miss the mark and create what they thought the customer wanted, only to have it rejected and then have to do it over again. In some cases this process repeats many, many times before the software is “good enough” or either the developer, or the customer, simply gives up. A good set of agreed-upon requirements at the outset of the effort helps to ensure that what the customer gets is what they were expecting, and if planned properly, when they expected to get it.
If you would like to learn more about software engineering and software requirements I would suggest the following books: Also, don’t forget to check out the texts referenced in this article.
Code Complete (2nd ed.), Steve McConnell, 2004, Microsoft Press
Software Requirements (2nd ed.), Alan M. Davis, 1993, Prentic Hall
Object-Oriented Software Engineering, Ivar Jacobson, 1992, Addison-Wesley
1 National Institute of Standard and Technology, The Economic Impacts of Inadequate Infrastructure for Software Testing, Planning Report 02-03, May 2002
2 Robert N. Charette, “Why Software Fails,” IEEE Spectrum Online, September 2005
3 Writing Effective Use Cases, Alistair Cockburn, 2001, Addison-Wesley
4 Use Case Modeling, Kurt Bittner and I. Spence, 2003, Addison-Wesley