[Brooks87] Brooks Jr., Frederick P., "No Silver Bullet: Essence and Accidents of Software Engineering," in Computer, Vol. 20, No. 4, April 1987, pp.10-19, also in Software Engineering, M. Dorfman and R. H. Thayer (eds.), IEEE Computer Society Press, 1997. pp. 13-22.

Frederick P. Brooks, Jr.

U. of North Carolina at Chapel Hill

(Author of The Mythical Man Month, 1975)

No Silver Bullet

Improvements in computer hardware dev.:

• six orders of magnitude performan-price gain in 1st 30 yrs.

• two fold gains every two years in price and in performance

• due to shift from assembly industry to process industry.

There is no "silver bullet" -- no single technology or management technique to make even one order of magnitude improvement in productivity, reliability, or decrease in complexity. Anomaly HW progress is so fast, and SW so slow.

The improvement can only be achieved by a disciplined, consistent effort to develop, propagate and exploit a set of techniques.

Analogy -- a carpenter is able to build efficiently only with a set of tools, not just a hammer.

[Brooks87]

Why is software development so difficult?

First look we'll look at the nature of SW systems then at the properties of the bullets proposed:

According to Aristotle, difficulties can be separately into

• essence -- difficulties inherent in nature of the entity

essence of SW is a construct of interlocking concepts:

data sets

relationships among data items

algorithms

invocation of functions

abstract conceptual construct is representation independent

• accidents -- difficulties are not inherent

representation dependent, e.g., syntax errors

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems. [Brooks87]

• inherent properties of essense of SW systems:

• complexity

• conformity

• changeability

• invisibility

[Brooks87]

Essence of Software Systems:

Complexity

(1 of 2)

• No two parts are alike, if they are similar we make them into a subroutine. In this respect software systems differ from computer hardware, buildings, or cars.

• The very large numbers of states in computers make them more complex that most things that are built. It's hard to conceive, describe, and test them.

• There are orders-of-magnitude more states in SW systems than computers. Difficult to enumerate, much less undestand, all the possible states of a program -- unreliability.

• SW scale-up results in nonlinear increases in complexity and size:

scaling-up software is not merely repetition of the same elements in larger sizes, it is necessarily an increase in the number of different elements. In most cases the entities interact with each other in some nonlinear fashion, and the complexity of the whole increases much more than linearly. [Brooks87]

[Brooks87]

Essence of Software Systems:

Complexity

(2 of 2)

Essential complexity --> technical problems:

Essential complexity of

communication among team-->flaws, cost overruns, delays

function-->difficulty invoking functions, hard to use programs

structure

-->hard to extend programs without introducing side effects

-->unvisualized states that constitute security trap doors

 

Essential complexity --> management problems:

--> hard to get an overview of the problem

--> impedes conceptual integrity

--> hard to find and control all the loose ends

--> creates tremendous learning and understanding burden

--> personnel turnover --> disaster

[Brooks87]

Essence of Software Systems:

Conformity

 

Einstein argued there must be simplified explanations of nature because God is not capricious or arbitrary. No such faith conforts the software engineer. [Brooks87]

 

Software must be produced by and run within an organization. Conformity to organizational politics/policy introduces complexity

Much of thee complexity is arbitrary complexity without rhyme or reason by the many human institutions and systems to which his interfaces must conform [Brooks87]

 

Much complexity comes from required conformation to other interfaces and can not be simplified out by any redesign of the software.

[Brooks87]

Essence of Software Systems:

Changeability

 

• Unlike software, manufactured things are infrequently changed after manufacture; they are superseded by later models, or essential changes are incorporated into later-serial-number copies of the same basic design. Call-backs of automobiles are really quite infrequent.

• Buildings do get changed, but the high cost of change, understood by all, serve to dampen the whims of the changers.

• All successful software gets changed.

• users who like the basic function and invent new uses for it at the edge or beyond the original domain

• successful software survives beyond the normal life of the machine for which it was written

• application, users, laws, and machines change continually forcing change on the software product

[Brooks87]

Essence of Software Systems:

Invisibility

 

• Software is invisible and unvisualizable

• Architects use floor plans --> spaces, traffic flows, views.

Mechanical engineers use scale drawings of parts.

Chemists use stick-figure models of molecules.

Geographers use maps to represent land.

Computer engineers use diagram schematics for silicon chips

• Software engineers have no good geometric representation.

Use several general directed graphs, which are not planar nor hierarchical, to visualize software structure:

• flow of control

• flow of data

• patterns of dependency

• time sequence

• name-space relationships

• We can't see the "big picture". It impedes the process of design within the mind of the designer and severely hinders communication among the designers, users, and managers.

• The fact we can't build a solid model of the software system gives the customer the illusion that the system should be flexible and easy to change.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of bullets proposed

High-level languages

• progressive use of high-level languages

brings about increases in

• productivity (by at least a factor of 5),

• reliability,

• simplicity, and

• comprehensibility

• abstract program = conceptual constructs:

• operations

• data types

• sequences

• communication

• concrete machine program deals with

• bits

• registers

• conditions

• branches

• channels

• disks, etc.

• using a high-level languages eliminates a whole level of accidental complexity that was never inherent to the program.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of bullets proposed

Time Sharing

• Time sharing vs. batch programming

• produced a major improvement in:

(but less than high-level language use)

• productivity

• quality of the product

• overcomes essential difficulties, preserves

• immediacy --> don't forget minutiae

• train of thought

• overview of complexity

• overcomes accidental difficulties, e.g.,

• faster turnaround

• beyond response time passes the human threshold of noticeability (~100ms), no benefits are expected

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of bullets proposed

Unified Programming Environments

• Unix and Interlisp where the 1st widely used

• Address accidental difficulties from using individual programs together as opposed to one integrated environment

• improved productivity by integral factor from use of

• integrated libraries

• unified file formats

• pipes

• filters

• stimulated development of toolbenches and integrated tool environments

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Ada

and other high-level language advances

 

• Ada philosophy

• modularization

• abstract data types

• hierarchical structuring

• Ada's greatest contribution will be that switch-ing to it occasioned training programmers in modern software-design techniques

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Object-oriented programming

 

• There are two separate, orthogonal ideas under the name of object-oriented programming:

• abstract data types

• an object's type should bedefined by

• a name,

• a set of proper values, and

• a set of proper operations

• NOT its storage structure

• the storage structure should be hidden

• hierarchical types

allow one to define general interfaces that can be further refined by providing subordinate types.

• Both allow higher-order expression of design and remove accidental difficulty

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Artificial Intelligence (AI)

• There are two AI definitions:

(1) The use of computers to solve problems that previously could only be solved by applying human intelligence.

(2) The use of a specific set of programming techniques known as heuristic or rule-based programming

[Parnas85]

• Advances in the first area are problem specific, in image recognition and speech recognition.

The hard thing about building software is deciding what one wants to say, not saying it. No facilitation of expression can give more than marginal gains.

[Brooks87]

• Advances in the second area are discussed in the next section under Expert Systems.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Expert Systems

(1 of 2)

• Definition of an Expert System:

A program that contains a generalized inference engine and a rule base, takes input data and assumptions, explores the inferences derivable from the rule base, yields conclusions and advice, and offers to explain its results by retraicing its reasoning for the user. The inference engines typically cna deal with fuzzy or probabilistic data and rules, in addition to purely deterministic logic. [Brooks87]

• Advantages of programmed algorithms:

• Inference-engine technology is developed in an application-independent way, and applied to many uses.

• The changeable parts of the application-peculiar materials are enclosed in a uniform fashion and tools are provided for developing, changing, testing and documenting the rule base. This regularizes much of the complexity of the application itself. [Brooks87]

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Expert Systems

(2 of 2)

• Most important advance offered is the separation of the application complexity from the program itself.

• Expert systems could suggest

• interface rules

• advise on testing strategies

• remember bug-type frequencies

• offer optimization hints

• The most powerful contribution will be to give access to the inexperienced programmer to the experience and accumulated wisdom of the best programmers

• Tool could disseminate best practice.

The gap between the best and the average software engineering practice is wider than in other disciplines.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

"Automatic" Programming

• Definition of Automatic Programming:

the generation of a program for solving a problem from a statement of the problem specifications.

Automatic programming has always been a euphemism for programming with a higher-level language than was presently available to the programmer. [Parnas85]

• Technique of building generators is very powerful, and routinely in used where

• problems are characterized by few parameters

• there are alternative, well-known methods or solutions

• extensive analysis has led to explicit rules for selecting

solution techniques, given problem parameters

• generators available: sorting and integrating diff. eq.

The system permits direct specification of the problem, and assess the parameters, choose from a library of methods of solution and generate the programs.

• Limited possibility for generalization.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Graphical or Visual Programming

(1 of 2)

• Graphical or Visual Programming:

the application of computer graphics to software design.

• Efforts mostly use flowcharts as the basis of program design. Brooks claims that:

• flowchart is a very poor abstraction of software structure.

• flowchart is useless as a design tool.

• programmers draw flowcharts after, not before, writing the programs they describe.

• A limitation is the current screens: too small, in pixels, to show both the scope and the resolution of any seriously detailed software diagram

desktop metaphor => airplane-seat metaphor

 

[See G. Raeder, "A Survey of Current Graphical Programming Techniques," Computer (special issue on visual programming), R. B. Graphton and T. Ichikawa (guest eds.), Vol. 18, No. 8, August 1985, pp. 11-25.]

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Graphical or Visual Programming

(2 of 2)

Problems:

• Hard to visualize SW along > 1 dimension:

Have several views of the program:

• control flow • variable-scope nesting

• data flow • hierarchical data structures

• time sequence • variable cross-references ...

• Can't superimpose all the diagrams

• It's difficult to extract a global view

• The VLSI success with visual programming does not extend to SW design because chip design is a layered 2-D description whose geometry reflects its realization in 3-space. A software system design is not.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

Program Verification

• Program verification tries to eliminate the errors at the source in the system-design phase

• Tries to enhance productivity and product reliability by proving designs correct before the effort is poured into implementing them and testing them.

• Verification is labor intensive, only a few substantial programs have ever been verified.

• Verification does NOT = error-proof.

Math proofs can contain errors, so can't eliminate testing

• Perfect program verification = program meets specification. The hardest part of the sw task is arriving at a complete and consistent specification, and much of the essence of building a program is in fact the debugging of the specification.

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Environment and Tools

 

• Biggest productivity advances

• hierarchical file systems

• uniform file formats to allow uniform program interfaces

• generalized tools

• Language-specific smart editors can only facilitate freedom from syntactic errors and simple semantic errors.

• Biggest gain is the use of integrated database systems to keep track of the myriad of details that must be recalled accurately by the individual programmer and kept current for a group of collaborators on a single system.

[Brooks87]

[Brooks87]

Accidental Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Workstations

 

• Composition and editing of programs and documents are fully supported by today's speeds.

• An increase in the power and memory capacity of the individual workstation will help compilation time but otherwise can not be expected to drastically enhance programming productivity.

[Brooks87]

Essential Difficulties of Software Systems:

Properties of potential

silver bullets proposed

Mass-Market Software

• Cheaper to buy than to build from scratch.

The most radical possible solution for constructing software is not to construct it at all. [Brook87]

• Software tools can be bought off-the-shelf.

• cheaper than hiring a programmer.

• documented and maintained better than home-grown SW

• in 1950's and 60's industry users would not use off-the-shelf packages, now widely used due to the hardware/ software cost ratio. In the 1960's hardware cost($2M), a program for $250K was not out of line in the budget.

• many users now operate their own computers and, although not able to write new programs, are adept at solving new problems with off-the-shelf applications.

• single most powerful sw-productivity for an organization is to equip workers with personal computers and good generalized office suite of programs and for laboratory scientists, generalized math and statistical packages.

• Marketplaces for individual software components have been proposed

(see Future: Component-Based Software p. 53 in these notes)

[Brooks87]

Essential Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Requirements Refinement and Rapid Prototyping

(1 of 2)

• Hardest problem:

The hardest single part of building a SW system is deciding precisely what to build. No other part of the conceptual framework is as difficult as establishing the detailed technical requirements, including all the interfaces to people to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.

[Brooks87]

• Most important function of SW builder:

iterative extraction and refinement of the product requirements from the client.

it is really impossible for a lient, even working with a sw engineer, to specify completely, precisely, and correctly the exact requirements of a modern sw product before trying some versions of the product. [Brooks87]

 

[Brooks87]

Essential Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Requirements Refinement and Rapid Prototyping

(2 of 2)

• One of the most promising work and one that attacks the essence, not the accidents of the sw problem, is the development of approaches and tools for rapid prototyping.

• Prototype software system:

one that simulates the important interfaces and performs the main function of the intended system, while not necessarily being bound by the same

• hardware speed,

• size, or

• cost constraints.

Prototypes typically perform the mainline tasks of the application, but make no attempt to

• handle the exceptional tasks,

• respond correctly to the invalid inputs, or

• abort cleanly.

Allow clients to see the conceptual structure and test for consistency and usability.

 

[Brooks87]

Essential Difficulties of Software Systems:

Properties of potential

silver bullets proposed

 

Great Designers

• Good design practices can be taught.

• Great designs come from great designers.

Very best designers produce structures that are

• faster, • smaller,

• simpler, • cleaner, and

• produced with less effort.

• The differences between the great and the average programmer approach an order of magnitude.

• Great designers and great managers are rare:

Grow your own great designers:

• systematically identify top designers as early as possible. Best are often not most experienced

• Assign a mentor, device & maintain career-dev plan on file, include apprenticeships with top designers, adv. formal education, short courses, solo design & lead assn.

• provide opportunities for growing designers to interact