Wednesday, April 13, 2011

Mythical Man Month Ch 16-17

Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition
By: Frederick P. Brooks Jr.
Publisher: Addison-Wesley Professional
Pub. Date: August 02, 1995



Chapter 16 : No Silver Bullet - Essence and Accident in Software Engineering

In a supplementary chapter that was originally a separate paper titled, "No Silver Bullet," Brooks addresses why he thinks that there will be no single advancement in the field of Software Engineering will provide an order of magnitude improvement in itself. Relating this to his titular example of the silver bullet that kills the werewolf, the werewolf is "magnitude improvement in productivity" and the magic silver bullet is the method used to attain it.

He brings this subject up with the purpose of dispelling myth and vain hope with reality that can lead to real progress in making software development easier and more productive.

Brooks claims that the difficulty in writing/building/growing software can be separated into two distinct categories:
- Essence : difficulties inherent in the nature of software
- Accident : difficulties that pertain to the production of software but are not inherent

Brooks states that, "I believe the hard part of building software to be specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation."

From there cooper lists and describes what comprises the essential difficulties of software engineering:

- Complexity : Programming is the most complex activity done on the most complex of human artifacts. Furthermore, programs aren't made by many repeating parts but are composed of many different elements so increasing the size of the program exponentially increases the complexity. Also, abstracting away the complexity in design destroys the essence of the program.

- Conformity : Software must often conform to other interfaces and cannot be simplified by design. Furthermore, the basic elements of the programming don't necessarily have any unifying principles like in other fields.

- Changeability : Software must conform to the pressures of businesses, clients, laws, and the perception that it is the easiest thing to conform.

- Invisibility : Software is neither tangible nor can it be easily visualized in a way that would lend itself to solid and consistent design or conceptualization.

Next, Brooks addresses some the advances in the field that reduce the accidental difficulties:

- High Level Languages : Higher levels of abstraction decreased the cognitive load and made software design easier and more productive by allowing the programmer to work with conceptual constructs (part of the essence) rather than the accidental difficulties that allow the program to conform to the machine.

- Time-Sharing : By having more access to the computer and the program, programmers could maintain their grasp of the complexities of the system and spend less time on re-familiarizing themselves.

- Unified Programming Environments : These allow easy communication and interfacing of multiple programs on a single machine.

Finally, Brooks comes to the various technologies and methodologies that purport to be the silver bullets:

- Ada and other advances in high-level languages: advocates of Ada hope to advance the language to provide a general-purpose high level language that emphasizes modularity, abstract data types, and hierarchical structuring. Brooks debunks Ada as a silver bullet in that with the introduction of higher level languages, most of the improvement to be had has already been exploited. However, he does concede that the training Ada would give to programmers in using high level languages could have an impact.

- Object Oriented Programming : OOP advocates the use of abstract and hierarchical types, called classes, and that each of those should be given a name and a valid set of values and valid methods that can be called on them. Brooks holds the most hope for this bullet and says that these remove a "higher order sort of accidental difficulty," but nonetheless they remain accidental difficulties.

- AI : AI acts a rather broad silver bullet that tends to slip into two definitions: 1) Using computer intelligence to solve what was previously done by human intelligence.
2) Use of expert knowledge to devise a program to solve a problem like they would.

Brooks notes that once a problem in AI is understood, the particular AI problem slips into definition 2, namely, expert systems.

- Expert Systems : Assuming that an expert system could be devised from a brilliant expert in software engineering that could explain his recommendations in a way that could be put in a generalized knowledge base, it would still only address the accidental difficulties in areas like debugging but the essential difficulties would remain untouched.

- "Automatic Programming" : A pipe dream. Any automation of software engineering would require the target problem to have the following properties:
1) It could be characterized by a few well understood parameters
2) The problem has many known methods to solve it
3) The correct solution could be selected given the parameters
Unfortunately, these properties rarely hold.

- Graphical Programming : Brooks claims that an effective solution would require software to be easy to visualize as well as have enough workspace to adequately design with the entire scope in view, neither of which appear possible.

- Program Verification : Brooks dashes the hopes that large jumps in productivity could be attained by proving program to be correct in design before implementation by making the simple observation that program verification is time consuming and nigh impossible for sufficiently complex programs and is thus not viable.

- Environments and Tools : They can provide incremental improvements on accidental difficulties, but they miss the mark on essential difficulties.

- Workstations : increases in machine power don't reduce essential difficulties, although increased power is nice.

After all this, Brooks provides a few worthwhile solutions that attack the essence:
- Buy Versus Build : The beast way to increase productivity is to not build at all.

- Requirements refinement and rapid prototyping : The customer never knows what they really want so much effort must be expended to find what they actually want, this saves time in the end and reduces complexity, makes the program more visible to the client, and provides the right perception of how changeable the program actually is.

- Incremental Development : Brooks likens software engineering to growing a complex biological structure that evolves over time. By growing small parts that always work and adding complexity and functionality in continuously there is a reduction in complexity and programmers are happy to see success in a system that always works.

- Great Designers : Brooks makes a final plea to nurture great designers that can build great software and makes a few recommendations on how to find and provide incentives to these people.

Chapter 17 : "No Silver Bullet" Refired

In this chapter, Brooks addresses the criticisms to his No Silver Bullet paper just before the 10 year mark of his predictive limit expired.
First, he notes that, indeed, no silver bullet has yet to be found and claimed to be as such by a no-biased 'bona fide' cash-paying customer.

Brooks then clarifies his definition of accident as the part of development pertaining to the implementation process. Then he addresses the fact that NSB claims that the accidental part of the work is 9/10 of the total work in developing software. Because of this assumption, he claims that complete reduction of the accidental difficulties could not produce an order of magnitude increase in productivity. He concedes that measurement is needed though.

Brooks debunks the claims that solving the essential difficulties are hopeless by stating that "not all complexity is inevitable" and that much of his proposed solutions tried to avoid complexity, such as growing the program incrementally.

Brooks clarifies that the purpose of his paper wasn't to depress or give to all a spirit of gloom, but to provide a sense of realism so that real progress could be made.

Brooks then dispels arguments against the division of software difficulties into essence and accident and moves to support the movement for "The Vanilla Framework" that purports to increase the visual development of software.

Brooks concedes that quality is a good target that may give rise to productivity and that the perception customization has a strong impact on whether to "build or buy."

Brooks explains that the reason OOP has lingered is that despite the power of the concepts provided by modularity, encapsulation, and inheritance, realizing the power of these things incurs a high initial cost and late returns.

Brooks then expounds on the difficulties of reusability stating that in order for reuse to occur, many well defined libraries must be available, understood, generalizable, and cheap to use.

In the end, so silver bullet has yet been found.

Discussion :

These chapters cover a stunning array of topics as they relate to software engineering but Brooks does a capital job of cutting through to the heart of the difficulties and I think his classification is spot on.

Although it may seem "gloomy" that there's no "silver bullet" his arguments do a wonderful job exposing the real targets for steady improvement in the field. With that said there is still the lingering optimism that makes me wish for exponential leaps in progress that contradicts my sense of reason in this matter.

It is interesting to see how long people have been talking about the use of AI and OOP in the field as the panaceas to our frustrations and to see how far they've actually come in the time spent between chapters 16 and 17.

Brooks recommendation for growing software incrementally and using rapid prototyping sound suspiciously similar to many agile development methodologies like Extreme Programming and I can't help but wonder how many advocates of agile were inspired by Brooks' writing.

No comments:

Post a Comment