Wednesday, April 13, 2011

Mythical Man Month Ch 18-19

Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition
By: Frederick P. Brooks Jr.
Publisher: Addison-Wesley Professional
Pub. Date: August 02, 1995

Chapter 18 : Propositions of the Mythical Man-Month: True or False?

In this chapter, Brooks briefly reviews the main points for each chapter and adds a few annotations here and there.

Overall I've already written about everything here, so I'll make comments in my discussion section.

Chapter 19 : The Mythical Man-Month after 20 Years.

Brooks asks the question, "Why has Mythical Month persisted," and continued to sell after 30 years (40 years now). He claims that its longevity is attributable to the fact that Mythical Man Month is primarily concerned with team management and development and that many of the dilemmas addressed are sociological issues whereas the technology is merely a stage the issues play out on.

Brooks goes on to list portions of the book that are still true today:

The Central Argument: Conceptual Integrity and the Architect - Conceptual integrity is needed for productively deliver quality software products and an architect is needed to ensure that integrity.

The Second-system effect: Featuritis and Frequency-Guessing - Programs made to be general can suffer from "featuritis" (or feature creep) that lowers overall quality and consideration should be taken to avoid it. Brooks recommends guessing at frequencies of use for different functions and features.

Next, Brooks discusses several large changes in the industry since the book was first written:

The Triumph of the WIMP Interface -
WIMP stands for Windows, Icons, Menus, Pointers. Brooks points out this as an outstanding example of understanding the concept of the system and designing according to the concept. The WIMP concept was built of the metaphor of the work desktop and what operations it needed. Cut, paste, copy, type, use of hands, nouns and commands, etc... The WIMP interface also provided a way to cater to novice users and power users by allowing more efficient "power commands" (via keyboard) to be discovered with the mouse menus. Brooks also makes a bold prediction that voice navigation will replace the WIMP paradigm.



Brooks goes on to rescind his statement that managers should plan to "thrown one away." He does this because he had based the statement on the assumption that development would proceed along a waterfall development methodology where feedback and illumination into the inconsistencies in the system would occur when one was built already. Instead, under an incremental model, throwing a system away is unnecessary because early feedback occurs early and inconsistencies revealed in early prototypes and releases.

Brooks admits that he was wrong in disagreeing with Parnas' ideas to conceal the inner workings of modules on the basis that modularization allows higher level conceptualization of the design of the program and can even lead to increased code reuse.

Brooks provides some data found since the original publication that indicates that the man-month is indeed mythical and is encouraged that the industry is looking toward improving the people and social institutions rather than the tools to increase productivity.

Next Brooks observes the largest change: The microcomputer revolution that brought computing to the common man for cheap and empowers them to both be creative and provide demand incentives to quality generalized software.

Brooks goes on to note that OS systems can now be listed on two hands and that reuse has been facilitated by "metaprogramming" where third party developers provide the flexibility for individual programs by programming additional functionality. Brooks claims that metaprogramming directly addresses the essential difficulties of software engineering.

Brooks closes by claiming that software engineering (if it should be called that) is still in its infancy and that perhaps more unifying concepts and theories will be developed that will pull it out of the tar pit.

Discussion :
The last chapter that addresses the changes is an essential addition for the modern reader and unifies the book quite nicely.

Brooks description of featuritis complements Coopers description of feature creep but it seems that they recommend different reasons for why it occurs. Brooks also resembles Brooks (or perhaps its the other way around) in that they both call for a position in the team that emphasizes design before coding. I suspect Brooks has read his Mythical Man Month...

Brooks description of the triumph of "WIMP" is slightly amusing since I have personally have grown up in a world where that is all that has existed (mostly). It is difficult to imagine doing "desktop work" that consists of cutting and pasting of actual physical documents and honestly, it's a bit eye opening and I'm glad I live in the age that I do. I brought up that last point to my girlfriend who was with me at the time and asked her to imagine copying and pasting documents and she asked, "What do you mean," and "people actually did that?!"
Although it's obvious the terms came from using paper, that realization just doesn't hit most people and it really is a sign of how the times have changed and how people think about the objects they use on a day to day basis (that is to say, almost none).

It is also exciting to note the change brought about by the microcomputer revolution, and once again, I'm grateful for the time I live and study computer science in.

Mythical Man Month Ch 16-17

Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition
By: Frederick P. Brooks Jr.
Publisher: Addison-Wesley Professional
Pub. Date: August 02, 1995



Chapter 16 : No Silver Bullet - Essence and Accident in Software Engineering

In a supplementary chapter that was originally a separate paper titled, "No Silver Bullet," Brooks addresses why he thinks that there will be no single advancement in the field of Software Engineering will provide an order of magnitude improvement in itself. Relating this to his titular example of the silver bullet that kills the werewolf, the werewolf is "magnitude improvement in productivity" and the magic silver bullet is the method used to attain it.

He brings this subject up with the purpose of dispelling myth and vain hope with reality that can lead to real progress in making software development easier and more productive.

Brooks claims that the difficulty in writing/building/growing software can be separated into two distinct categories:
- Essence : difficulties inherent in the nature of software
- Accident : difficulties that pertain to the production of software but are not inherent

Brooks states that, "I believe the hard part of building software to be specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation."

From there cooper lists and describes what comprises the essential difficulties of software engineering:

- Complexity : Programming is the most complex activity done on the most complex of human artifacts. Furthermore, programs aren't made by many repeating parts but are composed of many different elements so increasing the size of the program exponentially increases the complexity. Also, abstracting away the complexity in design destroys the essence of the program.

- Conformity : Software must often conform to other interfaces and cannot be simplified by design. Furthermore, the basic elements of the programming don't necessarily have any unifying principles like in other fields.

- Changeability : Software must conform to the pressures of businesses, clients, laws, and the perception that it is the easiest thing to conform.

- Invisibility : Software is neither tangible nor can it be easily visualized in a way that would lend itself to solid and consistent design or conceptualization.

Next, Brooks addresses some the advances in the field that reduce the accidental difficulties:

- High Level Languages : Higher levels of abstraction decreased the cognitive load and made software design easier and more productive by allowing the programmer to work with conceptual constructs (part of the essence) rather than the accidental difficulties that allow the program to conform to the machine.

- Time-Sharing : By having more access to the computer and the program, programmers could maintain their grasp of the complexities of the system and spend less time on re-familiarizing themselves.

- Unified Programming Environments : These allow easy communication and interfacing of multiple programs on a single machine.

Finally, Brooks comes to the various technologies and methodologies that purport to be the silver bullets:

- Ada and other advances in high-level languages: advocates of Ada hope to advance the language to provide a general-purpose high level language that emphasizes modularity, abstract data types, and hierarchical structuring. Brooks debunks Ada as a silver bullet in that with the introduction of higher level languages, most of the improvement to be had has already been exploited. However, he does concede that the training Ada would give to programmers in using high level languages could have an impact.

- Object Oriented Programming : OOP advocates the use of abstract and hierarchical types, called classes, and that each of those should be given a name and a valid set of values and valid methods that can be called on them. Brooks holds the most hope for this bullet and says that these remove a "higher order sort of accidental difficulty," but nonetheless they remain accidental difficulties.

- AI : AI acts a rather broad silver bullet that tends to slip into two definitions: 1) Using computer intelligence to solve what was previously done by human intelligence.
2) Use of expert knowledge to devise a program to solve a problem like they would.

Brooks notes that once a problem in AI is understood, the particular AI problem slips into definition 2, namely, expert systems.

- Expert Systems : Assuming that an expert system could be devised from a brilliant expert in software engineering that could explain his recommendations in a way that could be put in a generalized knowledge base, it would still only address the accidental difficulties in areas like debugging but the essential difficulties would remain untouched.

- "Automatic Programming" : A pipe dream. Any automation of software engineering would require the target problem to have the following properties:
1) It could be characterized by a few well understood parameters
2) The problem has many known methods to solve it
3) The correct solution could be selected given the parameters
Unfortunately, these properties rarely hold.

- Graphical Programming : Brooks claims that an effective solution would require software to be easy to visualize as well as have enough workspace to adequately design with the entire scope in view, neither of which appear possible.

- Program Verification : Brooks dashes the hopes that large jumps in productivity could be attained by proving program to be correct in design before implementation by making the simple observation that program verification is time consuming and nigh impossible for sufficiently complex programs and is thus not viable.

- Environments and Tools : They can provide incremental improvements on accidental difficulties, but they miss the mark on essential difficulties.

- Workstations : increases in machine power don't reduce essential difficulties, although increased power is nice.

After all this, Brooks provides a few worthwhile solutions that attack the essence:
- Buy Versus Build : The beast way to increase productivity is to not build at all.

- Requirements refinement and rapid prototyping : The customer never knows what they really want so much effort must be expended to find what they actually want, this saves time in the end and reduces complexity, makes the program more visible to the client, and provides the right perception of how changeable the program actually is.

- Incremental Development : Brooks likens software engineering to growing a complex biological structure that evolves over time. By growing small parts that always work and adding complexity and functionality in continuously there is a reduction in complexity and programmers are happy to see success in a system that always works.

- Great Designers : Brooks makes a final plea to nurture great designers that can build great software and makes a few recommendations on how to find and provide incentives to these people.

Chapter 17 : "No Silver Bullet" Refired

In this chapter, Brooks addresses the criticisms to his No Silver Bullet paper just before the 10 year mark of his predictive limit expired.
First, he notes that, indeed, no silver bullet has yet to be found and claimed to be as such by a no-biased 'bona fide' cash-paying customer.

Brooks then clarifies his definition of accident as the part of development pertaining to the implementation process. Then he addresses the fact that NSB claims that the accidental part of the work is 9/10 of the total work in developing software. Because of this assumption, he claims that complete reduction of the accidental difficulties could not produce an order of magnitude increase in productivity. He concedes that measurement is needed though.

Brooks debunks the claims that solving the essential difficulties are hopeless by stating that "not all complexity is inevitable" and that much of his proposed solutions tried to avoid complexity, such as growing the program incrementally.

Brooks clarifies that the purpose of his paper wasn't to depress or give to all a spirit of gloom, but to provide a sense of realism so that real progress could be made.

Brooks then dispels arguments against the division of software difficulties into essence and accident and moves to support the movement for "The Vanilla Framework" that purports to increase the visual development of software.

Brooks concedes that quality is a good target that may give rise to productivity and that the perception customization has a strong impact on whether to "build or buy."

Brooks explains that the reason OOP has lingered is that despite the power of the concepts provided by modularity, encapsulation, and inheritance, realizing the power of these things incurs a high initial cost and late returns.

Brooks then expounds on the difficulties of reusability stating that in order for reuse to occur, many well defined libraries must be available, understood, generalizable, and cheap to use.

In the end, so silver bullet has yet been found.

Discussion :

These chapters cover a stunning array of topics as they relate to software engineering but Brooks does a capital job of cutting through to the heart of the difficulties and I think his classification is spot on.

Although it may seem "gloomy" that there's no "silver bullet" his arguments do a wonderful job exposing the real targets for steady improvement in the field. With that said there is still the lingering optimism that makes me wish for exponential leaps in progress that contradicts my sense of reason in this matter.

It is interesting to see how long people have been talking about the use of AI and OOP in the field as the panaceas to our frustrations and to see how far they've actually come in the time spent between chapters 16 and 17.

Brooks recommendation for growing software incrementally and using rapid prototyping sound suspiciously similar to many agile development methodologies like Extreme Programming and I can't help but wonder how many advocates of agile were inspired by Brooks' writing.

Wednesday, April 6, 2011

Mythical Man Month Ch 13-15

Mythical Man-Month, The: Essays on Software Engineering, Anniversary Edition
By: Frederick P. Brooks Jr.
Publisher: Addison-Wesley Professional
Pub. Date: August 02, 1995

Chapter 13 : The Whole and the Parts

In this chapter Brooks considers how to approach system testing. First he makes the elementary prescription of thoroughly of thoroughly designing the system out first which he claims, will illuminate gaps in the understanding of how the system will execute and interface with other parts through definition and specification.

Primarily Brooks advocates a top-down design process where the high level functioning and reason for the program is first enumerated as modules and then each of these parts drilled down into in "refinement steps" until there is no room for ambiguity and data representation and algorithms can be clearly defined.

Brooks then mildly advocates "structured programing" that avoids GO-TO statements but is hesitant to dismiss them completely. Using these structures reduces the risk of logical errors.

From there brooks discusses component debugging techniques in system development and begins by discussing the history of debugging.
The history starts with on-machine debugging where machine time was extremely valuable and that careful structuring of the program with planned stops were included to reveal bugs as quickly as possible was done.
Next in the history were memory dumps where "high-speed printers" were attached and programs were run until a check failed. Debugging and finding the error occurred after the machine ran.
Next was the "snapshots" method of where selective dumping and tracing were used.
Finally he mentions interactive debugging and comes to the conclusions that programmers were not realizing the full benefits of interactive debugging sessions because the sessions for debugging weren't thoroughly planned for.

Finally, Brooks moves on to system debugging (rather than component debugging). He recommends that debugged components should be used since the effects of a known bug can change in system testing.
He also commends the use of scaffolding and use of dummy components and says that the use of scaffolding actually saves time and produces a better system than if none were used at all.
Brooks also proposes that software system testing needs a "purple-wire" technique to highlight issues found in the code that matches up with the project documentation.
Finally he recommends that components should be added to the system tests one at a time so that bugs and effects are thoroughly found and attributed to the correct components.
Lastly, Brooks claims that release updates should be quantized and releases should be large and released rather infrequently for team stability.


Chapter 14 : Hatching a Catastrophe

In this chapter, Brooks explains how a projects find themselves far behind schedule and how to avoid it.

His explanation is that schedule slippage happens one day at a time rather than by catastrophic events and that the slippage isn't fully realized because problems aren't reported since no likes to be the bearer of bad news, or as Sophocles put it (switching the perspective of shame to one of blame, "None love the bearer of bad news."

To avoid slippage, Brooks recommends that milestones be "sharp" or clear and concise so that no one, in trying to avoid shame or blame, will provide overly positive reports on progress. With sharp milestones, no one can provide a falsehood about progress and progress itself is well understood.

If fuzzy (vague) milestones are used, then the milestones become millstones that tend to persist and reduce team morale.



Next, Brooks addresses the purportedly common excuse: "The Other Piece Is Late, Anwyay" and exhorts managers to get their teams to hustle and avoid those one day slips. Therefore, the manager should use critical path schedules or PERT charts where any slip moves the end date on the documentation.

Brooks claims that managers must also know their role and respect first line managers roles. Namely, the boss's role is twofold:
1) Reduce the role of conflict and inspire sharing of status (for info)
2) "Yank the rug back" (get progress going and make changes.)

To fulfill this role, bosses should make clear demarcations between when they are getting information and when they are making changes so that their workers will share their progress voluntarily and truthfully.

Chapter 15 : The Other Face

In this chapter, Brooks advice on writing code and programs that communicate to the end users and to other programmers that will modify the code later.

In order for an end user to use a program, it must make documentation about the following available:
Purpose
Environment
Domain and Range of input
Functions and algorithms used
Input-output formats
Operating Instructions
Options
Running Time
Accuracy and Checking of output

In order to believe (trust) a program, brooks recommends that releases should include a number of tests cases that can be run to show how the program reacts.
The test cases should:
illustrate commonly encountered data,
extreme cases that work,
and cases that lie beyond the extreme and are erroneous.

Lastly, the program should include documentation on how to modify the program that should provide a clear, sharp overview of the internal structure.
This can be provided through:
Flowcharts (limited)
Descriptions of algorithms
Explanation of the layout
Overview of the pass structure
Discussion of modifications planned in the original design

Brooks claims that flowcharts should only be used at the high level and are largely unnecessary.

Brooks moves on to say that programs should be self documented within the code itself with good comments, variable names, and paragraphs like description of the code within the code. Self-documented code reduces the overall need for external documentation and save time overall.

Brooks expresses limited concerns about the size of the source code with self-documented code but remains optimistic overall about the method.

Discussion :


Brooks commitment to top-down design of a system is met with agreement in my mind since it allows abstraction to occur and the designer is able to "see the forest through the trees," while still being able to discuss the details of each part.

Brooks lack of commitment in shunning go-to statements is rather funny in retrospect since they've all but disappeared in current times.

His recommendations on system testing are quite thorough and overall commendable.

His observations on the nature and mechanics of schedule slippage are perhaps the most useful thing about this round of reading and his recommendations on providing sharp milestones and splitting information gathering from actions on changing for the boss roles will surely not go unheeded by those who read it.

The Other Face chapter shows the most age in that the need to talk about self-documentation is largely a basic matter in computer science education.
He describes without explicitly stating that we should make code readable, use good variable names, and not get tied up too much on external documentation.

Overall, it was a good read.