Newsletter of the Boston SPIN
Issue 6, September/October 1995
Return to Newsletter Index
NOV 14 (_SECOND_ Tuesday)-- Boston SPIN Monthly Meeting
Jack Ferguson (SEI) on
The Software Acquisition Capability Maturity Model
6:30 PM (refreshments), 7:00-8:30 PM (meeting)
GTE, Building #5, 77 A Street, Needham, MA
(Admission Free, Wheelchair accessible)
DEC 19 -- Boston SPIN Monthly Meeting, will feature Tim Lister
MAY 20-23,1996 -- 8th Software Engineering Process Group (SEPG) Conference, Atlantic City, N.J.,
"Broadening the Perspective for the Next Century"
Back to top
SEPG95 CONFERENCE MATERIALS STILL AVAILABLE
The Boston SPIN still has some full sets of the Software Engineering Process Group SEPG95 Conference Materials (including Tutorials). We will give you a full set (including a handy tote bag) for a $10 donation to the Boston SPIN.
You can obtain the SEPG 95 materials in one of three ways:
- Pick up the materials at a Boston SPIN meeting,
- Pick up the materials at the address below,
- Send us a check for $10.00 plus postage and a return address, and we will send the materials to you.
For further information such as POSTAGE COSTS ($6-$21 domestic), contact:
ESC/ENS (Bldg. 1704) (TAI), Attn: Charles J. Ryan
5 Eglin Street, Hanscom AFB, MA 01731-2116
Work: 617-377-8324 Fax: 617-377-8325
Back to top
by Rachel Silber
The Craft of Software Testing: Subsystem Testing Including Object-Based and Object-Oriented Testing
Prentice Hall PTR, ISBN 0-13-177411-5, (1995)
553 pp., including index, glossary, appendices, bibliography.
Software test development has long been an art, poorly mapped and poorly understood compared to product development. Brian Marick's The Craft of Software Testing provides explicit techniques for subsystem testing, and is a valuable resource for the testing practitioner.
Subsystem testing straddles the responsibilities of programmers and software testers. The audience for this book is the technical professional who is primarily responsible for testing their own or another's software. To make the best use of this book, a reading knowledge of C or C++ is required. Marick also addresses the maintenance programmer who wishes to know how to assure the validity of changes to existing code. There are no philosophical or management-oriented defenses of the need for this rigorous approach to testing; Marick assumes that his audience already knows how difficult good testing is.
The approach in The Craft of Software Testing begins with inspection of both specifications and code to produce a list of test requirements. Test requirements are combined to create test specifications, ideally with a full specification of the expected output. A large variety of inputs in testing, critical to good testing, is easy to ensure with this methodology.
Once the fully-specified tests are implemented, Marick recommends measuring the coverage attained by the test suite and augmenting the tests based on those results. Bugs found through the test suite and through code inspections also feed back new requirements and new tests to the test suite. Each stage of the process enriches the test suite and serves as a safety net for possible omissions in previous steps.
Test coverage is an important metric in subsystem testing, but Marick avoids making coverage an end in itself. Lack of coverage of a certain area of code may reveal an area where the specification was unclear or where the tester missed "clues" about the behavior of the system. Coverage will not reveal errors of omission in the specification or the code, which may be of greater importance; detecting these errors challenges the knowledge and creativity of the tester.
Part One walks through an extensive example of the methodology described here, including source code for a small subsystem. Applying the techniques to code shows clearly how to extract meaningful test requirements and test specifications from the subsystem under test. Parts Two and Three discuss applying the methodology within existing projects and organizations, specifically addressing the common situation of testing under time constraints. In Part Four, Marick covers some particular testing challenges, such as subsystems that are partially specified with an input grammar or with a state machine model.
Object-oriented software is now common, but most testing technology still assumes a procedural orientation. The tester of object-oriented software will find explicit assistance in Part Four with testing object-based and object-oriented software. Each class under test will generate its own requirements. Inheritance, and extensions to systems that support inheritance, add their own testing problems. In addition to the new code that's written, the uses of existing code may change; this is both the power and danger of inheritance. A logical approach to testing for derived classes and dynamic binding is presented.
Marick succeeds in putting recent testing research into a form practical and available to the tester whether or not one has the resources to conduct a research project of one's own. It is refreshing to have an author acknowledge the extreme time constraints under which much testing takes place. By suggesting ways to integrate a new methodology piecemeal into an existing process, Marick empowers individual contributors to make substantial improvements within their organization.
The author's evident enjoyment of both the intellectual rigor and the creative challenges of testing at its best enlighten and enliven the presentation of often difficult material. I expect this book will become an indispensible tool for many testing practitioners, and a classic in the software quality field.
Back to top
May 1995 Meeting Report
by Ed Maher, courtesy of Digital Equipment Corporation
The Importance of Culture in Implementing the CMM
Stu Jeans (Manager of the Software Engineering Process Department at Lockheed Sanders)
This presentation did not have as much to do with Culture as I expected. It was more a description of a successful process improvement program -- one that got them to Level 3. The significance of culture in their program was that they recognized that understanding culture is an important first step in any process improvement program. In the Sanders experience, this understanding led them to acknowledge problems that were getting in the way of the success of the entire organization (not just the software group).
During this presentation he made frequent references to the IMA (Implementation Management Associates) course material and recommended the book: "The Unwritten Rules of the Game" by Peter Scott-Morgan(McGraw-Hill).
Stu believes that the hardest part of process improvement is in understanding the culture of the organization that you are trying to change. He defines culture as being the organization's values, unwritten rules, and behaviors.
Successful process improvement programs are usually those that drive the change in the same direction as the culture. Obviously, to do this, the culture first needs to be understand. The underlying reason for these things should also be understood (they usually exist with good reason). Once this occurs, any improvement initiative can then take these into account -- not only acknowledge them, but leverage them.
It is important for an organization to understand its values and to reinforce those things that they do value. He described a situation where a company was trying to implement some new practices to make them more predictable. However, they continued to reward the good fire fighters and for the most part not recognize the teams that were meeting their commitments. They were demonstrating what they valued (and it was not predictability).
So, what did they do at Sanders? First they went looking for the unwritten rules; they found three that were relevant to what they were doing:
- Projects operate in stovepipes (poor intergroup communication).
- Customers value flexibility in requirements introduction.
- Intergroup coordination represents considerable change.
This led them to the revelation that focusing on software process improvement (to the exclusion of the other engineering processes) was not enough. If you are developing embedded software systems, you can not succeed with process improvement if you only have a mature software organization.
Once they recognized this problem, they took what they learned about software maturity (mostly based on the CMM) and applied it to the other contributing disciplines: systems engineering, hardware engineering, mechanical engineering, configuration management, quality assurance, and program management.
There was a question on how difficult it was implementing software-based maturity concepts within non-software disciplines. He said that it really was not -- they customized the SEI software assessment questionnaire for each discipline and were pleased with how well it transferred and applied.
He spent a fair amount of time describing the Sanders organization structure. I will not go into it in detail. It included cross- organizational mentor teams for all the major disciplines, as well as a Systems Engineering Process Department and his Software Engineering Process Department.
His Software Engineering Process Department was responsible for process maintenance (documentation, tools, methods, and training), process support, and maintenance of project data.
In response to a question, he described the goals of his Software Engineering Process Department as follows:
- Become more knowledgeable in process engineering, understand how things are done, and understand how change can be crafted.
- Be a force in the organization. Do not work toward "process for process' sake". Understand that process improvement is only done if it is to improve the business goals and to make money.
- Be more pervasive within the organization. (They also have a goal to get to Level 4)
Someone asked if having a central or corporate process function isn't too "Ivory Tower". He acknowledged that as a something that they are constantly watching out for. The use of Mentor Teams staffed from the engineering organizations helps to keep this in check. They also go out of their way to stay in touch and involved with all of the engineering functions.
They have a standard software process architecture that covers the whole product life cycle and applies to all disciplines. They currently are working under four process models that are used depending on the size of the project and whether it had to be DoD compliant. Each process model has its own set of governing standards. They are moving to one standard process with tailoring guidelines so that each project can define a process that is consistent with the organization's standard process as well as all appropriate standards. The cross-organizational Mentor Teams are responsible for the process definition/content. His group (the Software Engineering Process Department) manages the documents.
He was asked why they were moving toward one process model. He said that it was to align themselves better with the Level 3 concept of having an organizational standard software process that includes guidelines for creating a project specific process. This is not really changing the process that would be followed in each instance -- it is only changing how the processes are defined and managed.
Right now their process documentation is paper. They are moving to state-of-the-art technology for documenting, communicating, and managing their process documents.
They have an organizational training program that explicitly focuses on project domain topics and process topics. The courses that people take are tied to their responsibilities.
He identified a few things that he felt were central to their success:
- Establishment of a document hierarchy
- Including descriptions of the intergroup relationships within this hierarchy
- Gaining greater understanding of the culture to help manage the resistance to change
- Staying the course
He also said that the involvement of senior management at Sanders was great -- and was crucial to their success.
Back to top
Some questions that came up:
Q: What do you do when the unwritten rules go directly against the goals of the proposed change?
A: First you have to really get an understanding of the unwritten rules -- including an understanding of how and why they came into being. Next, you need to explicitly communicate the rule along with the consequences, and define the process change around it. This usually works. The problems occur when you do not know these rules or if you choose to ignore them.
Q: (Related to the last question) What do you do if the unwritten rule is: "Change is bad"?
A: Same answer. You need to understand why that is the case and deal with it.
Q: How do you ensure that the culture does not change out from under you?
A: This can be a problem -- especially if reorganizations or corporate mergers throw two different cultures together. You just have to be aware of the possibility and react. Constant change is the way of the world.
Q: How does all of this fit in with the philosophy of "doing more with less"?
A: You need to prioritize. Coordinating all of your process improvement activity so that you make conscious decisions as to where you should spend your process improvement dollar helps to ensure that the money spent is well spent.
Q: How do new tools and/or software environments fit in?
A: They have a dedicated group that analyzes and methodically introduces new tools (It sounded a lot like what is intended by the CMM Level 5 Technology Change Management Key Process Area)
Q: What does he think about how the SEI concepts (including the CMM) apply in the commercial environment?
A: It applies if it is used as a guide -- not as a cookbook.
Back to top
The Boston SPIN is a forum for the free and open exchange of software process improvement experiences and ideas. Meetings are usually held on third Tuesdays, September to June.
We thank our sponsors: GTE and BULL INFORMATION SYSTEMS.
For information about SPINs in general, including HOW TO START A SPIN, contact:
DAWNA BAIRD of SEI, (412) 268-5539, email@example.com.
Boston SPIN welcomes volunteers and new sponsors.
For more information about our programs and events contact:
CHARLIE RYAN, Technical Assessments, Inc.,
ESC/ENS (Bldg 1704), 5 Eglin St, Hanscom AFB MA 01731-2116;
(617) 377-8324; FAX (617) 377-8325; firstname.lastname@example.org.
IN THE SPIN is published monthly September to June, but not always (q.e.d.). Letters, notices (including job postings), and contributed articles are welcomed. Articles do not necessarily represent the opinions of anyone besides their authors.
IN THE SPIN is available only by email.
TO SUBSCRIBE send email address to email@example.com (Ron Price).
SEND letters-to-editor, notices, job postings, calendar entries, quips, quotes, anecdotes, articles, offers, and general correspondence to Sallie Satterthwaite, (508) 369-2365, firstname.lastname@example.org, or to Jim Holloway, (617) 848-1180, email@example.com.
Our WEB HOME PAGE is at http://www.cs.uml.edu/Boston-SPIN/
The following will also work: http://www.cs.uml.edu/Boston-SPIN/index.html
Return to Newsletter Index
Back to top