Newsletter of the Boston SPIN
Issue 10, May 1996
Return to Newsletter Index
JUNE 18 (Tuesday): Boston SPIN annual elections and program.
Dr. Chuck Myers (SEI):
Managing Resistance to Software Process Improvement
6:30 PM (refreshments), 7:00-8:30 PM (meeting)
Last meeting until September.
GTE, Building #5, 77 A Street, Needham, MA
Info: Ken Oasis, (617) 563-4197, or e-mail: email@example.com
JULY 10 (Wednesday): BCS Software Quality Group: Hot Topics Forum
Bring your problems, and great ideas you are willing to
share. Co-sponsor: ASQC Software Division.
7PM, BCS office, 101 First Ave, Waltham
Info: John Pustaver, 508-443-4254
We have 2 separate email lists, one for just membership (this newsletter) and one for LOTS of announcements that we receive from process organizations and forward out. To add yourself to the announcements list send email to Charlie Ryan firstname.lastname@example.org
Back to top
APRIL 1996 Meeting Report
by Ed Maher, courtesy of Digital Equipment Corporation
Welcome to the Next Level -- Approaches to Achieving Higher SEI Capability Maturity Levels
Stan Ostrowski, Dynamics Research Corporation (DRC);
Kevin Porter, Computing Devices International (CDI)
Arthur D. Little
Both of these presentations involved the experiences of organizations in the government sector successfully implementing process improvement in response to customer demand. They also both involved a focus on going from Level 1 to Level 2 on the SEI's Capability Maturity Model for Software scale.
[As background, Level 2 is known as the "Repeatable Level". An org operating at level 2 has basic project management disciplines in place and is likely to repeat past successes on similar projects. The Key Process Areas at level 2 (along with a very high level summary) are:
- Requirements Management; this involves the formulation, agreement, tracking, and control of the software requirements
- Software Project Planning; this involves estimating, establishing commitments, and planning the software project activities.
- Software Project Tracking & Oversight; involving the tracking of activities and deliverables against plan; including how corrective actions are taken and how changes are communicated
- Software Subcontract Management; addressing the evaluation, selection, and management of subcontractors
- Software Quality Assurance; this involves the objective assurance that applicable standards, policies, and procedures are being followed
- Software Configuration Management; addressing the management, control, and communication relating to the software work- products -- for example, the requirements, designs, code, plans, tools, and builds ]
Back to top
Kevin Porter (CDI) spoke on: "Striving for Maturity -- A Branch Office's Journey"
Kevin's organization is a reasonably small branch office (about 38 people) that has accountability to their Corporate offices in Minnesota. There is a centralized corporate culture based on an ongoing relationship with the Department of Defense (DoD). Over the last four years the branch has had two formal Software Process Assessments (SPA) and a mini-assessment. These were corporate- driven with the branch being an active participant in the process. The most resent SPA was this year, and they now have an action plan with committed resources.
He believes that they have now implemented change that meets the goals associated with level 2. They are in the process of institutionalizing these changes and have not yet been appraised as being at that Level. The branch has two full-time people working on process improvement (plus support teams).
Previously, they had gone through a number of process improvement initiatives with little success. Kevin felt that this was most likely due to their customer not being all that interested in paying for process improvement; the focus was on the delivery of the contracted end-products. This has now changed. As Michael Reed told the SPIN last June, the Air Force (One of CDI's major customers) is moving toward requiring that all software suppliers be appraised at Level 3 on the SEI CMM scale.
He next outlined some of the innovative things that they are doing to help improve their process maturity.
- Given that they are co-located with a subcontractor who is already at Level 3, they can see first-hand how a Level 3 org operates.
- They are part of a Software Productivity Consortium which provides them with easy access to other people's experiences, guidelines, and templates which they can tailor for their own environment.
- Even though they're currently only shooting for Level 2, they are keeping an eye on many of the goals associated with Level 3 as way to reduce subsequent rework.
- He spoke highly of Lotus Notes as an easy-to-use and effective way to communicate. They also used it as a document repository.
Kevin also mentioned a challenge that they had because they had to exist in the corporate environment, but at the same time they wanted to quickly implement changes to their local processes. One thing they recognized was that it was more efficient to implement the changes as addenda to the corporate policies and procedures (P&Ps). The alternative would have been to spend a lot of cycles trying to get the corporate P&Ps changed.
He mentioned a number of factors which led to their success. These included:
- An appropriate focus on process improvement as a journey and not a goal
- Getting committed resources
- The use of a Process Asset Library (using Lotus Notes)
Another technique that they used was to work on getting the most visible critics on their side as champions. He also reiterated something that Naomi Karten stated last month; working well as a team and with people is critical to success. They have been so successful that the corporate process people recognize and use them as a resource.
Some of the obstacles that they have had to deal with include:
- People frequently want to see reliable return-on-investment data before they'll commit.
- It was very difficult for them to effect change on the corporate policies and procedures with which they were required to conform.
- The bottom line and the reward structure continue to revolve around shipping product and not process improvement.
CDI is now in the process of institutionalizing the things that they put in place. One thing that they are observing is that people do not relate well to formal "Policies and Procedures" (P&Ps). With this in mind, they are focusing their actions on providing people with checklists, templates, and process documents. These are consistent with the more formal P&Ps, but relate more obviously to the day-to-day activities and deliverables. The people who have to follow the change do not have to spend time understanding or interpreting the P&Ps. They also have integrated process training (including CMM training) into the standard menu of training.
Back to top
Stan Ostrowski (DRC) spoke about their very aggressive software process improvement program, which started in 1994. The company employs about 900 people; 120 in software. They have $85M in sales. Their customers include: DoD, Treasury, GSA, and State Government. Most of their work is custom development or system engineering. They have been independently appraised at Level 2 and were recognized as achieving the goals associated with the Level 3 "Peer Reviews" key process area. Subcontract Management (a level 2 KPA) is not applicable to their work.
As with Kevin, their primary motivation for process improvement as based on demand from their customers. He felt that in the past they were an effective Level 1 organization, and that this success was primarily due to the heroics of individuals.
Some of the problems that they came across in the past when trying to improve the process were founded in something that is helping them now -- a strong customer pulling the strings. In the past, the customer would say that they aren't paying for things like Software Quality Assurance or Configuration Management -- they are paying for product! Now the customer is requiring those things and more. DRC was also experiencing a lot of the same problems as many orgs: resistance to anything that looks at all like bureaucratic standardization; people not having the time; and fear that creativity is being stifled.
Stan mentioned that the nature of going from Level 1 to Level 2 is such that you can't really calculate quantifiable Return on Investment. When you're at Level 1, you don't know the cost of things. Because of this, he is always looking for subjective data that can demonstrate how things are better. Some of the things he has observed are:
- It was obvious that the planning and tracking process was improved.
- The documented processes have resulted in fewer surprises, made training easier, and the common terminology has improved communication.
- The reviews which they put in place have reduced errors and have also improved communication.
- The use of a traceability matrix has been a big help in managing requirements throughout the project.
He then talked about some of the things that they did to successfully introduce this change:
- The first thing he mentioned was management commitment. This commitment was backed up with on-going support, funding, and infrastructure.
- One of their key development projects committed people to work on the improvement. Stan felt that the process improvement would not have succeeded without the strong and proactive Project Leader on that particular development project.
- All the process action teams had representation from across the group. This allowed each team to get diverse perspectives on what they were doing, and created a communication channel back out to the org.
- They selectively used the services of a process improvement consultant.
- Focusing on the specifics of getting to Level 2; they didn't just focus on producing Policies and Procedures -- they defined the actual processes, used pilots, and put a lot of attention into training.
- Since they were focused on achieving Level 2, they also conducted a "readiness assessment" a couple of months before the real thing. He said that there would have been no reason to have the formal assessment if they knew they were going to fail.
[ This makes sense given the business goal of achieving a Level 2 assessment. It also conveys that they are not as interested in the assessment as a way of helping to prioritize process improvement. ]
From the time that management committed to the [process improvement] program until the actual assessment, the program took total of 23 months. This included about four months to develop an action plan, five months to develop the policies and procedures, and six months of pilot activities. Their next steps are to start looking at the Goals of Level 3 (which involves focusing more on the organization as opposed to individual projects).
Joe Puffer then spoke briefly. He identified some common themes from the two presentations: they both needed commitments; their motivation was business-driven; they had tried and didn't succeed with improvements in the past; and culture & people issues were important. They also both mentioned that they are now considered experts within their companies.
Back to top
QUESTIONS & ANSWERS (paraphrased)
Q: To what extent were their customers involved in the process change? Did they need any training?
A: Stan said that DRC was internally focused in response to a customer demand for process maturity. They didn't need to train the customers. However, the customer couldn't help noticing the increased emphasis on review and approval of the requirements.
Kevin mentioned that the DoD has a fairly formal interaction with their development process, but that it didn't really change from the customer perspective and they did not need any education.
Q: Did any important metrics/measures change as they improved?
A: Kevin pointed out that metrics had been fairly gross (LOC was being collected) so that they don't have the ability to see if any measures have changed or improved. They are now collecting process metrics.
Stan said that the only change was going from not having any metrics to having some.
Q: In developing processes aligned with the CMM key process areas, how did you handle the redundancies across the KPAs?
A: Stan said that they kicked off their process action teams and took an approach similar to the analysis and design of software. Each team produced a hierarchy of documents and conducted review cycles (across the action teams) at each phase. The redundancies would be noticed and dispositioned during one of the first of these reviews.
Kevin said that they used a traceability matrix to keep track of all the process changes. This helped keep them straight and avoid redundancies.
Q: How do you get the engineers to do it (change)?
A: Joe mentioned that people often will resist change because they have always been doing it the current way. The key is to introduce change in a way that allows them a chance to try it and succeed. Involve the people that resist in the action teams and the Pilots. This is all about managing change -- if people don't see themselves failing, their resistance is less likely to build.
Stan said that one factor that helped was that they staffed the process improvement projects with individuals that were respected within the org and had skills that contribute to the process. The people involved in the change were not just those that happened to be free at the time.
Q: How did you conduct the pilots?
A: Stan said they had pilot activities for each key process area. They solicited new projects to try different aspects of each proposed process. In a seven month period they had 15 concurrent pilots. He felt that the pilots were very successful -- once these people tried it as a pilot, they were not going back to the old ways. He also mentioned that they enticed pilot participants because the pilots got more personalized training, orientation, and mentoring in the new processes.
Q: (For Stan) When going for Level 2, why did you also target the "Peer Reviews" (Level 3) key process area?
A: Most projects had been doing some form of peer-review, so it was much easier than any of the other Level 3 Key Process Areas. Also, it was where they could have the biggest return on investment (savings from the early identification of defects).
Back to top
QUALITY WEEK '96:
A Personal Review by Johanna Rothman, Rothman Consulting Group, Inc.
Quality Week '96 was a week of contrasts. About 800 people attended, representing about 400 companies. (Yes, Microsoft was there!) Of the people I met, about 20% were doing a number of the activities in the CMM levels 2,3,4 in the areas of project planning and management, metrics, and configuration management. The others were interested in changing what they were doing, to be more effective.
To my surprise, there were still a large number of people who believe that an SQA group must be organizationally independent, and be the police and gatekeepers of the product and the product development process. Then, these same people were surprised when the product was "thrown over the wall" at them. I had several long discussions with some of these people, to understand what I perceived as a contradiction.
John Marciniak summarized the National Software Council meeting which immediately preceded Quality Week. It appears that the NSC is focusing its efforts on education of software folks to better meet industry's needs, and education of the public and lawmakers about software.
Tom Gilb gave a fascinating talk about inspections and some of the successes he has had in inspecting requirements and contracts to dramatically improve product quality.
Leon Osterweil (UMass-Amherst) discussed why software is always under test, and how to use customers positively to test your software (in addition to rigorous in-house testing) in his keynote address.
Watts Humphrey's described the use of the Personal Software Process to create very low-defect software.
Back to top
OTHER CONFERENCE HIGHLIGHTS
I attended a number of interesting sessions. Chuck House, while with Centerline last year, gave about 50 seminars to a variety of software companies. He presented some fascinating data about what these companies are and are not doing. According to Chuck's data, the managers at these companies thought they were doing things better than the individual contibutors did. And, inadequate scheduling and planning with resulting problems in development and test were high on the list of concerns. Chuck asks, "If we all know *what* we are supposed to do (via the CMM), why can't we do it?" His perspective is that the CMM is not sufficiently real- world, and that getting to a particular level does not mean you actually get any value from the activities.
Another session, billed as the "Great Debate" between Boris Beizer (well-known author of testing books) and Tom Gilb (inspection maven) was a dud. Boris and Tom agreed that inspection and testing are complementary and perform different functions.
A panel session with Robert Hodges (Sybase), Cem Kaner (test book author and consultant), Brian Marick (development test book author and consultant), Bob Birss (AT&T), and Melora Svoboda (consultant) was a fascinating discussion about when to do what kinds of testing to save time and money. Hodges talked a bit about the assertion-based testing that is performed on Sybase database products. Kaner discussed the legal need to adequately test to prevent litigation. Marick and Birss discussed developer-based testing and SQA-organization-based testing.
The effectiveness and improvement of process and testing were the most frequently discussed topics, providing a range of possibilities to attendees. The big "aha" that I learned was about the proper use of sampling for inspection purposes. My talk on metrics was well attended and prompted a lot of discussion about when to use data and if and when to use intuition. All in all, the attendees had an opportunity to learn a tremendous amount of information from a variety of perspectives.
Johanna can be reached at
Back to top
EXCEL, Inc. 255 Independence Drive Hyannis, MA 02601
Employment Opportunities in Quality:
Principal Quality Engineer
Establish and maintain internal and external quality system standards, procedures, and quality plans. Lead cross fundamental teams in problem solving of process and product quality issues. Project management in the implementation and maintenance activities associated with ISO 9001-94 system. Certified ISO9001 Lead Assessor or ASQC Certified Quality Engineer, software background preferred.
Supplier Quality Engineer
Statistics, root cause analysis, and corrective actions. ISO 9000 background and SMT experience. Supplier quality contact and data reporting. First article inspection and work instruction generation. Experience in printed circuit manufacturing.
Senior Manufacturing Quality Engineer
Hands on PCB manufacturing experience. Quality Process experience. Establish and maintain internal manufacturing quality system standards, procedures and quality plans. Lead cross functional teams in problem solving activities and provide project management with ISO 9001-94 system. Certified ISO 9001 Lead Assessor or ASQC certified Quality Engineer.
Senior Quality Engineer (Corporate Quality)
Plan, develop, implement, oversee, and report on efforts in support of Excel's goal to achieve ISO 9000 Certification. BSEE and 10 years quality engineering experience.
Contact: John Scott, (508) 862-3273
The Renaissance Network 65 Franklin Street, Boston, MA 02110
We (The Renaissance Network) are a small, specialized placement firm, focused on creating a good career fit between technically talented professionals and the high-tech companies in the Boston area.
We are currently searching for someone with strong software development experience and with the following qualifications, for an interesting Boston-based PERMANENT position with a company experiencing a high rate of growth and opportunity.
- Strong communication skills, enjoys personal contact with users during development phase
- 1-3 years experience development experience with SQLWindows (Gupta), Oracle and/or Sybase.
- Familiar with GUI programming and design. Knowledge of UNIX and TCP/IP desirable.
- Project management experience would be a real plus.
The Renaissance Network 65 Franklin Street,
Boston, MA 02110
Voice (617) 357 0600,
FAX (617) 357 0602
Back to top
QUIPS, QUOTES, ANECDOTES
"After two years of interviewing industry experts in the US and Japan about their perspectives on trends and issues the software research team has identified a set of issues that will shape its future, including patents, immigration, trade, antitrust, the global talent pool, software quality, piracy, and, of course, the ongoing march of technology and the resulting emergence of new platforms.
"No issue is more pressing or of more universal concern among those interviewed than the management of the software development process. Creating software is still more like a craft than true engineering or manufacturing. Methods for meeting changing requirements, achieving timely delivery, estimating the effort involved and assuring quality are all still in their infancy, and, despite careful attention by serious minds, have not improved much in 25 years!"
(From the Bay Area SPIN & East Bay Software Quality Association announcement of a program about Stanford University's Computer Industry Project (SCIP) -- a long-term, interdisciplinary study of the production and consumption of information technology.)
Back to top
The Boston SPIN is a forum for the free and open exchange of software process improvement experiences and ideas. Meetings are usually held on third Tuesdays, September to June.
We thank our sponsors, GTE and Raytheon. We also thank U/Mass at Lowell for hosting our Web page, and Digital Equipment Corporation for Ed Maher's SPIN meeting reports.
For information about SPINs in general, including ***HOW TO START A SPIN***, contact: DAWNA BAIRD of SEI, (412) 268-5539, email@example.com .
Boston SPIN welcomes volunteers and sponsors.
For more information about our programs and events contact:
CHARLIE RYAN, The Software Engineering Institute (SEI)
Technical Assessments, Inc.
ESC/AXS (Bldg 1704), 5 Eglin St, Hanscom AFB MA 01731-2116;
(617) 377-8324; Fax (617) 377-8325; firstname.lastname@example.org
IN THE SPIN is published monthly or bimonthly September to June. Letters, notices (including job postings), and contributed articles are welcomed. Articles do not necessarily represent the opinions of anyone besides their authors. We do not publish advertisements or job searches, but we gladly publish job postings.
IN THE SPIN is available only by email. TO SUBSCRIBE send email address to email@example.com.
SEND letters-to-editor, notices, job postings, calendar entries, quips, quotes, anecdotes, articles, offers to write, and general correspondence to Sallie Satterthwaite, 508-369-2365, firstname.lastname@example.org. If possible, please format input as Courier text with explicit line breaks and a maximum line of 65 characters.
Send SPIN Doctor questions to Judy Brodman, email@example.com .
Our WEB HOME PAGE is at http://www.cs.uml.edu/Boston-SPIN/
Return to Newsletter Index
Back to top