Quality Certifications for Offshore Developers
The standards cover
-
Activities to be defined in an enterprise's quality management system (QMS)
-
Management responsibility
-
Resource management and measurement
-
Analysis and improvement
Individual enterprises' interpretations and internal implementations determine how the standards are applied and interpreted externally, as well as the significance of any certification achieved.
ISO 9000 provides a definition of the terms and references used to ensure semantic commonality in the use of these standards by various companies. ISO 9001 defines the requirements for the QMS, and is the standard used to assess and ensure an enterprise's ability to deliver on customer requirements consistently. This standard is often used by services companies to indicate their ability to deliver on customer requirements. This is the only standard within the ISO 900x family that can be certified through third party consulting organizations and certification agencies. ISO 9004 is a guideline standard that assists in achieving continuous process improvement through the QMS.
Future Developments
The ISO 900x standards are being rolled up into a combined ISO 9000-2000 standard, and future vendor certifications will increasingly refer to ISO 9000-2000 rather than the separate standards (9001 and so on). Two other ISO standards are potentially relevant to the provision of application outsourcing services, but neither of them has anywhere near the "mind share" of the 900x family. The first is ISO 12207, which has been expanded by the Institute of Electrical and Electronics Engineers. ISO 12207 takes an ISO 9000 approach by attempting to standardize the product development and maintenance life cycles. The second is ISO 15504, which is the culmination of the much-anticipated and long-running Software Process Improvement and Capability Determination (SPICE) effort that began in the early 1990s. SPICE seems to have taken the tenets of ISO 9001 and ISO 12207 and combined them with those of the Software Engineering Institute's (SEI's) CMM. In fact, the SEI participated in the SPICE development group. Although ISO 15504 is receiving some attention, at this time, the CMM clearly has more mind share.
Software Engineering Institute
The SEI (www.sei.cmu.edu), which is operated by Carnegie-Mellon University, focuses exclusively on software engineering and associated disciplines. It is a research and development center funded by the U.S. federal government and sponsored by the U.S. Department of Defense (DOD).
The Capability Maturity Model
The SEI's CMM is a rigorous methodology and standard for software development based on five levels. Achieving CMM Level 5 is widely acknowledged as a strong indicator of quality software development processes. Only about 70 companies worldwide have publicly acknowledged that they have achieved Level 5 certification. Of these companies, 50 are in India.
It is important to note that the CMM standards are descriptive (they describe what must be done), rather than prescriptive (how to do it). A vendor can, therefore, define a specific method of execution for a prescribed CMM process in a manner that is not the best possible implementation of that process. Therefore, CMM standards certification in no way guarantees that a vendor's internal implementation of these standards is best in class in any way. CMM certifications can be achieved for specific locations, business units, functions or the entire organization. Enterprises that are evaluating ESPs need to understand at which level the certification has been achieved.
The People Capability Maturity Model (P-CMM)
The SEI has also developed the P-CMM, which is fast gaining traction among ESPs. The P-CMM framework also has five levels and specifically addresses the development of capabilities of the people within a software development organization. It also provides a strong foundation for assessing the "people policies" of an organization and overall organizational workforce development. Even fewer companies globally have attained P-CMM Level 5 than CMM Level 5.
Future Development
The SEI has evolved a next-generation standard called the Capability Maturity Model Integration (CMMI) standard, which integrates the CMM standards discussed earlier, and signifies a higher level of process capability and maturity. The SEI's sponsor (the DOD) wants CMMI to supersede the older standards, and the SEI is in discussions with all stakeholders about plans to retire the current CMM standards at the end of 2003. Commercial application development organizations, however, aren't so sure. We believe that these issues will be resolved by the end of 1Q03, and that the SEI will likely bow to DOD pressure. As a result, ESPs that are at CMM Level 5 already, or others that are working toward it, will probably choose to focus on achieving CMMI certifications and capabilities. Enterprises will likely see increasing references to CMMI instead of (or in addition to) CMM - or a greater emphasis on using ISO 15504. As described above, ISO 15504 aims to combine the best of ISO 900x and CMM in a single standard.
Six Sigma
Six Sigma is a rigorous and a systematic methodology that uses information (management by facts) and statistical analysis to measure and improve an enterprise's operational performance, practices and systems. Its purpose is to identify and prevent "defects" in manufacturing and service-related processes. The term "Six Sigma" was coined from the underlying statistical principles, where "sigma" measures the standard deviation of a population of defects. In the context of a total quality management methodology for software development, Six Sigma translates into no more than 3.4 defects per million "opportunities." Six Sigma is quite possibly the most-focused quality technique for measuring critical processes. It also gives the best direct feedback to an enterprise of the degree of control it has on its overall quality. The Six Sigma methodology was "invented" and first implemented by Motorola ( http//mu.motorola.com ) in the mid-1980s. Since then, it has been used extensively in manufacturing enterprises, but has only recently been taken up by software companies as a means of addressing the application
development process and defects in coding. Organizations are not certified in Six Sigma; however, there are people certifications (for example, green belt and black belt). This makes it difficult, if not impossible, to compare Six Sigma organizations. The heart of the Six Sigma approach is the DMAIC Model. This is positioned as a systematic method for analyzing and improving business processes. The model consists of five phases, from which it derives its name
- Define opportunities
- Measure performance
- Analyze opportunity
- Improve performance
- Control performance
A key Six Sigma measure is defects per million opportunities, which is then used to calculate the Six Sigma metric of actual achievement and performance.
Other Quality Standards
In addition to the key certification schemes and standards described earlier, there are other standards or quality references that enterprises are likely to see in ESPs' marketing material and sales proposals.
BS7799/ISO17799
BS7799/ISO17799 is one of the most widely recognized security standards in the world. The heightened focus on security by enterprises worldwide will likely lead to an increasing number of software vendors adopting this certification standard.
Malcolm Baldrige Quality Award
The Malcolm Baldrige Quality Award (www.quality.nist.gov) was instituted by the U.S. government and the National Institute of Standards and Technology to promote excellence through quality in U.S. enterprises. The Baldrige criteria focus on performance excellence for the entire organization within an overall management framework.
Deming Model
The Deming Model (www.deming.org) is based on the fundamental principles of statistical quality control. These principles were defined by the renowned quality consultant and expert Dr. W. Edward Deming and popularized primarily in Japan through his teachings there in the 1950s. Typically, references companies make to Deming will most likely be in the context of the Deming prize, which is awarded annually by the Union of Japanese Scientists and Engineers to a company that excels in quality.
Balanced Scorecard Approach
The balanced scorecard is a new approach to strategic management. It was developed in the early 1990s by Dr. Robert Kaplan (Harvard Business School) and David Norton of Balanced Scorecard Collaborative (www.bscol.com). The balanced scorecard is less of a quality system and more of a management feedback system. Feedback on internal business processes and external outcomes is used to continuously improve strategic performance and results.
Assessing Application Outsourcers' Certifications
Application outsourcers' pursuit of quality and quality certifications has a dual purpose - to improve the quality of an ESP's functions and processes, and to provide differentiation in the marketplace. When assessing ESPs, enterprises should conduct due diligence to assess the relative emphasis an ESP gives to each of these purposes. Enterprises should verify that a particular standard, or a particular certification level, results in higher-quality processes and software. However, using the CMM as an example, there is clear evidence that the higher the CMM level, the greater the improvement in performance in terms of average defects per function point and hence lower costs. Therefore, if an enterprise were using a Level 3 vendor vs. a Level 2 vendor for a time-and-material project, it could expect to see the cost differential in the actual cost of the project. If the project is turnkey or on a fixed cost basis, the underlying benefit is that the vendor should pass on the benefits of the lower costs to the client in terms of a more-competitive price structure. Vendors with high-quality certifications that positively affect their internal cost structures can and should further "raise the bar" on the competition by passing on the benefits of their reduced cost structure directly to clients.
When assessing application outsourcers' quality claims, enterprises should
-
Review the ESP's internal documented statistics on defects and compare these with the certification levels.
-
Regardless of the certification level, be wary of an ESP that is not tracking these statistics or is unwilling to allow a review of them.
-
Check associated productivity improvements that, while not formally documented, are supported anecdotally. Ask the ESP to provide evidence of the productivity improvements achieved through the improved processes and state how these improvements will directly
benefit the enterprise. -
Ensure that there is a mechanism for transferring the ESP's knowledge about process capability to the enterprise to improve the enterprise's internal process capability.
-
Consider engaging the ESP in a separate exercise aimed specifically at improving the enterprise's internal development processes and readiness for CMM or CMMI.
Key Facts
-
The SEI's CMM is a major quality tool in software development
-
The SEI's P-CMM is being used as an additional certification and quality standard to aid in the software development effort through enhanced personnel management of the development workforce.
-
Other emerging and relevant standards that are being used by the software industry are Six Sigma for statistical process control-based quality and BS7799 for security-related measures and certification.
-
Higher levels of certification achieved through strong process capabilities lead to enhanced end results.
In conclusion Quality and strong quality certifications should be a factor in evaluating any vendor's suitability for an application outsourcing relationship. This is even more critical in the case of an offshore provider - the risks in offshore application outsourcing can be mitigated to some extent through quality internal processes implied by a vendor's certification. This will also give the enterprise additional confidence in a provider's methodologies and processes. Quality certification levels provide one important dimension for assessing the competence of an offshore provider. However, significant due diligence is required to assess the true value-added benefits that a vendor is able to deliver by virtue of its quality certification and to ensure that the certification is backed up by consistently delivered high-quality processes.
