Login | Register
My pages Projects Community openCollabNet


Open software projects should use engineering techniques



Background


In the software industry there are certain engineering techniques used by most of the well known software organizations. As the field of software engineering matures, organizations are discovering that they can develop software better, if they apply certain key practices to the development effort. These practices improve the development effort in many ways. Four of the most noticeable are: increasing quality, decreasing cost, increasing control, and increasing predictability[1].


Increasing quality means the final product has less defects than it would if these practices are not used. Quality in software is the opposite of the amount of defects in software. A defect is any property that a software product has, that the designers and developers do not want it to have. Usually in open software projects defects are referred to as bugs. Using these engineering practices can increase the quality of the final product significantly.


Decreasing cost means less rivalrous resources are used while a product is being developed. A rivalrous resource is any resource that the team has a limited amount of, as opposed to a non-rivalrous resource that can be used as much as needed without depleting that resource. An example of a rivalrous resource in an open project is developer time. An example of a non-rivalrous resource is GCC, a free (as in freedom) compiler collection licensed under the GPL. Using well known engineering practices can decrease the overall cost of a development effort significantly.


Increasing software quality and decreasing cost are grouped together by the SEI and referred to as improving the effectiveness of the software development effort.


Increasing control (accuracy) means that the targeted results of software efforts are achieved, on average, more by organizations that use the practices, than those that do not. For example, if you take 10 projects, and measure the average difference of the scheduled release dates and the actual release dates, that average will be closer to zero when these practices are used. Using well known engineering practices can increase the overall control of development significantly.


Increasing predictability (or decreasing variability) means that the final products of a set of software efforts are usually closer to their targeted results when these these practices are used, than when they are not. This includes the functional features that the creators have imagined, the performance of the product, the ease of use they had in mind for users, and the final look of the interfaces. Using these engineering practices can increase the predictability of the development effort significantly.


The difference between control and predictability can be hard to grasp. One way to think about it is mathematically. When control is increased, the average of the differences between the targeted results and the actual results is closer to zero. When predictability is increased, the sum of the differences between the targeted results and the actual results is closer to zero.


The SEI at Carnegie Mellon University has been working on software engineering since 1986 [2]. With help from the Department of Defense and the Mitre Corporation, they have created a set of key practices that they believe an organization should apply, to be considered mature, as well as a model to measure this maturity. The model is known as the Capability Maturity Model (CMM), and the practices are knows as the key practices of the CMM. These practices are widely accepted by the industry as the best known way to improve their software development process. The techniques used by an organization to create a software product are referred to as that organizations software process. When an organization applies the key practices of the CMM, they are using a mature software process.



Why open projects should use mature process


Some people feel that open projects should not bother with good process because they have no obligation to finish their work at all, let alone on time or with any given level of quality. Some argue that it's a waste of time because a lot of the engineers are not being paid, and no one can force them to use mature process. Some people argue that open projects do implement a good software process, that produces higher quality products at a faster rate than closed organizations. None of the arguments are valid.


The argument that people who work on open projects have no obligation to use a mature software process makes sense, if the product is only going to be used for research, or if the engineers were just having fun and won't be using their products in systems with any critical nature. The fact is that today, and in the future, that is not the case. Open software organizations and many other individuals are pushing very hard to get their products used in critical systems. Many are saying that governments should be using only open software.


The argument that no one can force open developers to follow a set process only makes sense if there is no motivation for the engineer to do what the others on the project, and the users, want. That is certainly not true for successful open projects. Many open software developers these days are actually working for organizations and getting paid to develop the software. Many are also volunteering, but they can certainly be motivated to follow process. If they are motivated enough to put in the time to work on the project in the first place, all it takes is to educate them on the benefits of a mature process and it seems likely that they will utilize it. The project leaders have the power to require the use of good process, and not accept the contributed work that has been developed immaturly. Some will be stubborn and refuse to adapt, and they are free to keep hacking away in an ad hoc manner, but they should not be creating systems that people depend on. Some of them are even skilled enough to create software with few defects in that ad hoc manner, but as soon as they leave the project, there is little help for others to continue the work that they did. In a mature organization, the process will help others continue working on the product, even when the hero that could always make it happen, is gone.


The argument that open projects already use a good software process is partially true. But it's easy to see that not all open projects do, and even the ones that do, tend to do it implicitly, rather than defining a great process for all to see and then sticking with it as the project is developed. The apache project actually uses a fairly mature process [3]. It is probably not an coincidence that Apache is one of the most successful, reliable, and predictable open software projects that exists.


The truth is, it would be a great thing if all software were open software, however it is also true that software that people rely upon, no matter how critical, should follow industry best known practices to improve quality and reliability. And is always better to create a product at lower cost. I am not arguing that the open software that exists today is low quality or unpredictable, but it is easy to see how using a mature software process along with the best open software development methods could have benefits.


Software process experts generally agree that high quality software can be created by immature organizations, and there are many obvious examples of this, especially in open software projects. But the empirical evidence shows that using a mature software process will significantly increase an organizations chances of creating high quality software at low cost that achieves targeted results.



How open software can benefit from mature process


There are a number of ways that open software can benefit from using a mature software process. Assuming a process is used that improves the quality, cost, control, and predictability of open software, one can imagine significant benefits to the open software community.


Increasing the quality of open software would help give potential users more confidence in it as a whole. Currently, while some open software is high quality, a lot of it is loaded with defects. A common belief is that open software usually has less defects than closed software, because the source code is available for all to see and critically review. The truth is that there is no conclusive evidence to prove or disprove that idea[4]. An idea that can be taken as fact, is that mature software process is one of the best methods closed software organizations have to minimize the amount of defects in the software they create. It makes sense that the same methods would also help keep the number of defects lower in open software efforts. Everyone knows that it's much easier to remove a defect when the defect is discovered earlier in development rather than later. Mature process helps engineers find defects earlier, while the massive reviewing that open software seems to be benefiting from now tends to find the defects later, almost always after a work has already been done and code has already been written. I believe that using mature process will help minimize the amount of defects in open software, as it does in closed software.


Decreasing the cost of developing open software would help improve the amount of free software that is available, and help open software be created faster. The most obvious point here is that developers would have more time to do useful work. Time is the most expensive resource that open software consumes when being developed. Any tool that helps developers be more efficient is very valuable. Open software developers have a habit of ignoring certain parts of development because they only have time to do what is absolutely necessary, like writing code that works. If using a mature software process can help developers do more with the time that they have, it means that they will have extra time to do the other important parts of development, like good testing and creating documentation. It also would allow fewer developers to accomplish the same amount of work, freeing up some developers to do other work that may have not ever been done.


Increasing the control of free software efforts would help more open software projects to meet their targeted results. An example of a project that could have benefited from increased control would be the Mozilla project. The actual release of Mozilla 1.0 was more than a year after the targeted date. It is very difficult for an open project to set a targeted release date, because they may not know how many developers will be working on the project or how much time each will spend developing. Since this is a weak point in open software development, it only makes sense to do anything we can to improve it.


Increasing the predictability of free software efforts would help keep open software efforts more uniform. For instance, one open software project may take a lot less time to achieve a milestone than planned, while another open project of similar size might take a lot more time to achieve a similar milestone than planned. The institutionalization of projects, keeping their results more uniform, would be a good thing for open software. For example, I would really like to have a good open Java 2 environment that I can build and distribute Hoot with. At this time one does not exist that I know of, and it's very difficult for me to know when one will be available. If open projects were more predictable, I would feel more comfortable in planning my work based on the targeted results of the work to create an open Java 2 platform.


The four metrics mentioned above: quality, cost, control, and predictability, are important for most all software. The vast majority of the closed software companies realize this and are trying very hard to operate with a high level of maturity. If open software is to beat closed software based on pure technical superiority, it is important that the open software community utilize the industry standard techniques that help the closed organizations develop software better.



How Hoot will help


Hoot will provide two important tools to open software developers: knowledge on how to use a mature process, and tools to help facilitate that use.


The knowledge will come in the form of organizational, or generic, procedures for satisfying the key processes of the CMM v1.1, as well as educational literature. Other training materials may be created as part of Hoot to help project leaders understand the basics of mature process and the benefits they can expect from using one.


The tools to help facilitate that use will be an integrated system, that does as much as possible to make the use of a mature software process both convenient and fun for open software developers. See the Hoot home pages for details on this system.



Copyright 2002, Ryan Boder





For references, see the ListOfReferences.sxw file in the source repository.

[1] 1, section 2.4

[2] 1, To the Reader

[3] 3, Case studies

[4] 3