Open
software projects should use engineering techniques
Background
In
the software industry there are certain engineering techniques used
by most of the well known software organizations. As the field of
software engineering matures, organizations are discovering that they
can develop software better, if they apply certain key practices to
the development effort. These practices improve the development
effort in many ways. Four of the most noticeable
are: increasing quality, decreasing cost, increasing control,
and increasing predictability[1].
Increasing
quality means the final product has less defects than it would if
these practices are not used. Quality in software is the opposite of
the amount of defects in software. A defect is any property that a
software product has, that the designers and developers do not want
it to have. Usually in open software projects defects are referred to
as bugs. Using these engineering practices can increase the quality
of the final product significantly.
Decreasing
cost means less rivalrous resources are
used while a product is being developed. A rivalrous resource is any
resource that the team has a limited amount of, as opposed to a
non-rivalrous resource that can be used as much as needed without
depleting that resource. An example of a rivalrous resource in an
open project is developer time. An example of a non-rivalrous
resource is GCC, a free (as in freedom) compiler collection licensed
under the GPL. Using well known engineering practices can decrease
the overall cost of a development effort significantly.
Increasing
software quality and decreasing cost are grouped together by the SEI
and referred to as improving the effectiveness of the software
development effort.
Increasing
control (accuracy) means that the targeted results of software
efforts are achieved, on average, more by organizations that use the
practices, than those that do not. For example, if you take 10
projects, and measure the average
difference of the scheduled release dates and the actual release
dates, that average will be closer to zero when these practices are
used. Using well known engineering practices can increase the overall
control of development significantly.
Increasing
predictability (or decreasing variability) means that the final
products of a set of software efforts are usually closer to their
targeted results when these these practices are used, than
when they are not. This includes the functional features that the
creators have imagined, the performance of the product, the ease of
use they had in mind for users, and the
final look of the interfaces. Using these engineering practices can
increase the predictability of the development
effort significantly.
The
difference between control and predictability can be hard to grasp.
One way to think about it is mathematically. When control is
increased, the average of the differences between the targeted
results and the actual results is closer to zero. When predictability
is increased, the sum of the differences between the targeted results
and the actual results is closer to zero.
The
SEI at Carnegie Mellon University has been working on software
engineering since 1986 [2]. With help from the Department of Defense
and the Mitre Corporation, they have
created a set of key practices that they believe an organization
should apply, to be considered mature, as well as a model to measure
this maturity. The model is known as the Capability Maturity Model
(CMM), and the practices are knows as the key practices of the CMM.
These practices are widely accepted by the industry as the best known
way to improve their software development process.
The techniques used by an organization to create a software product
are referred to as that organizations software process. When an
organization applies the key practices of the CMM, they are using a
mature software process.
Why
open projects should use mature process
Some
people feel that open projects should not bother with good process
because they have no obligation to finish their work at all, let
alone on time or with any given level of quality. Some argue that
it's a waste of time because a lot of the engineers are not being
paid, and no one can force them to use mature process. Some people
argue that open projects do implement a good software process, that
produces higher quality products at a faster rate than closed
organizations. None of the arguments are valid.
The
argument that people who work on open projects have no obligation to
use a mature software process makes sense, if the product is only
going to be used for research, or if the
engineers were just having fun and won't be using their products in
systems with any critical nature. The fact is that today, and in the
future, that is not the case. Open software organizations and many
other individuals are pushing very hard to get their products used in
critical systems. Many are saying that governments should be using
only open software.
The
argument that no one can force open developers to follow a set
process only makes sense if there is no motivation for the engineer
to do what the others on the project, and the users, want. That is
certainly not true for successful open projects. Many open software
developers these days are actually working for organizations and
getting paid to develop the software. Many are also volunteering, but
they can certainly be motivated to follow process. If they are
motivated enough to put in the time to work on the project in the
first place, all it takes is to educate them on the benefits of a
mature process and it seems likely that
they will utilize it. The project leaders have the power to require
the use of good process, and not accept the contributed work that has
been developed immaturly. Some will be stubborn and refuse to adapt,
and they are free to keep hacking away in an ad hoc manner, but they
should not be creating systems that people depend on. Some of them
are even skilled enough to create software with few defects in that
ad hoc manner, but as soon as they leave the project, there is little
help for others to continue the work that they did. In a mature
organization, the process will help others continue working on the
product, even when the hero that could always make it happen, is
gone.
The
argument that open projects already use a good software process is
partially true. But it's easy to see that not all open projects do,
and even the ones that do, tend to do it implicitly,
rather than defining a great process for all to see and then sticking
with it as the project is developed. The apache project actually uses
a fairly mature process [3]. It is probably not an
coincidence that Apache is one of the most successful,
reliable, and predictable open software projects that exists.
The
truth is, it would be a great thing if all software were open
software, however it is also true that software that people rely
upon, no matter how critical, should follow industry best known
practices to improve quality and reliability. And is always better to
create a product at lower cost. I am not arguing that the open
software that exists today is low quality or unpredictable, but it is
easy to see how using a mature software process along with the best
open software development methods could have benefits.
Software
process experts generally agree that high quality
software can be created by immature organizations, and there
are many obvious examples of this, especially in open software
projects. But the empirical evidence shows
that using a mature software process will significantly increase an
organizations chances of creating high quality software at low cost
that achieves targeted results.
How
open software can benefit from mature process
There
are a number of ways that open software can benefit from using a
mature software process. Assuming a process is used that improves the
quality, cost, control, and predictability of open software,
one can imagine significant benefits to the open software community.
Increasing
the quality of open software would help give
potential users more confidence in it as a whole. Currently,
while some open software is high quality, a lot of it is loaded with
defects. A common belief is that open software usually has less
defects than closed software, because the source code is available
for all to see and critically review. The truth is that there is no
conclusive evidence to prove or disprove that idea[4]. An idea that
can be taken as fact, is that mature software process is one of the
best methods closed software organizations have to minimize the
amount of defects in the software they create. It makes sense that
the same methods would also help keep the number of defects lower in
open software efforts. Everyone knows that it's much easier to remove
a defect when the defect is discovered earlier in development rather
than later. Mature process helps engineers find defects earlier,
while the massive reviewing that open software seems to be
benefiting from now tends to find the defects later, almost
always after a work has already been done and code has already been
written. I believe that using mature process will help minimize the
amount of defects in open software, as it does in closed software.
Decreasing
the cost of developing open software would help improve the amount of
free software that is available, and help open software be created
faster. The most obvious point here is that developers would have
more time to do useful work. Time is the most expensive resource
that open software consumes when being developed. Any tool that helps
developers be more efficient is very valuable. Open software
developers have a habit of ignoring certain parts of development
because they only have time to do what is absolutely necessary, like
writing code that works. If using a mature software process can help
developers do more with the time that they have, it means that they
will have extra time to do the other important parts of development,
like good testing and creating documentation. It also would allow
fewer developers to accomplish the same amount of work, freeing up
some developers to do other work that may have not ever been done.
Increasing
the control of free software efforts would help more open software
projects to meet their targeted results. An example of a project that
could have benefited from increased control
would be the Mozilla project. The actual
release of Mozilla 1.0 was more than a year after the
targeted date. It is very difficult for an open project to set
a targeted release date, because they may
not know how many developers will be working on the project or how
much time each will spend developing. Since this is a weak point in
open software development, it only makes sense to do anything we can
to improve it.
Increasing
the predictability of free software efforts would help keep open
software efforts more uniform. For instance, one open software
project may take a lot less time to achieve a milestone than planned,
while another open project of similar size might take a lot more time
to achieve a similar milestone than planned. The institutionalization
of projects, keeping their results more uniform, would be a good
thing for open software. For example, I would really like to have a
good open Java 2 environment that I can build and distribute Hoot
with. At this time one does not exist that I know of, and it's very
difficult for me to know when one will be available. If open projects
were more predictable, I would feel more comfortable in planning my
work based on the targeted results of the
work to create an open Java 2 platform.
The
four metrics mentioned above: quality, cost, control, and
predictability, are important for most all
software. The vast majority of the closed software companies realize
this and are trying very hard to operate with a high level of
maturity. If open software is to beat closed software based on pure
technical superiority, it is important that the open software
community utilize the industry standard techniques that help the
closed organizations develop software better.
How
Hoot will help
Hoot
will provide two important tools to open software developers:
knowledge on how to use a mature process, and tools to help
facilitate that use.
The
knowledge will come in the form of organizational, or generic,
procedures for satisfying the key processes of the CMM v1.1, as well
as educational literature. Other training materials may be created as
part of Hoot to help project leaders understand the basics of mature
process and the benefits they can expect from using one.
The
tools to help facilitate that use will be
an integrated system, that does as much as possible to make the use
of a mature software process both convenient and fun for open
software developers. See the Hoot home pages for details on this
system.
Copyright
2002, Ryan Boder
For
references, see the ListOfReferences.sxw file in the source
repository.
[1]
1, section 2.4
[2]
1, To the Reader
[3]
3, Case studies
[4]
3
