Peer review is seen as an important quality assurance mechanism in both
industrial development and the open source software (OSS) community. The
techniques for performing inspections have been well studied in industry; in
OSS development, peer review practices are less well understood.
In contrast to industry, where reviews are typically assigned to specific
individuals, in OSS, changes are broadcast to hundreds of potentially
interested stakeholders. What is surprising is that this approach works very
well, despite concerns that reviews may be ignored, or that discussions will
deadlock because too many uninformed stakeholders are involved.
In this work we use a multi-case study methodology to develop a theory of OSS
peer review. There are three research stages. In the first stage, we examine the policies of 25 OSS projects to understand
the review processes used on successful OSS projects. We also select six
projects for further analysis: Apache, Subversion, Linux, FreeBSD, KDE, and
Gnome. In the second stage, using archival records from the six projects, we construct
a series of metrics that produces measures similar to those used in traditional
inspection experiments. We measure the frequency of review, the size and
complexity of the contribution under review, the level of participation during
review, the experience and expertise of the individuals involved in the review,
the review interval, and number of issues discussed during review. We create
statistical models of the review efficiency, review interval, and
effectiveness, the issues discussed during review, to determine which measures
have the largest impact on review efficacy. In the third stage, we use grounded theory to analyze 500 instances of peer
review and interview ten core developers across the six projects. This approach
allows us to understand why developers decide to perform reviews, what happens
when reviews are ignored, how developers interact during a review, what happens
when too many stakeholders are involved during review, and the effect of
project size on the review techniques. Our findings provide insights into the
simple, community-wide mechanisms and behaviours that developers use to
effectively manage large quantities of reviews and other development
discussions.
The primary contribution of this work is a theory of OSS peer review. We find
that OSS reviews can be described as (1) early, frequent reviews (2) of small,
independent, complete contributions (3) that, despite being asynchronously
broadcast to a large group of stakeholders, are reviewed by a small group of
self-selected experts (4) resulting in an efficient and effective peer review
technique. / Graduate
| Identifer | oai:union.ndltd.org:uvic.ca/oai:dspace.library.uvic.ca:1828/3258 |
| Date | 27 April 2011 |
| Creators | Rigby, Peter C |
| Contributors | Storey, Margaret-Anne, German, Daniel M |
| Source Sets | University of Victoria |
| Language | English, English |
| Detected Language | English |
| Type | Thesis |
| Rights | Available to the World Wide Web |
Page generated in 0.003 seconds