Reviewing peer review

The anonymous, closed peer review system has produced too many published errors and is itself under review. The New York Times Magazine article, "Trial and Error" notes that the problems range from outright bias to the newness of emerging disciplines, which use evolving methods that might fall short of the kind of accuracy expected from published science journals.

In contrast, the prestigious British Medical Journal switched to nonanonymous reviewing in 1999 and currently publishes reader responses at each paper's end. It results in a "few bores," who appear to nitpick the results, but so far the benefits outweigh the risks of the completely closed system.

Meanwhile, the completely open Wikipedia, which has suffered its share of criticism for sloppiness, has seen its open source review model validated by a recent Nature article. It found that Wikipedia's accuracy compared favorably to the online Encyclopedia Britannica when it came to science entries. 

The New York Times Magazine article concludes that opening the peer review process "stands to maintain rigor, turn review processes into productive forums and make publication less a proprietary claim to knowledge than the spark of a fruitful exchange."


Technorati tags: peer review, Wikipedia