That transformation was behind the recent decision by the prestigious 60-year-old Shakespeare Quarterly to embark on an uncharacteristic experiment in the forthcoming fall issue — one that will make it, Ms. Rowe says, the first traditional humanities journal to open its reviewing to the World Wide Web.
Mixing traditional and new methods, the journal posted online four essays not yet accepted for publication, and a core group of experts — what Ms. Rowe called “our crowd sourcing” — were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal, due out Sept. 17.
The Shakespeare Quarterly trial, along with a handful of other trailblazing digital experiments, goes to the very nature of the scholarly enterprise. Traditional peer review has shaped the way new research has been screened for quality and then how it is communicated; it has defined the border between the public and an exclusive group of specialized experts.
Today a small vanguard of digitally adept scholars is rethinking how knowledge is understood and judged by inviting online readers to comment on books in progress, compiling journals from blog posts and sometimes successfully petitioning their universities to grant promotions and tenure on the basis of non-peer-reviewed projects.
The quarterly’s experiment has so far inspired at least one other journal — Postmedieval — to plan a similar trial for next year.
Just a few years ago these sorts of developments would have been unthinkable, said Dan Cohen, director of the Center for History and New Media at George Mason University. “Serious scholars are asking whether the institutions of the academy — as they have existed for decades, even centuries — aren’t becoming obsolete,” he said.
Each type of review has benefits and drawbacks.
The traditional method, in which independent experts evaluate a submission, often under a veil of anonymity, can take months, even years.
Clubby exclusiveness, sloppy editing and fraud have all marred peer review on occasion. Anonymity can help prevent personal bias, but it can also make reviewers less accountable; exclusiveness can help ensure quality control but can also narrow the range of feedback and participants. Open review more closely resembles Wikipedia behind the scenes, where anyone with an interest can post a comment. This open-door policy has made Wikipedia, on balance, a crucial reference resource.
Ms. Rowe said the goal is not necessarily to replace peer review but to use other, more open methods as well.
In some respects scientists and economists who have created online repositories for unpublished working papers, like repec.org, have more quickly adapted to digital life. Just this month, mathematicians used blogs and wikis to evaluate a supposed mathematical proof in the space of a week — the scholarly equivalent of warp speed.
In the humanities, in which the monograph has been king, there is more inertia. “We have never done it that way before,” should be academia’s motto, said Kathleen Fitzpatrick, a professor of media studies at Pomona College.
Ms. Fitzpatrick was a founder of the MediaCommons network in 2007. She posted chapters of her own book “Planned Obsolescence” on the site, and she used the comments readers provided to revise the manuscript for NYU Press. She also included the project in the package she presented to the committee that promoted her to full professor this year.
Many professors, of course, are wary of turning peer review into an “American Idol”-like competition. They question whether people would be as frank in public, and they worry that comments would be short and episodic, rather than comprehensive and conceptual, and that know-nothings would predominate.
After all, the development of peer review was an outgrowth of the professionalization of disciplines from mathematics to history — a way of keeping eager but uninformed amateurs out.
“Knowledge is not democratic,” said Michèle Lamont, a Harvard sociologist who analyzes peer review in her 2009 book, “How Professors Think: Inside the Curious World of Academic Judgment.” Evaluating originality and intellectual significance, she said, can be done only by those who are expert in a field.
The most daunting obstacle to opening up the process is that peer-review publishing is the path to a job and tenure, and no would-be professor wants to be the academic canary in the coal mine.
Although initially cautious, Mr. Galey said he is now “entirely won over by the open peer review model.” The comments were more extensive and more insightful, he said, than he otherwise would have received on his essay, which discusses Shakespeare in the context of information theory.
Advocates of more open reviewing, like Mr. Cohen at George Mason argue that other important scholarly values besides quality control — for example, generating discussion, improving works in progress and sharing information rapidly — are given short shrift under the current system.
“There is an ethical imperative to share information,” said Mr. Cohen, who regularly posts his work online, where he said thousands read it. Engaging people in different disciplines and from outside academia has made his scholarship better, he said.