2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
ISSUE NO. 4
Rethinking Peer Review in the Age of Digital
Humanities
Roopika Risam
For academics, double-blind peer review processes remain the gold standard for
validating scholarly work. The value accrued by scholarship has traditionally flowed
mono-directionally from peer review. In the hierarchies that govern academic hiring
and tenure and promotion practices, the single-authored monograph from the
distinguished scholarly press sent out for review upon completion occupies a position
of prominence. Among shorter forms, the prestigious academic journal provides
readily legible markers of academic quality. Yet, for scholars working in digital formats
or within digital humanities, conventions governing the gatekeeping of “scholarly”
work feel increasingly mismatched to the digital milieu. Therefore, digital scholarship
requires consideration of the factors distinguishing it from print scholarship, along
with a new approach to validating scholarship that emerges from and respects the
specificities of digital work.
Rethinking peer review in the age of digital academe is a task that goes beyond the
question of medium or platform to a question of epistemology. That which we call
“digital scholarship” is not simply print scholarship gone digital but raises questions of
genre and gives rises to its own conventions. One of the first significant interventions of
digital platforms for academic work was the rise of “e-journals.” These academic
journals, published electronically rather than in print, generally mirror print journals
in both issue structure and article format. E-journals brought with them a host of
responses, from skepticism to excitement[1], accompanied by concern that recourse to
the digital would decrease scholarly merit – as though there were little distinction
between an academic journal online, with a review board and review process, and a
blog. Yet, scholarly publishing has responded to the affordances of the digital: we have
seen greater interest in open access and a boom in new journals that use digital
platforms to distribute articles ahead of the publication lags that accompany print
journals. Yet, many e-journals reproduce the hierarchies and values of print knowledge,
relying on traditional notions of what academic working looks like.
On the other hand, “digital scholarship” is its own animal, a chimera that defies the
conventions of print scholarship. Three principle differences between digital and print
scholarship in the humanities require a radical revision to how we review and assess
scholarly production and to how scholarly work accrues value: digital scholarship is
https://adanewmedia.org/2014/04/issue4-risam/ 1/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
often collaborative, digital scholarship is rarely finished, and digital scholarship is
frequently “public.” Each of these qualities of digital work invites particular concerns
for review, rendering digital scholarship not readily legible to tenure and promotion or
hiring committees.
1) Digital scholarship is often collaborative. 
Whereas creation and distribution of print knowledge in the humanities is usually a
solitary task, digital scholarship is often collaborative and challenges the familiar image
of the print academic in her hermitage, toiling in obscurity. The relatively greater
frequency of collaboration in digital scholarship is a function of platform. Producing
digital scholarship requires us to draw not only on the interpretive skills in which we
were trained but a range of skills we may not possess, particularly as they pertain to the
use and development of digital components of scholarship. Moreover, digital
scholarship requires perhaps the most valuable of commodities for academics: time.
Currently, I co-direct Digitizing Chinese Englishmen with Adeline Koh of Richard
Stockton College. Koh initiated the project, which addresses colonial silences in 19th-
century digital humanities scholarship by making available the Straits Chinese
Magazine, a colonial English-language magazine published in Singapore from 1897-
1907. Working alone, Koh found the project moving slowly. Given the nature of digital
scholarship and its questionable value for tenure and promotion, those of us who work
in the digital milieu often find we must steal time from tenure and promotion-worthy
work to advance our projects. These challenges are even more acute for those of us who
work outside of universities with very high research activity, with higher teaching loads
and few institutional resources to support our research. In light of these challenges,
Koh invited me to co-direct the project with her. I brought to the project not only my
time and labor, but also my own skill set in TEI. Beyond project directors or principal
investigators, other participants – programmers, graduate students, interns – often play
critical roles in the creation of digital scholarship but traditional citation and reward
structures of academe are not configured to acknowledge these contributions. Despite
the efficiency of combining skills and human capital, collaboration, while expected in
scientific or social science fields, raises concerns within the humanities, particularly
over how to evaluate individual contributions to collaborative projects for tenure and
promotion.
2) Digital scholarship is rarely finished. 
The adage for traditional scholarship is that it is never finished – one simply finds an
acceptable stopping point. While this is true of digital scholarship as well, digital
https://adanewmedia.org/2014/04/issue4-risam/ 2/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
projects may exist in phases, may be perpetually in-progress, and may never have an
acceptable stopping point. The end point for digital scholarship is frequently a moving
target, and there is often no single event in the process of creating digital scholarship
that is comparable to the act of submitting a manuscript for review. Therefore, digital
projects require new approaches to linear conventions of scholarly time. For example,
completion of a phase of a digital project may be an appropriate moment for
evaluation. Yet, such assessments will unlikely be the conferral of a mark of scholarly
validation on the project but the source of additional information and feedback that
might, in turn, be folded into the next phase of the project. Further, digital projects are
difficult for search committees or tenure and promotion committees to compare to
print scholarship because a CV line identifying a digital project cannot adequately
communicate the amount of intellectual labor the project represents. Digital projects
may require ongoing efforts for sustainability and preservation, whereas books and
articles are presumed to be finished. The nature of intellectual labor itself comes into
question as digital scholarship requires us to account for continued time, new versions,
and preservation efforts that are less relevant to the production of traditional print
scholarship.
3) Digital scholarship is frequently public. 
Although more academics have taken to blogging and social media to share, promote,
and collaborate on their work, scholarship in the humanities relies largely on private
labor taking place behind the closed office doors. Public components of print
scholarship are prescribed by academic ritual: invited talks, symposia, conference
panels. The “public” nature of these acts of scholarship have been subject to question as
well, with academics arguing that a conference talk is not, in fact, “public.”[2]
Conversely, digital scholarship, particularly projects engaging web-based
infrastructure, often become public early in development. This is in part because of the
affordances of a web presence for digital work, the orientation of digital humanities
work towards praxis, and the fact that many tools and platforms are web-based.
Whereas a scholar producing a print article can delay making the work public for as
long as she wishes, there is often relatively less agency for the digital scholar.
Differentials of privacy and publicness for print and digital scholarship cleave to the
levels of status and prestige they accrue. Print scholarship, usually not open access,
lives within academic spaces, behind gatekeepers like paywalls, databases, and library
archives. Its privacy is guaranteed by barriers that accrue capital for distributors.
Conversely, digital scholarship may be relatively more easily available, without the
mediating force of transactional capital preventing access. The monetary barriers to
https://adanewmedia.org/2014/04/issue4-risam/ 3/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
accessing print scholarship accord it quantifiable financial value – albeit one that
provides returns for the gatekeeper rather than the author – and with financial value
print scholarship accrues intellectual value. Thus the “publicness” of digital scholarship
grants it relatively less value in the academic machine.
Based on these descriptors, digital scholarship threatens to displace a benign sort of
academic discourse that does not trouble the value and status of print knowledge. What
we must guard against in the creation and evaluation of digital scholarship, however, is
reproduction of the fetishism of print that undergirds academic disciplines. We cannot
assume that the standards of traditional scholarship can be easily translated for digital
scholarship. Scholars in arts and media have been raising related issues of evaluation
for decades. Those of us invested in digital scholarship must build on historically
radical moves to continue troubling the relationship between print scholarship and
academic status and value.
Digital scholarship, therefore, is best understood as part of an ongoing trend in
academic discourse prevalent enough to require rethinking the production of academic
value. Yet, we must be wary of fetishizing the digital as well. As the work of scholars in
conversation with communities like #transformDH, Global Outlook::Digital Humanities,
and Postcolonial Digital Humanities have argued, digital humanities risks reproducing
gaps and silences in knowledge production around issues of difference and is not
inherently free from biases. Even among traditional funding streams for digital
humanities, such as the National Endowment for the Humanities, digital projects on
canonical topics tend to receive greater sanction. Therefore, we need to attend to the
ways that digital scholarship itself risks perpetuating more conservative elements of
disciplinarity and canon as we look for ways to evaluate digital projects.
There is, in fact, growing interest in making the value of digital scholarship legible to
tenure and promotion committees and search committees [3]. For example, we might
look at tracking citations, grants, and usage statistics. These metrics demonstrate
engagement with digital projects in ways that are not necessarily available for print
scholarship. On the DHCommons editorial board, we are working to develop a review
journal for digital projects that takes into account the specificities of digital scholarship.
Anvil Academic, a partnership between CLIR and NITLE, is pioneering a platform for
digital publishing that would be an indicator of peer review for digital projects. Ada
itself forges new directions in peer review using both expert beta readers and
community review. Journal of Digital Humanities experiments with peer review by
https://adanewmedia.org/2014/04/issue4-risam/ 4/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
drawing content from Digital Humanities Now before review begins. #DHTHIS provides
a platform for crowdsourced evaluation of digital work. Efforts continue in many
directions to define evaluation for digital work. Perhaps the most important insight
when encountering digital scholarship, however, is to attend to the particulars of the
digital – the possibilities for collaboration, new approaches to completion, and their
public nature – viewing these not as limitations but affordances.
—CITATION— 
Risam, R. (2014) Rethinking Peer Review in the Age of Digital Humanities. Ada: A
Journal of Gender, New Media, and Technology, No.4. doi:10.7264/N3WQ0220
(http://dx.doi.org/10.7264/N3WQ0220)
This article has been openly peer reviewed at Ada Review
(http://adareview.fembotcollective.org) .
 (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en_US)  
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs
3.0 Unported License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en_US)
Footnotes    ( returns to text)
1. Howard, J. 2010, Wed. April 7. “Scholars Increasingly Embrace Some,
but Not All, Digital Media.” The Chronicle of Higher Education.
http://chronicle.com/article/Scholars-Increasingly-Embrace/64982/
(http://chronicle.com/article/Scholars-Increasingly-Embrace/64982/)
2. Kolowich, S. 2012, Tue. October 12. “The Academic Twitterazzi.” Inside
Higher Ed. http://www.insidehighered.com/news/2012/10/02/scholars-
debate-etiquette-live-tweeting-academic-conferences
(http://www.insidehighered.com/news/2012/10/02/scholars-debate-etiquette-live-tweeting-
academic-conferences)
3. Modern Language Association. 2012. “Guidelines for Evaluating Work
in Digital Humanities and Digital Media. Modern Language Association.
https://www.mla.org/guidelines_evaluation_digital
(https://www.mla.org/guidelines_evaluation_digital)
PEER REVIEWED
https://adanewmedia.org/2014/04/issue4-risam/ 5/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
Roopika Risam (https://adanewmedia.org/author/rrisam)
Roopika Risam is Assistant Professor of English and English Education at Salem State
University. Her research examines postcolonial and African diaspora literatures and the role
of digital humanities in mediating between them. Her work has recently appeared in Left
History, Digital Humanities Quarterly, and First Monday and is forthcoming in South Asian
Review and DSH: Digital Scholarship in the Humanities.
7 THOUGHTS ON “RETHINKING PEER REVIEW IN THE AGE OF DIGITAL
HUMANITIES”
Alex Gil
APRIL 27, 2014 AT 12:16 PM
This is one of the clearest and most succinct states of the union on re-
viewing digital scholarship. It hits most of the points that came up during
the MLA2014 panel led by V. Szabo on Evaluating Digital Scholarship:
Candidate Success Stories.
I would add one more genre of evaluation and that is the NINES/ARC
model which has been around now for more than a decade, if I know my
history well. In order to get aggregated into that model you need to pass
a thorough review. They are the still viable precursors of the ones you list
above.
One model I would like to see, (and I cannot conrm nor deny that I’m
working on it) is leveraging the legitimacy and peer-review networks of
existing prestigious journals to shift sub-disciplines that are beholden to
them.
Pingback: Hybrid Pedagogy, Digital Humanities, and the Future of
Academic Publishing - Hybrid Pedagogy
Pingback: Introduction: Publication & Its Discontents | Ada v2 Design Test
Responsive
Pingback: What is Digital Humanities? (in 400 words or less) | A Wee
Blether | A Wee Blether
https://adanewmedia.org/2014/04/issue4-risam/ 6/7
2/24/2021 Rethinking Peer Review in the Age of Digital Humanities - Ada New Media
Pingback: #MLA15 Presentation » Digitocentrism
Pingback: Tales from a Silver Medalist: Publishing an Interactive,
Collaborative Article in JITP |
Pingback: Hybrid Pedagogy, Digital Humanities, and the Future of
Academic Publishing - Hybrid Pedagogy
Copyright © 2012-2021. All work on this website is distributed
under a Creative Commons license. The default license for the
content on Ada is a Creative Commons Attribution-
NonCommercial-NoDerivs 4.0 Unported License. Individual
article copyright terms may differ. Please refer to each article for
its license.
Ada: A Journal of Gender, New Media, and Technology 
ISSN 2325-0496
https://adanewmedia.org/2014/04/issue4-risam/ 7/7