Blog Post

DH User Testing: What's Been Done? (Adding to a Growing List of Resources)

Today's post over on my blog is "DH User Testing: What's Been Done? (Adding to a Growing List of Resources)". I'm not going to cross-post all of it here as it is footnote heavy, and my WP footnote formatting doesn't play nice with Drupal (on which this site is built)... but it's basically a curated list of research (DH and not) related to DH user testing, sorted by focus and field and including some comments on the state of digital evaluation. I haven't gotten my more recent researches into DH user testing into sharable shape yet, so I'd be happy to hear in the comments about any more recent (last three years) DH user studies (as well as interesting earlier work I've missed).

I've also been moving the DH user testing citations from my Zotero library into the impressive "Assessment and Evaluation" collection in the Zotero Digital Humanities library Dan Cohen started--that's another great resource to check out.

Here's to finding out more about the usage of our scholarly efforts!

“We have little empirical data about how these resources are being used… the general lack of knowledge about level and quality of use… has been identified as a pressing concern.”- 

- Harley, D. et al. (2006). Use and Users of Digital Resources: A Focus on Undergraduate Education in the Humanities and Social Sciences. Center for Studies in Higher Education.



Good topic! I've been working through this same literature recently, although from a slightly different angle. I work on a project focused on how a couple different user populations use and evaluate digitized resources (rather than DH projects or tools) emerging from projects like Google Books. I've found very few research studies that focus specifically on this particular use (beyond Sukovic, who is on  your list). There's also a lot of slippage in what types of projects, tools, and resources are being evaluated in these studies, and what kind of users.

Here are a few others:

Warwick, C. L. H., Terras, M., & Nyhan, J. (2012). Digital Humanities in Practice. Claire Warwick's first chapter, "Studying users in digital humanities," is available as a pdf. It's mostly lit review, wtih a few UK projects given at the end as case studies (e.g. the LAIRAH project), but it might be another good starting point for thinking about users and use...

Gibbs, F. & Owens, T. (???) "Building Better Digital Humanities Tools." Says "forthcoming in DHQ"...

Meyer, E. (2011). Splashes and Ripples: Synthesizing the Evidence on the Impacts of Digital Resources. London: Joint Information Systems Committee (JISC) Report.

There's also a lot of stuff around humanists' research practices, and the use or non-use of electronic resources...




Thank you for these, Melissa. What is being studied (editions? archives? websites? tools?) and who is being studied (scholars, students, library partons...) are definitely tricky things with such research. (And as I'm posting about later this morning, even what user testing means--use, usability, and usefulness--or other values--isn't always clear/different factors are valued by different people.)

I tried to show this variance a bit with my post by dividing up my limited citations by library users, scholars, etc., but I think it would be really neat to take the DH Zotero Assessment/Evaluation collection once it's better populated and tag the different citations under categories for form, user type, and what factor is being analyzed.

I noticed you're working with Paul Conway--he was my advisor too! (For the M.S.I., not the Ph.D., though.) I'd enjoy reading more about your work.