Blog Post

A digital dissertation technical update: Infinite Ulysses project

This post originally appeared on LiteratureGeek.com in September 2014.

The core functionality of my website is the ability to highlight pieces of the Ulysses text and leave annotations on them (questions/answers, interpretations, contextualizations, notes); to achieve this, I'm building off Annotator.js—Javascript code that lets you do what I just described, with a variety of plugin code that lets you add neat features like a heatmap of where the most annotations appear on a text (e.g. on what pages did readers have the most questions? what pages had the most contentious interpretations?). Since the site itself is built with Drupal, I'm using the Drupal modules already built to let Annotator.js work with, for example, Drupal's database as the place to store readers' annotations.

Not as easy as it sounded (is it ever?)

That the Drupal log under the skull and crossbones.That's the Drupal logo under the skull and crossbones. This was on my home dry-erase board for a while...[/caption] My original project plan was to spend a lot of time coding the look and behavior of the reading interface, with the site functionality not re-inventing various wheels, but instead making connections among existing sets of code that allow textual annotation and social commentary, voting, ranking, etc. In practice, the existing Drupal-module versions of Annotator.js are buggy. Two big examples (for me):

  • the Drupal Annotation modules (Annotation and Annotator) don't store tags added to annotations, so any tags added by a reader disappear forever on refreshing or changing the page (and tags really need to be treated as entities in their own right, and tied into the core Drupal taxonomy); and
  • the code that sets up an individual textual annotation to be an "entity" (a Drupal type of content—such as a blog post, book page, or comment—that other existing Drupal modules automatically know how to act on) isn't completely working either (e.g. you can't add fields—such as a field to add tags to an annotation, fixing the tag storage problem—with the current code).

Having everything not go as smoothly as planned was actually great, though! I had a realistic goal in terms of what I needed to get done by the end of the summer to stay on track (have the site's main functions almost all working, ready to be completely finished at the end of September), and I wasn't worried about doing different work than expected to achieve those goals. Having the Drupal annotation modules need more work than expected meant I got to spend the summer trying various things to solve these issues, and learning a lot in consequence—like creating non-basic custom Drupal entities. For example, I learned how to make a custom entity fieldable (able to have fields, like a place where you can add tags and have them autocomplete if they match existing tags) and add some fields to it, both by writing code ("programmatically") as well as via the Drupal admin interface (after making the annotation visible to the core Field UI module). I also read through the Annotator code so many times that I became much more comfortable with it, so I'll be able to augment it in the future far more easily. In the end, it looks like I'll be building off the code fix created by another DH developer who was kind enough to start an online conversation with me about our mutual work improving textual annotation in Drupal (more on that project once they've decided to make their code public). I went back and forth about whether I wanted to build off their solution or keep hacking at my own, but in the end I hadn't been expecting to need to make those code fixes in the first place, so I decided I'd keep my remaining coding time this year free for expected work with the actual reading interface, like making a heatmap of the annotations and porting other existing Javascript/Coffeescript plugins to work with the Drupal version of Annotator.

Other summer work

Screengrab of many many tabs open across the top of my browser. Screengrab of many many tabs open across the top of my browser. A fairly normal view when I'm coding![/caption] Patching. I submitted an extremely simple patch to the Drupal Annotation module (fixing a misspelled variable) as a way of learning the process; my patch was accepted into the module. Now I know how to create patches when I want to share further work with others using Drupal Annotation. Installation profiles. I'm currently working on two separate local versions of the Infinite Ulysses website: one to work on improving the Annotation function code, and one for work on everything else (e.g. user profiles, site content, theming). As I work on the site's annotation abilities, I fairly often want to reinstall the site as a slightly quicker alternative to weeding all the references to annotation code out of the database (I should set up uninstall hooks to do this for me eventually...), or to see whether the code works correctly when installing the modules into a new site. Keeping the annotation work on a separate site lets me do this without losing my other work on the site. But! I don't want to start from a standard Drupal blank site every time I need to reinstall my annotation code; I want a sample page of Ulysses to test the code on, user permissions already set up, modules installed, and various other things configured that I'd rather not do manually every time I reinstall the site. Installation profiles are a way to avoid starting from square one with a Drupal site, instead automatically pre-configuring the site as you'd like it to be (think of it like a module that tells the site to configure and install things a certain way during the site installation process). When I visit install.php, I now have a "Infinite Ulysses Pop-Up" option in addition to standard or minimal Drupal installations. Eventually, this profile will turn into a distribution, such that anyone can use it to install a Drupal site pre-configured to act as a participatory digital edition. Annotation sidebar. The Annotator.js code I'm working with displays annotations as pop-ups hovering over the text, but from experience with my UlyssesUlysses prototype and usability research, I prefer having annotations authored and displayed in a sidebar. This will also be handy for the tablet version of the website, as you'll be able to slide the annotation sidebar off and on the page like a drawer. I've been playing around with a couple ways of achieving this: modifying this existing CoffeeScript or this extension of that code (demo'd here) to work with Drupal, creating the sidebar by resetting the current CSS on annotation pop-ups and styling them into a sidebar, or a tactic such as this repo which purports to display annotations as Drupal comments. Book-page styling. I've been vacillating a bit as to the final display structure of the novel: I'm pretty sure I either want to have one page per chapter of the book and infinite scroll to put all the text of a chapter on one page, or have separate pages to match up with each page of the 1922 text of the novel (which I'm using). This is something I might end up doing A/B testing on, showing the text to some visitors one way and to other visitors the other way, and comparing their reactions and stats such as time spent on the site. Here are some questions I'll need whatever solution I use to address:

  • How many annotations can load on a page at a time? How does this limit or non-limit affect site speed?
  • How do annotations load on a very long page that only loads partial sections of the text at a time (when they're scrolled to, i.e. infinite scroll)?
  • How will bookmarking on the site work—will a reader's last visited and furthest page in the book be marked as where they should start next, or will users have the option (or necessity) to bookmark one or more pages themselves?
  • If I do offer one long scrolling webpage per chapter of the novel, can I easily generate bookmarks that jump a reader down to the last line they were reading?
  • What other ways can users chart their progress through the book, such as badges per chapter, or a progress bar for pages read?
  • What if readers want to read it out of order, or only some parts online (and mark parts they read offline as completed)? What if they want to glance ahead without losing the bookmark at where they actually want to pick up reading next?

Long-term thinking. I met with the artist and IP lawyer behind the excellent web/iPad comic of Ulysses (Ulysses Seen). We're all keenly interested in opening Ulysses to more public reading and conversation; they want a way for comic readers to be able to jump from the comic to the text, and I'm also interested in supporting the comic's readers in enjoying the text—so we've been discussing combining our websites into one super-accessible, textual and visual Ulysses edition, after I've defended my dissertation. This is also a good way for me to think about how I'll manage hosting costs and moderation effort in the future. Other summer activities included lots of reading (as much as I read, I'm always adding more links to my reading list every time I go on Twitter), researching the social modules I want to use on annotations, and drafting site content. In particular, I've been looking into how existing communities (especially online ones where people submit text and/or converse) have succeeded in building friendly spaces, self-moderating, creating community guidelines, and fostering an inclusive and welcoming atmosphere. Would you like to volunteer as an early user of the Infinite Ulysses site? Sign up here!

109

2 comments

Just a brief note (and a brief kudos on not yet throwing out Drupal and rolling your own) -- I haven't really kept up on this for a while (read: 3-5 years) but my guess is that infinite scrolls and infinite pages aren't too compliant with ADA / 508 suggestions. Though I can use a mouse easily, I generally find myself on a laptop with just my trackpad + keyboard, and using only the keyboard except when I have to or I'm doing something else (think eating something while scrolling through a page to look for something). At the very least, it's a slight inconvenience for me; for others that rely entirely on keyboard use or screen readers, it may be a much bigger issue. This becomes a more pressing issue for those that may want to use the project in class. 

Again, older Drupal user (I quit around when 7 was going stable), but Drupal tends to run quite slow already. The infinite scroll on a page could cause some interesting hiccups with multiple queries to annotations in the DB. Perhaps this could be allievated with either more hardware or some clever page-caching, but I don't know anything about that subject matter.

All that said, still really interested in seeing where the project goes. I think it'd be a great resource. 

130

Hi, Jesse! Thank you for reading and for your really helpful response.

Keyboard-only use of the site is an important feature, and the keyboard vs. infinite scroll issue you raise is a great point. I was planning on conducting the "mouse-bucket challenge" suggested here as part of my work once my final coding on the annotation behavior was done, and it's great to be reminded of that (I've made some coding and layout decisions since I originally wrote/posted this back on my personal blog). I was originally thinking I'd do some A/B testing to figure out what page layout readers preferred, but I think there are just too many issues with infinite scroll—the huge issue of accessibility you mention, auto-creating bookmarks for the last page read, and preventing the terrible accidental scroll when I get around to releasing the tablet version being some other issues.

There's something to be said to syncing the online book to the individual pages of (one of the many possible editions of) Ulysses rather than a long scrolling page, too. The project already has some other big accessibility challenges—screen-reading annotated text, and accessibility for a site founded on a Javascript tool—so it's nice to avoid other accessibility roadblocks like infinite scroll where I can. I've found user studies and universal design work on library catalogs to be really applicable to digital edition design so far—and I'm hoping my dissertation can increase the resources for others trying to design accessibily for the digital humanities. 

103