Blog Post

Introduction and Plan of Attack

Hi all! My name is Peter Delgobbo and I’m currently an English Literature MA student at Syracuse University. My primary research interests (in the broadest possible terms) include critical theory, Marxism, post-war European film, and digital humanities, along with a some dabbling in new media, speculative realism, science fiction, modernism, history of the book, and contemporary literature. However, I’m not here to bore you with yet another list of research interests; I want to use this introductory post to explain a project I’ve been working on for the last few months and which I intend to further develop in future HASTAC blog posts.

Last semester, I took a class on digital humanities with Chris Forster (my HASTAC mentor) which introduced me to the concept of media visualization, specifically as performed by Lev Manovich. For those of you unfamiliar with his work, Manovich is the founder of the Software Studies Initiative, a working group which uses computational analysis to generate visualizations of gigantic corpuses of images—every front cover of Time Magazine, for instance, or 2.3 million Instagram photos. I was most interested, however, in Manovich’s visualizations of Dziga Vertov’s films. He wrote custom macros for a program called ImageJ in order to create various montages and graphs of frames from Man with a Movie Camera. 

I wondered if I could replicate his results with other films, using a much larger sample size. Instead of using one or two frames per shot, I wanted to use every frame in the entire film (this obviously proved too unwieldy and I ended up using something like every tenth frame). I also wanted to apply Manovich’s method to multiple films by a single director in an attempt to make some claims about the auteur’s visual style, or whatever. Here’s an example of what I came up with.

Manovich would call this type of visualization a “montage.” It represents 22752 frames from Stanley Kubrick’s 1999 film Eyes Wide Shut, arranged chronologically from left to right and top to bottom. I can also use an ImageJ macro to extract metadata from individual frames and graph them on a Cartesian coordinate system.

In this case, the median hue of each frame is plotted along the x-axis, while the median saturation is plotted along the y-axis. You can see the full collection of images generated for this project here.

The main problem with my work so far is that, although it looks pretty, it’s useless. Although you can occasionally come up with something mildly interesting using this method—such as graphing all of the primarily blue shots from Kubrick’s late color films—I haven’t yet been able to produce anything that feels really worthwhile or exciting. Over the next few months, I hope to ameliorate this situation. Here’s a rough battle plan of things I’d like to do with these visualizations:

* Make them smarter. I have an idea of how to use a program called shotdetect to automatically generate folders containing the constituent frames of every shot, which might allow me to make visualizations which communicate more information than color.
* Make them faster. ImageJ is written in Java and takes an astronomical amount of time and computer resources to process each visualization. Python/ImageMagick might be a better option.
* Make them more useful. I know how to visualize color, but how can I visualize time? Composition? Camera movement? Density? Manovich has worked out very small-scale answers to these problems as they appear in Vertov, but I’d like to develop a system that is a bit more universally applicable.
* Make them more interactive. Imagine opening a montage and clicking on a single frame. The rest of the image fades away, leaving only the shot in which this frame appears. Click a button and the shot rearranges itself into a filmstrip. Click another and a list of other shots of similar length in the film appears. Perform a search to find all the other shots in the director’s corpus with a similar color composition. 

Who knows how much of this is possible, let alone practical; nevertheless, this feels like the right direction for this sort of work. I want to make these visualizations into actually useful tools which can help critics analyze art, rather than becoming new, equally impenetrable works of art themselves. Stay tuned, and let’s see what we can do!



No comments