Blog Post

From Cardi B to Stranger Things: Reading Sounds, Captioning Meaning

closed captioning icon

This is the third post in a series leading up to my HASTAC webinar “From Cardi B to Stranger Things: Using Popular Culture in the Writing Classroom.” Disability Studies and Disability Rhetoric ask us to rethink how we arrange our classrooms (physical and virtual) and the course content we provide. Rather than a reactive measure, I have started to look at the insight of disability studies/rhetoric as an opportunity to rethink how I approach teaching "meaning-making" in my courses. For example, providing alternative text for images not only makes web content available for students using screen-readers, it also offers an opportunity to think differently about the images we choose to include in our multimodal compositions. Additionally, I now provide more audio content to compliment my written content, and I allow students to complete responses using audio/video more freuqently. This raises questions I was not remotely asking, such as: Why am I drawn to a particular image in my blog posts and slides? How do the images I have chosen to include "work" with my text to create meaning? I also found myself asking: Why does an audio transcript have to be "boring"? Just how bad is YouTube captioning? How hard is it to make my own captions? What is it like to navigate a blog post using a screen-reader? As much as we may feel we are a "visual culture," the Web itself is awash in text (it always has been). What is interesting about text specifically is that it does not have to be such an "ocular-centric" experience, text lends itself well to repurposing with various technologies to reach across various modalities. As with my previous lesson ideas, this ultimately comes down to students thinking about their relationship with their texts (broadly defined) and their various audiences -- attending to these various experiences not only makes their writing more accessible, it makes for more thoughtful and engaging compositions.

Origin: Metis and Reading Sounds

My second year in my program, I presented an earlier article by Jay Dolmage that eventually developed into his foundational text, Disability Rhetoric. Dolmage positions Metis, "cunning intelligence" in the Ancient Greek sense, as an embodied knowledge and rhetorical sidestepping of hegemonic conceptions of normalcy. Put differently, bodies are "disabled" only according to dominant culture's ever-shifting definition of normalcy. Disability, then, is not a deficiency but rather a different way of knowing, one that considers what "goes without saying." I think of metis as asking us to attend to those things we accept without thinking of it, those day to day moments that we go on "auto-pilot" because everything is working for us as expected and without incident. I can certainly give someone an answer for why I made these nonconscious decisions when asked to, but I am not readily attending to these reasons in my routine thinking.

Often times when we think of disability, the popular image is wheelchair access to a building; the physical barriers that make it harder and/or impossible for some of us to navigate spaces. But access works in many different ways. Sean Zdenek's Reading Sounds is an important project that looks at the world of captioning and the rhetorical choices captioners make that inevitably shape the meaning deaf and hard-of-hearing film/television viewers can make. Often times this important work is an afterthought, something outsources to a company after the video is shot and edited rather than something considered from onset of the production. That is not to say there aren't a few notable exceptions like Timur Bekmambetov director of Night Watch (2004).

Gif image of the more dynamic captions in a scene from the film Night Watch (2004) as discussed below.

Figure 1. Gif image of the more dynamic captions from a scene from the film Night Watch (2004)

While in traditional captioning, we would have white text on a black border appearing in the center of the screen, what Bekmambetov did was make his captions a crucial part of his scene design and post-production (he did the captioning himself). In the scene above we have a character being thrown against a wall; his screams NO! NO! STOP! move with him and break apart themselves from the sheer "momentum" of hitting the wall. The whole thing feels like a comic book, so much so that I don't feel like I am losing anything in not having the actual sound. The whole thing reminds me of Scott McCloud's fascinating book Understanding Comics.

As it pertains to the writing classroom, Zdenek is calling for different kind of intertextuality, one in which captioners develop important cultural literacy to use in their creation of captions:

Cultural literacy for captioners is the ability to be an ideal reader of the text, to recognize the intertextual associations hat every text creates with prior text. Not text is an island...[and] Allusions can be subtle and easy to miss...when you don't have the cultural literacy to recognize them. (221)

When I think of my students, I think of the wealth of knowledge and experience they already bring to the classroom--the various "texts" (books, posts, films, songs, etc) they bring with their thinking in the classroom. And yet, these are things they don't regularly, consciously think about but certainly draw on and deploy when they make meaning.

Using Captioning to Teach Meaning-Making

This lesson has a few different phases to it and can therefore be stretched across multiple class periods (or not!). The first phase examines how image and text works together: how changing the text provided with an image can change our uptake of that image. I usually have students read this excellent blogpost about image descriptions by Mel Finefrock. Finefrock focuses on our reasons for selecting the images we include in our digital compositions, something that not only would help blind readers but anyone engaging with their content. In class, I have students practice changing the meaning of an image by changing the text presented with it. I find children's books to be a fun and approachable way to engage with this practice.

Slide featuring the image and text from the children's book 'Where the Wild Things Are' as discussed below.

Image of a slide that asks the question "How do these captions change how we view max" and provides the same scene from 'Where the Wild Things Are' but with different captions as discussed below.

As demonstrated in the slides above with a scene from the children's book Where the Wild Things Are, in the original text, Max is kind of a terror. In this part of the book he is clearly jumping off the stairs and towards his dog, fork in hand. This image is meant to illustrate his disruptive behavior, something that aligns with his mother calling him a "Wild Thing" and then sending him to bed without dinner. Perhaps the punishment fits the crime in this case. But what if we change the text to say something like "His mother called and he excitedly went to dinner" or "And he chased the wicked Dog that bit his sister" how does that change the way we interpret Max as a character? I'd argue he is still "Wild" but less of a terror and more of an energetic, but not necessarily menacing, child. Regardless, there is an interesting relationship between image and text here, one that impacts our creating meaning out of the story. From here I provide students with other examples from children's books and task them with coming up with captions that noticeably change our uptake of the scenes depicted. Alternatively, I have students examine internet memes to the same effect and them have them create their own.

The next phase of the lesson has students engage with video and captioning. Sometimes as a bridge, I might have students add music to their image/text compositions from earlier to see if they can change the uptake of their compositions yet again by adding sound (light music for levity, industrial music to imply something sinister, etc). The subsequent image/text/sound compositions provide a lot of space for critical play. Regardless, before captioning I have students read a few posts from Zdenek's blog where he accounts for his experiments in more creative captioning (his "Subtitles As Visual Art" OR "Cripping Close Captioning" are good primers). We then discuss some of the interesting best practices he suggests, and I demo captions I created for a short scene from a movie using the VLC Media Player and a Text Document (see this Tech Junkie article for instructions on how to do this). Captioning takes a long time to do, especially if you want to do it well. For this reason, I find short video clips (5 mins and under, the shorter the better) that I have groups of 2-3 students practice captioning. There should be some dialogue and other sounds (music, ambient noise, etc.) to allow students to practice prioritizing which sounds to caption when you can't caption everything. Teaser trailers and short television clips work really well for this and are a small download (see the Thor Ragnarok teaser trailer below for a good example). You can give all groups the same video clip or have each group work on a different one (I have had success with both arrangements).

Teaser Trailer for the film 'Thor: Ragnarok'

While two minutes of video may not seem like a lot, this could take the majority of a 50 minute class period. To help with this, I usually create a subtitle text file (called a .srt file) template for students where they only have to fill in the subtitle itself and the time signature range it will appear on in the video. Also keep in mind that students will be watching their video over and over again, so make sure they have headphones (and that it is something they can watch multiple times without losing their cool). Not only does this activity help students to become more aware of the labor it takes to create accurate and helpful captions, it also helps them to become far more aware of all the different sounds they regularly rely on to make sense of the video clip. Not just how the music establishes a mood but how ambient/background noises helps us to understand continuity from scene to scene, one moment to the next. Students start thinking about how long to have dialogue on the screen and which sounds are important for setting up other scenes/moments later on.

Image of a slide listing the best practices for captioning. 1) Use a template. 2) Add Time Signatures after determining captions 3) Watch it through and note where your captions linger too long or disappear too quickly 4) adjust time signatures 5) Share with another group.

Essentially, I give space for students to think about how we use various modalities together to make meaning. How image, text, and sound work together to make meaning in ways that we do not always consciously attend to. I stress that accessibility is not simply a retrofit, a quick fix and afterthought, but rather represents a unique opportunity to help them become critical and attentive content producers that consider the various experiences different audiences will have engaging with their writing. This all works to provide more angles for them to consider their compositional design choices from in addition to critically examining the unaddressed able-ism that permeates the popular culture they regularly engage with.

The Lesson Plan

Components: VLC media player, Text File, and downloaded copies of short, 1-5 minute video clips (movie trailers work well).

NOTE: headphones would also be a good choice as they will be watching their clips several times.

Procedure: In pairs (or up to 3) students will caption the essential sounds from their trailer. As a class view each captioned trailer and/or have groups swap their captioned trailers.

Discussion: Which sounds ended up being the most essential to meaning-making? What additional things did you notice about the video that you didn’t before?

Dolmage, Jay Timothy. Disability Rhetoric. Syracuse University Press, 2014.

Zdenek, Sean. Reading Sounds: Closed-Captioned Media and Popular Culture. The University of Chicago Press, 2015.



No comments