As stated in the call for the 4th Annual Digital Media and Learning (DML) competition (Badges for Lifelong Learning), digital badges present an opportunity to design new systems in which “achievements are inspired, recognized, translated across contexts, and displayed and managed across the web.” Already, it seems clear that the employment of effective badging systems could represent a critical focus for future digital media and learning innovation. In particular, badges can serve as both inspiration for future learning in informal learning environments, and as recognition of expertise in ways that are not easily captured with traditional means of assessment. And yet, as Halavais (2012) argues, "badging" necessitates ethical considerations; it is critical to understand how these systems have salience and are contested within the learning communities they serve.
Thus, this proposal focuses on understanding nascent badging practices “in the wild” (Hutchins, 1995), in order to connect learning practices of “just plain folk” (Lave & Wenger, 1992) to actionable design directives. While the current work on badges presents many enticing models for driving change in assessment and credentialing, it is still unclear how specific badging designs may work within the informal contexts that designers seek to connect to. There is still a potential gap between the designed systems of many badge projects and the informal, collaborative, contentious, and interest-driven (Ito, et al., 2009) forms of learning that pervade online participatory culture (Jenkins, 2006). We need to close this gap, grounding the design and development of future badging systems with empirical studies of expertise in interest-driven affinity spaces (Gee, 2004; Hayes and Duncan, 2012).
The proposed project is a 1-year investigation of badging and recognition systems in online affinity spaces, consisting of a multi-level analysis of the functions of social expertise within these communities. By capturing, characterizing, and categorizing how knowledge and expertise are intertwined within these sites of learning, the aim is to develop grounded recommendations for the design of future badging systems. Questions to consider include: How are recognition systems already implemented in online participatory cultures? In what ways other than overt badging systems is expertise signified and shared within online spaces? How are social forms of expertise within these spaces related to potentially concomitant constructs, such as legitimacy, credibility, and trust? How can the continued study of online affinity spaces inform future badging systems?
As part of a larger effort to understand digital gaming as a key site of digital media and learning practices, Steinkuehler & Duncan (2009) elaborated the informal discursive and scientific reasoning practices that were part of “gamer talk” in online contexts. Duncan (2010a; 2010b) further extended this approach to investigate design thinking. Common to these approaches was a dual-level coding methodology (Duncan, 2010c) in which both the content coding of verbal data (Chi, 1997; Weber, 1990) and d/Discourse analysis (Fairclough, 1995; Gee, 2010) approaches were applied, with the intent of first identifying generalizable practices, then deeply characterizing individual cases of meaning-making.
This proposal employs a similar analytical method to investigate six online affinity spaces, focusing explicitly on the social nature of expertise within these spaces. Sites for the study will range in form (e.g., social news sites, social media networks, online discussion forums) and content (e.g., educational reform communities, hobbyist networks, social change initiatives). With the goal of determining how recognition systems are implemented and social expertise is otherwise employed within these spaces, a wide variety of sites for investigation will be entertained.
The project will consist of three major stages: (1) A determination of sites of research, identifying existing formal and informal recognition systems present in affinity spaces; (2) collection of data, pilot work to develop grounded codes for capturing social expertise within these spaces, followed by the application of the coding scheme to the data corpus; and (3) the development of design recommendations, the release of a project white paper summarizing the recommendations, plus outreach to the DML community through the online public forum. The proposed project would be conducted between May 6, 2013 and May 31, 2014.
Stage One: May, 2013 through September, 2013
The initial task will be identification of sites under study. Sites will be prioritized for study where recognition systems have already been implemented as markers of expertise (e.g. “beta readers” in many fan fiction communities a la Black, 2007, or structures that support professional programming mentorship on sites such as stackoverflow.com). Additionally, informal displays of expertise found in popular affinity spaces will be investigated for their potential to drive reconsiderations of recognition systems. These potentially include:
• Reddit — the “front page of the internet,” a popular central social news site. Reddit features multiple “subreddits," topical discussions ranging from knitting to politics to computer programming, in which experts and amateurs regularly interact
• Twitter — the premiere “microblogging” social media site. Networks within Twitter include clustered conversations around individuals/organizations, active ongoing hashtags (e.g., #edchat), and popular Twitter lists
• Online discussion forums — While Gee’s (2004) original conception of the affinity space was built from analyses of gaming-related discussion forums, discussion forums of many hobbyist topics (from car repair to music fandom) will be entertained.
Additionally, from August, 2013 to September, 2013, the project would host a survey on the online public forum, distributed through the DML community to solicit additional suggestions of recognition systems and sites for further study. During this stage (August, 2013), graduate student researchers will join the project for the remainder of Stages Two and Three.
Stage Two: October, 2013 through February, 2014
After the determination of six sites of study, pilot studies will be conducted on anonymized text culled from each site. Data corpuses will consist primarily of affinity space textual interactions, with additional features of posts (date, time, quotes/retweets) and contributors (avatar choice, posting metrics) also captured for future analysis.
Pilot studies will familiarize the researchers with each affinity space, as well as the intellectual tasks engaged upon by members within each (a la Steinkuehler & Chmiel, 2006). From these pilots, an initial coding scheme will be developed to capture how social expertise is enacted within each space (e.g., how expertise and knowledge are intertwined, how the social space recognizes it, and how identity is performed within each).
The specific means of determining social expertise for each space will be developed through a bottom-up process of qualitative coding, with the goal of first identifying methods of signifying and sharing expertise within interest-driven learning spaces, then a deeper look at the "microstructure" of expertise within them. Several driving interests and concerns will guide this process, focusing on constructs that have been shown to be important in understanding engagement in interest-driven affinity spaces. Foremost will be the identification of the mastery of skills, content, and social facility. Do participants in affinity spaces recognize such mastery, and if so, in what ways does the display of mastery shape and influence learning?
Beyond mastery, other factors will be considered, including:
• Discourse (Steinkuehler & Duncan, 2009; Kuhn, 1992): How is expertise implicated in how participants talk to one another? Is knowledge construction seen as a collaborative and/or competitive task?
• Affinity (Gee, 2003; Gee, 2004): How is knowledge of, and affinity for a particular topic expressed within an affinity space? Do ad hoc affinity groups (Gee, 2003) manifest in relation to expertise?
• Credibility (Fogg, 2002): How can we uncover how trust is developed and shaped both by and within online affinity spaces? How do designed elements of affinity spaces influence the perceived credibility of experts within them?
• Participation (Ito, et al, 2009): Can "genres of participation" work as a lens to understand the performance of expertise in these spaces? How does expertise within affinity spaces span both online and offline spaces?
By September 2013, researchers will have refined pilot results to develop content coding schemes, aimed at capturing both regularities across affinity spaces as well as within each affinity space. Common codes for all affinity spaces will be determined, with individual sets of codes to capture practices unique to each site. Additionally, Google Hangout, Skype and telephone interviews with active members of affinity spaces under study will be conducted during this period (participants will be compensated for their participation), supplementing the coding of online interactions.
Stage Three: March, 2014 - May, 2014
Quantitative and qualitative analyses (content coding, Discourse analysis) will shape the final white paper. Researchers will summarize results as well as propose concrete design directives based on these results to better inform the next stage of badge development. The online public forum will serve the goal of disseminating the white paper (professionally formatted, intended for a mass audience). Additionally, the site will host a continuing discussion between the researchers and badging developers. During this stage, significant outreach will be underway, communicating results and design recommendations back to badging developers (via the 2014 DML conference) and academic audiences (via the American Educational Research Association).
Public Forum Online
The public forum proposed here will be an online public forum in the form of a dedicated website (based on WordPress, featuring a project blog and plug-ins linking to predominant social media sites). The online public forum will serve as a site for the distribution of the project’s culminating white paper, while also providing a venue for ongoing analysis and dissemination of study results.
Akin to the extensive Daedalus Project (Yee, 2009), ongoing analyses of data from the study will be shared with fellow academics and designers over the term of the project. With the goal of providing open access and open critique of the study as it is conducted, Badges for Lifelong Learning awardees will be given the opportunity to be active participants within the online public forum at several stages: (1) by having a voice during the solicitation of affinity spaces to investigate; (2) by providing ongoing critique of results as they are released on the forum; and (3) through persistent design-oriented discussions, in which designers and developers of future badging systems will be encouraged to discuss design directives uncovered by our analyses.
The online public forum will host periodic webinars with researchers, participants from affinity spaces under study, and badging system designers. The public forum, and the project in general, seeks to connect lived experience, research, and design on these recognition systems to further the design of future badging system designs and foster potential new partnerships.
Black, R. (2007). Fanfiction writing and the construction of space. E-Learning and Digital Media 4(4), 384-397.
Chi, M. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences 6(3), 1997, 271-315.
Duncan, S. C. (2010a). Gamers as Designers: a framework for investigating design in gaming affinity spaces. E-Learning and Digital Media 7(1), 21-34.
Duncan, S. C. (2010b). World of Warcraft and “the World of Science”: Ludic play in an online affinity space. In T. Wright, D. Embrick, and A. Lukacs (Eds). Social Exclusion, Power and Video Game Play: New Research in Digital Media and Technology. New York: Lexington Press.
Duncan, S. C. (2010c). A dual-level approach for investigating design in online affinity spaces. In K. Gomez, L. Lyons & J. Radinsky (Eds.), Learning in the Disciplines: Proceedings of the 9th International Conference of the Learning Sciences (ICLS 2010) - Volume 2, Short Papers, Symposia, and Selected Abstracts. International Society of the Learning Sciences: Chicago, IL, 346-347.
Fairclough, N. (1995). Critical discourse analysis. Boston: Addison Wesley.
Fogg, B. J. (2002). Persuasive technology: Using computers to change what we think and do. San Francisco: Morgan Kaufmann.
Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.
Gee, J. P. (2004). Situated language and learning: A critique of traditional schooling. London: Routledge.
Gee, J. P. (2010). An introduction to Discourse analysis: Theory and method, 3rd. ed. New York: Routledge.
Halavais, A. (2012). A genealogy of badges: Inherited meaning and monstrous moral hybrids. Information, Communication, and Society 15(3), 354-373.
Hayes, E. R. and Duncan, S. C. (Eds., 2012). Learning in video game affinity spaces. New York: Peter Lang Publishing.
Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.
Ito, M., Baumer, S., Bittanti, M., boyd, d., Cody, R., Herr-Stephenson, B. … Tripp, L. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.
Jenkins. H. (2006). Convergence culture: Where old and new media collide. New York: NYU Press.
Kuhn, D. (1992). Thinking as argument. Harvard Educational Revview 62(2), 155–178.
Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.
Steinkuehler, C. & Chmiel, M. (2006.) Fostering scientific habits of mind in the context of online play. In Barab, S. A., Hay K. E., Songer N. B., Hickey D. T. (Eds.), Proceedings of the international conference of the learning sciences (ICLS 2006). Erlbuam, Mahwah NJ, 723–729.
Steinkuehler, C. A. and Duncan, S. C. (2009). Scientific habits of mind in virtual worlds. Journal of Science Education and Technology, 17 (6), 530-543.
Weber, R. (1990). Basic content analysis (quantitative applications in the social sciences). New York: Sage Publications.
Yee, N. (2009). The Daedalus Project: The psychology of MMORPGs. Online resource, located at http://www.nickyee.com/daedalus/.
Sean Duncan is an assistant professor in the Learning Sciences program at Indiana University and a Digital Media and Learning Research Competition on Badging and Badge Systems Development grantee based on his winning proposal Connecting Badges and Expertise in Interest-Driven Affinity Spaces. The Research Competition was funded by the Bill and Melinda Gates Foundation and administered by HASTAC, in collaboration with Mozilla.