Blog Post

Evaluating Microcredentialing in Public Networked Environments | Alex Halavais

Evaluating Microcredentialing in Public Networked Environments | Alex Halavais

New online environments have created new ways of measuring and communicating experience and learning within a community. Some of these metrics can be generated automatically--indicating, for example, how long a person has been an active member of a discussion forum. Others, like Wikipedia’s barn stars, represent peer or expert endorsement of skills or achievements. Until recently, these indicators have largely been used as shorthand within a community to assess potential partners or celebrate achievement. How might these indicators be useful in encouraging, navigating, and communicating learning experiences across broader publics?

The research proposed here explores ways in which such indicators might act as microcredentials, demonstrating achievement or knowledge across communities. Traditional credentials, including the university degree and professional licensure, have often been seen as a kind of passport. The university degree acts as more than just a celebration of achievement or mark of experience, it is used as a certification of learning that can open doors to a career options and to new learning opportunities. In some cases, as in the licensing of medical doctors or teachers, the certifications may go so far as to be part of a legal requirement for engaging in work in the area.

Microcredentials represent learning and experience at a smaller scale, and offer the possibility of more diverse, detailed, and dynamic records of learning experience. In some ways they follow the pattern of other forms of syndicated microcontent on the web (like blogs, tweets, and syndicated calendars), allowing for unstructured sharing and networking. Recently, there has been increasing interest in how badges might be displayed and what they might mean when they cross epistemic communities. Platform gamers can now easily accumulate and display badges from across the games that they play and members of larger communities increasingly are invited to “share” their badges publicly, in order to garner interest in the community, as well as promote individual achievements and cultivate a public persona. Various attempts at badge aggregation and syndication (including the Open Badge Infrastructure) make sharing of these credentials easier, as do the everyday social tools of the web: Twitter, Facebook, and LinkedIn among them.

The scholarly research directly on badges and achievements is surprisingly sparse, and much of it focuses on its mechanisms of motivation or assessment processes rather than what is done with the microcertification once earned. What work does focus on how achievements can “unlock” new resources or provide for “levelling” tends to be limited to how such indicators are used within an epistemic community, rather than how they might form “boundary objects,” translating experience and knowledge between and among communities.

Several areas of research, however, can help to inform a framework for understanding how microcredentials might be used in networked publics. Well before their incorporation into online interactions, badges and other visual certifications of authority represented complex signalling systems (Halavais, 2012). Donath (2007) has suggested that signaling theory is a fruitful way of thinking about interactions on social networking systems broadly. And of course many studies of traditional certification draw explicitly on signaling theory (e.g., Chan, 2011; Wilkin & Connelly, 2012), which in many ways found its start in a model for understanding how educational certification translated to job searches (Spence, 1973). Naturally, we are also interested in the other ways in which microcredentials might be deployed, including as a form of construction and presentation of self in networked publics.

To better understand the influence of badges across domains, and particularly in public networks, the following research questions are proposed:

1. How are badges, achievements, and other microcredentials currently shared? Which are shared widely, and what encourages their sharing?

2. What are the motivations for sharing these markers of experience and learning?

3. How are they interpreted by those who see the markers but may not be a part of the cognate communities? In particular, how do they influence gatekeepers for particular communities: those charged with recruiting and hiring new employees, and those who admit students in higher education?

4. Are there factors inherent to the design of a microcredential that make it more effective or useful in communicating a learning achievement?

The answers to these questions will naturally differ across types of communities; as with other innovations, the future is unevenly distributed. Gamers and programmers, for example, are more likely to be familiar with badges and achievements. As we move beyond a more general map of current usage in the first phase, we will engage communities that are the most active users of such credentials, and examine whether there are unique characteristics that make certain microcredentials more or less likely to diffuse to the broader public.

Plan of Work

Phase 1: Mapping the Territory (June - August, 2013)

To provide context for both the DML Competition Badges and for the broader badge infrastructure, during the first several months we will first create a public catalog of badges. Our focus here will be badges that are shared outside their initial communities. We will approach this in two ways.

The first will be querying the existing badge communities, including those involved in the Mozilla Open Badge Infrastructure and DML Competition Winners, and those public projects making use of new badge tools (the WordPress, Joomla, and Drupal plugins, the badger platform, etc.). We will be looking at what kinds of learning and experiences the badges represent, and what place cross-network sharing plays.

We will also conduct a large-scale content and framing analysis of open profiles on LinkedIn, Twitter, Facebook, and Google Plus, to provide an indication of what badges are being presented and in what context. This will require iterative human coding to recognize particular patterns, which will then be coded for automatic recognition where possible. Once a substantial number of profiles have been coded, a sample of these will be coded for framing: How do users present these certifications? Are they described in certain ways? Are there links or other materials to allow for verification, or to better understand the experiences that led to the granting of the badge, or the organization who has issued the badge?

While our primary intent in creating this map is to provide a resource for the later phases of the research, we also hope that a much-needed directory will serve as an ongoing resource for those interested in both researching and building badges, much as DMOZ and other projects did for the early web.

Phase 2: Signalling Surveys & Focus Groups (August - December, 2013)

Having established some of the ways in which microcredentials are being deployed, we will survey a sample of those who have presented them in public profiles to discover why they are being used, and what outcomes those who have used them feel have been achieved. We are particularly interested in pragmatic outcomes (i.e., whether these were presented with the intent of gaining access to employment or further formal or informal learning), but are also interested in other gratifications presenting badges publicly may provide. In order to better understand the reasons for presenting these signals publicly, we will engage a subset of the respondents in a follow-up online focus group.

In parallel, we will be surveying a sample of hiring professionals and academic admissions staff about whether and how they make use of such credentials. As with those who present the microcertifications, we will follow with online focus groups among a subset of the respondents.

Phase 3: Best Practices and Design Considerations (January - April, 2013)

In the final phase, we will draw on the lessons learned from the previous phase to create a set of design recommendations for those creating microcredentials, in order to ensure that they are effective in providing credible, public indicators of learning experiences. We will test these best practices experimentally, presenting subjects with manipulated public profiles to determine what factors improve both the likelihood to present microcredentials in public settings, and the likelihood that they will be accepted as accurate or lead to inferences about the suitability of a candidate for employment or further education.

Dissemination

We propose dissemination of our research in several ways. Our ongoing work will be presented in a public blog, as will the index and map of public badges. We will invite public participation in updating that ongoing indexing project.

We will host a public workshop in October, described in a following section, and present our work at the DML conference, as well as the American Sociological Association meeting. Our intention is to publish articles based on each phase of the research, and also to produce a culminating white paper that provides a summary of the research as a whole, and the implications for educators, learning researchers, and those responsible for designing microcredentials.

Public Forum Narrative

We plan to host a pre-conference workshop on the University of Denver campus before the Association of Internet Researchers’ October 2013 conference. Our aim will be to bring together researchers with an interest in online credibility, authority, credentials, learning, and networked organization to discuss the rise of microcredentials. Co-locating with Internet Research conference allows us to leverage the broad interdisciplinary spectrum of social scientists in attendance. As part of this meeting we will provide access to our data, and discuss ways in which it may be analyzed and used to move research forward in this area. We will encourage attendees to present their own cognate research, and help to create a collective research agenda moving forward. We hope the meeting will serve as a catalyst for those who have similar research interests in online authority.

To help encourage participation in the workshop, we have budgeted for travel for three invited guests to act as featured presenters and co-organizers. We will identify these three during the summer of 2013, based on those doing current research in the area.

Citations

Chan, A. L. J. (2011). Effectiveness of Translator Certification as a Signaling Device: Views from the Translator Recruiters (pp. 31-48). R. Sela-Sheffy & M. Shlesinger (eds.), Identity and Status in the Translational Professions. Amsterdam: John Benjamins.

Donath, J. (2007). Signals in social supernets. Journal of Computer-Mediated Communication, 13(1), article 12.

Halavais, A. (2012). A Genealogy of Badges: Inherited Meaning and Monstrous Moral Hybrids. Information, Communication & Society, 15(3): 354-373.

Spence, M. (1973). Job Market Signaling. The Quarterly Journal of Economics, 87(3): 355-374.

Wilkin, C. L. and Connelly, C. E. (2012). Do I Look Like Someone Who Cares? Recruiters’ Ratings of Applicants’ Paid and Volunteer Experience. International Journal of Selection and Assessment, 20: 308–318.

*****

Alex Halavais is associate professor in the School of Social and Behavioral Sciences at Arizona State University and a Digital Media and Learning Research Competition on Badging and Badge Systems Development grantee based on his winning proposal Evaluating Microcredentialing in Public Networked Environments. The Research Competition was funded by the Bill and Melinda Gates Foundation and administered by HASTAC, in collaboration with Mozilla. 

Research Competition applicants were asked to submit proposals for empirical and theoretical research that support and inform the design, development, and deployment of digital badges and badge systems across a diverse range of learning content, institutions, and approaches, including the Gates Foundation supported Project Mastery Sites, as well as research focused on the efficacy of Teacher Mastery badging projects.
 
110

1 comment

Anonymous (not verified)

Thanks alot for sharing this article! It was very helpful. We often face this problem in deciding the rates <a href="http://gamealo.net/avatar/game-avatar-225" rel="dofollow" rel="nofollow">.</a><br />. This article gave us an idea about deciding the rates and how to charge from our customers.

85