Digital Badges: An Annotated Research Bibliography v1

Printer-friendly version
This annotated bibliography is a first step toward organizing literature about digital badges, open badges and badge systems. This domain involves multiple streams of literature from education, learning sciences, library and information science, reputation systems, and systems design. The bibliography includes peer-reviewed and non peer-reviewed articles, blog posts, news articles, white papers, videos, wikis and FAQs. We acknowledge that digital badges are an emerging topic and we have attempted to include a full spectrum of viewpoints. In light of this, we have chosen to provide descriptive rather than evaluative annotations.
 

Table of Contents

Journal articles (peer-reviewed)

Blogs

News Articles

Other

White Papers and Reports

Non Peer-Reviewed Journal Articles

Secondary Peer-Reviewed Journal Articles

Books

Digital Badges: An Annotated Research Bibliography selected and annotated by Sheryl Grant and Kristan E. Shawgo and published on HASTAC on February 27, 2013. (Looking for the Badges for Learning Research collection? Click here.)

This bibliography has been curated by Sheryl Grant, Director of Social Networking for the HASTAC/MacArthur Foundation Digital Media and Learning Competition, and PhD student at the School of Information and Library Science (SILS) at the University of North Carolina at Chapel Hill (UNC-CH) and annotated by Kristan Shawgo, HASTAC Special Projects Manager and Ci-BER Library Liaison, and recent MSLS graduate from SILS at UNC-CH. 

Why a Badges Bibliography? Following the  flood of response to the HASTAC/MacArthur Foundation's Badges for Lifelong Learning initiative in 2011, and the release of Mozilla's Open Badges Infrastructure, HASTAC decided to assemble this Badges Bibliography v.1 as a humble attempt to organize the universe of knowledge about digital badges. More importantly, we hope this bibliography will come to represent a cross-disciplinary approach that inspires questions, perspectives, and approaches to badges that reflect the inherently collaborative nature of badge systems. 

This annotated bibliography is a first step toward organizing literature about digital badges, open badges and badge systems. This domain involves multiple streams of literature from education, learning sciences, library and information science, reputation systems, and systems design. The bibliography includes peer-reviewed and non peer-reviewed articles, blog posts, news articles, white papers, videos, wikis and FAQs. We acknowledge that digital badges are an emerging topic and we have attempted to include a full spectrum of viewpoints. In light of this, we have chosen to provide descriptive rather than evaluative annotations.

Currently, the bibliography is heavy on certain theoretical perspectives and certain approaches to this topic. We intend for this to be a collaborative bibliography and welcome submissions. If you have sources or streams of literature you would like to add to this bibliography, please send an e-mail to Kristan Shawgo at kristan.shawgo(at)duke.edu or Sheryl Grant at sheryl.grant(at)duke.edu. Please provide the citation(s) in APA format. As of February 2013, the bibliography currently contains more than 160 references.

How to cite: Grant, S. & Shawgo, K.E. (2013). Digital Badges: An Annotated Research Bibliography. Retrieved from http://hastac.org/digital-badges-bibliography

return to top of page

Journal articles (peer-reviewed)

Abramovich, S., Schunn, C., & Higashi, R. (2013). Are badges useful in education?: It depends upon the type of badge and expertise of learner. Education Tech Research Development. Retrieved from: http://download.springer.com/static/pdf/503/art%253A10.1007%252Fs11423-013-9289-2.pdf?auth66=1364398853_eaf4db44b712278a1380306b6d651ce6&ext=.pdf

Categories: badges, assessment, motivation, intelligent tutors

Annotation: This article discusses how badges are touted as an alternative assessment that can increase learner motivation. The researchers considered two distinct models for educational badges; merit badges and videogame achievements. To begin unpacking the relationship between badges and motivation, they conducted a study using badges within an intelligent-tutor system for teaching applied mathematics to middle-school students. Their findings indicate that badge earning could be driven by learner motivations and that systems with badges could have a positive effect on critical learner motivations. However, badge acquisition patterns were different across learners with different levels of prior knowledge. Different badge types also affected different learners motivation. Additionally, the researchers believe that their findings are compatible with the research finding that extrinsic motivators have a negative influence on learning. The implication for educational badge designers is that they must consider the ability and motivations of learners when choosing what badges to include in their curricula. The authors believe their findings exist as one piece of the large research base needed to understand educational badges.

---

Adams, J., & DeFleur, M. (2006). The acceptability of online degrees earned as a credential for obtaining employment. Communication Education, 55, 32-45.

Categories: credentialing, professional development

Annotation: This research article focuses on how employers in charge of hiring decisions view online vs. traditional classroom learning.  They posit that the reputation of the degree granting agency makes a difference and that accreditation has traditionally been used to demonstrate educational integrity, instructional quality as well as certainty about specific guidelines and standards. For their study, letters were sent to hiring managers from eight different metropolitan areas asking if the managers would hire person A - 100% traditional classroom learning, Person B - 100% virtual learning, or Person C - 50/50 split between traditional and virtual learning.  They found that in the case of Person A vs. Person B, 96% chose A and in the case of Person A vs. Person C, 75% chose A. They also gathered qualitative responses about accreditation, interaction and classroom experience, reputation and quality, skills and experience. Though it was written in 2006 before MOOCs, digital badges, and programs like MITx altered the landscape of online learning, the study’s results may still hold great significance in regards to the importance of credentialing and brings into question whether digital badges earned by potential employees might hold value to hiring managers.

----

Adams, T., Demaiter, E. (2010). Knowledge workers in the new economy: Skill, flexibility, and credentials. In McMullin, J. and Marshall, V. (Eds.). Aging and Working in the New Economy: Changing Career Structures in Small IT Firms (pp. 119-142). Northhampton, MA: Edward Elgar Publishing.

Categories: credentialing, higher education, professional development

Annotation: This chapter focuses on importance or lack of importance of credentialing in the field of information technology (IT). The authors describe IT as a field where someone with a PhD is working next to someone without a degree. They conducted a quantitative web-based survey data and found that in the field of IT, credentialing may be more important than skills in regards to hiring, because achieving a formal education is seen as providing the learner with skills, socialization, contacts, and credentials. In addition, they found that younger workers and women may benefit even more from credentials within the field. Includes a review of theories about skill and education in regards to employment.

----

Anderson, D., Lucas, K. B., & Ginns, I. S. (2003). Theoretical perspectives on learning in an informal setting. Journal of Research in Science Teaching, 40(2), 177-199.

Categories: informal learning, theory, K-12

Annotation: This article focuses on the human constructivist theory of learning as an important tool for “investigating the development of knowledge and understanding emergent from experiences in informal settings” (p. 177). The constructivist theory of knowledge is based on a theory of knowledge scaffolding-- that we take in new knowledge and then integrate it into our next “level” of learning. The authors were interested in how formal (traditional classroom) and informal (visit to a museum) learning together would affect knowledge construction for students. For their research study, they had students 11-12 years in age complete concept maps and interviews in three different stages -- prior to a visit to a Sciencentre in Brisbane, Australia, pre-visit and post-visit plus a classroom related discussion. Their results included seven emerging categories of knowledge transformation: addition, emergence, progressive differentiation, disassociation, recontextualization, merging, and development of personal theories. This article will be of interest to those researching the integration of informal and formal learning and constructivist theory for K-12 students.

----

Antin, J. (2009). Motivated by information: Information about online collective action as an incentive for participation. In Proceedings of the ACM 2009 International Conference on Supporting Group Work (pp. 371-372). Sanibel Island, FL: ACM.

Categories: motivation

Annotation: This article looks at motivational factors in online collective action sites like Wikipedia and YouTube.  The authors posit that we must take individual operational knowledge into account when considering motivation. For example, someone who sees an error in a Wikipedia entry may want to edit the entry, but they choose not to to edit, because they simply don’t know how to edit the page.  The authors tested two hypotheses: “(1) individuals contribute more in collective action situations when they believe that they are highly competent, and; (2) they contribute more when they believe that they are more competent than other group members“ (p. 372). It could be reasoned that if their hypotheses hold true then a digital badge provided to signify competence could increase motivation to participate in online collective action sites.

----

Antin, J. & Churchill, E. (2011). Badges in social media: A social psychology perspective. Human Factors, 1-4.

Categories: achievement, motivation, gamification, reputation systems

Annotation: The article looks at the intersection of Human Computer Interaction (HCI) and social psychology theories as they apply to digital badges and recognize the impact of Wikipedia, StackOverflow and Foursquare on the current landscape of badges. The authors discuss social psychological functions for badges in social media, which include 1) goal setting: users set learning goals for themselves; 2) instruction: can be used to learn about a new website or idea; 3) reputation: represent interests, expertise and interactions of earner, 4) status/affirmation: advertise achievements to others and personal affirmation; and 5) group identification: shows shared experiences or shared interests. The article acknowledges that badges are emergent and require further research.

----

Antikainen, M., Makipaa, M., & Ahonen, M. (2010). Motivating and supporting collaboration in open innovation. European Journal of Innovation Management, 13(1), 100 - 119.

Categories: motivation

Annotation: This article posits that motivational factors must be understood in order to create successful open innovation communities. For their research study, the authors posed questions to users and maintainers of the open innovation communities CrowdSpirit, FellowForce, and Owela. They found that fun was the biggest motivating factor for contributors and that monetary rewards were not very effective in increasing motivation. The article includes an extensive list of references and many easy-to-read and useful tables, including a table of motivational factors for online community contributions, such as altruism, reputation, and enhancement of professional status, ideology, etc., along with references to the authors who wrote about the factors and a table of their research results that includes suggested tools and methods to increase collaboration.

----

Arkes, J. (1999). What do educational credentials signal and why do employers value credentials? Economics of Education Review, 18, 133–141.

Categories: credentialing, professional development, higher education, motivation

Annotation: This research article focuses on how employers can “learn about workers' pre-college cognitive abilities by observing the educational credentials they obtain, and to assess whether employers value credentials because they signal these cognitive abilities” (p. 133).  For this study, participants (all men) were gathered from the National Longitudinal Survey of Youth (NLSY). NLSY participants completed the Armed Services Vocational Aptitude Battery (ASVAB) prior to attending college and then 13 years later author, Arkes selected participants from this group who were currently employed and not in school to complete self-reporting about their educational and work achievements, resulting in 1064 participants. The author makes the assumption that educational attainment signals pre-existing (before college) ability. Based on test performance and self-reporting, “the results show statistically significant partial associations between pre-college ability and a high school diploma, college attendance, a bachelor's degree and an advanced degree, but not an associate's degree” (p. 134).  “In conclusion, I find evidence indicating that a national test measuring cognitive abilities could convey part of, but certainly not all of, the information provided by workers acquiring credentials to signal their abilities” (p. 141).

----

Baker, D.P. (2011a). “Forward and backward, horizontal and vertical: Transformation of occupational credentialing in the schooled society.” Research in Social Stratification and Mobility, 29: 5-29.

Categories: credentialing, theory

Annotation: This article develops a “new theoretical argument about why over the past century a rapid and robust educational transformation of occupational credentialing has occurred and what the current intensification of this trend means for sociological inquiry into social stratification” (p. 6).  It does not test a hypothesis but looks at theories on credentialing (human capital and education-as-myth), current ways credentialing plays out in educational institutions (horizontal institutionalization, vertical institutionalization, forward educational credentialing and backward educational credentialing) and looks toward the future of credentialing. This article would be useful for those interested in the theories around educational credentialing from a sociological (social stratification) or economic perspective.

----

Baker, D.P. (2011b). The Future of the Schooled Society: The Transforming Culture of Education in Postindustrial Society. In Hallinan, M.T. (ed.), Frontiers in Sociology of Education (pp. 11-34). Dordrecht, Netherlands: Springer.

Categories: credentialing, higher education, formal vs. informal education

Annotation: This chapter provides a history of the education system in the post-industrial era and posits that the “already robust global culture of education will likely intensify its transformation of postindustrial society” (p. 13). The author examines how education has become institutionalized and how it impacts social stratification with an overarching theme that  “academic achievement assumes greater importance than specific skill mastery” (p. 19). He argues that education is now about cognitive development and critical thinking rather than learning specific skills signified by the decrease in vocational training. The chapter also focuses on earning traditional degrees as credentialing, our society moving towards a meritocratic system (based on academic diplomas) and addresses the fear of credentialing fraud and diploma mills.

----

Barak, A. & Rafaeli, S. (2004). On-line question-posing and peer-assessment as means for web-based knowledge sharing in learning. International Journal of Human-Computer Studies, 61(1), 84–103.

Categories: assessment, higher education

Annotation: This article focuses on moving to a culture of assessment rather than testing, which they believe addresses the process of learning. They address self-assessment, peer assessment, and achievement assessment.  “MBA students carried out an on-line Question-Posing Assignment (QPA) that consisted of two components: Knowledge Development and Knowledge Contribution.” The students authored questions and answers and rated their peers’ responses and the authors conducted a content analysis of this process. They found that those who developed and contributed higher level questions, earned a better grade on their final exam and they also noted that not many students participated in ranking their fellow students.  They concluded by saying, “this research suggests that on-line question-posing activity may enhance meaningful learning and that on-line peer-assessment activity may enhance communities of learners.” 

----

Chan, C., Bhandar, M., Oh, L., & Chan H. (2004). Recognition and participation in a virtual community. Proceedings of the 37th Hawaii International Conference on System Sciences.

Categories: motivation, rewards 

Annotation: This research article is about the effects of recognition of participation in a non-commercial virtual communities with the Yahoo! Group “Academic Dress”, a virtual community that has academic discussion about academic dress, as its exploratory case study. The literature review includes a discussion on success factors in virtual communities- trust, anonymity and sense of community and two conceptual foundations: Theory of Information Sharing & Social Exchange Theory and Recognition & Participation in Virtual Community. Their methodology included electronically conducted interviews and qualitative survey.  Their study identified three forms of recognition: identification, expertise and tangible recognition and they found that members with a strong identity in the group felt a stronger sense of community and being recognized as an expert motivated further contributions. This study addresses interesting implications for motivation, recognition and reward that could be achieved through digital badges.

----

Cheng, R. & Vassileva, J. (2006). Design and evaluation of an adaptive incentive mechanism for sustained educational online communities. User Modeling and User-Adapted Interaction, 16(3-4), 321-348.

Categories: motivation, rewards, higher education

Annotation: This article examines incentive mechanisms in online communities through the example of an online, resource-sharing community called “Comtella.”  It begins with a brief literature review that considers issues of lack of user participation vs. information overload, conducting qualitative examination of content, and the process of peer-review. They proposed two mechanisms for looking at user incentive: a mechanism encouraging users to rate resources and an adaptive reward mechanism (different rewards for different levels of participation) and conducted their study with participants from a college course, “Ethics and Information Technology.” The found that incentivized users were more likely to rate articles, rated articles were more likely to be chosen to be summarized, that incentivized users contributed content earlier than non-incentivized users and they were more satisfied with their experience overall. This article would be of interest to those looking at incentives and motivations in online communities and how ratings effect user interest.        

----

Cheshire, C. & Antin, J. (2008). The social psychological effects of feedback on production of internet information pools. Journal of Computer-Mediated Communication, 13, 705-727.

Categories: motivation

Annotation: This article was co-authored by an assistant professor and a doctoral student (respectively) at Berkeley’s School of Information and focuses on what feedback mechanisms can motivate user contributions on sites like Wikipedia and YouTube with a specific focus on social psychological motivation. The authors define information pools as the gathering of information data online from many different sources to be accessed by others. The article begins with a useful literature review of articles about motivations for online contributions and they note that previous studies have found that positive ratings create greater chance for further contributions. Their study was about social psychology incentives for contributions and they focused on the effects of gratitude, historical reminder and relative rankings on the amount of future user contributions.  They found that these intrinsic motivation were statistically significant.

----

Daswani, N., Golle, P., Marti, S., Garcia-Molina, H., & Boneh, D. Evaluating Reputation Systems for Document Authenticity.

Categories: reputation systems

Annotation:This article focuses on document authenticity that is delivered in a peer-to-peer (P2P) system and how a document’s authenticity can be ensured more accurately through a reputation-based system. The author’s explain that, “a reputation-based system uses a node’s past interactions to weigh the opinions of other nodes when retrieving an authentic document” (p. 1). The article delves into specific details of what to consider in setting up a reputation system tool (metadata, reputation matrix, threat model and goals and metrics), provides samples of reputation systems and closes with research questions to create further discussion on using reputation systems for determining document authenticity.

----

Deci, E.L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social Psychology, 18,105-115.

Categories: motivation, rewards

Annotation: This is an article about an experiment on the effects of external rewards on intrinsic motivation. It begins with a literature of experiments and learning theories- the first set showing that external rewards decrease intrinsic motivation and the second showing it increases intrinsic motivation.  Here they investigators also made a distinction between financial reward (used in their first two studies) and verbal reward (used in their third study) to see if these methods would have different effects on motivation.  Their sample sizes were small, but they did find overall that their subjects became less motivated over time when provided financial reward and became more motivated over time when provided verbal reward.  This article would be of interest to those looking at the possible intrinsic and extrinsic motivation inherent in digital badges and those thinking about the importance of intentions in digital badging.

----

Deci, E.L. (1972). Intrinsic motivation, extrinsic reinforcement, and inequity. Journal of Personality and Social Psychology, 22, 113-120.

Categories: motivation, rewards, badge design

Annotation: This research article is a follow-up to the author’s previously published study, Effects of Externally Mediated Rewards on Intrinsic Motivation. Author, Edward Deci, concluded that his previous experiment needed a bigger sample size and showed interesting but not statistically significant results about the impact of financial reward and verbal reward on intrinsic motivation.  In this research study, he conducted his experiment with a larger sample size and discovered the same results- subjects became less motivated over time when provided financial reward and became more motivated over time when provided verbal reward.  This article would be of interest to someone thinking about motivation and design for digital badges.

----

DiFranzo, D., Graves, A. (2011). A farm in every window: A study into the incentives for participation in the Windowfarm Virtual Community. In Proceedings of WebSci 2011. Koblenz, Germany: ACM.

Categories: motivation

Annotation: Windowfarm is a multi-user blogging site for those who create Windowfarms, which are indoor vertical hydroponic farming system. The investigators explained that in most virtual communities there are a small set of people who provide most of the contributions and they were interested in exploring user participation and motivation in this virtual community from various viewpoints and theories: Psychology, Sociology, Mass media, Economics and Cognitive Science. Various models of motivation: intrinsic vs. extrinsic; functional psychology’s Value-expressive, Social adjustive, Utilitarian, and Knowledge; Clary’s motivational categories for volunteering: Values, Social, Understanding, Career, Protective, and Enhancement; recent experiment on motivation in contributing to Wikipedia found the factors to be fun and ideology; sociology: sense of efficacy, commitment and anticipated reciprocity were among the motivations reported in virtual communities, social psychology model of Sense of Community and so forth. For their experiment, they sent a survey to the 20,000 members, research database records and created a behavior classification system.  They found that a users’ feelings of integration and fulfillment of need created the strongest sense of community for them. The main motivational factor they found was self-satisfaction and self-improvement. They found overall that it followed closest to Wikipedia framework overall with fun and ideology being top motivators.        

----

Easley, D. & Ghosh, A. (2013). Incentives, gamification, and game theory: An economic approach to badge design. ACM EC conference. Retrieved from: http://www.arpitaghosh.com/papers/EC-final.pdf

Categories: gamification, badge design

Annotation: *Community member contributed content* Badges, or equivalent rewards such as top-contributor lists that are used to recognize a user’s contributions on social media websites, clearly appear to be valued by users who actively pursue and compete for them. However, different sites use different badge designs, varying how, and for what, badges are awarded— some sites such as StackOverflow award badges for meeting fixed levels of contribution, while others like Amazon and Y! Answers reward users for being amongst some top set of contributors on the site, corresponding to a competitive standard of performance. Given that users value badges, and that contributing to a site requires effort, how badges are designed will affect the incentives— and therefore the participation and effort— elicited from strategic users on a site. We take a game-theoretic approach to badge design, analyzing the incentives created by widely-used badge designs in a model where winning a badge is valued and effort is costly, and potential contributors to the site strategically decide whether or not to participate, and how much total effort to put into their contributions to the site. Finally, we ask whether or not a site should explicitly announce the number of users winning a badge; the answer to this question is determined by the curvature of the value of winning the badge as a function of the number of other winners.

----

Europa, Summaries of EU Legislation (2006).  Key Competencies for Lifelong Learning, Retrieved from: http://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11090_en.htm.

Categories: lifelong learning

Annotation: This is a summary of a law proposed in the European Union to create a reference tool for key competences for lifelong learning and to ensure these competences are integrated into strategies and infrastructure.  The eight key competences they list are: communication in the mother tongue, communication in foreign languages, mathematical competence and basic competences in science and technology, digital competence, learning to learn, social and civic competences. sense of initiative and entrepreneurship, and cultural awareness and expression. reference tool, basis for action at a community level. The summary explains that these key competences “provide added value for employment, social cohesion or young people (European Youth Pact), which explains the importance of lifelong learning in terms of adapting to change and integration.” This is an interesting look at how lifelong learning is being considered and integrated into policy initiatives and law.

----

Falk, J. H., Storksdieck, M., & Dierking, L. D. (2007). Investigating public science interest and understanding: evidence for the importance of free-choice learning. Public Understanding of Science, 16(4), 455-469.

Categories: lifelong learning

Annotation: This is a research article about school-learning and lifelong learning in the field of science, where the researchers conducted phone interviews with the public about where they learn about science.  They “argue that the contribution of school-based science learning to the long-term public understanding of science is limited, particularly for the majority of Americans who do not go on to post-secondary schooling.” The results showed a pretty even split for those who self-reported learning about science through the classroom and out of the classroom. They found that “43 percent of the public’s self-reported science understanding derives from leisure time, free-choice learning; science understanding was primarily acquired for reasons related to personal interest, need and/or curiosity.” This article has interesting implications for the importance of lifelong learning. It is important to note that while it was published in 2007, the phone surveys examined this study were conducted in 1997 and 2000. The authors acknowledged the impact of the advent of the internet between the two studies, which in all likelihood have a greater impact on their findings today.

----

Farzan, R., DiMicco, J., Millen, D., Brownholtz, B., Geyer, W., Dugan, C. (2008). Results from deploying a participation incentive mechanism within the enterprise. In Proceedings of 26th Annual ACM Conference on Human Factors in Computing Systems (pp. 563-572).

Categories: motivation

Annotation: This is a research article that examines the impact of incentive mechanisms (rewards and reputation) on user participation on social networking sites using the example of a social networking site for employees at IBM called “Beehive.” The authors explain incentive systems, such as incenting with rewards, by reputation, by goal-setting, for community benefit and for self benefit. They designed a points based rewards system that would provide both individual reward and increase reputation based on their content analysis of the site and collected both quantitative and qualitative data.  The system provided rewards in the form of an image (a bee- with different colors associated with different levels) with a descriptor: new bee, worker bee, busy bee and super bee (lowest to highest). They discovered an initial bump in content contribution by incentivized users, which leveled out over time and found that users would contribute to bump themselves up to the next level and then cease contributing. This is an interesting and easy to read article about motivation and incentivizing in social networks, plus it shows an example of earning digital badges in an experiment.     

----

Fazio, L. (1985). The Delphi: Education and assessment in institutional goal setting. Assessment and Evaluation in Higher Education, 10(2), 147–158.

Categories: assessment

Annotation: This article is about the Delphi process, a flexible method of evaluation, that helps avoid confrontation by allowing anonymous feedback and helps move an organization forward in its goals. The process “focuses on collating the aggregate judgments of a number of individuals who speculate on the present and the future and who have either similar or diverse backgrounds” (p. 149). This research study looks at faculty and administrators’ perceptions of institutional need at the University of Medicine and Dentistry in New Jersey. The three Delphi questionnaires used in this study are included in full and the article also contains a bibliography. The Delphi process could be used to evaluate educational effectiveness in both formal and informal learning.

----

Fehr, E. and Fischbacher, U. (2002). Why social preferences matter: The impact of non-selfish motives on competition, cooperation and incentives. The Economic Journal, 112, C-C33.

Categories: motivation

Annotation: This article posits that most economists believe people are motivated exclusively by material self-interest, but they should also consider how social preference motivate behavior and that material incentives may be harmful. The authors conduct research to look at heterogenous motivations, including reciprocal fairness, inequity aversion - both of which are altruistic motivations, competition vs. cooperation, and also spiteful or envious preferences- where people are only concerned about others getting less. Contains extensive references section. This article looks at various motivational factors and could be used to inform badge system design.

----

Friedman, E., Resnick, P., & Sami, R., (2007). Manipulation-resistant reputation systems. In ds. Nisan, N.; Roughgarden, T. & Tardos, E. (eds.). Algorithmic Game Theory (pp. 677-697). Cambridge, MA: Cambridge University Press.

Categories: reputation systems, credibility

Annotation: This chapter addresses methods of manipulating reputation systems - through whitewashing, phantom feedback/sybil attacks and incorrectly reported feedback - and how to create a system that is free of manipulation beginning with a discussion of the importance and uses of reputation systems.  They suggest to avoid whitewashing through “pay your dues” methods, incorrectly reported feedback through a moderated “center” that reviews contributions, and incentives for honesty and to avoid Sybil attacks by using Pagerank and max flow strategies. The chapter is part of the book, “Algorithmic Game Theory” and therefore contains many complicated algorithms for creating manipulation-resistant reputation systems, which may be difficult to understand for the lay reader.

----

Ghosh A., McAfee, P. (2011). Incentivizing high-quality user-generated content. In Proceedings of the World Wide Web conference. Hyderabad, India: ACM.

Categories: game theory, reputation systems, motivation

Annotation: This article examines how to create motivate user generated content in online forums that is both high in quality and quantity and is also easy to contribute.  They are using game theory and setting exposure as their primary motivator for contributed content. For their study they used “the solution concept of a free entry Nash equilibrium: as in a standard Nash equilibrium, the quality chosen by each participating contributor must be a best response to the qualities chosen by the remaining participants”, in this case a thumbs up/thumbs down “vote” given by readers of the content. Motivation, game theory (and its algorithms) and reputation systems are all covered in this article as they apply to user generated content.

----

Halavais, A.M. C. (2012). A Genealogy of badges: Inherited meaning and monstrous moral hybrids. Information, Communication and Society, 15(3), 354-373. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/1369118X.2011.641992. Open access version from 2011: http://alex.halavais.net/a-genealogy-of-badges

Categories: assessment, reputation systems, motivation

Annotation: The author provides a history of analog badges and digital badges and how they’ve been used for marking authority, expertise, experience (or achievement), and identity.  He also briefly addresses false badges and how badges may be misinterpreted by those outside of the badge issuer’s community, before pointing the discussion towards Jane Jacob’s “Systems of Survival” as a framework for effective digital badging. He explains that it is important to think about the intended function of badges, because, for example, “when badges are used as a way to certify knowledge and learning, the danger of monstrous hybridization becomes part of learning networks and educational institutions.” This article is an excellent look at the history of badges, both analog and digital, and a reminder of the importance of considering how badges are being employed.   

----

Hedlund, J., Forsythe, G., Horvath, J., Williams, W., Snook, S., & Sternberg, R. (2003). Identifying and assessing tacit knowledge: Understanding the practical intelligence of military leaders. The Leadership Quarterly, 14(2), 117-140.

Categories: informal education, learning theory

Annotation: This research paper looks at Tacit Knowledge (TK) in the domain of military leadership. TK is likened to professional intuition and is based on informal learning, everyday experience, practical application and personal goals. They had a three-part methodology- 1. interviewed military leaders at three different levels- platoon leaders, company commanders and battalion commanders - for examples of TK, 2. they narrowed down the list of TK attributes and created assessment tools and 3. then administered the assessment tools to new leaders. They found a great overlap with existing ideas about leadership, but also found establishing credibility and trust to be specifically important TK for military leaders. Readers may connect the concept of Tacit Knowledge with the importance of informal education and practical application being touted by many digital badge advocates.

----

Hickey, D., & Granade, J. (2004). The influence of sociocultural theory on our theories of engagement and motivation. In D. M. McInerney & S. Van Etten (Eds.), Big theories revisited: Research on sociocultural influences on motivation and learning (Vol. 4, pp. 223–247). Greenwich, CO: Information Age.

Categories: motivation, (sociocultural) theory

Annotation: This chapter looks at the influence of sociocultural theory on theories of motivation. The authors posit that knowledge is socially defined, participation in knowledgeable activity transforms that knowledge plus any associated goals and values and that the internalization of values and standards through the process of knowledge acquisition drive motivation in learning. The idea of internalizing knowledge and “the assumption that using knowledge transforms or changes both learners and the knowledge” (p. 242) has strong implications for motivation in learning.

----

Hickey, D., & Zuiker, S. (2005). Engaged participation: A sociocultural model of motivation with implications for educational assessment. Educational Assessment, 10(3), 277–305.

Categories: motivation, educational reform, assessment, (sociocultural) theory

Annotation: This article discusses an innovative approach to educational motivation “that embraces the core sociocultural assumption that knowledge ultimately resides in the context of its use, in the practices of “knowledge communities,” in which case, learning involves meaningful participation in the practices that characterize a community” (p. 278) and the implications of this approach on education assessment. It provides a detailed overview of sociocultural theories on motivation and the reconciliation between individual and social activity - aggregative and dialectical and its stated purpose is to raise questions about educational reform, assessment and motivation rather than provide answers. “With the support of the U.S. National Science Foundation, a 3-year study was initiated in 2000 to explore the issues in motivation and assessment summarized previously. The study involved a 20-hr secondary genetics curriculum built around the GenScope (http://genscope.concord.org) modeling/simulation software (Horwitz & Christie, 2000). The project’s central challenge was motivating lower-achieving ninth graders to learn introductory genetics.” The data is still being analyzed, but initial findings have been positive.

----

Hintz, R. (2009). Science Education in the Boy Scouts of America. Dissertation. Retrieved from Proquest: http://www.eric.ed.gov/ERICWebPortal/search/recordDetails.jsp?ERICExtSearch_SearchValue_0=ED513200&searchtype=keyword&ERICExtSearch_SearchType_0=no&_pageLabel=RecordDetails&accno=ED513200&_nfls=false&source=ae

Categories: analog badges, badge standards, motivation, professional development

Annotation: This is a 350-page dissertation written by a PhD candidate studying science education. The author compared merit badge standards for the Boy Scouts of America to the National Science Education Standards with specific attention paid to the “geology” merit badge.  She explains that learning associated with earning Boy Scout badges is both tactile and visual, learning styles, which may aid in retention over time. She found that 85% of the badges met at least one National Science Education Standard and concluded that Boy Scout participants learn science through earning merit badges, camping and other outdoor activities. She conducted interviews with former Scouts who are currently scientists and these scientists believe participating Boy Scouts and earning badges has helped them in their career. She makes a good connection between standards and success, though it would have been interesting to also look into the acceptance of badges by potential employers and educational institutions and how earning badges and being a scout helped them with their success (Did it help them with motivation? self-confidence? knowledge?). 

----

Huffaker, D. & J. Lai (2007). Motivating online expertise-sharing for informal learning: The influence of age and tenure in knowledge organizations. In Proceedings of IEEE International Conference on Advanced Learning Technologies, (pp. 595-599).

Categories: motivation

Annotation: This is a research article about what motivates user participation in online forums and looks specifically at the staff online forum for IBM.  It begins with discussing previous studies on online user participation with conflicting (intrinsic vs. extrinsic) findings on motivational factors. Participants were sent a survey about expertise-sharing and were asked what would motivate them to contribute and their preferred method of feedback. They found the primary motivator for older contributors to be based in “helping the community” and the primary motivator for younger contributors to be based in a desire for management approval/recognition. In their second study, they asked participants to contribute to the online forum and provided community interest and self-interest (or neither for the control group) as their motivation, but they garnered no significant results in this part of the study.           

----

Hull, G. A., & Greeno, J. G. (2006). Identity and agency in non-school and school worlds. In Learning in Places the Informal Education Reader. New York: Peter Lang Publishing.

Categories: lifelong learning, motivation, K-12

Annotation: This chapter challenges two assumptions about formal and informal learning. The first is that informal and formal learning must be different kinds of learning and the second is that informal learning is solely a supplement to formal learning. They posit that traditional classroom learning is often set up in a way that does not allow children and teens to successfully connect with the subject matter and teachers, which therefore lessens their learning experience. Their chapter focuses on the concept of formation of personal identity in learning and they believe that informal learning creates shifted identities that increase motivation, improve interpersonal interactions and work to disrupt uneven power dynamics.  The chapter provides examples of their concepts in real-life situations, explores language literacy and learning math skills and argues that we should focus on ways that best support learning.

----

Hummel, H., Burgos, K., Tattersall, C., Brouns, F., Kurvers, H., & Koper, R. (2005). Encouraging contributions in learning networks using incentive mechanisms. Journal of Computer Assisted Learning, 21, 355-365.

Categories: motivation, (social exchange) theory

Annotation: This research article looks at motivating factors and incentives for contributions on online learning networks. The article posits that the quality of learning is rooted in practice (constructivist approach), informed participation, and social exchange (with the best value at the lowest cost).  The authors identified four motivators for learning network contributions: personal access or anticipated reciprocity, personal reputation, social altruism and tangible rewards. Other theories explain motivators as individual (personal access) vs. interpersonal (altruism) and hard rewards (personal access) vs. soft rewards (altruism) with the idea that altruistic motives drive better content. After conducting a pilot study, the researchers studied motivation through first establishing a control time period, then providing an incentive and then removing an incentive on their Learning Network, LN4LD. They found the highest motivation level occurred when they were providing an incentive and that this motivation decreased significantly when it was removed. It is of note that the study only had a 13.6% response rate and 17 participants overall. This is an interesting look at how incentives or lack of incentives motivate online contributions in knowledge building spaces.

----

Katz, I., & Assor, A. (2007). When choice motivates and when it does not. Educational Psychology Review, 19, 429-442.

Categories: motivation, (self-determination) theory

Annotation: The article focuses on choice in an educational setting and the self-determination theory (SDT) of motivation which centers around three needs that are at the heart of intrinsic motivation: autonomy, the need for relatedness, and the need for competence. The authors detail these three needs of SDT and how they are integrated into the classroom setting through choice.  The article has an extensive bibliography.

----

Keppell, M., Au, E., Ma, A., & Chan, C. (2006). Peer learning and learning-oriented assessment in technology-enhanced environments. Assessment & Evaluation in Higher Education, 31(4), 453-464.

Categories: assessment, higher education

Annotation: This article is a qualitative study with three lecturers and their students about peer learning and assessment in the higher education classroom.  It begins by outlining peer learning, assessment and technology-enhanced learning environments and then takes a close look at three case studies that include these concepts. Case 1 involved peer sharing and critique of the group design project using the Blackboard Learning Management System, in Case 2 first year participants in the Art Education Foundation module were asked to use the Blackboard Learning Management System as part of a learning-oriented assessment strategy and the third case study was a design project for primary-school teachers that focused on the design of a virtual learning community (VLC) and involved peer learning using face-to-face groups and virtual groups using the Blackboard Learning Management System. The article concludes by asserting that peer assessment is crucial for lifelong learning.     

----

Krieger, M., Stark, E., & Klemmer, S. (2009). Coordinating tasks on the commons: Designing for personal goals, expertise and serendipity. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI '09) (pp. 1485-1494). New York: ACM.

Categories: badge design, motivation

Annotation: This is a research article on Wikipedia and task management. The abstract notes that, “despite the availability of tag-based community-wide task assignment mechanisms, informants reported that self-directed goals, within-topic expertise, and fortuitous discovery are more frequently used than community-tagged tasks.” Their qualitative study sought to answer three questions: How do people choose tasks to complete on Wikipedia, how does that vary amongst levels of participants and what does that mean for tool design? In their discussion section the authors address five significant points of insight they gained- there is a bottom-up structure, template tags do not function well as items to accomplish next, there is a lack of triage, it is difficult to attain contextual support and there is a disconnect between individual and site tasks. In response, the authors designed and implemented a task management system called “WikiTasks.”

----

Kriplean, T., Beschastnikh, I., & McDonald, D. (2008). Articulations of wikiwork: Uncovering valued work in Wikipedia through Barnstars. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (pp. 47-56). San Diego, CA: ACM.

Categories: motivation

Annotation: This is an article about a badging system employed by Wikipedia, called Barnstars, that provides “personalized tokens of appreciation” to participants. “A barnstar is an image accompanied by a short and often personalized statement of appreciation for the work of another editor” (p. 49) and they are used to recognize the work of others. The authors conducted a content analysis of the distributed barnstars and identified the main reasons they were received (listed from highest to lowest percentage) - editing work, social and community support actions, border patrol, administrative, collaborative actions and disposition, and meta-content work. They included qualitative data along with their descriptions of these categories. They conclude that, “an important direction for online communities research is to formulate recommendation and reputation mechanisms that help members identify trustworthy partners for interaction, deal with newcomers who do not yet understand community norms, and encourage desirable activity” (p. 55) and suggest that further studies be conducted.      

---- 

Lepper, M. R., & Gilovich, T. (1982). Accentuating the positive: Eliciting generalized compliance from children through activity-oriented requests. Journal of Personality and Social Psychology, 42(2), 248-259.

Categories: motivation

Annotation: This research article focuses on children’s compliance (consequence-oriented and activity-oriented) to adult requests with special attention paid to educational compliance.  The authors found that most previous studies focused on consequence-oriented compliance - offering extrinsic reward for completed activities - whereas they hypothesize that using activity-oriented compliance - making the activity itself more enjoyable - would be more effective. They experimented with preschool students employing two different activity-oriented strategies- fantasy-transformation strategies and goal-setting strategies and compared the results to a control group. Students were found to be more compliant and enjoyed their activities more when provided with either fantasy-transformation strategies and goal-setting strategies.  

----

Luther, K., Diakopoulos, N., & Bruckman, A. (2010). Edits & credits: Exploring integration and attribution in online creative collaboration. In Proceedings of the 28th of the International Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '10) (pp. 2823-2832). New York: ACM.

Categories: reputation systems, motivation

Annotation: This is a qualitative study about motivation and reputation in online creative collaboration. Their methodology included interviewing online creative collaboration participants (leaders and/or artists) working on collaborative (collab) projects on Newgrounds, a flash animation portal. In the case of Newsgrounds, “collab” leaders must determine which artist’s submissions to integrate into a final design and how they should be credited or attributed for the final product. In their interviews, they learned that leaders and artists believe that it is important to acknowledge online contributions and the authors suggest that  an acknowledgement of contributions combined with a system for providing a commendation for contributions, like “barnstars” on Wikipedia, would be the most beneficial in motivating creative online contributions. The article contains reference, images and examples from Newsgrounds.

----

Marti, S., Garcia-Molina, H. (2006). Taxonomy of trust: Categorizing P2P reputation systems. Computer Networks, 50(4), 472-484.

Categories: reputation systems

Annotation: The authors state that their “primary goal is to provide a useful taxonomy of the field of peer-to-peer reputation design.” The article defines common concepts in peer-to-peer (P2P) reputation systems, such as transactions and Distributed Hash Tables, and outlines assumptions and constraints, including user behavior, adversarial powers, threat models and environmental limitations. The authors also identify sources for gathering information (system identities, information sharing, sources of information, information integrity, dealing with strangers) about users that help determine someone’s “reputation” given by their peers in an online forum as well as how an individual’s reputation can affect motivation through incentives and punishment. “A reputation system is generally composed of three basic components: gathering behavioral information, scoring and ranking peers, and rewarding or punishing peers“ (p. 482). 

----

 

McDaniel, R., Lindgren, R., & Friskics, J. (2012). Using Badges for Shaping Interactions in Online Learning Environments. In 2012 IEEE International Professional Communication Conference (IPCC). New York: IEEE. Retrieved from http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6408619 

Abstract: In this paper, we describe an online course management system (titled Adventures in Emerging Media) designed to allow students to choose their own pathways through learning content (a choose-your-own-adventure online course). In addition to providing students with additional agency and narrative prompts, we also used badges, or achievements, to promote specific types of student behaviors. This study provides data collected from approximately 200 students enrolled in this online digital media course in which badges were used to incentivize targeted student behaviors, such as taking an exam within a certain timeframe or responding to student work with especially helpful feedback. In addition to a brief analysis of relevant achievement assessment data, we also describe our approach to the mechanics of achievement design and show some of the elements of design and layout used to incorporate the achievements into a learning management system.

Annotation:  This article reflects upon the process of using a badging strategy in a gamified course in a  Design and Art school (University of Central Florida).  It reports on the behaviours that emerged around students efforts to acquire badges and engage in learning activities.  Some of the interesting observations relate to the way students saw peer achievement as a motivator – when a friend acquired a badge they were more motivated to chase that badge themselves.  It reports a somewhat ambivalent reception to the badges by students but suggests some future directions for investigation around gender differences, instant gratification, and motivational dimensions.

----

McDonald, D., Javanmardi, S., & Zachry, M. (2011). Finding patterns in behavioral observations by automatically labeling forms of Wikiwork in Barnstars. Accepted paper from WikiSym, 15-24.

Categories: Wikipedia, motivation

Annotation: “This research begins to address how we can leverage community based behavioral observations; observations that people make of those around them.” The researchers tested their question- are behavioral observations reliable?- on distributed “barnstars”, which are images accompanied by descriptive appreciation about users (Wikipedians) and their contributions on Wikipedia that can be given by any person using Wikipedia.  In essence, barnstars are badges given to exemplary contributors on Wikipedia. This article found that barnstars were most frequently awarded for different activities observed by individual users, though they might be awarded for the same type of activity. For example, barnstars are awarded to one Wikipedian for their editing skills, but they are awarded for different articles that they edited, not all for the same one. The authors believe this helps support their idea that behavioral observations are reliable and they suggest that these behavioral observations be leveraged to learn about and improve collective online contribution sites like Wikipedia.

----

Meyers, R. B. (2005). A pragmatic epistemology for free-choice learning. Environmental Education Research, 11(3), 309-320.

Categories: learning theories, informal learning

Annotation: This article is a critical review of free-choice learning (FCL)  theory, considers many alternative theories including absolutism, skepticism, pragmatism, Vygotskian epistemology and ultimately suggests the use of John Dewey’s learning theory as an underpinning for FCL. Their critique predominantly lies in the lack of guidance that FCL provides for educators. Dewey’s theory “puts the emphasis on the educator creating learning experiences for the learner to reconstruct their knowledge” (p. 319) and uses a model of scaffolded learning.

----

Miettinen, R. (2006). Epistemology of transformative material activity: John Dewey's pragmatism and cultural-historical activity theory. Journal for the Theory of Social Behavior, 36(4), 389- 408.

Categories: learning theory

Annotation: This is an article about learning theory that focuses on John Dewey’s pragmatism and cultural-historical activity theory and secondly on Lev Vygotsky modern take on Marx’s materialistic concept of practice.

----

Mortensen, M. F., & Smart, K. (2007). Free-choice worksheets increase students' exposure to curriculum during museum visits. Journal of Research in Science Teaching, 44(9), 1389-1414.

Categories: K-12, informal learning, motivation

Annotation: This article addresses how to bridge the free choice learning experience of visiting a science museum with curriculum in the K-12 classroom through the use of a guided worksheet. The authors consider constructivist theories of learning; prior knowledge, interests and beliefs; choice and control; sociocultural theories and theories of motivation (classroom & grades- extrinsic, museum learning- intrinsic) and use these methods (save extrinsic reward)  as recommendations in constructing the worksheet. They carried out their experiment through a school visit to the North Carolina Museum of Natural Sciences and found that students using “Chaperone Guides” were statistically more likely to have classroom curriculum related discussions while viewing focus displays at the museum. This article would be of interest for anyone looking for an almost step-by-step guide to create tools to integrate formal and informal learning experiences and anyone interested knowing more about learning theories and motivation. Examples of the museum’s chaperone guides can found here: http://naturalsciences.org/education/learning-resources/chaperone-guides

----

Neiswandt, M., & Shanahan, M.C. (2008). "I just want the credit" - Perceived instrumentality as the main characteristic of boys' motivation in a grade 11 science course. Research in Science Education, 38(1), 3-29.

Categories: motivation, sociocultural theory, K-12

Annotation: Researched from the perspective of science literacy by academics who also possess knowledge of educational psychology, the authors examine motivation in science learning through sociocultural theory, future time perspective and achievement goal theory. They conducted a full school year qualitative (interviews, observation, content analysis of curriculum) case study with ten 11th grade male students that sought to answer these two research questions: “1. What kind of goals do students adopt in a science class that is promoted as meeting students’ need for a senior science credit? 2. Are students’ goals influenced by goal messages discussed in the classroom and/or transported through the general school climate and school mission? If so, which goal messages do they integrate into their own goal structure and to which degree do they integrate them?” Their results found five common themes from students about the course: “dead end,” “just a credit,” “for non-science people,” “not a real science course,” and “everyday knowledge accumulation.” Their analysis included looking at how the teaching approach and stated course design (includes language about the course being “for non-science people”) affected students’ motivation and perceptions and found that most students were extrinsically motivated and externally regulated.

----

Olneck, M. (2012). Insurgent credentials: A challenge to established institutions of higher education. Paper presented to Education in a New Society: The Growing Interpenetration of Education in Modern Life at Radcliffe Institute for Advanced Study, Harvard University, Cambridge, Massachusetts, April 26-27, 2012. Retrieved from: http://hastac.org/documents/insurgent-credentials-challenge-established-institutions-higher-education

Categories: theory, higher ed

Annotation: This paper establishes the need to develop sociological explanations for the recent development of badges of certification of skill and knowledge mastery as possible substitutes for, or supplements to, conventional college and university degrees. It also calls for sociologists to attempt to anticipate the impact of these developments on the configuration of U.S. higher education. Among the sociologically significant features of badges that it identifies are the openness of badges to certifying knowledge and skills acquired outside of formal education institutions, the emergence of badges in a context of credential inflation, the challenge badges pose to reigning sociological explanations for the role of credentials in the labor market, and the departure of badges from formal degrees’ hierarchical classification of individuals. The paper concludes that badges may very well be positioned as “alternatives” to dominant forms of credentials, that they are more likely to generate reactions among non-elite institutions of higher education than among elite institutions, and that they may have the contradictory effects of challenging the dominance of academic knowledge and institutions, while, simultaneously, expanding the “schooling form” to activities and spheres previously beyond the reach of schooling.

----

Oliver, P. (1980). Rewards and punishments as selective incentives for collective action: Theoretical investigations. American Journal of Sociology, 85(6): 1356-1375.

Categories: theory, motivation

Annotation: This article focuses on selective incentives that the author describes as the processes “when actors reward or punish each other to motivate or sustain cooperation in some form of collective action” (p. 1356), an analysis of Mancur Olson and closes with a brief discussion of implications. Though many critics disagree with his conclusions, Mancur Olson’s theory, rooted in economics, is that people won’t “buy” the public good and will only contribute to groups/the community if selective incentives are involved. The author examines Olson’s theory in depth as well as theories of his critics and these theories are also presented as mathematical equations. The section on the “dynamics of positive selective incentives” could be particularly useful to those interested in motivation and digital badges.

----

Rafaeli, S., Ariel, Y., and Barak, A. (2008). Online motivational factors: Incentives for participation and contribution in Wikipedia. In Psychological Aspects of Cyberspace (pp. 243-267). New York, NY: Cambridge University Press.

Categories: motivation, online communities, Wikipedia

Annotation: This article begins with the history of the most popular wiki (“online, web-based collaborative writing environment“)- Wikipedia, its Wikipedians (contributors) and a literature review of articles written about Wikipedia. The authors explore motivational factors, including many theories about contributing to online communities rooted in psychology, Wikipedian gratification, sense of community and potential for interactivity. The authors suggest further study into participation on Wikipedia (1) professional versus nonprofessional participation; (2) constructive, confrontational, and vandalistic participation; (3) continuous versus one-time participation; (4) anonymous versus identifiable participation; (5) content contribution, community involvement, and (silent) participation in the form of lurking.” (p. 262). Though is article is focused on participation on Wikipedia, it serves as a great and thorough overview into theories of motivation in online communities.

----

Rafaeli, S., Raban, D., & Ravid, G. (2005). Social and economic incentives in Google Answers. In GROUP05 - Sustaining Community: The Role and Design of Incentive Mechanisms in Online Systems Workshop. Sanibel Island, FL.

Categories: motivation

Annotation: This research article examines Google Answers (no longer functioning as of 12/19/2012), an online forum where anyone can pose a question with an amount they are willing to pay for an answer and pre-approved experts will respond for incentives, including a $20 on average reward per answer, “star” rating, and feedback from 6/2002 to 10/2004. They found that the rate of answer response was strongly correlated with financial incentives and that a correlation between response and social-based incentives was only demonstrated through the presence of feedback/comments, not “star” ratings. This is a preliminary study with a very brief section of results and no discussion section.

----

Rashid, A. M., Ling, K., Tassone, R.D., Resnick, P.,  Kraut, R., & Reidl, J. (2006). Motivating participation by displaying the value of contribution. In Proceedings of CHI 2006 Conference on Human Factors in Computing Systems (pp. 955-958).

Categories: motivation

Annotation: This research paper looks at motivation in online community contributions using the movie recommendation website, MovieLens, as their test-bed.  The authors augmented MovieLens to display estimated values of contributions by giving movies 1 to 3 smiley faces to see how that affects motivation and they define the user groups who would benefit in four ways: self, small group the user has affinity with, small group the user does not have affinity with and the entire user community. Participants in four different groups (plus a control group) they were rating movies for based on the above defined user groups (self, entire user community, etc.). They were asked to rate 150 movies for their chosen group with embedded information about the amount of value (signified by smiley faces) they would provide “their community” through their ratings and then complete a brief survey. Participants reported that they were most motivated by whether they’d seen a movie or not and secondly by how many smiley faces a movie was given. Participants were more likely to rate movies for group similar to them, but not more likely to rate movies for the benefit of themselves. This article could be helpful in thinking about motivation and badge design that reflects participant’s motivation.

----

Thom-Santelli, J., Cosley, D., & Gay, G. (2010). What do you know?: Experts, novices and territoriality in collaborative systems. In Proceedings of the 28th international conference on Human factors in computing systems (pp. 1685-1694). Atlanta, GA.

Categories: motivation, tag system design

Annotation: This research article explore territoriality in social online communities and the differences between how experts and novices may react. For their experiment, they asked novices and experts to participate in collaborative social tagging (using MobiTags) of museum displays and then to rate the tags of others and they found that experts were more territorial over their own tags and more likely to vote down the tags of others. “encouraging feelings of ownership may motivate experts to contribute more to something they value” (p. 1685) - relevance to digital badges;  This article could have interesting implications for digital badges: Could badges motivate experts to contribute more to online communities and/or could earning digital badges make people more territorial over their own content?

----

Vassileva, J. (2003) Motivating participation in peer to peer communities. In Petta, P., et al. (Eds.) Engineering Societies in the Agents World III (ESAW 2002), Lecture Notes in Computer Science Volume 2577 (pp. 141-155).

Categories: motivation, peer to peer

Annotation: This article looks at a system in development called, Comutella, that is a peer-to-peer system at the University of Saskatchewan where students can share files/resources and services with each other. The author describes some potential motivators for contributing in a peer-to-peer community site, including seeing the system as useful/valuable, getting their questions answered and through additional incentives.  The following is a great descriptor of methods of motivations in a community. “There are four principle ways of motivating users to participate in a community: by trying to influence the user's feelings (of guilt, of belonging or owing to the group) to stimulate her altruism towards the community, by rewarding the user with visibility / reputation in the group depending on his/her contribution, by allowing the user to develop relationships with other users in the community (one would do a favor to a friend, which one might not want to do for anonymous unknown people), and by providing an economical model which ensures incentive for user contribution, (e.g. better quality of service, priority in the queues). It is likely that choosing the appropriate way of motivation depends: on the personality of the user, and on the nature or the user's interest in the area and the group (community). Thus, the same user can be altruistic in one group, motivated by reputation in another group and by economic rewards in a third group” (p. 144). This article is useful to thinking about digital badge design and motivations that may drive potential badge earners.

----

Werquin, P. (2010). Recognition of non-formal and informal learning outcomes is high on policy agendas: Executive summary. Organization for Economic Co-operation and Development (OECD). Lifelong Learning in Europe, 3, 142-149. Retrieved from http://www.oecd.org/dataoecd/9/16/41851819.pdf

Categories: lifelong learning, recognition of informal learning, assessment

Annotation: This article by Patrick Werquin, Directorate for Education in the Education and Training Policy Division of the Organization for Economic Co-operation and Development (OECD), comes across as a breath of fresh air and still seems extremely relevant despite it being published in 2008. It begins with definitions, rationale, challenges and threats associated with recognizing informal or non-formal learning and explains that establishing set definitions that are agreed upon is challenging because our ideas about learning are changing so rapidly.  He begins by explaining the benefits of informal learning, saying that “recognising what people know or can do – regardless of where they have acquired these skills, knowledge and competences – is indeed likely to be a strong incentive for them to resume learning formally as they will not have to start from scratch. This also cuts the traditional costs (time, tuition fees, transportation costs, etc.) and opportunity costs (forgone earnings, etc.) of formal learning” (p. 143. He also notes that the “recognition of non-formal and informal learning is seen as having the potential to foster the knowledge economy agenda by improving the level of qualifications of individuals, making them happier and better workers, and improving their standard of living” (p. 143). He then explains that the challenges to recognizing informal learning include stakeholders, assessment, standards, cost, limited data, and take-up (people are slow to engage in this type of program) and threat to recognizing informal education is the lack of evidence that these informal learning systems & recognition “work.” Though his excitement about the ability of informal learning to create a more equitable economy, encourage further formal learning, develop human capital and self-esteem is evident, he encourages an approach of pragmatism for those touting informal education and recognition.     

----

Winters, D., & Latham, G. P. (1996). The effect of learning versus outcome goals on a simple versus a complex task. Group & Organization Management, 21(2), 236-250.

Categories: learning theory, motivation

Annotation: This research article focuses the impact of different types of goal setting on performing tasks.  They hypothesized that small tasks would benefit the most from outcome goals and new big tasks would benefit most from learning goals. Their study was conducted with 114 undergraduate level business students and they found their hypotheses to be true. They explain the practical implications of the study as this: people with prior knowledge or skill of a task will be motivated by outcome goals and learning goals should be set if someone wants to learn something new. The question this brings up is whether badges function as an outcome goal or a learning goal.

----

Yu, J., Zhenhui, J., Chan, H. (2011). The influence of sociotechnological mechanisms on individual motivation toward knowledge contribution in problem-solving virtual communities. IEEE Transactions on Professional Communication, 54(2), 152-167.

Categories: badge system design, motivation

Annotation: The authors of this article focus on motivation in problem solving virtual communities (PSVCs) noting that not all members contribute, a large percentage just lurk. The article explores functional motivation, motivation by goals, motivation based on perceptions of environment, Expectancy-value theory, and finally volunteerism motives and takes a closer look at functional motivations- volunteerism motives (altruistic and egoistic motives) and Expectancy-value theory motivations (perceived effectiveness of knowledge repositories, reputation system, salience of social identity, and pro-sharing norms).  For their experiment, they chose students and alumni of a university in Asia and they found that egoistic motives for contribution were significant. They explain that, “an important implication of the functional approach is that matching benefits with individual motivations results in positive outcomes. Hence, in order to encourage enduring knowledge contribution, virtual-community practitioners should provide mechanisms to meet the motivations of knowledge contributors” (p. 163). This article provides a theoretical perspective on how motivation may be influenced by the possibility of earning digital badges.

----

Zhuolun, L., Huang, K. & Cavusoglu, H. Can We Gamify Voluntary Contributions to Online Q&A Communities? Quantifying the Impact of Badges on User Engagement. Retrieved from http://www.krannert.purdue.edu/faculty/kkarthik/wise12/papers%5Cwise12_submission_131.pdf

Categories: motivation, gamification, reputation systems

Annotation: The authors suggest that gamification could be a successful model for motivating online user contribution on question and answer (Q&A) sites and that digital badges could help increase “answering questions, commenting, making revisions, and asking questions.“ Through their micro-level and macro-level analysis of the site “Stack Overflow”, a Q&A site for programmers, they found that digital badges did increase the first three actions but it does not increase asking questions, perhaps because there is already a reward system inherent in the process.

return to top of page

Blogs

Ash, K. (2012, June 13). Digital badges would represent students’ skill acquisition. Education Week. Retrieved from: http://www.edweek.org/dd/articles/2012/06/13/03badges.h05.html?tkn=ORSF1

Categories: K-12, motivation, game-based learning

Annotation: Katie Ash’s blog post provides a thorough and fair representation of digital badges from the viewpoints of both advocates and skeptics and how digital badges could impact K-12 learners. Digital badges could help K-12 students by providing a way to define their skills beyond standardized testing and the digital badges they earn could become part of their resumes and college applications to help them with their future educational and professional development. Badge advocates also say that it could lead to scaffolded learning, for example, once they’ve earned their Intro to HTML badge, they are encouraged to pursue the Intermediate HTML badge. On the other hand, skeptics ask how digital badges be integrated into the structure of the classroom or if they function solely outside of the classroom, will that make young people less interested in completing activities because it is now being quantified with external rewards? They also express concerns about the gamification of learning. The author focuses on the digital badge efforts of MOUSE as an example of the possibilities of badging for youth. This article is an excellent resource for those interested in how digital badges will affect K-12 learners.

----

Bill (2012, April 17). Badges, Portfolios, and Blending Formal and Informal Learning. FunnyMonkey. Retrieved from http://funnymonkey.com/badges-portfolios-blending-formal-and-informal-le...

Categories: assessment, professional development

Annotation: This blog begins with a useful diagram of how formal education, badges and self-documented achievements come together to form a someone’s personal work and education portfolio, which is followed up by a detailed description of each aspect of the diagram. The author asserts that digital badges provide a context for informal learning in a way that will help “outsiders” see the knowledge learned as “valid” and digital badges place the focus on learning, rather than assessment.  Special attention is paid to Mozilla’s Open Badge Infrastructure, including badge issuers and Mozilla’s “backpack” for “storing” digital badges.

----

Blair, L. (2012). The cake is not a lie: How to design effective achievements. Retrieved from http://www.gamasutra.com/view/feature/6360/the_cake_is_not_a_lie_how_to_...

Categories: game-based learning, badge design, motivation

Annotation: This article by Lucas Blair looks at achievement design features in gaming - keeping in mind what might motivate players - and discusses best practices for design. It examines measurement achievements vs. completion achievements, boring vs. interesting tasks, achievement difficulty and goal orientation. In each section, the author defines the topics, provides specific game examples, such as stars earned in Angry Birds, and makes suggestions for best practices. Contains a bibliography. Good article for those interested in digital badges in gaming, motivation and best practice in badge design.

----

Casilli, C. (2012, May 20). Badge system design: Seven ways of looking at a badge system. Retrieved from http://carlacasilli.wordpress.com/2012/05/20/badge-system-design-seven-w...

Categories: badge system design

Annotation: Carla Casilli's blog post provides seven ways of looking at a badge system (and its design) includes philosophical, conceptual, pedagogical, visual/aesthetic, technical, categorical and ownership. Within each category she provides a bullet point list of questions raised, such as technical’s siloed vs. shared, open vs. proprietary and system vs. single. She admits that including ownership within her list is including a bit of an outlier, but that it serves as a reminder that no system will be completely simplified and neat. Casilli’s blog post is provocative, rather than explanatory.

----

Casilli, C. (2012, May 21). Badge system design: What we talk about when we talk about validity. Persona. Retrieved from http://carlacasilli.wordpress.com/2012/05/21/badge-system-design-what-we...

Categories: validity, credibility, reliability, education

Annotation: Carla Casilli seeks to address concerns around the validity, credibility, and reliability of digital badging, attempting to answer common digital badge critic question, “How will we know that they’re worthwhile badges issued from reputable sources?” Ms. Casilli asks readers to consider their current assumptions about what is considered valid within any education system. For example: “Does the issuer have the authority to issue a particular badge? vs. Does anyone have the authority to teach a particular class? or Does the earning of a badge indicate that the learner has learned? vs. Does the completion of a class indicate that the learner has learned what they were supposed to?” Her blog is written to inspire discussion around these questions, rather than providing the answers and works well as a starting point for critical thinking on the issue of validity/credibility/reliability in digital badging. Includes cited references.

----

Casilli, C. (2012, April 27). Badge system design: Standardization, formalization and uniqueness. Persona. Retrieved from http://carlacasilli.wordpress.com/2012/04/27/badge-system-design-standar...

Categories: badge system design

Annotation: This blog post is part of an ongoing series on Open Badges and Open Badge Infrastructure by Carla Casilli with this one focusing on designing the badges themselves.  Ms. Casilli suggests creating a badge that can evolve over time to fit new systemic categories, while retaining context and uniqueness.  Includes cited references.

---- 

Casilli, C. (2012, April 24). Mozilla Open Badges legal and privacy considerations. Persona, Retrieved from http://carlacasilli.wordpress.com/2012/04/24/mozilla-open-badges-legal-p...

Categories: legal issues, K-12, higher ed

Annotation: Carla Casilli examines the intersection between FERPA, COPPA, and a digital badging goal of “lifelong learning.” Ms. Casilli emphasizes the importance of  identity protection for the individual badge earner and addresses Mozilla’s steps in ensuring this protection, including a link to their Legal FAQs page. She concludes her blog by defining COPPA and FERPA: “COPPA is the acronym for the Children’s Online Privacy Protection Act and it’s a US federal law designed to govern and protect children’s online privacy and safety.” “FERPA, also an acronym and also a US federal law, stands for the Family Educational Rights and Privacy Act. It’s aimed at providing parents with the right to protect the privacy of children’s education records.”

---- 

Casilli, C. (2012, August 24). Mozilla Open Badges: Building trust networks, creating value. Persona. Retrieved from http://carlacasilli.wordpress.com/2012/08/24/mozilla-open-badges-buildin...

Categories: badge design

Annotation: This post builds on one of Carla Casilli’s earlier posts, Badge system design: What we talk about when we talk about validity (included in this bibliography) and in it she discusses how the “currency” of digital badges stems from trust and value as well as the possibility for badges to develop new communities of trust. Casilli’s blog includes many helpful visual aids for thinking about badge design, including a visual model demonstrating the intersection of reliability, replicability, credibility, validation, certification, accreditation, verification, and authentication when developing trust in a system. She also addresses how these intersections may vary and grown through the use of the open badge infrastructure and the new dynamic possibilities that could arise from them.

---- 

Cross, S. & Galley, R. (2012, November 16). MOOC Badging and the Learning Arc. Oldsmooc. Retrieved from http://www.olds.ac.uk/development-blog/moocbadgingandthelearningarc

Categories: badge design

Annotation: This blog post is about the digital badging system that was designed for Open Learning Design Studio’s MOOC (OLDS-MOOC) “Learning Design for a 21st Century Classroom” and is written by Rebecca Galley, OLDS-MOOC Projects Officer, and Simon Cross, OLDS-MOOC evaluator. The creators of the course admit that they are unsure of how badges will impact their course, but demonstrate through their blog that their choices were well considered. In creating their badge system, they envisioned their course having a “learning arc” (just like a movie has a “story arc”) and where each badge would fit into the arc. The badges are available in three types: reaching an endpoint, cumulative achievement, and performing optional/independent study beyond the course.  The post includes diagrams of each type as well as a helpful table that explains the role of each badge and the badges’ benefits to the receiver/learner and to the creator/awarder. Here is a link to their badge system that’s compatible with Mozilla Open Badges. Their MOOC ran January 10 - March 13, 2013.

---- 

Davidson, C. N. (2012, February 21). Can badging be the zipcar of testing and assessment? DMLCentral. Retrieved from http://dmlcentral.net/blog/cathy-davidson/can-badging-be-zipcar-testing-...  

Categories: assessment, education, motivation, K-12, higher education

Annotation: Author, Cathy N. Davidson, starts her article by explaining her initial skepticism of digital badges, saying, “I was afraid that, rather than taking us ‘beyond educational technology,’ badges would throw our passionate digital learning community into yet another morass of over-priced tech.” After further consideration, she views badges not as an end to assessment concerns, but a way to shift a paradigm away from standardized testing and No Child Left Behind. In addition, the author addresses the Digital Media and Learning Competition’s “Badges for Lifelong Learning.”

---- 

Davidson, C. N. (2012, September 20). How do we measure what really counts in the classroom? Fast Company Co-exist. Retrieved from: http://www.fastcoexist.com/1680584/how-do-we-measure-what-really-counts-...  

Categories: assessment

Annotation: Cathy N. Davidson’s blog begins with a brief history of standardized testing program and she explains that studies have shown that standardized tests do not motivate learning. Instead of standardized testing, her post offers an alternative form of assessing student learning with Khusro Kidwai and Kyle Peck’s free eRubric assessment system. Their program could be used to create peer assessment or have students create the eRubric with the professor on the first day of class. She states that “all research on assessment shows we learn more if we understand, participate in, and agree with the basic learning or work goals we’re aiming at.” New system of assessment like the eRubric mentioned above and digital badges being developed at Top Coder, Stack Exchange and the Digital Media and Learning Competition allow for assessing things such as collaborative efforts and seeing a project through. She hopes that “soon, we may well have automated, easy, teacher-friendly, student-inspiring assessment systems that actually measure what we value and count the kind of knowledge and thinking that really do count in the classroom and in the real world.”

---- 

Davidson, C. N. (2011, October 9). Institution, know thyself! How members can decide what counts #dmlbadge. HASTAC. Retrieved from http://hastac.org/blogs/cathy-davidson/2011/10/09/institution-know-thyse...  

Categories: badge design, assessment

Annotation: The focus of this blog by Cathy N. Davidson is the importance of moving away from inherited rigid and hierarchical systems of assessment and towards creating new measures for learning that truly assess learning achievements. She states that, “without an inherited system of accreditation, members of the system can take the time to assess what they think is important, what they think counts, and how they think things should count. “ She is interested in the possibility of digital badges (as redesigned assessment measures) to truly assess learning achievements and believes that redesigning assessment systems also allows for institutional introspection. She explains that, “redesigning its internal systems of evaluation offers an institution--whether a business, a school, a class, a network, a community organization or something else-- an opportunity for collective, institutional introspection, for a conversation about values, merit, metrics, and the purpose of assessment for that organization and its members.” 

---- 

Davidson, C.N. (2011, September 6). Why badges? Why not? HASTAC. Retrieved from http://hastac.org/blogs/cathy-davidson/2011/09/16/why-badges-why-not  

Categories: assessment

Annotation: Cathy N. Davidson’s blog post about digital badges begins with a reflection on the Digital Media and Learning Competition’s Badges for Lifelong Learning launch event and its participants (Secretary of Education, Arne Duncan; Vice President of U.S. Programs at the MacArthur Foundation, Julia Stasch; Executive Director of Mozilla, Mark Surman, and NASA Administrator Charles F. Bolden) and is a call for individuals and organizations to participate in the competition.  She highlights the steps of the competition, including the role of HASTAC in the competition, with an emphasis on the goal of having “an array of actual, open, working and functioning systems of accreditation that will help all of us, in any situation, think about whatever method our institution now uses and to be able to see, in concrete detail, how things might be better by adopting this new system that these teams have worked on together.” She provides a full exploration of educational and institutional standards, ways of measuring or assessing learning and the role of badges in this process.

---- 

Davidson, C.N. (2011, March 21). Why badges work better than grades. HASTAC. Retrieved from http://hastac.org/blogs/cathy-davidson/why-badges-work-better-grades  

Categories: higher education, professional development

Annotation: Cathy N. Davidson’s blog explains the potential of a digital badge system through the example of a stellar student in one of her courses. She describes digital badges as being comprised of four main components- a badge represents the issuing organization/community, it is a visual representation of mastery, it provides motivation for further learning/mastery and it must be accepted by outside communities (legitimization) to work. She explains that, “badges are useful for certifying complex processes or skills that are not comprehended in our traditional grading systems”, like “interpersonal skills, collaborative skills, imagination, innovative, initiative, independence” that employers value, but are not expressed through traditional grading systems.  The blog is followed by comments with links to additional articles on badges and a representation of both badge promoters and detractors.

---- 

Duncan, A. (2011, September 15). Digital badges for learning: Remarks by Secretary Duncan at 4th annual launch of the MacArthur Foundation Digital Media and Lifelong Learning Competition. Retrieved from http://www.ed.gov/news/speeches/digital-badges-learning

Categories: higher education, K-12, assessment, open education

Annotation: This is the transcript of an address by U.S. Secretary of Education, Arne Duncan, about the Digital Media and Learning (DML) Competition’s “Badges for Lifelong Learning” and how it intersects with the White House’s current efforts in transforming education, including their National Education Technology plan titled, "Transforming American Education: Learning Powered by Technology" and the “American Jobs Act”, part of which will be focused on increased funding for educators. His speech addresses educational technology, assessment, and specifically mentions the creation of a digital badging system for U.S. veterans. Regarding the DML Competition, he stated, “we're excited that, this year, this competition will serve as a catalyst to advance the potential of digital badges. Badges can help engage students in learning, and broaden the avenues for learners of all ages to acquire and demonstrate – as well as document and display – their skills. Badges can help speed the shift from credentials that simply measure seat time, to ones that more accurately measure competency.” While he admits that there is not one simple solution to the current state of education in the United States, he believes that educational technology has the ability to change the landscape of teaching and education. This article will be of interest to anyone researching U.S. education policies.

---- 

Elkordy, A. (2012, October 24). The future is now: Unpacking digital badging and micro-credentialing for K-20 educators. HASTAC. Retrieved from: http://hastac.org/blogs/elkorda/2012/10/24/future-now-unpacking-digital-...

Categories: higher education, K-12, credentialing, open education, assessment, motivation, professional development

Annotation: This is an engaging, well-written blog post that includes a thorough description of digital badges and a well-sourced bibliography. It addresses the pro-badging community and detractors as well as how digital badging is being explored not only in technology communities, but other communities like the U.S. Department of Education, the U.S. Department of Veterans Affairs and school districts. The author explores the impact of digital badges on open education, educational assessment, motivation for learners, professional development for educator, Mozilla Open Badge Infrastructure (OBI) and credentialing. Ms. Elkrody’s blog is strongly recommended as a starting point for those interested in digital badges and education.

---- 

Goldberg, D. T. (2012, March 6). Badges for learning: Threading the needle between skepticism and evangelism. DMLCentral. Retrieved from http://dmlcentral.net/blog/david-theo-goldberg/badges-learning-threading...

Categories: assessment, motivation, lifelong learning

Annotation: The blog is written partly as a response to badge skeptic Mitch Resnick’s concerns about motivations in digital badging being extrinsic (Utilitarian), that learners will focus on accumulation of badges, rather than a more intrinsic (Kantian) goal of “learning for its own sake.” Mr. Goldberg asks whether these two motivations must be mutually exclusive: Can’t initial extrinsic motivation lead to intrinsic motivation? He also postulates that badging can serve as a prompt for curiosity due to  its  very innovative nature, its newness and “cool” factor being a catalyst for new learning. He states that they are not a fix-all, but they offer a new way of thinking about the process of lifelong learning.

---- 

Grant, S. (2012, October 5). Questions about badges in higher ed. Retrieved from http://hastac.org/blogs/slgrant/2012/10/05/questions-about-badges-higher-ed

Categories: higher education, assessment, badge design

Annotation: Sheryl Grant’s blog post focuses on examples of digital badge design and uses at colleges and universities. Her examples range from classroom-level to campus-level badging as well as including examples of digital badges for cultural events, information literacy certificates from academic libraries and badges for conference participation. She also addresses digital badging as a method of alternative assessment in education, including peer evaluation models coming from Peer to Peer Recognition of Open Learning in Education (P2PU) and Professor Kyle Peck from the Pennsylvania State University. Ms. Grant’s post includes articles about digital badges from mainstream media outlets, such as The New York Times and the Huffington Post and it closes with a series of questions to inspire further discussion about Amy McQuigge’s big question: How can colleges and universities use badges?

---- 

Grant, S. (2011, September 24). Unpacking badges for lifelong learning. HASTAC. Retrieved from http://hastac.org/blogs/slgrant/2011/09/25/unpacking-badges-lifelong-lea...

Categories: motivation

Annotation: Sheryl Grant’s blog post explores both the positive and negative reactions to digital badges complete with links to the articles she references. Much of the concern and applause for digital badges centers around the idea of motivation. In addition to the issue of motivation, Grant also examines the overlap or intersection between learning theories and online communities, online collaboration, social participation, making, authenticity, and the idea of beginning to re-think online lurking as reading and learning. She closes her provocative article with an open call for further discussion about digital badges and how they may effect our process of learning and our use of the internet.

---- 

Halavais, A. (2012). Badges: The skeptical evangelist. Retrieved April 10, 2012, from http://alex.halavais.net/badges-the-skeptical-evangelist

Categories: assessment, formal and informal learning, motivation, credentialing, game-based learning, education

Annotation: Is digital badging good or bad? Alex Halavais’ simple answer is “yes.” If they are poorly applied, they will be bad and if used well, they can lead to peer learning and authentic assessment. The author believes that badges could provide opportunities for peer assessment, a method of creating a personal narrative and a means to seek out people with specific knowledge within our growing online communities. One of his concerns lies in the connection between badges and gamification. He discusses Henry Jenkin’s critique of badges that it could become a “badgecopalypse,” meaning - what if badges just take over, instead of providing another way of thinking?, which could create a badge monopoly. In regards to the Digital Media and Learning Competition (DML) of 2012, he says, “I worry that by winnowing 500 applications to 30, we may have already begun to centralize what “counts” in approaches to badges. But perhaps the skeptical posts I’ve linked to here provide evidence of the contrary: that the competition has encouraged a healthy public dialog around alternative assessment, and badges represent a kind of ‘conversation piece.’“ The blog post provides an even assessment of the possible impact of digital badges.

---- 

Hickey, D. (2012, October 3). Incorporating Open Badges into a hybrid course context. Retrieved  from: http://remediatingassessment.blogspot.com/2012/10/incorporating-open-bad...

Categories: higher education, assessment, legal issues

Annotation: Dan Hickey’s blog is about his real-life application of a badging system as a method of peer assessment/evaluation for his doctoral course on educational assessment. He integrated the OBI-compliant digital badge system, ForAllBadges, into OnCourse, Sakai-based open-source CMS. In creating this system of digital badges, Mr. Hickey considered credibility (inside and outside of the classroom) and privacy (FERPA). He incorporated descriptive criterion for each badge that could be awarded and decided to not make the digital badges part of the student’s final grade.  His blog post will be of interest to people considering the application of digital badges within the classroom.

---- 

Hickey, D. T. (2012, March 11). Initial consequences of the DML 2012 badges for lifelong learning competition. Retrieved from http://remediatingassessment.blogspot.com/2012/03/initial-consequences-o...

Categories: assessment

Annotation: This brief blog looks at the application/judging process and initial consequences of the Digital Media and Learning Competition’s “Badges for Lifelong Learning.” More than 300 submissions from individuals and groups were submitted, evaluated and eventually matched up (and whittled down) into a few dozen “content and program” teams matched with “design and tech” teams for the final stage of competition.  Dan Hickey explains that while it is too early to tell what the eventual outcome of the competition cycle will be, the competition quickly demonstrated a high level of interest and submissions for a competition with a low ($2 million) prize offering and that the process of submitting an entry laid the groundwork for these initiatives to take off whether they were awarded funding or not.

---- 

Hickey, D. T.  (2012, October 21). Initial Questions about Digital Badges and Learning. Re-Mediating Assessment. Retrieved from http://remediatingassessment.blogspot.com/2012/10/daniel-hickey-this-pos...

Recognizing, Supporting, and Attracting Adult Learners with Digital Badges. The Evolllution. This was also published as: http://www.evolllution.com/program_planning/recognizing-supporting-and-a... )

Categories: motivation, assessment, education, legal issues, validity, theory

Annotation: This blog opens with a brief description of how the Digital Media and Learning Competition, “Badges for Lifelong Learning” has impacted how we think about digital badges since its announcement in late 2011. Dan Hickey explains that this “is an initial effort to identify the initial questions about learning that educators, programs, and schools might want to ask if they are thinking about issuing digital badges.” He poses questions such as, how will you introduce digital badges? how can you ensure the validity of badges? how will badges function- motivation, recognition? and so on. He also proposes that individuals and organizations who are thinking of employing digital badges should consider questions around assessment, proposed learning theories, refining the method of using digital badges, and protecting the privacy of badge recipients (FERPA and COPPA).

---- 

Hickey, D. T.  (2012, September 9). Intended Purposes Versus Actual Functions of Digital Badges. Re-Mediating Assessment. Retrieved from http://remediatingassessment.blogspot.com/2012/09/intended-purposes-vers...

Categories: motivation, assessment

Annotation: Dan Hickey’s post focuses on “the difference between the purposes we intend to use digital badges for and the actual functions that they ultimately serve.” He outlines what he sees as four different functions for badges: recognizing learning, assessing learning, motivating learning and evaluating learning. He acknowledges that the function of motivation is the most controversial for digital badges and his post includes a table to summarize the four functions. (He explains that digital badges can be used to recognize learning, motivate learning or both and that there is a difference between earning badges immediately upon completion of a learning experience/task vs. that learning experience/task being reviewed by a peer or by an expert upon completion. He addresses summative vs. formative vs. evaluative assessment in regards to digital badging.)

---- 

Hickey, D. T.  (2012, October 30). Introducing digital badges within and around universities. Re-Mediating Assessment, Retrieved from http://remediatingassessment.blogspot.com/2012/10/introducing-digital-ba...

Categories: higher education, assessment

Annotation: In the blog post, Dan Hickey explains why he believes digital badges will succeed at Indiana University (IU helped develop Sakai and was where the International Society for the Scholarship of Teaching and Learning launched from, IU’s President gave $8M for the IU Online initiative which is being headed up by a professor in the Instructional Systems Technology Program and one of their professors is national known as supporting MOOCs) and is a response to Sheryl Grant’s post on HASTAC about digital badges. It focuses on where to start with digital badges and explores the two avenues Ms. Grant mentioned of working in and around universities to implement badges. Hickey’s blog post details his own process of working around universities and he uses Purdue University’s new “Passport System” as an example of working in universities. His blog closes with portions of IU President McRobbie’s 2012 State of the University address in which he addresses both digital badges and MOOCs.

---- 

Hickey, D. T. (2012, March 1). Open badges and the future of assessment. Re-Mediating Assessment. Retrieved from http://remediatingassessment.blogspot.com/2012/03/open-badges-and-future...

Categories: higher education, assessment, motivation

Annotation: This blog post takes a theoretical look at the controversial topic of digital badges and motivation as well as taking a look at the Digital Media and Learning Competition, “Badges for Lifelong Learning.” The competition received 300 submissions and Dan Hickey wrote this blog just prior to attending the announcement of the competition’s winners. In it he also addresses the scope of the Competition, remarking about the connections it has made to the National Science Foundation, the US Department of Education and other federal agencies.

---- 

Hickey, D. T.  (2012, March 18). Some things about assessment that badge developers might find helpful. HASTAC. Retrieved from http://hastac.org/blogs/dthickey/2012/03/18/some-things-about-assessment...

Categories: assessment

Annotation: In this blog, Dan Hickey states that, “badge developers should consider the various goals for their badges, and the assumptions behind those goals” and he explores what those goals and assumptions may entail, using the Digital Media and Learning Competition, “Badges for Lifelong Learning” winners as helpful examples. He identifies three types of primary goals for using badges: showing what somebody has done or might be able to do (e.g, Badgework for Vets), motivate individuals to learn or do more (e.g., BuzzMath) and transform or create learning systems (e.g., MOUSE Wins!). He also identifies three primary assessment functions: summative functions (assessment OF learning), formative functions for individuals (assessment FOR learning) and transformative functions for systems (assessment AS learning) and goes on to explain the connection between different types of assessment functions and specific learning theories. Important article for those interested in assessment, badge design and learning theories.

----

Ito, M. (2012, March 7). Reflections on DML2012 and Visions of Educational Change. DMLCentral. Retrieved from http://dmlcentral.net/blog/mimi-ito/reflections-dml2012-and-visions-educational-change

Categories: innovation, theory of change, badges, connected learning

Annotation: Mimi Ito reflects on major themes and conversations at the 2012 Digital Media and Learning Conference, including the conference themes of innovation, technology, and education. Responding to conversations and blogs that focus on badges, Dr. Ito considers theories of change in education with this emerging technology, "First, a badge is only as good as the community, learning philosophy, and theory of change attached to it," and notes that badges can perform a "diversifying function in achievement metrics."

---- 

Jenkins, H. (2012, March 5). How to earn your skeptic badge. Retrieved from http://henryjenkins.org/2012/03/how_to_earn_your_skeptic_badge.html

Categories: professional development

Annotation: Author, Henry Jenkins, supports the idea of earning badges, informal learning and connected learning and accepts that badges can be a model for respecting informal learning, but not as the only model. However, he believes that people creating digital badges and pushing for their use may not be functioning with the same value set as those who would then earn the digital badges. He goes into detail about his main concerns about digital badges: young people are looking to learn without adult approval or standards (and badges would create those standards), gamification, could make informal learning more formal and hierarchical, may be overlooking other modes of participatory learning, badges simply do not work for everyone, and it’s unlikely that a single badge system will be adopted as the only option. Jenkins ends his article with a call for more research into digital badges.

---- 

Joseph, B. (2012, April 2). Badges for learning: An abridged history. Online Leadership Program: Global Kids. Retrieved from http://www.olpglobalkids.org/content/badges-learning-abridged-recent-his...

Categories: digital badges history, game-based learning

Annotation: Barry Joseph’s brief history of digital badges includes the XBOX 360 Gamerscore system, Eva Baker’s address “The End(s) of Testing”, the influence of alternative assessment and games & learning scholars, the release of white paper “An Open Badge Framework” by Peer to Peer University (P2PU) and Mozilla, and ends with HASTAC’s launch of the Badges for Lifelong Learning Competition in late 2011. Excellent and quick to read primer for those interested in the historical roots of the digital badge movement. 

---- 

Joseph, B. (2012, May 14). Designing a program-based badging system at Global Kids. Online Leadership Program: Global Kids. Retrieved from http://www.olpglobalkids.org/content/designing-program-based-badging-sys...

Categories: badge design

Annotation: In this blog post, Barry Joseph explains Global Kids’ process of creating a badge design system that will work best for the organization as a whole, the individual programs and for the badge earners. First, they created a “global” digital badge system based on Global Kids’ organizational outcomes and indicators and next they developed a more specific program based badging system. Each program group created a system of digital badges based on a set of learning achievements and skills they felt would be beneficial for their participants. The author provides a diagram to explain how they are integrating four types of achievement badges: hard skills, soft skills, knowledge and participation to comprise one “role” badge earned, using the example of “Game Designer” role badge.

---- 

Joseph, B. (2012, June 25). Six ways to look at badging systems designed for learning. Retrieved from http://www.olpglobalkids.org/content/six-ways-look-badging-systems-desig...

Categories: K-12, motivation, assessment, game-based learning, open education

Annotation: The content of Barry Joseph’s blog was created through an open access collaboration and presents six frameworks for examining digital badging for K-12 learners to explain why people align themselves with badges and what their goals are in using badges. The six frameworks include: badges as alternative assessment, gamifying education with badges, badges as learning scaffolding, badges to develop lifelong learning skills, badges as DML driver, and badges to democratize learning and each framework presented provides real-life examples corresponding to the framework and most include “vital voices”, a section that makes reference to corresponding literature/blogs, including critiques. Excellent resource for those wanting to learn about reasons for using digital badging with K-12 learners.

---- 

Knight, E. (2012, June 11) 7 Things You Should Know About Badges. EDUCAUSE Learning Initiative. Retrieved from http://www.educause.edu/library/resources/7-things-you-should-know-about...

Categories: professional development

Annotation: This write up about digital badges functions like an FAQ that starts with basics of what a digital badge is and moves onto the bigger questions of “What is the significance?” and “What are the implications for teaching and learning?” It also contains a scenario about an individual earning digital badges for professional development that helps illustrate the articles explanation of digital badges. A good starting point and quick read for someone who is not familiar with digital badges.

Knight, E. (2013, February 11). An Open Distributed System for Badge Validation. World of E's. Retrieved from http://erinknight.com/post/42841860849/an-open-distributed-system-for-badge-validation

Categories: validation, Open Badges Infrastructure

Annotation: Erin Knight of the Mozilla Foundation discusses the purpose of the Open Badges Infrastructure (OBI), which is to establish an open credentialing system as an alternative to the current formal education system. Because the OBI allows systems of assessments and credentials to be portable across the Web, it has the potential to unlock the current credentialing monopoly that formal education holds. In this blog post, Erin Knight reflects on a proposal and working paper, RFC: An Open, Distributed System for Badge Validation (Working Paper) that seeks input from the public about ways to provide validation in an open credentialing system such as the OBI.

---- 

Knight, E. (2010, December 27). Certification’ Revisited (#1). Retrieved from http://erinknight.com/post/2488034871/certification-revisited-1

Categories: motivation, assessment, open education

Annotation: Author, Erin Knight, works Mozilla Foundation on assessment in open peer learning environments and digital badging and she has written this post just prior to piloting a badge/assessment project with Peer to Peer University (P2PU). Her post provides an extended definition for digital badges and why she believes we need an alternative form of assessment and motivation in our current education system. She closes her post with a thorough list of questions about digital badges and how they could/will be employed.

---- 

Ledesma, P. (2011, July 10). Can badges offer viable alternatives to standardized tests for school evaluation? Retrieved from http://blogs.edweek.org/teachers/leading_from_the_classroom/2011/07/can_...

Categories: assessment, K-12, open badge system, motivation

Annotation: Patrick Ledesma compares and contrasts digital badges with K-12’s current system of standardized testing. He addresses the policy implications for moving to a digital badging system, including the idea that badging might make it easier for the public to understand more about the learning process of students. Mr. Ledesma suggests that digital badges could be a good method of developing critical thinking in students and better learning motivation, while also carefully considering the challenges this new system could create. The article contains a great illustration of the process of earning and displaying digital badges as well as thoughtful comments by readers.

---- 

Lovejoy, T. Badges: The way the new learner learns. Digital Learning Environments. Retrieved from http://www.guide2digitallearning.com/blog_tonia_lovejoy/badges_way_new_l...

Categories: professional development

Annotation: Author, Tania Lovejoy, notes that people are increasingly completing webinars, workshops and doing other types of online learning outside of the traditional classroom environment to prepare themselves for the workforce and badges provide a way to represent those skills. She wrote this blog as a recently returned Peace Corps volunteer and explains her personal learning system has been rooted in her work, rather than a graduate degree and system of badging will allow for a better representation of her professional experience. Her blog includes references to P2PU, Khan Academy, Mozilla’s Badge Project and the Digital Media and Learning Competition.

---- 

Reid, A. (2011, September 15). Welcome to badge world. Digital Digs. Retrieved from http://www.alex-reid.net/2011/09/welcome-to-badge-world.html

Categories: motivation, validity/accrediting

Annotation: In Alex Reid’s blog post, he voices his concerns about digital badges, especially in regards to motivation and reward. He believes the problem lies with corporations and the federal government, not students and schools, because employers are the ones seeking credentials, but is still concerned with using badges to commodify experiences.  He explains that, “the commodification of learning was already quite clear in the Reagan era when we stopped thinking of higher education as a social good and instead defined it as an individual's investment in his/her human capital.” He also raises the issue of open badges and accreditation and posits that while earning digital badges is currently a free endeavor, that it will cost money to earn in the long run. And lastly, he discusses his concern about digital badges being driven by extrinsic motivation and how that will negatively impact creativity and problem-solving skills.

----

Resnick, M. (2012, February 27). Still a badge skeptic. HASTAC. Retrieved from http://hastac.org/blogs/mres/2012/02/27/still-badge-skeptic

Categories: motivation, critique, education

Annotation: In this blog post, Mitchel Resnick, Professor of Learning Research at the MIT Media Lab, discusses why he is still a badge skeptic.  His main concern lies in the issue of motivation in earning badges - that students will focus on accumulation of badges (extrinsic motivation) rather than on the process of learning (intrinsic motivation).  Mr. Resnick cites the “Scratch Online Community” as an alternative tool focusing on intrinsic motivation for learning that engages students to become contributing community members. His blog post created a great dialogue about motivation in badging from some key players in the digital badge movement.

----

Zellner, A. Badges as goals: Achievement goal theory. Retrieved from http://www.andrea-zellner.com/archives/903

Categories: motivation, critique, learning theory

Annotation: This is part of a series of blogs by Andrea Zellner written in response to the Digital Media and Learning Competition’s Badges for Lifelong Learning and a course she is taking called, Current Issues in Motivation and Learning. In this blog, she addresses the difference between achievement goals -students focusing on their own abilities- and performance goals -students focusing on besting other students. She argues that first orientation values the process of learning and therefore have more positive learning outcomes. She states that, “badge systems are certainly patterned after similar systems within digital games, where these goal orientations have resulted in the negative outcomes associated with performance goal orientations” and closes with concerns she has heard expressed about people gaming the system at Khan Academy and their use of ranking systems. Contains brief list of references.

----

Zellner, A. Badges as gold stars: The behavioral view of motivation and learning. Retrieved from http://www.andrea-zellner.com/archives/881

Categories: motivation, learning theory

Annotation: This is part of a series of blogs by Andrea Zellner written in response to the Digital Media and Learning Competition’s Badges for Lifelong Learning and a course she is taking called, Current Issues in Motivation and Learning. In this post, she discusses thinking about badges as a rewards system instead of an assessment system and looks to B.F. Skinner’s theory of operant conditioning. If badges are provided as a reward, they will be encouraged to try to earn another badge- the badge acts as a reinforcer and the variable-ratio reinforcement schedule, where the “reinforcer is presented at intermittent times after the behavior is demonstrated”, has been found to be the most effective. Zellner notes that this only works if the digital badge actually proves to be a positive reinforcer and she expresses concern that providing rewards such as digital badges, may decrease intrinsic motivation overall. Contains brief list of references.

return to top of page

News Articles

Carey, K. (2012, March 13). “The higher education monopoly is crumbling as we speak.” The New Republic. Retrieved from  http://www.tnr.com/article/politics/101620/higher-education-accreditation-MIT-university 

Categories: higher education

Annotation:While many companies have folded over time, universities have been experiencing longevity and staying power. But author, Kevin Carey, believes the advent of online education, such as MITx and Stanford’s online offerings on Artificial Intelligence and the creation of Udacity may change that. He does not believe traditional degrees will be immediately superseded, but they are an example of the theory of “disruptive innovation.” The remainder of his article gives an in-depth look at this theory created by Harvard business school professor Clayton Christensen and what it means for the traditional model of university-level education.

----

Carey, K. (2011, May 15). The quiet revolution in open learning. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/The-Quiet-Revolution-in-Open/127545/ 

Categories: assessment, professional development

Annotation:This article is about US President Barack Obama’s proposed community college reform agenda for increased funding of community colleges that was overturned by Congress in 2010 and the plan that the Departments of Labor and Education came up with in its stead. It was proposed that community colleges seeking federal funding would be required make their course materials open education resources (OERs). The author surmises that because these courses and materials will be linked with the federal government and its standards, that employers may view the completion of this online coursework with the same level of quality as a course completed in a traditional classroom setting. However, these courses will not provide learners with “credit” and the author brings in the subject of digital badges as a form of signified proof of informal learning.

----

Carey, K. (2011, May 18). We do in fact need some (non-stinking) badges. The Chronicle of Higher Education.  Retrieved from https://chronicle.com/blogs/brainstorm/we-do-in-fact-need-some-non-stinking-badges/35447 

Categories: professional development, assessment

Annotation:This article is a follow-up to Kevin Carey’s article “The Quiet Revolution in Open Learning” and a response to a critical comment from said article. In it, Carey provides two examples of professions that can and often do function outside of the system of formal education- lawyers and journalists, respectively. He discusses the bar exam as an example of an open badge earned – “an independent assessment that, in theory, anyone can take, based on a distinct body of knowledge and skills determined by a professional guild outside of higher education” and that journalists’ bodies of work serve as their credentials.

----

Crotty, J.M. (2012, January 26). Why get a pricey diploma when a bleepin’ badge will do? Forbes. Retrieved from http://www.forbes.com/sites/jamesmarshallcrotty/2012/01/26/the-end-of-the-diploma-as-we-know-it/ 

Categories: motivation, assessment, higher education

Annotation:This article is a cheeky contribution from James Marshall Crotty, who commonly writes about the education sector, about the advent of digital badges.  Mr. Crotty concedes that the current education/work system that values a four year university degree over all other forms of education is somewhat lacking, but his tone implies that he doesn’t necessarily view badges as the answer. His articles mentions the support of digital badges by industry and education reform organizations like Mozilla, MacArthur Foundation and the Digital Media and Learning Competition, Microsoft, Khan Academy, Peer-to-Peer University (P2PU) and others and also gives space for detracting arguments, including concerns about extrinsic motivation. He ends his article with a discussion of the validity of badges, by saying, “the biggest knock on badges, however, is that they are another shallow gimmick that fails to solve an age-old problem: quality assurance.”

----

Davidson, C.N. (2012, February 7). Badges: a solution to our teacher evaluation disaster.  Washington Post. Retrieved from http://www.washingtonpost.com/blogs/answer-sheet/post/badges-a-solution-to-our-teacher-evaluation-disaster/2012/02/06/gIQAHiNbvQ_blog.html#pagebreak 

Categories: assessment

Annotation: Cathy N. Davidson’s article in the Washington Post begins with Google’s experience of evaluating the qualities of their managers and their surprising results. Google’s flexibility and willingness to accept their results, rather than demand certain standards be met has allowed them to continue being an innovative company. Need to ask the right questions – “All the data in the world doesn’t matter if you are collecting one kind of information but the real problem or virtue lies elsewhere.” Davidson posits that end-of-term, multiple-choice tests are not asking the right questions and an alternative assessment that measures learning innovation is necessary.  She brings in the example of Stack Exchange, a site for web programmers - an innovative community who is buoyed by peer assessment and earning digital badges. She closes with a discussion of how standardized testing has been found to dis-incentivize learning, whereas badges can incentivize learning and the work HASTAC, Mozilla and the MacArthur Foundation are doing on digital badges.

----

Eisenberg, A. (2011, November 19). For job hunters, digital merit badges. The New York Times. Retrieved from http://www.nytimes.com/2011/11/20/business/digital-badges-may-highlight-job-seekers-skills.html 

Categories: professional development, motivation, formal vs. informal learning

Annotation:This news article by Annie Eisenberg has one of the best and easiest to understand explanations of the purpose of digital badges. Includes quotes from Connie M. Yowell, Director of Education Grant-making at the MacArthur Foundation and Cathy N. Davidson, Co-founder of HASTAC. Ms. Yowell describes digital badges as the, “glue to connect informal and formal learning in and out of school.”  The article provides a simplified explanation of the metadata ascribed to each digital badge through the example of a prospective employer clicking on an e-badge awarded for prowess in Javascript and seeing detailed supporting information, including who issued the badge, the criteria and even samples of the work that led to the award. In addition, the author discusses the structure of the MacArthur Foundation’s Digital Media and Learning Competition’s Badges for Lifelong Learning, explaining that money is awarded for winning projects and also given to Mozilla to develop a digital badge structure as well as acknowledging that the organization, Topcoder, also has a competition for digital badges.

----

Fain, P. (2012, December 7). Digital Badging for Veterans. Inside Higher Ed. Retrieved from: http://www.insidehighered.com/news/2012/12/07/website-recognizes-military-skills-digital-badges 

Categories: professional development

Annotation: Paul Fain’s article is about the BadgesforVets.org website and its goal of translating skills and knowledge learned in the military into digital badges that veterans can display on their résumé or LinkedIn profile. The site will be focused on credentials, not new assessment measures and the badges will be completely unconnected from the concept of gamification.  Digital badges for military veterans is an interesting test case, because of the military has been recognized for providing “standardized, verifiable and quality training programs.”  The article closes with examples of digital badges, such as “communications” and “civil affairs”, each including descriptive content.

----

Kilb, S. (2011, November 6). Credentials, the next generation.  New York Times. Retrieved from http://www.nytimes.com/2011/11/06/education/edlife/credentials-the-next-generation.html?_r=2 

Categories: assessment

Annotation: This is a brief article in the “Education Life” section of the New York Times about digital badges that includes quotes from two prominent people in the field of badges- Secretary of Education Arne Duncan and Connie Yowell, Director of Education at the MacArthur Foundation.  Secretary of Education Arne Duncan explained the federal governement’s take on digital badges, saying “badges can help speed the shift from credentials that simply measure seat time to ones that more accurately measure competency.” Connie Yowell explained the MacArthur Foundation’s reason for supporting a digital badge movement by saying, “we really believe that we’re launching a national conversation about what skills matter and how those skills get assessed.”

----

Lewin, T. (2012, March 4). Beyond the college degree: online educational badges.  New York Times. Retrieved from: http://www.nytimes.com/2012/03/05/education/beyond-the-college-degree-online-educational-badges.html?_r=5& 

Categories: higher education

Annotation: This is a brief article in the education section of The New York Times about MOOCs and opportunities for alternative credentialing. David Wiley of Brigham Young University posits that as soon as alternative credentials become as easy to interpret for employers as standard academic credentials (degrees), they will change the face of the current higher education system. The main focus of the article is on the possibility of cheating the system of online learning programs and the plans for in-person testing (at a cost) that have been proposed by Udacity and MITx to reduce the chances of gaming the system.

----

O’Shaughnessy, L. (2011, October 4).  Digital badges could significantly impact higher education. U.S. News & World Report. Retrieved from http://www.usnews.com/education/blogs/the-college-solution/2011/10/04/digital-badges-could-significantly-impact-higher-education 

Categories: higher education, assessment

Annotation: Lynn O’Shaughnessy’s article discusses the implications of the results found in the book written by two academics, Academically Adrift: Limited Learning on College Campuses that found a high percentage of students were not gaining the critical thinking skills that seeking a college degree is supposed to impart.  Universities have a monopoly on credentials and digital badges are a new system that could “shake-up” the monopoly of the current higher education system. Her article mentions Mozilla, MacArthur Foundation, NASA and Secretary of Education, Arne Duncan all as interested parties in digital badges.

----

O’Shaughnessy, L. (2011, October 3). Forget the college degree: earn digital badges instead. CBS News. Retrieved from http://www.cbsnews.com/8301-505145_162-37246793/forget-the-college-degree-earn-digital-badges-instead/?tag=mwuser 

Categories: professional development, higher education

Annotation: This is a brief article on CBS news by Lynn O’Shaughnessy, who writes about the financial costs of college, about the potential of digital badges to disrupt the monopoly of higher education and degrees.  It is an introduction to how digital badges can be used/displayed by badge earners and how employers may be able to learn more about a job candidate through the badges they have earned. Her article includes information about Mozilla and the MacArthur Foundation’s roles in the development of digital badges.

----

Parry, M. (2012, January 1). MIT will offer certificates to outside students in online courses. Chronicle of Higher Education. Retrieved from http://chronicle.com/article/MIT-WillOffer-Certificates-to/130173/ 

Categories: higher education, open education, assessment

Annotation: This blog by Marc Parry outlines the inception of MITx, when MIT decided to offer certificates to students outside of the university for completing their online courses. With this new inclusion, MIT will now offer a three tier system for learning- traditional MIT degree, MITx certificates and free OpenCourseWare materials. In the new MITx, courses are free but there is a fee for the certificate and the certificates will not be issued under the name MIT itself. The author suggests that this new endeavor will serve as a case study for how to sustain projects when they are free and may serve to break down the barriers between the general public and larger educational institution.

----

Scott, A. (2012, June 8). An alternative to the college degree? American Public Media: Marketplace Life. Retrieved from http://www.marketplace.org/topics/life/alternative-college-degree 

Categories: professional development, credentialing

Annotation:This six-minute podcast and transcript from American Public Media’s Marketplace Life begins with sound bites about the importance of education for the American people from Mitt Romney and Barack Obama during the 2012 presidential campaign. Amy Scott talks about the ubiquitous nature of employers looking for employees with a college degree and how “economists talk about a college degree as a signaling device, a way to show employers you're a smart, responsible person people.” But the landscape is being changed by alternatives methods of credentialing and employers increased disappointment with their applicant pool’s lack of creative thinking skills, which leads them to question the value of college degrees and look for better ways to measure skills. She notes that employees are also looking for cheaper ways to gain skills and online courses offered through Straight Line program, MITx and Harvard may be an answer. The podcast also includes a section about digital badges, including an interview with Cathy N. Davidson of HASTAC.

----

Selingo, J. (2011, September 22). Think different? Not on college campuses. Huffington Post. Retrieved from http://www.huffingtonpost.com/jeff-selingo/think-different-not-on-co_b_977072.html 

Categories: professional development 

Annotation: Jeff Selingo opens his article with a discussion of how Apple revolutionized the world of technology beginning with their “Think Different” campaign and explains that higher education and credentialing may also be in for a big change. Enter, digital badges and the “boost” they received from the MacArthur Foundation, Mozilla and HASTAC in the Digital Media and Learning Competition’s “Badges for Lifelong Learning.” He argues that with increased complaints from employers that new college graduates are not meeting their expectations, if badges are used correctly they may change what employers seek as qualifications. He does note that badge issuers will need to “earn the trust of employers” for a badging system to work.

----

Wadhwa, V. (2011, 15 October). An open letter to Peter Thiel. The Washington Post. Retrieved from http://www.washingtonpost.com/national/on-innovations/an-open-letterto-peter-thiel/2011/09/14/gIQAUexrjL_story.html 

Categories: higher education

Annotation: This is a letter written by Vivek Wadhwa, senior research associate with the Labor and Worklife Program at Harvard Law School and an executive in residence/adjunct professor at the Pratt School of Engineering at Duke University, to Peter Thiel, most commonly known as the co-founder of PayPal, after their debate about the value of higher education that aired on the show “60 Minutes.” Vivek Wadhwa believes that achieving an academic degree is paramount while Peter Thiel believes is not necessary and instead promotes learning outside of the academy in his own program for “Thiel Fellows.” In the letter, Wadhwa encourage Thiel to use his financial means and the ingenuity he showed in revolutionizing banking with PayPal to use technology to revolutionize the education system. Wadhwa also mentions education through gaming, open courses at MIT and Harvard and making education affordable as steps toward improving the education system.

----

Watters, A. (2011, November 15). How will Mozilla's Open Badges project affect higher ed? Inside Higher Education. Retrieved from http://www.insidehighered.com/blogs/hack-higher-education/how-will-mozillas-open-badges-project-affect-higher-ed 

Categories: higher education, DML competition, Mozilla, assessment, gamification

Annotation: This blog is a preliminary response to the 2011 announcement of the Digital Media and Learning Competition, “Badges for Lifelong Learning.” Audrey Watters explains that digital badges are merit-earned badges with an open infrastructure framework and a chance for people to earn “credit” for learning done outside of the official classroom setting. She provides Peer to Peer University’s (P2PU) and Mozilla’s pilot project to award badges for P2PU School of Webcraft as an example of how digital badging can work and briefly explains Mozilla’s open badging structure.

----

Watters, A. (2011, August 10). Mozilla’s Open Badges project: A new way to recognize learning. MindShift. Retrieved from http://blogs.kqed.org/mindshift/2011/08/mozillas-open-badges-project-a-new-way-to-recognize-learning/ 

Categories: higher education, professional development, assessment

Annotation: This article highlights Mozilla’s work with Peer-to-Peer University (P2PU) on creating an open badge system. The main premise behind the idea of digital badges is that the institutions and organizations traditionally responsible for accreditation no longer match the realities of what learning looks like today. The author explains that a degree does not necessarily indicate your skill proficiency and earning digital badges is a means of gaining skills and then showing those skills to potential employers- skills that are not necessarily represented by a degree. An open badge system means that anyone would be able to create a badge and that everyone would have a place to put their badges. Author, Audrey Watters, regularly reviews educational technology. The article includes a concept map of the Open Badges System Infrastructure as well as example badges from Mozilla.

----

Watters, A. (2011, September 8). Master a new skill? Here’s your badge. O’Reilly Radar. Retrieved from http://radar.oreilly.com/2011/09/open-badges-project-learning-education.html 

Categories: motivation, game-based learning

Annotation:This article is an interview with Erin Knight, Project Director for the Open Badges Project at Mozilla, about Mozilla’s Open Badge Initiative of merit earned digital badges with an open framework.  Ms. Knight answered questions about the impetus for the Open Badge project, the system’s technology infrastructure, benefits for badge users and badge earners, the link between Open Badges and gamification, and the future of Open Badges. This article provides good insight into Mozilla’s plan for digital badges through the lens of the key player in the Open Badge movement at Mozilla. 

----

Young, J.R. (2012, January 8). Badges’ earned online pose challenge to traditional college diplomas. Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Badges-Earned-Online-Pose/130241/ 

Categories: higher education, professional development, game-based learning

Annotation: Digital badges can be used to certify and recognize “soft skills” learned - skills that may be unclear by looking at a potential employees diploma, transcripts and traditional grades.  Digital badges are not only being provided by organizations focusing on computer programming skills, for example, but universities are also considering using them (MITx) and they are being applauded by many education reformers. While the author recognizes and details the many benefits of digital badges, he also weighs the challenges including resume overload and employers having to evaluate what a badge means. The article contains quotes from Cathy Davidson and Sheryl Grant of HASTAC and Belle S. Wheelan, president of the Commission on Colleges of the Southern Association of Colleges and Schools.

----

Young, J.R. (2012, October 14). Grades out, badges In, The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Grades-Out-Badges-In/135056/ 

Categories: assessment, higher education, professional development, motivation

Annotation: Jeffrey Young’s article considers digital badges as a replacement or supplement to the traditional “broken” grading system at universities.  It features content from Erin Knight at Mozilla about their open badge system, from Dan Hickey, associate professor of learning sciences at Indiana University at Bloomington, about the “collaborative engagement” badge he is using in one of his college courses and from Bill Watson, professor at Purdue University, about the 8-10 badges representing learning objectives that he has created for his online course on learning-systems design.  Mr. Young introduces all of these ideas with a focus on how future employers can learn more about their potential employees through digital badging. Interesting articles for those interested in how digital badges can be integrated into college courses to emphasize skills learned to future employers.

----

Young, J.R. (2012, January 21-22) Merit badges for the job market. The Wall Street Journal. Retrieved from: http://online.wsj.com/article/SB10001424052970204301404577170912221516638.html?KEYWORDS=merit+badges 

Categories: professional development, K-12, higher education, credentialing

Annotation: Jeffrey R. Young’s blog post focuses on the impact of digital badges on professional development and employment. The post addresses the use of digital badges in relation to the Digital Media and Learning Competition- “Badges for Lifelong Learning,” MITx, Peer to Peer University (P2PU), and Khan Academy. He explains some critics’ concerns about digital badges and professional development, including credentialing and how hiring managers will interpret/consider digital badges.

return to top of page

Other

HASTAC/Digital Media and Learning Competition curators. (2012). Badges for Lifelong Learning. Retrieved from: http://digitalbadges.hastac.org/

Annotation: Scoop.it is a curation tool for online content (including blog posts, articles, visual media and more). This Scoop.it page contains more than 450 articles that were curated by HASTAC on the subject of digital badges.

----

HASTAC. (2011). Digital badges for lifelong learning: An open conversation (Video file). Retrieved from: http://www.youtube.com/watch?v=iqVidWPVBKA 

Annotation: Professional 5 minute video about “Badges for Lifelong Learning” that was created by HASTAC, featuring leaders in the field of digital badges.  Good visual/audio resource about the Digital Media Learning Competition’s “Badges for Lifelong Learning Competition” and for learning about the digital badge movement from the mouths of many of its leaders.

----

MozillaWiki (2012). Badge FAQ’s. Retrieved from: https://wiki.mozilla.org/Badges/FAQs 

Annotation: This is an extremely detailed FAQ wiki about digital badges created by Mozilla that includes general information about badges, Open Badge Infrastructure, badge pilots and conceptual framework and the questions Mozilla is currently exploring around digital badges.

----

Open Michigan wiki, https://open.umich.edu/wiki/Badges 

Categories: higher education, open education

Annotation: This is a Wiki created by Open Michigan for their submission to the Digital Media and Learning Competition, “Badges for Lifelong Learning.” Their wiki states that their “intent behind the badges project is to harness the excitement created by those who share or advocate for sharing scholarly material and use that to gain momentum in the open education movement at the University of Michigan.” It was created for the purpose of administering badges to University of Michigan students for their participation in dScribe and for their faculty and staff involved in open access, open education and Open Educational Resource (OER).  It includes sections on their research, design, Drupal module and the pilot test for their system.

----

Peck, K. (2012). Teacher Learning Journeys v3.0: Prezi presentation. Retrieved from http://prezi.com/blogb7d5qpxh/teacher-learning-journeys-v30/?auth_key=f29aca32eb1792ab75c8827c726bcefcae2fe801 

Categories: assessment, professional development, education

Annotation: Professor Kyle Peck’s, Pennsylvania State University, Prezi presentation on the concept of “Teacher Learning Journeys”, which demonstrate an educator’s learning path and professional development in the field. His Prezi explains the idea of a learning “passport” with “stamps” and “badges” to recognize the educator’s accomplishments - stamps are for simple accomplishments like completing a webinar, while badges demonstrate learning, incorporation into the classroom and then reporting on that process. He presents an innovative idea for using badges complete with illustrations and the step-by-step process.

----

Mozilla. What Are Open Badges? Retrieved from http://www.openbadges.org/en-US/ 

Annotation: This website is an “Open Badges” tutorial created by Mozilla that includes a Badge 101 quiz where you can earn a Mozilla badge.  The website also has “portals” for potential badge issuers, earners and displayers which all utilize Mozilla’s Open Badge Infrastructure (OBI). There is also an FAQ link on the front page that answers questions about: badges, badge systems, OBI and Mozilla’s relationship with the Digital Media and Learning (DML) Competition.

----

Mozilla. Mozilla Open Badges. Retrieved from https://wiki.mozilla.org/Badges 

Annotation:This Mozilla Wiki serves as an introduction page to Open Badges. In addition to a brief description of their Open Badge Project complete with an informative concept map, it provides a list of ways to get involved with digital badging and a list of resources for learning more about digital badges.

return to top of page

White Papers and Reports

Ito, M. Gutierrez, K. Livingtone, S. Penuel, B. Rhodes, J., Salen, K., Schor, J., Sefton-Gree, J., & Watkins, S.C. (2013).  Connected Learning: An Agenda for Research and Design. MIT Press. Cambridge, MA. Retrieved from: http://dmlhub.net/publications/connected-learning-agenda-research-and-design

Categories: learning theory, education, reform

Annotation: This report is a synthesis of ongoing research, design, and implementation of an approach to education called “connected learning.” It advocates for broadened access to learning that is socially embedded, interest-driven, and oriented toward educational, economic, or political opportunity. Connected learning is realized when a young person is able to pursue a personal interest or passion with the support of friends and caring adults, and is in turn able to link this learning and interest to academic achievement, career success or civic engagement. The connected learning model is based on evidence that the most resilient, adaptive, and effective learning involves individual interest as well as social support to overcome adversity and provide recognition. This report investigates how we can use new media to foster the growth and sustenance of environments that support connected learning in a broad-based and equitable way. This report also offers a design and reform agenda, grounded in a rich understanding of child development and learning, to promote and test connected learning theories.

----

Mozilla Foundation and Peer 2 Peer University, in collaboration with The MacArthur Foundation  (2011). Open badges for lifelong learning. Retrieved from: https://wiki.mozilla.org/images/b/b1/OpenBadges-Working-Paper_092011.pdf 

Categories: motivation, open education, assessment, badge design

Annotation:This white paper written by the Mozilla Foundation and Peer to Peer University (P2PU) (following their pilot on digital badges) opens up with scenarios of four different learners- learners who could benefit from their informal education being acknowledged and rewarded and presents digital badges as the solution. They explain the ability of badges in capturing and translating the learning across contexts, encouraging and motivating participation and learning outcomes, and formalizing and enhancing existing social aspects of informal and interest-driven learning. They also thoroughly address the ideas of connected learning, new skills and literacies, badge system framework, assessment, badge design and display. The paper closes by re-visiting the four scenarios and integrating digital badges into their story and then outlining the next steps for “pushing badges into the ecosystem.” It gives an in-depth and easy to read view into what the Mozilla Foundation and P2PU believe can be and will be achieved through digital badges.

----

Schwartz, D. & Arena, D. (2013). Measuring What Matters Most. MIT Press. Cambridge, MA. Retrieved from: http://dmlhub.net/publications/measuring-what-matters-most

Categories: assessment

Annotation: This report makes the argument that, If a fundamental goal of education is to prepare students to act independently in the world—in other words, to make good choices—an ideal educational assessment would measure how well we are preparing students to do so. Current assessments, however, focus almost exclusively on how much knowledge students have accrued and can retrieve. In this report, Daniel Schwartz and Dylan Arena argue that choice should be the interpretive framework within which learning assessments are organized. Digital technologies, they suggest, make this possible; interactive assessments can evaluate students in a context of choosing whether, what, how, and when to learn.

----

Shute, V., & Ventura, M. (2013). Stealth Assessment: Measuring and Supporting Learning in Video Games. MIT Press. Cambridge, MA. Retrieved from: http://dmlhub.net/publications/stealth-assessment-measuring-and-supporting-learning-video-games

Categories: assessment

Annotation: Valerie Shute and Matthew Ventura discuss problems with such traditional assessment methods as multiple-choice questions, review evidence relating to digital games and learning, and illustrate the stealth-assessment approach with a set of assessments developed for and embedded in the digital game Newton’s Playground. These stealth assessments are intended to measure levels of creativity, persistence, and conceptual understanding of Newtonian physics during game play. Finally, they consider future research directions related to stealth assessment in education.

return to top of page

Non peer-reviewed papers

Baker, E. (2007). The end(s) of testing. Educational Researcher. 36(6), 309–317. Retrieved from http://www.aera.net/uploadedFiles/Publications/Journals/Educational_Researcher/3606/09edr07_309-317.pdf 

Categories: assessment

Annotation: In 2007, Eva Baker, the President of American Educational Research Association (AERA) and Professor and Director at the University of California, Los Angeles, Center for Research on Evaluation, Standards and Student Testing, gave the Presidential Address at AERA’s annual conference. After exploring a wide range of problems with the current use of assessments within schools, she focused on her key recommendation: the development of Merit badge-like “Qualifications” that certify accomplishments, not through standardized tests, but as “an integrated experience with performance requirements.” Such a system would apply to learning both in and out of school and support youth to develop and pursue passionate interests. Baker envisioned youth assembling their Qualifications to show to their families, to colleges, to employers, and to themselves. Ultimately, Baker believed “the path of Qualifications shifts attention from schoolwork to usable and compelling skills, from school life to real life.”

----

Goligoski, E. (2012). Motivating the learner: Mozilla’s Open Badges program. Access to Knowledge, 4(1). Retrieved from http://ojs.stanford.edu/ojs/index.php/a2k/article/viewFile/381/207 

Categories: motivation, critique

Annotation: Emily Goligoski’s (Stanford University) article provides a relatively fair assessment of Mozilla’s Open Badges program from touting their use of transparency to the inclusion of a well-researched section on public concerns about digital badges from critics- Alex Reid, Patrick Ledesma and James Marshall Crotty. Mozilla’s goal in using digital badges is to legitimize informal learning and they offer access to the source code of their free, open access badge infrastructure.  The article includes images of digital badges, an illustrated “infrastructure map” designed by Mozilla for badge users and provides a detailed list of Mozilla’s intention for badges and they are supposed to work.  Her assessment includes a section on funding for digital badge initiatives, including mention of the MacArthur Foundation and HASTAC, a list of other open education initiatives, such as MIT’s OpenCourseWare, Peer-to-Peer University (P2PU), and OERCommons, a section on user review and assessment, and the possibilities for indexing and referencing digital badges.

return to top of page

Secondary Peer-reviewed journal articles

Ardichvili, A., Page, V., & Wentling, T. (2003). Motivation and barriers to participation in virtual knowledge-sharing communities of practice. Journal of Knowledge Management, 7(1), 64-77.

Assor, A., & Connell, J. P. (1992). The validity of students' self-reports as measures of performance affecting self-appraisals. In D. H. Schunk & J. L. Meece (Eds.), Student perceptions in the classroom (pp. 25-47). Hillsdale, NJ: Lawrence Erlbaum Associates.

Bandura, A., & Schunk, D. (1981). Cultivating competence, self-efficacy, and intrinsic interest through proximal self-motivation. Journal of Personality and Social Psychology, 41, 586-598.

Burke, M. & Kraut, R. (2008). Mopping up: Modeling Wikipedia promotion decisions. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW '08) (pp. 27-36). New York, NY: ACM.

Burke, M., Marlow, C., & Lento, T. (2009). Feed me: Motivating newcomer contribution in social network sites. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI '09) (pp. 945-954). New York, NY: ACM.

Cheshire, C. 2007. Selective incentives and generalized information exchange. Social Psychology Quarterly, 70, 82-100.

Eisenberger, R., & Cameron, J. (1996). Detrimental effects of reward: Reality or myth? American Psychologist, 51(11), 1153-1166.

Garfield, S. (2006). Ten reasons why people don’t share their knowledge. KM Review, 9(2), 10-11.

Golder, S., & Donath, J. (2004). Social roles in electronic communities. Paper presented at Association of Internet Researchers 5.0.

Golding, B., Brown, M., & Foley, A. (2009). Informal learning: A discussion around defining and researching its breadth and importance. Australian Journal of Adult Learning, 49(1), 34-56.

Haiyi, Z., Kraut, R., Wang, Y., & and Kittur, A. (2011). Identifying shared leadership in Wikipedia. In PART 5, Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI '11) (pp. 3431-3434). New York, NY: ACM.

Halfaker, A., Kittur, A., Kraut, R. and Riedl, J. (2009). A jury of your peers: Quality, experience and ownership in Wikipedia. In ACM WikiSy., Orlando, FL: ACM.

Heng-Li, Y. & Lai, C. (2010). Motivations of Wikipedia content contributors. Computers and Human Behavior, 26(6), 1377-1383.

Higashi, R., Abramovich, S., Shoop, R., Schunn, C. (2011). The roles of badges in the Computer Science Student Network.  Paper presented at the 2012 Games+Learning+Society Conference. Madison, WI.

Hodges, D. (1998). Participation as dis-identification with/in a community of practice. Mind, Culture, and Activity, 5(4), 272-290.

Josang, A., Ismail, R., & Boyd, C. (2007). Survey of trust and reputation systems for online service provision. Decision Support Systems, 43(2), 618-644.

Kittur, A. & Kraut, R. (2010). Beyond Wikipedia: coordination and conflict in online production groups. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10) (pp. 215-224). New York, NY: ACM.

Koh, B., Kim, Y., Butler, B., Bock, G. (2007). Encouraging participation in virtual communities. Communications of the ACM, 50(2), 69-82.

Kollock, P. (1999). The economies of online cooperation: Gifts and public goods in cyberspace. In M. Smith and P. Kollock (eds.) Communities in Cyberspace (pp 220-239). London: Routledge.

Makino, J. (1988). Productivity of research groups. Relation between citation analysis and reputation within research communities. Scientometrics, 43(1), 87– 93.

Puca, R., & Schmalt, H.D. (1999). Task enjoyment: A mediator between achievement motives and performance. Motivation and Emotion, 23, 15-29.

Resnick, P., Zeckhauser, R, Friedman, R., & Kuwabara, K. (2000). Reputation systems. Communications of the ACM, 43(12), 45– 48.

Seijts, G. H., Latham, G. P., Tasa, K., & Latham, B. W. (2004). Goal Setting and Goal Orientation: An Integration of Two Different Yet Related Literatures. Academy of Management Journal, 47(2), 227-239.

Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., et al. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 1-21.

Shaffer, D. W., Squire, K. R., Halverson, R., Gee, J. P., & Academic Advanced Distributed Learning Co-Laboratory. (2005). Video games and the future of learning. Phi Delta Kappan, 87(2), 104.

Starkey, L. (2011). Evaluating learning in the 21st century: A digital age learning matrix. Technology, Pedagogy and Education, 201(1), 19-39.

Xiong, L. and Ling, L. (2004). PeerTrust: Supporting reputation-based trust for peer-to-peer electronic communities. IEEE Transactions on Knowledge and Data Engineering, 16(7), 843-857.

return to top of page

Books

Bjornavold, J. (2000). Making learning visible: Identification, assessment and recognition of non-formal learning in Europe. Lanham, MD: Bernan Associates.

Bowker, B. & Starr, S. (2000) Sorting Things Out: Classification and Its Consequences (Inside Technology). Cambridge, MA: MIT Press.

Collins, R. (1979). The Credential Society: An Historical Sociology of Education and Stratification. New York, NY: John Wiley.

Davidson, C. N. (2011). Now you see it: How the brain science of attention will transform the way we live, work and learn. New York, NY: Penguin Group.

Davidson, C. N., & Goldberg, D. T. (2009). The future of learning institutions in a digital age.  Cambridge, MA: The MIT Press.

Falk, J. H., & Dierking, L. D. (2002). Lessons without limit: how free-choice learning is transforming education. Walnut Creek, CA: AltaMira Press.

Farmer, R., Glass, B. (2010). Building Web Reputation Systems. Sebastapol, CA: O'Reilly Media.

Kohn, A. (1993). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes. Boston: Houghton Mifflin

Lepper, M., and Greene, D. (Eds.). (1978). The Hidden Costs of Reward: New Perspectives on the Psychology of Human Motivation. Hillsdale, NY: Erlsbaum Publishers.

Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education theory, research, and applications (2nd ed.). Upper Saddle River, NJ: Prentice Hall.

return to top of page

 

PDF - May 6, 2013