Blog Post

Design-based Research Project: Who Built America? Teacher Mastery Badge System | Jim Diamond

Design-based Research Project: Who Built America? Teacher Mastery Badge System | Jim Diamond

HASTAC’s Digital Media and Learning Teacher Mastery award provides an opportunity to develop solutions for two commutual challenges in online teacher professional development (oTPD). First, to create a sustainable online program that helps teachers learn new methods of promoting student inquiry and the practice of Common Core literacy skills, and that provides them with usable classroom materials. Second, to design a system of personally and publicly meaningful badges that reflect teachers’ professional growth and that contribute to the development of the online community by fostering collaboration and mentorship. In the Who Built America? (WBA) teacher mastery badge system, neither set of activities is fully separable from the other. While social studies teachers develop classroom-teaching materials as part of the badging process, they will contribute to the growth and sustainability of the professional development program. In turn, the program will provide a framework for acquiring badges as credentials and give them currency within a broader educational context of peers and school districts.

The current DML research competition affords an additional opportunity to evaluate the WBA project’s effectiveness with respect to developing Common Core-related teaching competencies and to understanding how the activities underlying the badge system offer opportunities for professional growth. Education Development Center’s Center for Children and Technology (EDC|CCT) proposes a one-year, design-based study to investigate the effectiveness of the WBA badge system in helping social studies teachers develop Common Core literacy-aligned lessons and activities. WBA has three objectives:

1. To design an online professional development system that provides history and social studies teachers opportunities to develop, receive feedback on, and implement lessons that promote student inquiry and the practice of Common Core literacy skills;

2. To recognize teacher professional growth using a system of digital badges that represent their activities and achievements; and

3. To create an online community of practice that supports teachers as they acquire new instructional design and pedagogical skills.

To evaluate how effectively the project achieves these objectives and to gather evidence for the relationship between the project’s conceptual framework and changes in teachers’ practices as a result of participation, EDC|CCT will investigate the following questions:

Q1. What role does working toward and achieving WBA badges play in the following aspects of social studies teachers’ professional development: continuing participation in the project; peer perceptions of a teacher’s professional capabilities; school and district administrators’ willingness to treat badges as professional credentials; and teachers’ interests in developing new skills?

Q2. In what ways do badge-related activities (including unit design, peer collaboration, and assessment by master teachers) influence the development of an online teacher professional development community of practice?

Q3. In what ways, if any, do middle- and high-school social studies and history teachers’ lessons and activities addressing Common Core-related literacy standards change as a result of their participation in the WBA professional development program?

To answer the first question, we will use data from online surveys administered as teachers achieve badges and conduct a teacher focus group and interviews with school administrators at the end of each project phase. To answer the second question, we will analyze the online communications among teachers, master teachers, and project administrators, as well as comments on the surveys and during the focus groups. Finally, to answer the third question, we will analyze matched, pre/post samples of teacher lesson plans and student work: 15 teachers in Phase 1 and 25 teachers and 50 students in Phase 2. We will assess lessons and student work using the NYS Common Core literacy rubrics. At the conclusion of this evaluation we will synthesize findings from each of the questions, discuss observed relationships between badging and learning outcomes (teacher and student) for this project, and present recommendations for integrating badging activities into future oTPD projects.

Research method rationale

As a formative, context-dependent research method, design-based research (DBR) is a valuable methodological framework for evaluating a nascent professional development and badge system. In DBR projects, successive cycles of design, analysis, and redesign based on formative findings contribute to successful implementations of domain-specific products that address the needs of specific populations, which is essential, given the complexity of education settings (Borko, 2004; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Dede, Ketelhut, Whitehouse, Breit, & McCloskey, 2009). Because EDC|CCT is already an assessment consultant to the WBA project and has extensive institutional expertise in formative research, we are well-positioned to collaborate with the American Social History Project and Electric Funstuff (the project leads) by providing them with formative findings that inform iterative designs of the badging activities and professional development framework.

Awarding digital badges as credentials that have external validity and professional cachet beyond the immediate context in which they are granted (e.g., a video game) is new, especially in oTPD. Thus, understanding how the activities associated with badging will influence teachers’ professional growth, as well as the sustainability of a “self-replicating” (through the granting of Master History Teacher badges) professional development community, necessitates a multi-phase study. The same is true of learning how participating teachers see the role of badges in their professional development, as well as how educators beyond the WBA community regard the possession of digital badges as signifiers of achievement and expertise. To investigate the relationship between the badge system design and changes in teachers’ practices, as well as the longer-term and longer-range implications of participation in the WBA project for the role of badges among various levels of education communities, we propose a two-phase research project. The table below includes the activities for each phase, but briefly:

• Phase 1: Pilot with 25–50 New York City history/social studies teachers

• Phase 2: Ongoing study with 75–100 history/social studies teachers, including at least three master teachers from Phase 1; pre/post collection of teacher lesson plans and student work; end-of-year focus groups with teachers in two districts and interviews with school administrators


The findings from this design experiment will contribute to three areas. First, the Common Core literacy standards are new and little is known about how teachers will learn to integrate them into their current repertoire. This is especially true of social studies and history teachers, who typically have less training in literacy instruction than do their English/language arts and elementary school counterparts (Moje, 2008). The findings will provide valuable insight into how to help teachers integrate discipline-specific reading and writing instruction into their practices. Second, digital badges as markers of professional development are also new and the findings will provide evidence for the relationship between the badge system’s activities and educators’ perceptions of badges as certifying credentials. Finally, the study also will assist in deepening developers’ understanding of how to create an oTPD program that provides teachers with ongoing support and mentorship as they learn new skills and implement them with their students, an issue of critical importance to teachers (Barnett, 2002).


June 2013: Memo to HASTAC on findings from Phase 1

December 2013: Memo to HASTAC on findings from Phase 2; public forum in New York City that brings together WBA project administrators, teachers from Phases 1 and 2, and other local badge providers to discuss outcomes-to-date; MacArthur Foundation Digital Media and Learning Report, summarizing findings from Phases 1–2 and discussing implications for Common Core literacy instruction and digital badges

Public Forum Narrative

In October 2013 the New York City Digital Badging Symposium will bring together Who Built America? (WBA) project administrators, social studies teachers from Phases 1 and 2 of the project, other local education badge providers (e.g., Global Kids, MOUSE Squad, and other school teachers), and organizations interested in badges to discuss project outcomes-to-date, as well as other badge providers’ experiences. The following three questions will frame the symposium:

1. How effective has the process of achieving badges been in helping learners develop new skills and in creating recognizable credentials?

2. What have the challenges and achievements been in establishing a badge ecosystem? In what ways are badge providers making connections with other education providers to expand badges as credentials?

3. Going forward, how can badge providers continue to build connections between themselves and other institutions, such as schools or corporations, to promote the value of their digital badges? What elements will they need to expand (e.g., more robust assessment or methods by which to develop badge communities) to promote the efficacy of badges as credentials?

The symposium will include three panels: a panel of WBA teachers and project administrators discussing badge activities as they relate to developing new pedagogical methods focusing on the Common Core; a panel of other education-related badge providers to discuss their methods for awarding badges; and a panel of education-related badge providers and institutions interested in badges as credentials to discuss how they can build connections.


Barnett, M. (2002, April). Issues and trends concerning electronic networking technologies for teacher professional development: A critical review of the literature. Paper presented at the American Educational Research Association, New Orleans, LA.

Borko, H. (2004). Professional development and teacher learning: Mapping the terrain.Educational Researcher, 33(8), 3–15.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R. & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13.

Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60(1), 8–19.

Moje, E. B. (2008). Foregrounding the disciplines in secondary literacy teaching and learning: A call for change. Journal of Adolescent & Adult Literacy, 52(2), 96–107.


Jim Diamond is a researcher at the EDC/Center for Children & Technology in NYCand a a Digital Media and Learning Research Competition on Badging and Badge Systems Development grantee based on his winning proposal Design-based Research Project: Who Built America? Teacher Mastery Badge System. The Research Competition was funded by the Bill and Melinda Gates Foundation and administered by HASTAC, in collaboration with Mozilla.

Research Competition applicants were asked to submit proposals for empirical and theoretical research that support and inform the design, development, and deployment of digital badges and badge systems across a diverse range of learning content, institutions, and approaches, including the Gates Foundation supported Project Mastery Sites, as well as research focused on the efficacy of Teacher Mastery badging projects.

No comments