Some Things about Assessment that Badge Developers Might Find Helpful
- Intended Purposes Versus Actual Function of Digital Badges
- Introducing the DML Design Principles Documentation Project
- The Transcendent Potential of Digital Badges and Paradigm Shifts in Education
- Initial Consequences of the DML 2012 Badges for Lifelong Learning Competition
- Research Design Principles for Studying Learning with Digital Badges
I recently met with Greg Wilson, the founder of Software Carpentry to discuss how to assess the impact of teaching basic computer skills to other scientists to help them manage their data. Greg is as passionate about education as he is about programming. We discussedAudrey Watters’recent tweet regarding “things every techie should know about education.”
As an assessment researcher scrutinizing the funded and unfunded Badges for Lifelong Learning proposals, I thought I would try something more specific. In particular I want to explore whether distinctions that are widely held in the assessment community can help show how some of the concerns that people have raised about badges (nicely captured at David Theo Goldberg’s “Threading the Needle…” DML post). I have posted a longer version at Remediating Assessment.
While most of the proposals have multiple goals for their badges, there seems to be three types of primary goals for using badges.
1. Showing what somebody has done or might be able to do (e.g, Badgework for Vets).
2. Motivate individuals to learn or do more (e.g., BuzzMath).
3. Transform or create learning systems (e.g., MOUSE Wins!).
These three types of goals appear to correspond with the three primary assessment functions.
1. Summative functions (assessment OF learning.)
2. Formative functions for individuals (assessment FOR learning).
3. Transformative functions for systems (assessment AS learning)
Different assessment functions generally follow from different theories of knowing and learning, but these assumptions are often taken for granted.
1. Summative functions generally follow from conventional associationist views of learning as building organized hierarchies of very specific associations.
2. Formative functions follow from modern constructivist theories of learning as constructing conceptual schema by making sense of the world
3. Transformative functions follow newer sociocultural theories of learning as participating in social and technological practices.
These three assessment functions often conflict with each other in complex ways
In particular, summative functions often undermine formative and transformative functions. This is because ratcheting up the stakes associated with summative functions (i.e., the value of the badge) often requires assessments that are “indirect” and “objective” like an achievement test. As John Frederiksen and Allan Collins pointed out back in 1989, such assessments have limited formative and transformative potential, compared to more direct and subjective performance and portfolio assessments.
My point here is that badge developers should consider the various goals for their badges, and the assumptions behind those goals. Failing to do so can create “wicked” tensions that are impossible to resolve. This can be toxic to educational systems because stakeholders ascribe those tensions to other things (politics, laziness, culture, faddism, etc.).
In response to my first draft of this post, Greg summarized my point more succinctly and more generally:
People have different philosophical models of education (whether they realized it or not) and that is why they talk across each other so often.
Greg also inspired me to suggest the following contribution to Audry Watters top ten list of questions you can ask to find out if somebody really knows education, if you want to know if the now about educational assessment:
Do you understand the difference between summative, formative, and transformative functions of assessment and how they interact?
(A longer version of this post is available at Re-mediating Assessment)