UC Davis Agricultural Sustainability Institute (ASI) is developing a model platform for validating experiential learning within formal institutional contexts at the undergraduate level. In creating a learner-driven, content-rich badge system, ASI will establish a new model for bridging learning in and out of the classroom, and enable learners to better communicate their skills and competencies to a broad audience.
What are the 3 most important things about building a badge system you would share with another organization just getting started?
- Learn about design research and what that process entails. Think very carefully about your user experience.
- It’s really important to understand what’s already out there and who’s building things that you can borrow from or collaborate on. While we don’t need one badge platform or system to rule them all, there are many great examples to draw from and by supporting and investing in further development of those core examples, we can actually move the whole enterprise forward in a coherent way and still provide design resources for the greater ecosystem.
- Know that this is a longer and more resource intensive process that other types of curriculum development efforts might be. Be realistic about your expectations for the process and don’t be afraid to be agile and do things incrementally and iteratively if possible, with an eye for extensibility into future phases once you have proof of concept, a working prototype or some first element of the system for your students and faculty to play with and get comfortable adopting.
Who were you addressing with your badge system design?
What were your initial goals for the badges?
- Develop a tool that helps students and faculty visualize and concretize the fairly abstract process of engaging in and reflecting upon competency development and integration of learning experiences across disciplines, and from inside and outside of the classroom
- Help students and faculty organize their thinking around the nature of competency development, how it unfolds and where it takes place
- Create a system that helps individuals track progress over time and beyond the traditional grading and evaluation system contained within the course structure; this will provide students with a more comprehensive view of their growth and development and support them in the creation of an individual portfolio of work and collections of badges
- Create a system that provides data we can use to evaluate the strengths of our program and that will inform our adaptive curriculum management process.
- Design with extensibility in mind; serve as a model for other programs and future design efforts in the badges ecosystem
- Propose a theory of learning, motivation and social change through our system design, as follows:
- Learning is a socially mediated process and students need feedback from peers, mentors and experts to validate and solidify their learning.
- In order to integrate learning experiences happening across disciplines and in differing contexts and spaces, students need tools to support reflection and synthesis at a more holistic level. This system should facilitate that process.
- Students should be in control of how and to whom they present the products of their education, via portfolio and badging mechanisms. This allows for an interest-driven learning process that leads to the development of unique pathways for each student and unique presentations of their individual contributions before their wider community.
- Badges should be tied to rigorous standards for learning and achievement; those standards should be enforced not just by the administrative authorities within the major, but through participation and consensus mechanisms from within the learning community, which includes students, faculty, and other supporters and mentors of our students.
Did those goals change at all throughout the design process?
These are still our goals, though we’re focusing on achieving some of them in this initial design phase and others in future iterations. Specifically, we are currently focusing on the quality of the individual user experience and its supporting data model. We have ongoing questions about how to best implement the peer-to-peer features of our system, as well as how best to invite non-UC Davis users to participate in the SA&FS learning community via badges. For now, we are focusing of the individual student’s portfolio development and badge building aspects of the system. We hope to show the importance of some of those more social/community aspects of the system in a future phase.
What types of badges are you using (participation, skill, certification, etc.)? Are there levels or pathways represented in your badges? If so, please describe.
How were the criteria for the badges determined? What pedagogies (if any) informed the learning and badge system design?
What are 3 things you learned about badge system design?
User Experience. This is not always simple or intuitive from the curriculum designer’s perspective. Thinking through the user experience is a really critical part of the design process. Getting feedback and engagement from your users early in the design of the system is key. The system will likely fail if you don’t get this right.
Speaking different languages within the team. Communicating design ideas from content expert to design expert to programming expert is something that requires skill and experience. An interest in badges does not equal the capacity to think with agility and an innovative mindset about the potential of a badging system. Translating the complexities of a human-centered user experience into the logic of programming language takes time to get right and a great deal of refinement. This can be a longer and more resource-intensive process than what might be expected, especially for those content experts unfamiliar with web development in general and human-centered design in specific, or those web experts unfamiliar with progressive educational theory.
Designing to include faculty. A badge system can lead to some real efficiencies in the interactions between faculty and students, if designed with both parties’ needs in mind. If you want to be successful in realizing this potential, it’s really important to think completely through faculty user experience.
What would you do differently if you were to start over?
We have an opportunity to redesign the system right now, which is allowing us to go back to the drawing board on some aspects of the system, while sticking with those features we’ve identified as fully functional and desirable. We've attached the current system overview and will provide complete documentation of the redesign in our final narrative report. In a sense, we are currently asking ourselves what we’d do differently, and most of it has to do with starting from a user experience perspective, and including the faculty user experience in that thinking. We made some assumptions that faculty would be willing to deal with a less polished interface and a clunkier, back-end experience. Given the demands on their time, that doesn’t turn out to be a fair assumption. We’re looking at how to get input from our own faculty as well as those professors at other institutions who’ve contacted us about our badge system. We anticipate that this will help us rethink how faculty engage in providing feedback to students and how the badge-earning and issuing process will include faculty, what number of them is effective and to what extent.
What is left to do?
We have to finish building the system, including the aspects we’re in the process of redesigning, open it to user testing and then roll it out with students in the fall of 2013. Then, we need to figure out how the system might really be built upon and/or adopted by others. We need continue to engage in communication with faculty at other universities who are interested in our system, and look to a next phase of development that might source ideas from across institutions, and take a more collaborative, crowd-sourced design approach, with many of those interested faculty members participating. We’d be interested in taking the lead on that kind of project because we see a tremendous benefit for our own development and for the creation of a system or a code base that meets the needs of other programs, too.
What is left unanswered?
We still need to figure out whether our theories around learning and motivation really work in this context. Will students be motivated to give and get feedback from peers, mentors and experts in their field? Will they want to earn and share badges without us requiring them to do so? Will they be motivated to design new badges that reflect their own interests and body of work? We need to get to the next development phase so we can build out more features that will make the system hang together with a coherence that helps drive students into the pathways, spaces and practices that should encourage and motivate them to keep using the system.
What might help you continue to succeed?
What are the 3 main challenges to widespread adoption of your badge system for your organization?
We have to get to the point where what we have is ready to be made available to our students and to other programs interested in our work, and we aren’t there yet. Thus, development time has been a challenge because we want to engage users in the design process, but we don’t want to turn them off from using the end product because we didn’t give it enough time in development before we release for testing and they get frustrated by a bad user experience. It’s proving to be a more delicate balance than expected.
We need to become increasingly effective in our communication about the system and the design principles behind it. This is true within our program and to the academic and badge-curious community at large. We believe we are proposing a reimagining of learning within higher ed and it’s easy for people to misunderstand certain aspects of that proposition and the real practices behind it.
Resource questions. Will we be able to continue building to a full-scale implementation within our program? What internal and external resources will exist to help us move towards a wider-scale adoption at UC Davis? And for other institutions, what will be the funding model that will allow them to build on what we’ve done, or make a departure from it, in adopting badging practices within their programs?
What is your badge system testing strategy? How have you or will you be testing your badge system prior to deployment?
In late summer/early fall, we’ll be testing our badging system with our own students and opening it up to people who are interested in checking it out using an OpenID login. We’re hoping to get a wide range of users outside our program engaged in the testing so other programs can learn for themselves and them we can also benefit from their feedback. We’ll be communicating about this testing phase with the badges community well in advance.
What factors are the success of your badge system contingent upon?
Determining the right balance point for the social networking aspect of the system. How much feedback is useful? What types are most effective? How can we most easily moderate these processes?
Continued support within our institution, from administration and faculty.
Getting badges to be seen by their consumers (grad schools, employers etc.) as a desirable and easily discoverable commodity.
What have you done, or do you plan to do, to evaluate your badge earner community?
As a formal academic program, we have evaluation mechanisms that predate the badges, but we’re also very interested in looking at the analytics to see what kinds of learning experiences are most valuable to our students, which students are most motivated by and engaged in the system, and how they are using the badges to communicate about their work. These evaluations will help us iterate on badging design as well as curriculum design for the whole major. One of our goals as a program is to employ an adaptive curriculum management model, and the data from this system could help us take that work to a whole new level.
Please describe any impact your badge system may have already had on your organization and your learners.
How would you characterize the impact your badge system will have on the badge ecosystem?
Because of the way we’ve structured our thinking around curriculum design as it relates to badges, and because of our position on the curve of innovation and adoption within the movement, we believe we have significant opportunity to impact at least the higher ed ecosystem, and likely the broader badging community with our work. Additionally, our position as a Land Grant University and our exploration of future collaboration and alignment with what’s happening in the K-12 space through 4H and other organizations, means that we’re already engaging in dialogues with potential for impact and the ability to highlight learning pathways across educational institutions and sectors.
What plans do you have to scale your badge system?
All plans are contingent upon further resources, but we’re really looking at how to scale within our institution, how to create a design process that enables us to spread the development of the platform to other interested institutions that we’ve been in dialogue with, and we also envision working within the national network of similar institutions (INFAS) many of which support majors like SA&FS. We’d like to see if we can not only spread the adoption to other schools and programs like ours, but interconnect them so that students and faculty can share and provide a community of support around portfolio building and badge earning across institutions. We imagine, students in our programs going on to graduate work within that network, or perhaps taking key positions in the regions where these schools are located, and if they could foster those connections through this system, the impact and synergistic effects could be very big, for those individuals and for our field.
How successfully are you getting institutional buy-in, or adoption from your learners?
We’d like to get blog posts going on HASTAC, through the UC Davis Academic Technology Services EdTech blog, The Wheel, and on our ASI website. We also plan to continue communicating developments on our project through press releases and ongoing conversations with reporters at the Chronicle of Higher Ed and Inside Higher Ed. We have not yet decided to what extent we’ll be using Facebook and Twitter to communicate about our badges system.