Project Q&A With: Design Exchange

Project Q&A With: Design Exchange

Smithsonian’s Cooper-Hewitt and National Design Museum, in partnership with LearningTimes, has embarked on an initiative to integrate badging into its successful DesignPrep program for underserved NYC high school students. Badges will be awarded at increasing levels for student achievements in design disciplines and/or overall design thinking and competencies for in-person and web-based learning. Some of the highest level badges will be accredited by professional organizations such as the Council of Fashion Design in America (CFDA) and AIGA, the professional association for design— bolstering resumes and higher learning applications in addition to a high level badge in Design Thinking.

Through badging, Design Exchange provides Cooper-Hewitt with a means to track the college and career skills that students are gaining through DesignPrep and provides students with a formal recognition of their accomplishment. Successfully launched in November 2012, the program is piloting a multi-tiered system that will allow students to secure badges based on design disciplines and skills. This system provides the opportunity for students to study design at a lateral level before delving deeper into specialties.

To strengthen teen investment and promote youth ownership of the program, Cooper-Hewitt kicked off Design Exchange by hosting a graphic design workshop for a select 8 youth to design the graphic identity and badges for the program. The program was then rolled out in November at the Museum’s Teen Design Fair, attended by over 350 students. Teen Design Fair youth were encouraged to register with BadgeStack to receive a token badge for their participation in the Fair.

In spring 2013, higher level badges were piloted with a select group of 11 dedicated students participating in Cooper-Hewitt’s Hive Fashion program, which provided a holistic, in-depth study of the business of fashion design. To date the Museum has issued 55 badges and 121 quest tokens, recognizing the accomplishments of 21 students. Sample badges include (“Take Anything:” Students compiled an inspiration board for their fashion designs and posted them on the BadgeStack forum; and “Mining History:” Students researched and posted inspiring garments and costumes from the Metropolitan Museum of Art’s online catalog). Program feedback will be used to inform future Design Exchange implementations for a larger group of DesignPrep youth. Long term, the Museum’s focus is on deepening student investment in the platform and partnering with professional and educational organizations to acknowledge the badges we offer.

What are the 3 most important things about building a badge system you would share with another organization just getting started?

Iterate based on observation and feedback. The more badges we created the more we learned what worked and what still needed work. We developed many badges that were failures but allowed us to learn so much about our participants and about the nature of digital badges as they relate to us.
Listen to participants and stakeholders. The progress of our badges depends directly on how our participants respond and interact with them. Focus groups, round-table discussions, one-on-one interviews, and keen observations allowed us to learn from our primary audience and tweak our approach to badges to more readily suit them.
Think macro then micro. We first developed an ecosystem for our badges, which provided a framework or scaffold as we created each individual badge and tried different approaches or methods. Once you start creating badges, it can become very complicated and confusing if you don’t have some idea of where you are going or what you want to achieve with a certain badge or type of badge. By creating a framework first, then populating it with your created badges allows freedom of flexibility without chaos.

What is the audience for your badge system?

Underserved New York City high school students participating in Cooper-Hewitt’s DesignPrep program.

What were your initial goals for the badges? Did those goals change at all throughout the design process?

The goal of this project was to integrate badging into Cooper-Hewitt’s DesignPrep program for underserved NYC youth to provide proof of achievement at increasing levels for student achievements in design disciplines and/or overall design thinking and competencies. An additional goal was to utilize badging to promote connected experiences for “anytime, anywhere” learning activities that cultivate the pursuit of interest-based learning for college and career readiness. These goals remained unchanged throughout the design process.

How was the criteria for the badges determined. What pedagogies (if any) informed the learning and badge system design?

User-focused iterative design helped to determine the criteria of the badges. We worked with our students to develop the content and focus of the badges, using multiple methods of input to create the badges (design workshops, focus groups, and interviews). The badge content, as well as the development of the badges, centered around design thinking. We took into consideration the different elements of the design process and the methods of thinking about that process as the structure for our system of badges. As a result, our badges were created to correspond with the core principles of design: Engagement, Critical Thinking, and Problem Solving.

What are three things you learned about badge system design? What would you do differently if you were to start over?

We began trying to engage users in a “game-like” interface, where badges were easy to earn and plentiful; however, we realized that our audience was interested in more pre-professional badges. We learned that the students want to demonstrate serious skills and collect badges with credentialing that will matter for them as they apply to school and look for internships. They mentioned Adobe design programs, Photography, Drafting, and 3D modeling software as skills they would like to demonstrate through a badge.

We found collaborating directly with other HASTAC grantees on the development of our badges to be highly successful. Some of the most rewarding discussions and collaborations surrounding badges resulted from informal discussion with 1-2 other members of the badging community allowing troubleshooting and share-outs in a safe and manageable environment.

The success of our badges relied directly on how intimately they were related to our in-person programming. When students knew they were being evaluated on both their in-session work and their digital participation to determine internship placement at the end of the program, badge participation was high. However, when similar badges were offered in other workshops, but students were not told they were being evaluated and no tangible reward was offered, participation declined sharply.
If we started over, we would kick off the program by imparting the meaning and value surrounding the badge ecosystem. We would begin with a strong story to explain why badges were important or why they were of value to the students, in an effort to help student understanding and badge adoption.

What is left to do? What is left unanswered?

In order to provide students with a more “serious” offering of badges based on their desire to demonstrate technical skills and experiences that might be resume worthy, we are exploring the development of a set of core skills badges that will function across multiple design disciplines. This core will form the basis of new badge offerings aimed at demonstrating the technical skills students want to showcase.

In addition, the development of standardized badge values and credentialing would help us continue to succeed. Currently, each organization is developing a miniature ecosystem where their badges relate to each other and add value, but once they are pushed out to the OBI and other public forums, the measure of value for that badge is not easily discernible and it is difficult to compare or connect badge across disciplines and across organizations.

What are the 3 main challenges to widespread adoption of your badge system for your organization?

We have technical limitations we feel are hindering the adoption of our badges in a widespread way. A “clunky” platform makes understanding, earning, and sharing badges difficult. As a result, we are finding that students are frustrated when trying to navigate our website and earn badges.

In addition, clearly communicating the value and importance of badges to students in programs and potential partner organizations has also been a challenge.

Further we are challenged with creating the solid assessment of “hard skills” that students develop in our workshops. Many of these skills require a design professional to provide accurate assessment, and this extra element is not currently part of the program work flow.

What is your badge system testing strategy? How have you or will you be testing your badge system prior to deployment?

Our badges rely heavily on iteration, by testing through application and then reviewing and refining an idea in a further application. While we have spent significant time talking to students about our badge structure, our testing has all taken place in real-world situations. When we offer a badge we informally ask students for feedback and examine errors or mistakes in the upload process. We then adapt the structure and requirements in the next round of badges to account for the difficulties.

What is the success of your badge system contingent upon?

  1. Ease of use for our students is the primary factor in the success of our badge offering. Our students primarily access the Internet through a mobile device. In order for our badges to succeed in the long run we will need to focus on mobile access to learning about and earning badges.
  2. Badge acceptance as a formal measure of success, (recognized by other organizations or with applications to higher learning) would help to validate our badges and encourage students to adopt them because of their measurable real-world value. We would like to see the development of a network where badges are shared and accepted as a learning currency.
  3. The allotment of time and money for staffing and continued community management and badge system development. While the technical side of badge development is still relatively simple, we find that implementing a badging system requires a significant amount of time and a dedicated team of staff members.

What have you done, or do you plan to do, to evaluate your badge earner community?

The Museum hired professional evaluator IFC Macro, a recognized and respected name in the fields of evaluation and K-12 education, to evaluate Design Exchange. IFC Macro is developing a plan to analyze data and activities that will include evaluation instruments such as surveys and focus groups of participating students. This professional evaluation will inform future programming as well as serve as a documentation of the program’s success which can be presented to potential future program funders. In addition, we have utilized focus groups to formally evaluate our badge earner community to determine strengths and weaknesses of the programming. Integrating a teen voice in the program planning and implementation will provide for a stronger, more relevant program for youth.

Data gained through the focus group shows that teens have a positive reaction to the program. Specifically:

  • They like that they have to complete tasks/learn more on their own complete the badge.
  • They like that you can see other student’s badges -- the “friendly competition.”
  • They like that BadgeStack is a record of all the sessions they have been to/things they have learned.
  • They like that it’s easy to upload photos to the site.

What will you do with the results of your evaluations? What research, if any, will be based on these evaluations, and do you plan to publish the outcomes?

We have already used or formative evaluations to inform changes in the system as we progress, and we will continue to incorporate participant feedback as we work to develop new badge levels and skills sets. Our system of badges to this point can be seen as a beta test for a larger badging/rewards system that can be utilized museum-wide and provide a larger reward system for engagement with our galleries and museum community. This research will inform how we engage with other Smithsonian units and we are looking to combine badges with other systems of credentialing and reward already being used in some of those units.
The results of our evaluations will be presented at the American Association of Museums. We are also partnering with Deutsche Bank to advocate for broader implementation of badges across other educational organizations. As part of this partnership, Cooper-Hewitt is developing a professional development training for other organizations—utilizing our own experience to provide advice on technical implementation and badge systems design.

Please describe any impact your badge system may have already had on your organization and your learners.

Through an informal focus group, we learned that a number of students don’t have in-school arts programming, school so they particularly like participating in our teen programs and earning the badge credential to recognize their skills. In addition students are benefitting from the sense of pride they experience when they earn a new badge as it pushes them to want to earn more.

How would you characterize the impact your badge system will have on the badge ecosystem?

The nature of our program content as a design-based entity adds an interesting layer to the badges ecosystem because of the types of content and skills we are looking to credential. As the nation’s design museum, we offer truly unique programs for students to explore the design process and develop design thinking skills. Offering badges for these and related skills pushes the use of badges into new territory and provides valuable investigations into measuring and rewarding learning and work that traditionally has not been credited at the high school level.

What plans do you have to scale your badge system?

Currently, we are looking into scaling our badge system to all participants and visitors museum-wide. We are in conversation with our IT team and the director of digital media, to consider the linking of our badge structure to our visitor management system. In addition, we are exploring partnership badging opportunities with other Smithsonian units.

How successfully are you getting institutional buy-in, or adoption from your learners?

We find that investment in our badge platform from student participants is directly in proportion to the immediate benefit badges can have for their success in our programs and their educational future. We initially launched our badges at our annual Teen Design Fair, attended by almost 300 students. Out of these students, only a handful took the initiative to participate in our badge system as there was little incentive or perceived benefit from earning these “low stakes” badges. Later in the program, during our 24-week fashion badging pilot, we informed our students that their participation in the badges platform would be a determining factor in awarding paid internships after the end of the series. Because of the direct connection between online learning and physical measure or reward, all students joined the system and many earned all the available badges. As our program structure incorporates students with a variety of participation levels, we are working to adapt our badge offering to meet the needs of these various groups.

Once your badge system is built, how self-sustaining is it? How much do you anticipate maintenance to be?

Cooper-Hewitt plans to convert badge hosting to the open source Badge OS platform, with the cost of staffing and continued community management built into our youth grants.

For those who want to follow the development, implementation, and adoption of your badge system, what social media sites will you be posting updates to?

Updates to our current badges system will go on our Teen Design Facebook (www.facebook.com/teendesign) page. Other museum wide initiatives will be broadcast through our website (www.cooperhewitt.org), Facebook page (www.facebook.com/cooperhewitt) and Twitter feed (www.twitter.com/cooperhewitt).

No comments