One of the biggest concerns educators have about using games in education revolves around the question of assessment. “How do I know my students are learning anything in the game?” “How do I know what they’re learning?” “How do I know where they’re still struggling?” These are questions we hear from educators all the time. While many educators are sold on games as a way to build engagement and excitement in the classroom, many are still reluctant to whole-heartedly embrace them if they can’t prove their efficacy.

In truth, assessment is HARD. As a former classroom teacher, I certainly struggled with it. Sure I could give tests, and generate scores for my students, but even those assessments just scratched the surface of what learning was and wasn’t taking place in the classroom. Assessment is always difficult, regardless of the educational activity. Still, we feel learning games offer some interesting opportunities for assessment we’re confident instructors can get excited about, and eventually rally behind.

One approach to making sense of what’s going on in games for learning that we’ve adopted for some of our products is to find and report on Key Performance Indicators (KPIs.) Take Practice Marketing, our university-level principles of marketing simulation we’re developing with McGraw Hill for example. In this turn-based marketing simulation game, student teams compete against one another in the backpack market. Focused on the “Four Ps” of marketing, players must identify a target market segment, design a bag that meets its needs, price it appropriately, find distribution channels to carry it, and position it with an effective promotional and advertising strategy. Players whose marketing plans are well aligned do well, and teams are ranked in key areas such as market share, profitability, and overall bank balance.

While there are certainly winners and losers in a game such as Practice Marketing, most instructors are less interested in seeing who is at the top of the leader board, and more interested in understanding WHY students are achieving their results in the game. Instructors want to know what successful teams are doing correctly, and where struggling teams went off track. Our Sandstone gaming platform allows us to track every single mouse click from every single team. However, it’s not a lack of data that is the challenge. Rather, it’s finding and reporting the RIGHT data in a clear and concise manner for busy instructors that is the real challenge.

Unlike a multiple choice test, there is no one “RIGHT” way to play Practice Marketing. It’s a dynamic multiplayer simulation, with many variables at play, not the least of which is the talent level of your competitors. While such a style of game makes for a challenging, “no two games the same” experience, it makes it very difficult for the game to simply assign a letter grade for student performance.

What we’ve decided to provide instructors instead is a set of Key Performance Indicators, which are essentially key calculations that elucidate a player’s performance. Rather than focus on every move a player makes, these KPIs provide a short-hand report for instructors. On a quick glance, instructors can scan KPIs for each team playing the marketing sim, giving them actionable information about how to probe students. Some examples of KPIs in the marketing sim include:

• Product appeal to target segment – How well does the backpack players create match up to the preferences of the target segment they’ve gone after?
• Media Strength – How effective is promotional strategy players employ?
• Profitabilty – Given the costs of production, retail price chosen, and other spending, how profitable is the players’ company?

What these select KPIs do is hone in on the important measures that contribute to overall company performance. They provide more information than just the bottom line outcome or team rank in the game. They provide information that will allow instructors to ask teams targeted questions, and enable them to provide suggestions on how certain teams may be able to right the ship when they’ve gone off course. We feel these informed discussions are far more valuable than generating a simple letter grade.

Assessment will always be a challenge, not just for games, but for any educational activity. The good news: It’s a challenge many game designers, including the folks here at Muzzy Lane, are enthusiastic about taking on. For more on the topic of assessment, make sure to download our recent white paper on Feedback Loops in Games and Learning.