Blog Archives
“Stealthy Assessment” of Learning through Games
“SimCityEDU: Pollution Challenge, an educational version of the popular city-building video game, is a known quantity in a fresh wave of serious learning games that bridge the gap between instruction and assessment.
Released last November, the game—in which students acting as mayors must balance the growth of their cities with environmental impacts—tracks, analyzes, and feeds back to teachers through dashboards more than 3,000 different data points showing how well each student understands systems thinking.
“If a student builds one bus stop, then waits before strategically building other bus stops, he has an eye for problem-solving that I would not have gotten with a multiple-choice or written test,” said Matt Farber, a social studies teacher who beta-tested SimCityEDU with 6th graders at the 650-student Valleyview Middle School in Denville, N.J. “We used to try formal assessments every day, and then do a summative assessment at the end of a unit every two weeks and pretty much move on, but you don’t get a lot of reflection with that. Now, there’s iteration, which I hadn’t planned on. Students get competitive for their personal best.”That is the double benefit of games with embedded assessments, say those who develop and use them. They not only provide a deeper insight into understanding, allowing educators to more quickly identify students’ strengths and weaknesses, but they also thwart a growing disengagement from traditional forms of evaluation. Teachers commonly report that games with embedded assessments encourage students to look at failure as opportunity—a way of thinking that will serve them well as they grow up.
Expect to see more assessment-embedded video games in classrooms soon, experts predict.
GlassLab, a digital learning game-development studio based in Redwood City, Calif., and the creator of SimCityEDU, plans to develop five more serious learning games with embedded assessments over the next three years with grant money from the Bill & Melinda Gates and John D. and Catherine T. MacArthur foundations. “We’re on track, but frankly, we don’t think that’s enough,” said Jessica Lindl, the general manager of GlassLab, a project of the New York City-based nonprofit Institute of Play. “We want to empower and accelerate the entire market. At the end of our grant, we don’t want just six games. We want thousands of other games to be created.”
To read the full article by Robin L. Flanigan, click here;
http://www.edweek.org/ew/articles/2014/03/13/25games.h33.html
GlassLabs releases research on Psychometric Considerations in Game-based Assessment
Press Release: GlassLab Publishes Research on Game-based Assessment
For Immediate Release
February 6, 2014
Digital Games Can Improve Measurement of Student Learning With Continuous Assessment, According to New Research From GlassLab
New white paper offers framework for integrating game design and educational assessment
Redwood City, Calif. – February 6, 2013 — Researchers have figured out a new way to give teachers a dynamic portrait of a student’s learning in action, using video games. In a white paper released today by Institute of Play project GlassLab (the Games, Learning and Assessment Lab), a team of assessment data scientists, learning designers and game developers describe a multidisciplinary approach to designing a new type of classroom game — a game-based assessment. Game-based assessments can provide a rich understanding of the different factors that affect educational achievement and predict how a student’s performance might change over time.
The white paper, “Psychometric Considerations in Game-Based Assessment,” answers the provocative questions that stand in the way of realizing the full potential of games to transform learning and assessment: How can scientists make sense of the endless stream of data generated by a digital game — the entire spectrum from wayward mouse clicks to strategic choices in gameplay? How can psychometric data help game designers build better challenges to improve learning outcomes? And how can experts in diverse fields come together to build and test new game-based assessments?
“Game-based assessments may hold the promise of a richer, multi-dimensional portrait of student learning, but they also present a new frontier in assessment design, ripe with challenges and opportunities for psychometricians and game designers to explore collaboratively,” says co-author Robert Mislevy, a psychometrics consultant for GlassLab, pioneer of evidence-centered assessment design, and Frederic M. Lord Chair in Measurement and Statistics at ETS. “This paper provides a framework for the continued exploration of this new frontier and proposes a design approach for developing and testing new game-based assessments.”
“Psychometric Considerations in Game-Based Assessment” is the first publication from GlassLab, and contains findings from the development of the Lab’s first game-based assessment product, SimCityEDU: Pollution Challenge!, which launched in November 2013. GlassLab’s research and development efforts are made possible by the generous support of the John D. and Catherine T. MacArthur Foundation and the Bill & Melinda Gates Foundation.
A project of the nonprofit Institute of Play, GlassLab is an interdisciplinary partnership between leaders in commercial games and experts in learning and assessment to develop next-generation educational games. Co-authors of “Psychometric Considerations in Game-Based Assessment” include researchers from Institute of Play, Educational Testing Service, Electronic Arts, and Pearson’s Center for Digital Data, Analytics and Adaptive Learning.
The 160-page white paper is available for free download today from Institute of Play. To download the full white paper and Executive Summary as a PDF e-book, or to explore print-on-demand options, please visit http://bit.ly/glasslab-research.
The next white paper from GlassLab, scheduled for publication in Fall/Winter 2014, will detail GlassLab’s Evidence-Centered Game Design process for developing game-based assessments.
Making Meaningfull Learning Games
John Flinchbaugh/Flickr
Katrina Schwartz writes that:
“Proving the effectiveness of a game is no simple proposition. Games are all different. When students play a game that focuses on a particular skill, often called “drill-and-kill” games, they often see improved test scores. But those games don’t necessarily get at the higher order thinking skills that games have the potential to help develop. But more complex games often don’t offer clearly correlated outcomes that a teacher can use to prove effectiveness.
The debate has led some game developers and their funders to focus on how the game itself can be used to assess learning — how the game can become a test. But White says that also changes the nature of the game. When a child sits down to play a game, he assumes it’s “failure agnostic” and that allows him to experiment and meander his way through the game freely. When assessment is involved, it changes his relationship to the game. White isn’t against using games for assessment, but he doesn’t think anyone has really figured out how to accurately use the data that games provide to assess in a meaningful way.
Glasslab’s SimCityEDU, funded by the Gates and MacArthur Foundations, is the highest profile example of gaming for assessment. White says if they can crack the code he’ll license it from them, but it’s a lot easier to pull analytics and metrics for a single game than it is to generalize those principles into something that could be applied to all educational games. And, when assessment is involved, the price of developing a game skyrockets.” To read the full article on Mindshift, click here;
It is good to read a discussion of Higher Order Thinking Skills in games and gaming for assessment. These are two important issues that need more attention and development. Unfortunately these games have too often targeted the lower order thinking skills of Blooms’ taxonomy. In 2007, John Rice conducted a study which assessed higher order thinking skills in video games with the aim of developing a tool for parents and educators to rate the level of higher order thinking skills in video games. He found that the educational efficacy of these games varies greatly with respect to higher order thinking skill. (Rice, 2007, p. 88).
Rice began his study by examining; “lower level learning in so-called edutainment products and concludes with an example of an advanced social studies simulation that fosters higher order thinking. [Rice] identifies characteristics of highly cognitive virtual interactive environments and offers a detailed index and scoring rubric as a tool for teachers and preservice teachers to use when evaluating the tendencies a video game demonstrates toward encouraging higher order thinking in its participants (Rice, 2007, p. 87).”
When educators are looking for a game that does more than drill facts in to the brains of students, Rice’s rubric can serve as a helpful tool in selecting a game that lifts students to the top of Blooms’ taxonomy (1956).
References:
Bloom, B. S. (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. New York: Longmans Green.
Rice, J. W. (2007). Assessing Higher Order Thinking in Video Games. Journal of Technology and Teacher Education, 15(1), 87-100.
