Researchers Use Video Game to Crack the 'Language Code'

May 18, 2011 -

Carnegie Mellon University's Lori Holt and Sung-Joo Lim and Stockholm University's Francisco Lacerda are using video game training with a mock "alien" language to replicate the challenges of learning languages as an infant. The research found that listeners were to quick recognize word-like units. The study was funded by grants from the National Science Foundation, National Institutes of Health and Riksbankens Jubileumsfond.

To uncover how spoken sounds are decoded by the brain, the research team designed a video game narrated in deliberately distorted speech. The soundtrack (unintelligible babble in any language) was the only source of instruction for the 77 adult players in the study. After only two hours of play, the participants could reliably extract world-length sound categories from continuous alien sounds and apply that learning to advance through the game.

"Traditionally, when we study adult learning in the lab, it's nothing like how infants learn language," said Holt, professor of psychology at CMU and a specialist in auditory cognitive neuroscience. "This video game models for adults the challenge language learning poses to infants. This presents the opportunity to study learning in ways that are just not feasible with infants."

Lacerda, professor of phonetics and an expert in language acquisition, agrees that using video games is a promising new way to explore language learning.

"This is a wonderful opportunity to approximate the task facing infants by creating a setting where adults are forced to infer what the meaning of different sound elements might be, and to do it in a functional way."

The research has the potential to help researchers better understand and effectively treat a number of conditions including dyslexia and improving second language learning.

Lim, a graduate student in psychology at CMU and lead author of the study, has used the game to help adults learn English.

"Native speakers of Japanese can use this type of training to learn English consonants they have difficulty distinguishing," she said

Holt, director of CMU's Speech Perception and Learning Laboratory, is interested in taking the study further to determine how the video game and its alien soundtrack engage different areas of the brain to produce rapid learning. The next step is to investigate this by observing players with functional magnetic resonance imaging (fMRI) to view real-time brain reactions to the video game.

Researchers will present their findings at the Acoustical Society of America's annual meeting May 23-27 in Seattle.

For more information on research at Carnegie Mellon, visit www.cmu.edu/research/brain.


Comments

Re: Researchers Use Video Game to Crack the 'Language Code'

I love this stuff.  I majored in CS and minored in linguistics; Watson's performance on Jeopardy was one of the most fascinating things I've ever seen.

Re: Researchers Use Video Game to Crack the 'Language Code'

That is just too cool 

Re: Researchers Use Video Game to Crack the 'Language Code'

This is really cool. Like really cool. I want to try doing that, play an unfamiliar game in an unfamiliar language, too bad imports won't work on my consoles.

-Austin from Oregon

Feel free to check out my blog.

Forgot your password?
Username :
Password :

Shout box

You're not permitted to post shouts.
MechaTama31Infophile: Kind of like how you're criticizing these theoretical reactions before you've even read any? ;)09/01/2015 - 12:44am
PHX CorpI'll probaly Start the stream around 8PM Eastern08/31/2015 - 10:09pm
PHX CorpOk, see you guys Tomorrow on the GP Facebook Page, I'll be steaming either the first 2 megaman games(Through Megaman Legacy Collection) or Rare Replay as the first game on My twitch.tv page tomorrow While we wait for GP to Come back up later this week08/31/2015 - 10:01pm
james_fudgeAlso check out our Facebook page and chat there! https://www.facebook.com/gamepolitics08/31/2015 - 9:53pm
james_fudgeSee you all on the other side! Find me on Twitter :)08/31/2015 - 9:51pm
james_fudgeAllright, i'll mention this on the GP facebook page08/31/2015 - 9:49pm
PHX Corpand now it's ready to go for everyone08/31/2015 - 9:35pm
PHX Corpok, done http://www.twitch.tv/trustygemtwitch I have to put on one more finishing touch and it is ready to go08/31/2015 - 9:19pm
Andrew EisenFeel free to leave us suggestions on Facebook or Twitter too. We're going to be busy but we'll try our best to keep an eye on 'em.08/31/2015 - 8:59pm
Andrew EisenIt's an interesting idea though. If we do anything, we probably won't know until after the site goes offline so keep an eye on GP social media for announcements.08/31/2015 - 8:59pm
Andrew EisenYeah, we could use my Twitch chat box too. There's always IRC but we don't currently have a GamePolitics channel.08/31/2015 - 8:57pm
Goth_SkunkThough I think the limit is 9 at a time in the hangout, so anyone who can't get in would be stuck out in the 'on air' portion.08/31/2015 - 8:57pm
Andrew EisenFor the show, I'd like the chat open to anyone who wants to watch.08/31/2015 - 8:55pm
PHX CorpI could Set Up a Temporary chatroom on My twitch.TV page while GP is busy updating the site(since I'll be Fighting Megaman Legacy Collection on Xbox one)08/31/2015 - 8:54pm
Goth_SkunkI don't see a problem with inviting viewers. It's not like I'm advocating this to be an open forum, just something specific to GP members.08/31/2015 - 8:53pm
Andrew EisenThat's why I embed the chat box from my Twitch Page. Can't get chat on the YouTube page to work either.08/31/2015 - 8:49pm
Andrew EisenI do but I haven't seen a way to incorporate viewers to chat without specifically inviting them to the event.08/31/2015 - 8:49pm
Goth_SkunkThough I'm surprised you'd not be familiar with this, Andrew. Do you not use Google Hangouts when you do S.P.A.C.?08/31/2015 - 8:45pm
Andrew EisenGP Movie Night!08/31/2015 - 8:41pm
Big PermEveyone should be too busy with phantom pain to notice GP is down. Hail Vidya!08/31/2015 - 8:40pm

Be Heard - Contact Your Politician