Researchers Use Video Game to Crack the 'Language Code'

May 18, 2011 -

Carnegie Mellon University's Lori Holt and Sung-Joo Lim and Stockholm University's Francisco Lacerda are using video game training with a mock "alien" language to replicate the challenges of learning languages as an infant. The research found that listeners were to quick recognize word-like units. The study was funded by grants from the National Science Foundation, National Institutes of Health and Riksbankens Jubileumsfond.

To uncover how spoken sounds are decoded by the brain, the research team designed a video game narrated in deliberately distorted speech. The soundtrack (unintelligible babble in any language) was the only source of instruction for the 77 adult players in the study. After only two hours of play, the participants could reliably extract world-length sound categories from continuous alien sounds and apply that learning to advance through the game.

"Traditionally, when we study adult learning in the lab, it's nothing like how infants learn language," said Holt, professor of psychology at CMU and a specialist in auditory cognitive neuroscience. "This video game models for adults the challenge language learning poses to infants. This presents the opportunity to study learning in ways that are just not feasible with infants."

Lacerda, professor of phonetics and an expert in language acquisition, agrees that using video games is a promising new way to explore language learning.

"This is a wonderful opportunity to approximate the task facing infants by creating a setting where adults are forced to infer what the meaning of different sound elements might be, and to do it in a functional way."

The research has the potential to help researchers better understand and effectively treat a number of conditions including dyslexia and improving second language learning.

Lim, a graduate student in psychology at CMU and lead author of the study, has used the game to help adults learn English.

"Native speakers of Japanese can use this type of training to learn English consonants they have difficulty distinguishing," she said

Holt, director of CMU's Speech Perception and Learning Laboratory, is interested in taking the study further to determine how the video game and its alien soundtrack engage different areas of the brain to produce rapid learning. The next step is to investigate this by observing players with functional magnetic resonance imaging (fMRI) to view real-time brain reactions to the video game.

Researchers will present their findings at the Acoustical Society of America's annual meeting May 23-27 in Seattle.

For more information on research at Carnegie Mellon, visit www.cmu.edu/research/brain.


Comments

Re: Researchers Use Video Game to Crack the 'Language Code'

I love this stuff.  I majored in CS and minored in linguistics; Watson's performance on Jeopardy was one of the most fascinating things I've ever seen.

Re: Researchers Use Video Game to Crack the 'Language Code'

That is just too cool 

Re: Researchers Use Video Game to Crack the 'Language Code'

This is really cool. Like really cool. I want to try doing that, play an unfamiliar game in an unfamiliar language, too bad imports won't work on my consoles.

-Austin from Oregon

Feel free to check out my blog.

 
Forgot your password?
Username :
Password :

Shout box

You're not permitted to post shouts.
Andrew EisenHence the "Uh, yeah. Obviously."09/02/2014 - 12:53am
SleakerI think Nintendo has proven over the last 2 years that it doesn't.09/02/2014 - 12:31am
Andrew EisenSleaker - Uh, yeah. Obviously.09/01/2014 - 8:20pm
Sleaker@AE - exclusives do not a console business make.09/01/2014 - 8:03pm
Papa MidnightI find it disappointing that, despite the presence of a snopes article and multiple articles countering it, people are still spreading a fake news story about a "SWATter" being sentenced to X (because the number seems to keep changing) years in prison.09/01/2014 - 5:08pm
Papa MidnightAnd resulting in PC gaming continuing to be held back by developer habits09/01/2014 - 5:07pm
Papa MidnightI find it disappointing that the current gen of consoles is representative of 2009-2010 in PC gaming, and will be the bar by which games are released over the next 8 years - resulting in more years of poor PC ports (if they're ever ported)09/01/2014 - 5:06pm
Andrew EisenMeanwhile, 6 of Wii U's top 12 are exclusive: Mario 3D World, Nintendo Land, Pikmin 3, Mario Kart 8, Wonderful 101, and ZombiU. (Wind Waker HD is on the list too but I didn't count it.)09/01/2014 - 4:36pm
Andrew EisenLikewise, only two of Xbox One's top 12 are exclusive: Dead Rising 3 and Ryse: Son of Rome (if you ignore a PC release later this year).09/01/2014 - 4:34pm
Andrew EisenNot to disrespect the current gen of consoles but I find it telling that of the "12 Best Games For The PS4" (per Kotaku), only two are exclusive to the system: Infamous: Second Son and Resogun.09/01/2014 - 4:30pm
MaskedPixelantehttp://www.joystiq.com/2014/09/01/beyond-two-souls-ps4-trophies-emerge-directors-cut-reported/ MMM MMM, nothing quire like reheated last gen games to make you appreciate the 400 bucks you spent on a new console.09/01/2014 - 4:24pm
Andrew EisenThat's actually a super depressing thought, that a bunch of retweeters are taking that pic as an illustration of the actual issue instead of an example of a complete misunderstanding of it.09/01/2014 - 4:20pm
Andrew EisenObviously, the picture was created by someone who doesn't understand what the issue actually is (or, possibly, someone trying to satire said misunderstanding).09/01/2014 - 4:10pm
Papa MidnightPeople fear and attack what they do not understand.09/01/2014 - 4:04pm
Papa MidnightWell, let's not forget. Someone held their hand in a peace sign a few weeks ago and people started claiming it was a gang sign. Or a police chief displayed the hand signal of their fraternity and was accused of the same.09/01/2014 - 4:04pm
SleakerEither people don't understand that what the picture is saying is true, or the picture was created out of a misunderstanding of what sexism is.09/01/2014 - 3:52pm
Sleaker@AE ok yah that's where the kind of confusion I'm getting. Your tweet can be taken to mean two different things.09/01/2014 - 3:51pm
Andrew EisenSleaker - No. No, not even remotely. The pic attached to my tweet was not made by me; it's not a statement I'm making. It's an illustration of the complete misunderstanding of the issue my tweet is referring to.09/01/2014 - 3:13pm
Papa MidnightIn other news, Netflix states why it paid Comcast: http://money.cnn.com/2014/08/29/technology/netflix-comcast/index.html?hpt=hp_t209/01/2014 - 3:10pm
Papa MidnightAndrew Eisen: Sites like Tumblr make things seem much bigger than they are. A vocal extreme minority start complaining and things go as they do.09/01/2014 - 3:09pm
 

Be Heard - Contact Your Politician