A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution: A proof of concept study

S. V. Adamovich, K. August, Alma Merians, E. Tunik

Research output: Contribution to journalArticle

49 Citations (Scopus)

Abstract

Purpose: Emerging evidence shows that interactive virtual environments (VEs) may be a promising tool for studying sensorimotor processes and for rehabilitation. However, the potential of VEs to recruit action observation-execution neural networks is largely unknown. For the first time, a functional MRI-compatible virtual reality system (VR) has been developed to provide a window into studying brain-behavior interactions. This system is capable of measuring the complex span of hand-finger movements and simultaneously streaming this kinematic data to control the motion of representations of human hands in virtual reality. Methods: In a blocked fMRI design, thirteen healthy subjects observed, with the intent to imitate (OTI), finger sequences performed by the virtual hand avatar seen in 1st person perspective and animated by pre-recorded kinematic data. Following this, subjects imitated the observed sequence while viewing the virtual hand avatar animated by their own movement in real-time. These blocks were interleaved with rest periods during which subjects viewed static virtual hand avatars and control trials in which the avatars were replaced with moving non-anthropomorphic objects. Results: We show three main findings. First, both observation with intent to imitate and imitation with real-time virtual avatar feedback, were associated with activation in a distributed frontoparietal network typically recruited for observation and execution of real-world actions. Second, we noted a time-variant increase in activation in the left insular cortex for observation with intent to imitate actions performed by the virtual avatar. Third, imitation with virtual avatar feedback (relative to the control condition) was associated with a localized recruitment of the angular gyrus, precuneus, and extrastriate body area, regions which are (along with insular cortex) associated with the sense of agency. Conclusions: Our data suggest that the virtual hand avatars may have served as disembodied training tools in the observation condition and as embodied "extensions" of the subject's own body (pseudo-tools) in the imitation. These data advance our understanding of the brain-behavior interactions when performing actions in VE and have implications in the development of observation- and imitation-based VR rehabilitation paradigms.

Original languageEnglish (US)
Pages (from-to)209-223
Number of pages15
JournalRestorative neurology and neuroscience
Volume27
Issue number3
DOIs
StatePublished - Aug 10 2009

Fingerprint

Hand
Observation
User-Computer Interface
Parietal Lobe
Biomechanical Phenomena
Cerebral Cortex
Fingers
Rehabilitation
Magnetic Resonance Imaging
Body Regions
Brain
Healthy Volunteers

All Science Journal Classification (ASJC) codes

  • Neurology
  • Developmental Neuroscience
  • Clinical Neurology

Cite this

@article{ee99eb47b4d248fb8f049ba53b4a9834,
title = "A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution: A proof of concept study",
abstract = "Purpose: Emerging evidence shows that interactive virtual environments (VEs) may be a promising tool for studying sensorimotor processes and for rehabilitation. However, the potential of VEs to recruit action observation-execution neural networks is largely unknown. For the first time, a functional MRI-compatible virtual reality system (VR) has been developed to provide a window into studying brain-behavior interactions. This system is capable of measuring the complex span of hand-finger movements and simultaneously streaming this kinematic data to control the motion of representations of human hands in virtual reality. Methods: In a blocked fMRI design, thirteen healthy subjects observed, with the intent to imitate (OTI), finger sequences performed by the virtual hand avatar seen in 1st person perspective and animated by pre-recorded kinematic data. Following this, subjects imitated the observed sequence while viewing the virtual hand avatar animated by their own movement in real-time. These blocks were interleaved with rest periods during which subjects viewed static virtual hand avatars and control trials in which the avatars were replaced with moving non-anthropomorphic objects. Results: We show three main findings. First, both observation with intent to imitate and imitation with real-time virtual avatar feedback, were associated with activation in a distributed frontoparietal network typically recruited for observation and execution of real-world actions. Second, we noted a time-variant increase in activation in the left insular cortex for observation with intent to imitate actions performed by the virtual avatar. Third, imitation with virtual avatar feedback (relative to the control condition) was associated with a localized recruitment of the angular gyrus, precuneus, and extrastriate body area, regions which are (along with insular cortex) associated with the sense of agency. Conclusions: Our data suggest that the virtual hand avatars may have served as disembodied training tools in the observation condition and as embodied {"}extensions{"} of the subject's own body (pseudo-tools) in the imitation. These data advance our understanding of the brain-behavior interactions when performing actions in VE and have implications in the development of observation- and imitation-based VR rehabilitation paradigms.",
author = "Adamovich, {S. V.} and K. August and Alma Merians and E. Tunik",
year = "2009",
month = "8",
day = "10",
doi = "10.3233/RNN-2009-0471",
language = "English (US)",
volume = "27",
pages = "209--223",
journal = "Restorative Neurology and Neuroscience",
issn = "0922-6028",
publisher = "IOS Press",
number = "3",

}

A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution : A proof of concept study. / Adamovich, S. V.; August, K.; Merians, Alma; Tunik, E.

In: Restorative neurology and neuroscience, Vol. 27, No. 3, 10.08.2009, p. 209-223.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution

T2 - A proof of concept study

AU - Adamovich, S. V.

AU - August, K.

AU - Merians, Alma

AU - Tunik, E.

PY - 2009/8/10

Y1 - 2009/8/10

N2 - Purpose: Emerging evidence shows that interactive virtual environments (VEs) may be a promising tool for studying sensorimotor processes and for rehabilitation. However, the potential of VEs to recruit action observation-execution neural networks is largely unknown. For the first time, a functional MRI-compatible virtual reality system (VR) has been developed to provide a window into studying brain-behavior interactions. This system is capable of measuring the complex span of hand-finger movements and simultaneously streaming this kinematic data to control the motion of representations of human hands in virtual reality. Methods: In a blocked fMRI design, thirteen healthy subjects observed, with the intent to imitate (OTI), finger sequences performed by the virtual hand avatar seen in 1st person perspective and animated by pre-recorded kinematic data. Following this, subjects imitated the observed sequence while viewing the virtual hand avatar animated by their own movement in real-time. These blocks were interleaved with rest periods during which subjects viewed static virtual hand avatars and control trials in which the avatars were replaced with moving non-anthropomorphic objects. Results: We show three main findings. First, both observation with intent to imitate and imitation with real-time virtual avatar feedback, were associated with activation in a distributed frontoparietal network typically recruited for observation and execution of real-world actions. Second, we noted a time-variant increase in activation in the left insular cortex for observation with intent to imitate actions performed by the virtual avatar. Third, imitation with virtual avatar feedback (relative to the control condition) was associated with a localized recruitment of the angular gyrus, precuneus, and extrastriate body area, regions which are (along with insular cortex) associated with the sense of agency. Conclusions: Our data suggest that the virtual hand avatars may have served as disembodied training tools in the observation condition and as embodied "extensions" of the subject's own body (pseudo-tools) in the imitation. These data advance our understanding of the brain-behavior interactions when performing actions in VE and have implications in the development of observation- and imitation-based VR rehabilitation paradigms.

AB - Purpose: Emerging evidence shows that interactive virtual environments (VEs) may be a promising tool for studying sensorimotor processes and for rehabilitation. However, the potential of VEs to recruit action observation-execution neural networks is largely unknown. For the first time, a functional MRI-compatible virtual reality system (VR) has been developed to provide a window into studying brain-behavior interactions. This system is capable of measuring the complex span of hand-finger movements and simultaneously streaming this kinematic data to control the motion of representations of human hands in virtual reality. Methods: In a blocked fMRI design, thirteen healthy subjects observed, with the intent to imitate (OTI), finger sequences performed by the virtual hand avatar seen in 1st person perspective and animated by pre-recorded kinematic data. Following this, subjects imitated the observed sequence while viewing the virtual hand avatar animated by their own movement in real-time. These blocks were interleaved with rest periods during which subjects viewed static virtual hand avatars and control trials in which the avatars were replaced with moving non-anthropomorphic objects. Results: We show three main findings. First, both observation with intent to imitate and imitation with real-time virtual avatar feedback, were associated with activation in a distributed frontoparietal network typically recruited for observation and execution of real-world actions. Second, we noted a time-variant increase in activation in the left insular cortex for observation with intent to imitate actions performed by the virtual avatar. Third, imitation with virtual avatar feedback (relative to the control condition) was associated with a localized recruitment of the angular gyrus, precuneus, and extrastriate body area, regions which are (along with insular cortex) associated with the sense of agency. Conclusions: Our data suggest that the virtual hand avatars may have served as disembodied training tools in the observation condition and as embodied "extensions" of the subject's own body (pseudo-tools) in the imitation. These data advance our understanding of the brain-behavior interactions when performing actions in VE and have implications in the development of observation- and imitation-based VR rehabilitation paradigms.

UR - http://www.scopus.com/inward/record.url?scp=68149179351&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=68149179351&partnerID=8YFLogxK

U2 - 10.3233/RNN-2009-0471

DO - 10.3233/RNN-2009-0471

M3 - Article

C2 - 19531876

AN - SCOPUS:68149179351

VL - 27

SP - 209

EP - 223

JO - Restorative Neurology and Neuroscience

JF - Restorative Neurology and Neuroscience

SN - 0922-6028

IS - 3

ER -