Hostname: page-component-54dcc4c588-scsgl Total loading time: 0 Render date: 2025-10-07T01:12:17.385Z Has data issue: false hasContentIssue false
  • English
  • Français

What you see is what you get: visual scanning failures of naturalistic social scenes in schizophrenia

Published online by Cambridge University Press:  05 June 2020

Gaurav H. Patel Open the ORCID record for Gaurav H. Patel [Opens in a new window] ,
Sophie C. Arkin ,
Daniel R. Ruiz-Betancourt ,
Heloise M. DeBaun ,
Nicole E. Strauss ,
Laura P. Bartel ,
Jack Grinband ,
Antigona Martinez ,
Rebecca A. Berman  and
David A. Leopold
...Show all authors
Gaurav H. Patel*
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA Division of Experimental Therapeutics, New York State Psychiatric Institute, New York, USA
Sophie C. Arkin
Affiliation:
Department of Psychology, University of California Los Angeles, Los Angeles, USA
Daniel R. Ruiz-Betancourt
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA
Heloise M. DeBaun
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA
Nicole E. Strauss
Affiliation:
Palo Alto University, Palo Alto, USA
Laura P. Bartel
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA Division of Experimental Therapeutics, New York State Psychiatric Institute, New York, USA
Jack Grinband
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA Division of Experimental Therapeutics, New York State Psychiatric Institute, New York, USA
Antigona Martinez
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA Schizophrenia Research Division, Nathan Kline Institute for Psychiatric Research, Orangeburg, USA
Rebecca A. Berman
Affiliation:
Section on Cognitive Neurophysiology and Imaging, National Institute for Mental Health, Bethesda, USA
David A. Leopold
Affiliation:
Section on Cognitive Neurophysiology and Imaging, National Institute for Mental Health, Bethesda, USA
Daniel C. Javitt
Affiliation:
Department of Psychiatry, Columbia University Irving Medical Center, New York, USA Division of Experimental Therapeutics, New York State Psychiatric Institute, New York, USA Schizophrenia Research Division, Nathan Kline Institute for Psychiatric Research, Orangeburg, USA
*
Author for correspondence: Gaurav H. Patel, E-mail: ghp2114@cumc.columbia.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Impairments in social cognition contribute significantly to disability in schizophrenia patients (SzP). Perception of facial expressions is critical for social cognition. Intact perception requires an individual to visually scan a complex dynamic social scene for transiently moving facial expressions that may be relevant for understanding the scene. The relationship of visual scanning for these facial expressions and social cognition remains unknown.

Methods

In 39 SzP and 27 healthy controls (HC), we used eye-tracking to examine the relationship between performance on The Awareness of Social Inference Test (TASIT), which tests social cognition using naturalistic video clips of social situations, and visual scanning, measuring each individual's relative to the mean of HC. We then examined the relationship of visual scanning to the specific visual features (motion, contrast, luminance, faces) within the video clips.

Results

TASIT performance was significantly impaired in SzP for trials involving sarcasm (p < 10−5). Visual scanning was significantly more variable in SzP than HC (p < 10−6), and predicted TASIT performance in HC (p = 0.02) but not SzP (p = 0.91), differing significantly between groups (p = 0.04). During the visual scanning, SzP were less likely to be viewing faces (p = 0.0001) and less likely to saccade to facial motion in peripheral vision (p = 0.008).

Conclusions

SzP show highly significant deficits in the use of visual scanning of naturalistic social scenes to inform social cognition. Alterations in visual scanning patterns may originate from impaired processing of facial motion within peripheral vision. Overall, these results highlight the utility of naturalistic stimuli in the study of social cognition deficits in schizophrenia.

Keywords

Attentionface emotion recognitionmotionsocial cognitionvisual search

Information

Type
Original Article
Information
Psychological Medicine , Volume 51 , Issue 16 , December 2021 , pp. 2923 - 2932
Check for updates
Copyright
Copyright © The Author(s) 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Anderson, E. J., Tibber, M. S., Schwarzkopf, D. S., Shergill, S. S., Fernandez-Egea, E., Rees, G., & Dakin, S. C. (2016). Visual population receptive fields in people with schizophrenia have reduced inhibitory surrounds. The Journal of Neuroscience, 37(6), 1546–1556. http://doi.org/10.1523/JNEUROSCI.3620-15.2016.Google Scholar
Arnold, A. E. G. F., Iaria, G., & Goghari, V. M. (2016). Efficacy of identifying neural components in the face and emotion processing system in schizophrenia using a dynamic functional localizer. Psychiatry Research: Neuroimaging, 248, 55–63. http://doi.org/10.1016/j.pscychresns.2016.01.007.CrossRefGoogle ScholarPubMed
Bulzacka, E., Meyers, J. E., Boyer, L., Le Gloahec, T., Fond, G., Szöke, A., … Schürhoff, F. (2016). WAIS-IV seven-subtest short form: Validity and clinical use in schizophrenia. Archives of Clinical Neuropsychology, 31(8), 915–925. http://doi.org/10.1093/arclin/acw063.Google Scholar
Butler, P. D., Abeles, I. Y., Weiskopf, N. G., Tambini, A., Jalbrzikowski, M., Legatt, M. E., … Javitt, D. C. (2009). Sensory contributions to impaired emotion processing in schizophrenia. Schizophrenia Bulletin, 35(6), 1095–1107. http://doi.org/10.1093/schbul/sbp109.CrossRefGoogle Scholar
Butler, P. D., Zemon, V., Schechter, I., Saperstein, A. M., Hoptman, M. J., Lim, K. O., … Javitt, D. C. (2005). Early-stage visual processing and cortical amplification deficits in schizophrenia. Archives of General Psychiatry, 62(5), 495–504. http://doi.org/10.1001/archpsyc.62.5.495.CrossRefGoogle Scholar
Calderone, D. J., Martinez, A., Zemon, V., Hoptman, M. J., Hu, G., Watkins, J. E., … Javitt, D. C. (2013). Comparison of psychophysical, electrophysiological, and fMRI assessment of visual contrast responses in patients with schizophrenia. Neuroimage, 67, 153–162. http://doi.org/10.1016/j.neuroimage.2012.11.019.CrossRefGoogle ScholarPubMed
Chen, Y., Levy, D. L., Sheremata, S., & Holzman, P. S. (2004). Compromised late-stage motion processing in schizophrenia. Biological Psychiatry, 55(8), 834–841. http://doi.org/10.1016/j.biopsych.2003.12.024.CrossRefGoogle Scholar
Chen, Y., Palafox, G. P., Nakayama, K., Levy, D. L., Matthysse, S., & Holzman, P. S. (1999). Motion perception in schizophrenia. Archives of General Psychiatry, 56(2), 149–154.CrossRefGoogle Scholar
Corbetta, M., Patel, G. H., & Shulman, G. L. (2008). The reorienting system of the human brain: From environment to theory of mind. Neuron, 58(3), 306–324. http://doi.org/10.1016/j.neuron.2008.04.017.CrossRefGoogle ScholarPubMed
Dias, E. C., Van Voorhis, A. C., Braga, F., Todd, J., Lopez-Calderon, J., Martinez, A., & Javitt, D. C. (2020). Impaired fixation-related theta modulation predicts reduced visual span and guided search deficits in schizophrenia. Cerebral Cortex (New York, NY: 1991), 30(5), 2823–2833. http://doi.org/10.1093/cercor/bhz277.Google Scholar
Green, M. F., & Horan, W. P. (2010). Social cognition in schizophrenia. Current Directions in Psychological Science, 19(4), 243–248. http://doi.org/10.1177/0963721410377600.CrossRefGoogle Scholar
Green, M. F., Horan, W. P., & Lee, J. (2019). Nonsocial and social cognition in schizophrenia: Current evidence and future directions. World Psychiatry, 18(2), 146–161. http://doi.org/10.1002/wps.20624.CrossRefGoogle ScholarPubMed
Green, M. F., Horan, W. P., & Lee, J. (2015). Social cognition in schizophrenia. Nature Reviews Neuroscience, 16(10), 620–631. http://doi.org/10.1038/nrn4005.CrossRefGoogle ScholarPubMed
Heimberg, C., Gur, R. E., Erwin, R. J., Shtasel, D. L., & Gur, R. C. (1992). Facial emotion discrimination: III. Behavioral findings in schizophrenia. Psychiatry Research, 42(3), 253–265.CrossRefGoogle Scholar
Holdnack, J. A., Prifitera, A., Weiss, L. G., & Saklofske, D. H. (2015). Chapter 12. WISC-V and the personalized assessment approach. In Weiss, L. G., Saklofske, D. H., Holdnack, J. A., & Prifitera, A. (Eds.), WISC-V Assessment and Interpretation (pp. 373–414). New York: Elsevier IncGoogle Scholar
Jacoby, N., Bruneau, E., Koster-Hale, J., & Saxe, R. (2016). Localizing pain matrix and theory of mind networks with both verbal and non-verbal stimuli. Neuroimage, 126, 39–48. http://doi.org/10.1016/j.neuroimage.2015.11.025.CrossRefGoogle ScholarPubMed
Javitt, D. C. (2009). When doors of perception close: Bottom-up models of disrupted cognition in schizophrenia. Annual Review of Clinical Psychology, 5, 249–275. http://doi.org/10.1146/annurev.clinpsy.032408.153502.CrossRefGoogle ScholarPubMed
Johnston, P. J., Enticott, P. G., Mayes, A. K., Hoy, K. E., Herring, S. E., & Fitzgerald, P. B. (2010). Symptom correlates of static and dynamic facial affect processing in schizophrenia: Evidence of a double dissociation? Schizophrenia Bulletin, 36(4), 680–687. http://doi.org/10.1093/schbul/sbn136.CrossRefGoogle ScholarPubMed
Kohler, C. G., Martin, E. A., Milonova, M., Wang, P., Verma, R., Brensinger, C. M., … Gur, R. C. (2008). Dynamic evoked facial expressions of emotions in schizophrenia. Schizophrenia Research, 105(1–3), 30–39. http://doi.org/10.1016/j.schres.2008.05.030.CrossRefGoogle Scholar
Kusunoki, M., Gottlieb, J. P., & Goldberg, M. E. (2000). The lateral intraparietal area as a salience map: The representation of abrupt onset, stimulus motion, and task relevance. Vision Research, 40(10–12), 1459–1468. http://doi.org/10.1016/S0042-6989(99)00212-6.CrossRefGoogle ScholarPubMed
Leitman, D. I., Wolf, D. H., Ragland, J. D., Laukka, P., Loughead, J., Valdez, J. N., … Gur, R. C. (2010). ‘It's Not What You Say, But How You Say it’: A reciprocal temporo-frontal network for affective prosody. Frontiers in Human Neuroscience, 4, 19. http://doi.org/10.3389/fnhum.2010.00019.Google ScholarPubMed
Leitman, D. I., Ziwich, R., Pasternak, R., & Javitt, D. C. (2006). Theory of Mind (ToM) and counterfactuality deficits in schizophrenia: Misperception or misinterpretation? Psychological Medicine, 36(8), 1075–1083. http://doi.org/10.1017/S0033291706007653.CrossRefGoogle ScholarPubMed
Marsman, J.-B. C., Cornelissen, F. W., Dorr, M., Vig, E., Barth, E., & Renken, R. J. (2016). A novel measure to determine viewing priority and its neural correlates in the human brain. Journal of Vision, 16(6), 3–3. http://doi.org/10.1167/16.6.3.CrossRefGoogle ScholarPubMed
Martinez, A., Gaspar, P. A., Hillyard, S. A., Andersen, S. K., Lopez-Calderon, J., Corcoran, C. M., & Javitt, D. C. (2018). Impaired motion processing in schizophrenia and the attenuated psychosis syndrome: Etiological and clinical implications. American Journal of Psychiatry, 175(12), 1243–1254. http://doi.org/10.1176/appi.ajp.2018.18010072.CrossRefGoogle ScholarPubMed
McDonald, S., Bornhofen, C., Shum, D., Long, E., Saunders, C., & Neulinger, K. (2006). Reliability and validity of The Awareness of Social Inference Test (TASIT): A clinical test of social perception. Disability and Rehabilitation, 28(24), 1529–1542. http://doi.org/10.1080/09638280600646185.CrossRefGoogle ScholarPubMed
McDonald, S., Flanagan, S., Rollins, J., & Kinch, J. (2003). TASIT: A new clinical tool for assessing social perception after traumatic brain injury. The Journal of Head Trauma Rehabilitation, 18(3), 219–238.CrossRefGoogle ScholarPubMed
Morrison, K. E., Pinkham, A. E., Kelsven, S., Ludwig, K., Penn, D. L., & Sasson, N. J. (2019). Psychometric evaluation of social cognitive measures for adults with autism. Autism Research, 12(5), 766–778. http://doi.org/10.1002/aur.2084.CrossRefGoogle ScholarPubMed
Murray, J. D., Anticevic, A., Gancsos, M., Ichinose, M., Corlett, P. R., Krystal, J. H., & Wang, X.-J. (2014). Linking microcircuit dysfunction to cognitive impairment: Effects of disinhibition associated with schizophrenia in a cortical working memory model. Cerebral Cortex (New York, NY: 1991), 24(4), 859–872. http://doi.org/10.1093/cercor/bhs370.Google Scholar
Nieuwenhuys, R., Voogd, J., & van Huijzen, C. (2008). The human central nervous system (4 ed.). New York: Springer-Verlag.CrossRefGoogle Scholar
Nuechterlein, K. H., Green, M. F., Kern, R. S., Baade, L. E., Barch, D. M., Cohen, J. D., … Marder, S. R. (2008). The MATRICS Consensus Cognitive Battery, part 1: Test selection, reliability, and validity. The American Journal of Psychiatry, 165(2), 203–213. http://doi.org/10.1176/appi.ajp.2007.07010042.CrossRefGoogle ScholarPubMed
Okruszek, Ł, & Pilecka, I. (2017). Biological motion processing in schizophrenia – Systematic review and meta-analysis. Schizophrenia Research, 190, 3–10. http://doi.org/10.1016/j.schres.2017.03.013.CrossRefGoogle ScholarPubMed
Orbelo, D. M., Grim, M. A., Talbott, R. E., & Ross, E. D. (2005). Impaired comprehension of affective prosody in elderly subjects is not predicted by age-related hearing loss or age-related cognitive decline. Journal of Geriatric Psychiatry and Neurology, 18(1), 25–32. http://doi.org/10.1177/0891988704272214.CrossRefGoogle ScholarPubMed
Patel, G. H., Sestieri, C., & Corbetta, M. (2019). The evolution of the temporoparietal junction and posterior superior temporal sulcus. Cortex, 118, 38–50. http://doi.org/10.1016/j.cortex.2019.01.026.CrossRefGoogle ScholarPubMed
Pinkham, A. E., Penn, D. L., Green, M. F., & Harvey, P. D. (2015). Social cognition psychometric evaluation: Results of the initial psychometric study. Schizophrenia Bulletin, 42(2), 494–504.CrossRefGoogle ScholarPubMed
Russ, B. E., & Leopold, D. A. (2015). Functional MRI mapping of dynamic visual features during natural viewing in the macaque. Neuroimage, 109, 84–94. http://doi.org/10.1016/j.neuroimage.2015.01.012.CrossRefGoogle ScholarPubMed
Savla, G. N., Vella, L., Armstrong, C. C., Penn, D. L., & Twamley, E. W. (2013). Deficits in domains of social cognition in schizophrenia: A meta-analysis of the empirical evidence. Schizophrenia Bulletin, 39(5), 979–992. http://doi.org/10.1093/schbul/sbs080.CrossRefGoogle ScholarPubMed
Sereno, M. I., Dale, A. M., Reppas, J. B., Kwong, K. K., Belliveau, J. W., Brady, T. J., … Tootell, R. B. H. (1995). Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science (New York, NY), 268(5212), 889–893.CrossRefGoogle ScholarPubMed
Shepherd, S. V., Steckenfinger, S. A., Hasson, U., & Ghazanfar, A. A. (2010). Human-monkey gaze correlations reveal convergent and divergent patterns of movie viewing. Current Biology: CB, 20(7), 649–656. http://doi.org/10.1016/j.cub.2010.02.032.CrossRefGoogle ScholarPubMed
Sowden, S., Schuster, B., & Cook, J. (2019). The role of movement kinematics in facial emotion expression. Presented at the th International Conference on Educational Neuroscience. http://doi.org/10.3389/conf.fnhum.2019.229.00018/event_abstract.CrossRefGoogle Scholar
Sparks, A., McDonald, S., Lino, B., O'Donnell, M., & Green, M. J. (2010). Social cognition, empathy and functional outcome in schizophrenia. Schizophrenia Research, 122(1–3), 172–178. http://doi.org/10.1016/j.schres.2010.06.011.CrossRefGoogle Scholar
Taylor, S. F., Kang, J., Brege, I. S., Tso, I. F., Hosanagar, A., & Johnson, T. D. (2012). Meta-analysis of functional neuroimaging studies of emotion perception and experience in schizophrenia. Biological Psychiatry, 71(2), 136–145. http://doi.org/10.1016/j.biopsych.2011.09.007.CrossRefGoogle Scholar
Veddum, L., Pedersen, H. L., Landert, A.-S. L., & Bliksted, V. (2019). Do patients with high-functioning autism have similar social cognitive deficits as patients with a chronic cause of schizophrenia? Nordic Journal of Psychiatry, 73(1), 44–50. http://doi.org/10.1080/08039488.2018.1554697.CrossRefGoogle ScholarPubMed
Velthorst, E., Fett, A.-K. J., Reichenberg, A., Perlman, G., van Os, J., Bromet, E. J., & Kotov, R. (2017). The 20-year longitudinal trajectories of social functioning in individuals with psychotic disorders. American Journal of Psychiatry, 174(11), 1075–1085. http://doi.org/10.1176/appi.ajp.2016.15111419.CrossRefGoogle ScholarPubMed
Weschler, D. (1997). WAIS-III Wechsler Adult Intelligence Scale. San Antonio, TX: The Psychological Corporation.Google Scholar
White, B. J., Berg, D. J., Kan, J. Y., Marino, R. A., Itti, L., & Munoz, D. P. (2017). Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video. Nature Communications, 8(1), 14263. http://doi.org/10.1038/ncomms14263.CrossRefGoogle ScholarPubMed
Wilming, N., Kietzmann, T. C., Jutras, M., Xue, C., Treue, S., Buffalo, E. A., & König, P. (2017). Differential contribution of low- and high-level image content to eye movements in monkeys and humans. http://doi.org/10.1093/cercor/bhw399.CrossRefGoogle Scholar
Zaki, J., & Ochsner, K. N. (2009). The need for a cognitive neuroscience of naturalistic social cognition. Annals of the New York Academy of Sciences, 1167, 16–30. http://doi.org/10.1111/j.1749-6632.2009.04601.x.CrossRefGoogle ScholarPubMed
Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation, and landmark localization in the wild. Rhode Island: Computer Vision and Pattern Recognition (CVPR) Providence.Google Scholar
Supplementary material: PDF

Patel et al. supplementary material

Patel et al. supplementary material

Download Patel et al. supplementary material(PDF)
PDF 2.4 MB