Current Research 2017                                                      

William P. Seeley

Department of Philosophy

Bates College


Attentional Engines: A Perceptual Theory of the Arts (under contract).

Attentional Engines applies recent breakthroughs in cognitive science to some of the enduring questions of philosophical aesthetics. The emerging view in cognitive science is that top down influences from memory, attention, emotion, and motor expertise, and everyday motor skills play a broader and more constructive role in shaping the contents of ordinary perception than had been traditionally thought. These results suggests that we revisit standard positions in the philosophy of art regarding the nature of pictorial recognition, the ontology of sculpture, metakinesis and kinesthetic appreciation in dance, musical expression, motion picture comprehension, and the utility of psychology and neuroscience to explanations of the normative dimension of artistic appreciation. The net result is a novel, uniquely philosophical view of art that integrates research from psychology and neuroscience into contemporary discussions of a range of puzzles in philosophy of the arts.

Why do we find art so engaging? My proposal is that artworks are attentional engines. Artists and audiences are actively engaged in a collaborative back and forth exchange that has produced a variety of different categories of art defined by unique sets of stylistic devices and associated evaluative conventions. The different stylistic devices that define categories of fine art can thereby be thought of as perceptual strategies that direct a consumer's attention to features of a work critical for conveying its content and holding its there (Carroll & Seeley, 2013; Rollins, 2004; Seeley, 2013). The concepts that encode our knowledge of different categories of art can be thought of as recipes for skillfully engaging with artworks, as filters that guide how we attend to a work and shape how we perceive, experience, understand, and evaluate it. The reason we find art so engaging, then, is that artworks are a unique class of artifacts that are naturally fine tuned to a range of basic perceptual processes via the collaborative exchange between artists and audiences.

Of course, the fact that artworks are fine-tuned to perception doesn't, in itself, explain why are we express an interest in them. Why is it that we are so drawn to the often odd and abstract objects and events in this class of artifacts? I suggest that we start with an observation about ordinary perception. Only a small subset of the vast array of perceptual information available at a given time is salient to our current behavior. Selectivity is therefore a critical issue in ordinary perception. How do perceptual systems handle this issue? One thought is that minimal sets of perceptual features are sufficient for a coarse categorization of objects and events in ordinary contexts. Feedback from this kind of quick and dirty judgment about the identity of an object or event can then be used to fine-tune perceptual processing on the fly and direct attention to those features of the environment relevant to the current context. Importantly, these perceptual processes are integrated with those affective processes responsible for representing the significance of different categories of stimuli to the goals, interests, and well-being of the perceiver. In short, affective processes that are critical to explaining why we experience some aspects of our environment as valuable and phenomenally engaging also guide attention and shape perceptual processing at very early stages. The reason, then, that we find ourselves drawn to artworks is that they are attentional engines fine-tuned to a range of basic perceptual processes that include those affective processes that encode the significance, or value of objects and events to the goals and interests of the perceiver.

The goal of Attentional Engines is to articulate a model for understanding artworks in a range of different media as attentional engines and to demonstrate its relevance to discussions of a number of topical issues in the philosophy of arts including the nature of pictorial recognition, the dynamics of sculpture, kinesthetic understanding and appreciation in dance, musical expression, narrative understanding in film, and the normative dimension of artistic appreciation.


1. Selective attention and cross-modal perception in the arts.

A broad range of behavior is associated with cross-modal perception in the arts: we hear the sounds of a movie emanating from the location of the visual event it depicts; the perceived duration of a note we hear is influenced by the gesture with which we see it performed; motor skill and perceived task difficulty can influence the way we hear music and see the spatial layout of events depicted in pictures; and we sometimes report that the music we hear feels sad or exuberant. Philosophical explanations of cross-modal perception often make reference to neuroscientific discussions of multisensory integration in selective attention (Spence & Driver, 2004). This research demonstrates that superior colliculus plays a regulative role in attention, integrating unique modality specific visual, auditory, and somatosensory spatial maps into a common spatial framework for action, and that motor skill, emotional salience, and the semantic salience contribute to the integration of auditory, visual, and somatosensory information in ordinary perceptual contexts. I present a model for multisensory integration in our engagement with artworks derived from a diagnostic recognition framework for object recognition (Schyns, 1998) and a biased competition model for selective attention (Desimone and Duncan, 1995). The proposed model attributes a role to superior colliculus in a broader fronto-parietal attentional network that integrates sensory information, primes perceptual systems to the expectation of stimulus features salient to particular tasks (e.g. visual search, object identification, or reaching and grasping) at particular locations, and inhibits the perception of task irrelevant distracters. I argue that this model demonstrates cross-modal effects are the rule not the exception in perception and discuss ways in which it explains a range of cross-modal effects in our perceptual engagement with visual artworks and musical compositions.

2. Art, meaning, and perception: a question of methods for a cognitive neuroscience of art.

Neuroaesthetics might give us traction with aesthetic issues. However it can be seen to have trouble modeling the artistically salient semantic properties of artworks. This is a particularly trenchant problem for perceptual media, e.g. dance, music, and painting. So if meaning really matters…and it does…even in aesthetic contexts…the prospects for this nascent field are dim. I review research by Margaret Livingstone suggesting that the aesthetic features of Monet's Impressionist paintings emerge from the ways he used spatial frequency information to produce dynamic perceptual effects in viewers. I argue that this strategy cannot differentiate paintings from lab demonstrations and other experimental stimuli and so cannot account for the artistic salience of these features of the work. The possibility for a solution to this problem boils down to a question of whether or not we can get a grip on the kinds of constraints present and available to guide interpretive behavior in our engagement with works of fine art. I argue that biased competition models of selective attention can be used to solve this problem, generalize to the affective content of our responses to artworks, and so show that research in cognitive neuroscience is germane to the types of problems of interest within the philosophy of art.

3. Art, Appreciation, and a Crossmodal Model for Evaluative Perception

The target of this research is a standard skeptical argument about the utility of psychology and neuroscience to explanations of the normative dimension of artistic appreciation. However, the details of the discussion have a bearing on the question of evaluative perception more generally, as well as discussions of cognitive penetrability and high-level views of perceptual content. In each of these cases what is at issue is a question about the potential productive influence of learning, memory, and attention on the structure of perceptual experience. I extend the crossmodal model for perception in everyday contexts derived from a diagnostic recognition framework, a biased competition model for selective attention to recent discussions of the relationship between cognition, emotion, and perception  (Barrett and Bar, 2009; Pessoa and Adolphs, 2010; Pessoa, Kastner, and Ungerleider, 2002). The view that emerges from this discussion suggests that the affective and cognitive significance of environmental stimuli are integrated into perceptual experience via reciprocal unimodal, sensorimotor, and affective attentional circuits that link prefrontal brain regions associated with working memory and object recognition with the thalamus and sensory cortices. This integrated crossmodal model for perception dissolves concerns that research and methods from psychology and neuroscience are ill suited to explain the normative dimension of artistic appreciation, suggests that cognitive penetrability is the rule not the exception in perception, and supports a high-level content view of perceptual experience.


4. What is a category of art?

Philosophers often make reference to categories of art in explanations of attributions of artistically salient aesthetic qualities (Walton, 1974) and semantic content to works (Carroll, 1996; Levinson, 1992). Roughly speaking categories of art are defined by the loosely unified formal-compositional vocabularies, productive strategies, and evlauative conventions associated with the works of individual artists, schools, or historical periods. The assumption is that categories of art play a constitutive role in our engagement with artworks that is analogous to the role knowledge of the structure and function of object and event types plays in ordinary perceptual contexts. However, very little realistic attention is given to the computational structure of categories of art or how they might accomplish this role. Four competing theories of concepts are standardly identified in the concepts literature: classical necessary and sufficient conditions accounts, prototype accounts, exemplar accounts, and knowledge-based accounts (see Murphy, 2004). I argue that the use of categories of art in the literature most closely approximate knowledge-based accounts.

5. Movies as attentional engines (with Noël Carroll)

Artworks are attentional engines, or artifacts intentionally designed to direct attention to formal features that are diagnostic for their artistically salient aesthetic, expressive, and semantic content. This is nowhere more true than the movies. Moving pictures are constructed from a suite of formal and narrative devices carefully developed to capture, hold, and direct our attention, e.g. variable framing, erotetic narrative structure, and criterial prefocusing. These devices are tools for developing content by controlling the way information is presented throughout the duration of our engagement with a movie. In this respect moving pictures are analogous to visual routines used to direct and bias attention in natural behavior - they are artifacts used to deliver information on the fly as it is needed for the development of the narrative. These formal devices are designed to guide and control salient aspects of viewers behavioral responses to mass market movies. Therefore, we argue that it is no surprise to discover that that they are fine tuned to the architecture of emotion, perception, and cognition, and that their function can be modeled relative to the range of cognitive, affective, and neurophysiological processes that underwrite narrative comprehension, appreciation, and our emotional engagement with charcaters at the movies..

6. A sensiromotor model for audience engagement with dance.

The neuroscience of dance is a growing field of genuine interdisciplinary collaboration. The goal of this research is to evaluate differences in the approaches to dance taken by neuroscientists and psychologically oriented philosophers. Studies of dance in neuroscience of art have focused on correlations between affective responses and preference ratings among viewers (Calvo-Merino et al 2009). Philosophers of art draw a distinction between these types of subjective aesthetic responses and questions about interpretation, including questions about the ways artists' use formal strategies to express emotions and ideas (Carrol and Banes, 1982). Barbara Montero (2006) and Jonathan Cole (Cole & Montero, 2006)  have argued that dance recruits proprioception (i.e. awareness of the relative orientation of our limbs within our body space) as an aesthetic sense. This model for aesthetic responses to dance suggests that the formal strategies used by choregraphers to express emotions and ideas harness our natural tendency to simulate the observed actions of others. However, expertise effects in action understanding (Calvo-Merino, 2005, 2006) suggest that the phrases employed in dance may be a difficult to understand when they are interpreted simply as abstract choreographed movements. I argue that narrative devices (e.g. Bill T. Jones), conventional formal /compositional devices (e.g. George Balanchine or Merce Cunningham), and a general knowledge of artistic conventions (e.g. Yvonne Rainer) play a critical role in dance, enabling audiences to contextualize abstract choreographed movements as actions with intentional (cognitive) and expressive (emotional) content.

7. Crossmodal perception and musical expressiveness.

Musical works can also be construed as attentional engines. The local environment is replete with information, a small fraction of which is salient to an organism's current behavior. Cognitive systems are, in contrast, limited capacity systems. This entails that selectivity is a critical feature of perceptual systems. Biased competition models provide a mechanism to resolve this problem. Recurrent fronto-parietal and cortico-thalamic circuits facilitate crossmodal integration among unimodal perceptual, sensorimotor, and affective  processing systems, enhancing the encoding of expected task salient features, inhibiting the perception of distracting information, and modulating gut reactions to biologically significant stimuli (Duncan & Barrett, 2007). These crossmodal attentional networks focus attention on minimal sets of features diagnostic for, or sufficient to perceptually recognize, the shapes, identities, affordances, and emotional significance of task salient objects and events. I argue that musical works are fine-tuned to the operations of these crossmodal attentional networks which can therefore be used to model how they carry and convey emotionally expressive content.

      There is a philosophical puzzle associated with the expression of emotions by works of pure music. We, as human agents, express emotions when we publically display them in our behaviors, when our postures, gestures, and actions are expressive of our current emotions. Works of pure music are inanimate objects. They have no emotions of their own to express and they don't transparently resemble the bodily gestures through which we ordinarily express ours. Nonetheless, we regularly recognize expressive qualities in pure music. Thus the puzzle. The most obvious way to address this puzzle is to treat expressive musical works as communicative devices, as expressions of the emotions of some agent, e.g. the composer, the performer, or a fictional persona. However, this move sidesteps the real question: how do listeners recognize qualitative aspects of the emotional states of animate agents in an inanimate sound structure?

      A solution to this puzzle emerges from the conjuction of a contour theory of musical expressiveness (Kivy, 2002) and embodied appraisal theories of emotion (Robinson, 2005). Contour theory can be treated as an account of the perceptual cues that composers use to express emotions in music: the dynamic rhythmic and tonal contours of expressive musical works resemble both the perceived contours of expressive behaviors and the subjective feelings of emotions. Embodied appraisal theories provide a psychological mechanism to explain how listeners recover and recognize the expressive content of musical works: the ebb and flow of tension and release in the dynamic rhythmic and tonal contours of expressive music induces a range of autonomic responses in listeners that they recognize as resembling the unfolding dynamic contours of different types of emotion. The suggestion is that listeners recognize the expressive qualities of the music in much the same way they become aware of their emotional responses to ordinary stimuli -- via a process of cognitive monitoring, categorization, and appraisal of their own embodied autonomic responses to valenced objects, actions, and events.

      We can treat the puzzle of musical expressiveness as an information processing problem, as a question about how listeners acquire, represent, manipulate, and use information encoded in musical works to recognize its expressive content. Research using point light displays to study biological motion demonstrates that the dynamic contours of a bodily movements are sufficient to enable viewers to perceptually recognize the emotional states of actors independent of any other aspect of their visual appearance (Clarke et al, 2005; Dittrich et al, 1996). Research on the relationship between the expressive qualities of music and dance demonstrates that bodily movements and musical passages carry and convey the same expressive emotional content by virtue of exhibiting the same dynamic contours (Chapados and Levitin,2008; Krumhansl & Schenk, 1997; Vines et al, 2006). The perceived ebb and flow of tension and release in a musical passage is therefore a diagnostic cue to its expressive content which we recognize in the ebb and flow of arousal in our embodied autonomic responses to the work. I review this research and discuss how crossmodal attentional networks, e.g. sensorimotor circuits and connectivity between areas responsible for gut reactions (ventromedial prefrontal cortex, anterior cingulate cortex, and amygdala) and the attribution biological significance and valence (orbitofrontal cortex) to stimuli and auditory, somatosensory, and visual systems, can be used to model the way works of pure music carry and convey their emotionally expressive content.

8. A sensorimotor model for seeing sculpture.

The question of how to successfully imbue a static medium with the dynamics of animate agents has been a long standing problem in sculpture. I argue that this problem dissolves if one recognizes that posture, skin tension, and the relative relationship of the mass and configuration of muscles across joints are perceptual cues to the kinetics and dynamics biological movements. In ordinary perceptual contexts these cues enable viewers to perceptually interpret and predict the movements, goal directed actions, intentions, and often emotions of animate agents. In sculpture they are cues to the dynamics that enliven depicted figures. I illustrate this claim with discussions of Degas' sculptures of dancers, Rodin's Burghers of Calais, Raymond Duchamp-Villon's Le Grande Cheval, and David Smith's Australia , and discuss the degree to which it can be generalized to the use of generic motion cues in abstract sculptures, e.g. the dynamics of Calder's mobiles or the use of "speed lines" to articulate the horizontal motion of an abstract geometric figure in a cartoon.

9. Neuroscience and literature.

I explore models of discourse comprehension which suggest that we construct rich, multi-modal situation models for narrative comprehension that fill-in, or articulate, and stitch together the content of what is expressed in the sparse surface structure of speech, text, and hollywood films. I argue that this research supports embodied simulation models for semantic comprehension (Barsalou, 1999; Fisher & Zwann, 2006), narrative understanding and appreciation (Speer etal 2009; Zacks et al 2009), and empathy for characters in narrative fiction (Seeley, 2010) that integrate mechanisms, processes, and insights from cognitivist and theory-theory models.

10. Looking at Mona Lisa: a psychophysiological study.

It is often asserted that the aesthetic qualities of the Mona Lisa include the dynamic qualities of her expression. Margaret Livingston has argued that this is, in a sense, true (Livingstone, 2002). Leonardo used a technique called sfumato to render the expression of the figure in the painting, the corners of her mouth and eyes. Sfumato is a formal technique in which artists blur the sharp edges that define object features in a painting so that these boundaries disappear into the broad contours of soft, “smoky” shadow. Livingstone filtered a reproduction of Leonardo’s painting in order to separate out the low, middle, and high spatial frequency information used to depict Mona Lisa’s face. The sfumato contours that define Mona Lisa's smile were more apparent in the images representing low and middle spatial frequency information in the painting than the sharp lines of the high spatial frequency image. Therefore, critical formal features defining Mona Lisa's smile are depicted only in low and middle spatial frequency information. The spatial resolution of human vision decreases dramatically as one moves from the center of the visual field towards the periphery. This difference in spatial resolution between central, or foveal, and peripheral vision is explained by the fact that the receptive fields of peripheral retinal neurons are dramatically larger than those of their foveal counterparts. The result is that foveal neurons are sensitive to sharp, narrow luminance boundaries that carry high spatial frequency information, but are unable to register coarse, broad luminance gradients, like contours rendered in sfumato, that carry low and medium spatial frequency information. Conversely, the wider receptive fields of neurons in the peripheral field are well suited to record the latter category of contours, but are nearly blind to high spatial frequency information. Livingstone argues, as a result, that when one foveates on, or directs one’s attention to, Mona Lisa’s smile it disappears. However, the smile reappears in a viewer’s peripheral field when he or she looks away. Livingstone argues that Mona Lisa’s expression varies systematically with the eccentricity of a viewer’s gaze from the center of the painting. This suggests a way to test her theory about our engagement with the painting. Facial EMG studies using the International Affective Picture System have demonstrated that we mimic the expressions depicted in photographs of faces (Land, Greenwald, Bradley, and Ham. 1993). These studies have demonstrated that the perception of pictures depicting angry, fearful, or happy faces produces a similar facial pattern in participants, e.g. enhanced corrugator (frown) and zygomatic (smile) muscle activity for angry/fearful versus happy faces respectively. We hypothesize as a result that, if Livingstone’s theory is sound, one ought to find measurable changes in zygomatic activity a viewer’s gaze varies in eccentricity from Mona Lisa’s smile.

11. Empirical aesthetics and the philosophy of art.

The philosophy of art and empirical aesthetics are, to all outward appearances, natural bedfellows, disciplines bound together by complimentary methodologies and the common goal of explaining a shared subject matter. Philosophers are in the business of sorting out the ontological and normative character of different categories of objects, events and behaviors, squaring up our conception of the nature of things, and clarifying the subject matter of different avenues of intellectual exploration via careful conceptual analyses of often complex conventional practices. Psychologists have developed careful empirical methods for measuring and modeling behavior, methods that are fruitfully used in practice to test and evaluate hypotheses derived from our conception of the nature of our own cognitive and emotional engagement with the world. So, for all appearances, philosophy and psychology share in the common task of sorting and testing theories about the nature of art and artistic practices (e.g., what is an artwork, what is the nature of the productive practices involved in creating these kinds of artifacts, and what is the nature of a consumer's artistic engagement with these artifacts). Unfortunately appearances can be deceptive. Despite common calls for rapprochement, the two disciplines rarely meet. There are both methodological and ideological reasons for this rift, and they are not surprisingly related. I have explored some of the central sources of resistance on both sides of this divide, introduced a model for the possibility of rapprochement, and sketched the promise and pitfalls of current research in dance and film, where an active attempt at bridging the divide between philosophy and empirical aesthetics is underway.

12. Attention and Cognitive Control in Affective Perception for Embodied Appraisals.

Embodied appraisal theories like Prinz (2004) treat emotions as direct, noncognitive embodied responses to biologically and behaviorally salient aspects of the environment that are under environmental control. Prinz defines cognitive states and processes, in contrast, as mental states and processes that are under organismic control. Organismic control is, in turn, defined relative to top-down neurophysiological processes that originate in prefrontal cortex and are associated with executive control of behavior. Recent evidence from affective neuroscience shows that neurophysiological measures of emotional responsiveness to emotion laden facial expressions are modulated by the availability of endogenously controlled attentional resources in run of the mill high attentional load contexts. Endogenous shifts of attention are under the top-down control of neurophysiological processes originating in prefrontal cortex associated with executive control. This entails that embodied appraisals are, by definition, cognitive representations of the biological significance of environmental stimuli under organismic control contrary to Prinz' assertions.

13. Sparse, Narcissistic, and Under Attentional Control vs. the Unity of Consciousness.

I argue that the putative unity of conscious experience, traditionally one of the defining features of a philosophical theory of consciousness, is an illusion. I argue that evidence from the role sensorimotor attentional routines play in the guidance of everyday behavior and the perceptual performance of patients suffering from visual form agnosia suggest that this illusion emerges from the fact that attention is directed to just what perceptual information we need just as we need it in ordinary behavioral contexts, which entails that we rarely, if ever, experience any slip between goal directed cognitive and perceptual behavior. I illustrate this claim by discussing the way the unified experience of film narratives are constructed from the sparse, spatio-temporally disjoint, surface structure of cinematic sequences.

14. Capoeira & Clint Eastwood: two case studies for the methodological utility of art in neuroscience.

Current research in the cognitive neuroscience of visual art lies at the confluence of two broad research strategies: empirical aesthetics and aesthetic experimentalism (Carroll, Moore, and Seeley, forthcoming; Rollins, 2004). The central claim of empirical aesthetics is that the methods of psychology and related fields can be used to enhance our understanding of the nature of art and associated behavior. Aesthetic experimentalism is derived from the observation that artists’ productive strategies emerge from systematic explorations of the perceptual effects of different sets of medium specific formal cues (Gombrich, 1960). The central claim of aesthetic experimentalism is that we can learn about the operations of perceptual systems by examining the productive strategies of artists (e.g. the way visual artists develop and use formal techniques to convey information in their works). The conjunction of these two research strategies suggest that the relationship between the philosophy of art and cognitive neuroscience is a two way street. This claim is most often made in regards to the visual arts, and canonically for the works of naturalist painters who aim for realistic depictions of landscapes, events, and people. However, in these cases artworks are most often used to illustrate the contributions of underlying neurophysiological mechanisms to perception, not as novel stimuli in experiments (Cavanaugh, 2005; Zeki, 1999). In this research I examine two domains in which artworks are actively used as stimuli for experimental investigations in cognitive neuroscience: the use of dance in studies of the relationship between perception and action (Calvo-Merino et al, 2005; Calvo-Merino et al, 2006; Cross et al, 2006; Cross et al, 2009; Urgesi et al, 2007) and the use of Hollywood films to study the degree to which perceptual responses to ordinary events are synchronized across agents in natural contexts (Hasson et al 2008; Hasson et al, 2004). I argue that these case studies demonstrate the potential for a genuine rapprochement between philosophy and neuroscience in the study of art and cognition.

15. Imagination, picture perception, & narrative understanding.

a. Imagining Crawling Home. Philosophical accounts of narrative fiction can be loosely divided into two types. Participant accounts argue that some sort of simulation, or first-person perspective taking plays a critical role in our engagement with narratives. Observer accounts argue to the contrary that we primarily engage narrative fictions from a third-person point of view, as either outside observers or side participants to the depicted events. Recent psychological research suggests a means to evaluate this debate. The perception of distance and slope is influenced by the energetic (e.g., task difficulty) and emotional (e.g., anxiety) costs of actions (Proffitt, 2006). These effects are limited to increases in the costs of actions agents intend to perform themselves, generalize to cases where participants imagine acting, and demonstrate a role for tacit motor simulation in action planning. If the participant account is sound, one should, therefore, find similar effects across changes in the interpretation of the costs of actions depicted in static images. We asked people to copy the rough spatial layout of two paintings, Andrew Wyeth’s Christina’s World and Winter, 1946, across different interpretations of the costs of the actions they depict. We predicted that increasing costs would cause participants to draw distances as longer and hills as steeper. Our results confirm this prediction for the energetic, but not the emotional, costs of actions (although participants did expand the extent of the landscape in both contexts). In this paper we present the results of our study and argue that they demonstrate a significant role for simulation and first-person perspective taking in narrative understanding. Future directions involve using a larger range of images and stories to more carefully control for the effects of variance in the energetic costs of depicted actions. These results can also contribute to discussions of the role motor simulation plays in our general cognitive economy and debates about mental representation and embodied cognition in the philosophy of mind. This study was published as “Imagining Crawling Home: A Case Study in Cognitive Science and Aesthetics,” in Review of Philosophy and Psychology 1(3), 2010: 407-426

b. Whose Effectors, Your or Mine? Identification, Interpretation and Motor Simulation in Narrative Fiction. Energetic costs effects in slope and distance perception are effector specific, or limited to energetic costs for the particular muscle groups associated with the joints involved in anticipated actions. For instance, energetic costs associated with walking do not affect distance assessments in throwing contexts, energetic costs associated with throwing do not affect distance assessments in walking contexts, and energetic cost effects associated with anticipating reaching to a target with one’s right hand are not affected by the concurrent task of manipulating a keyboard with one’s left hand (Witt & Proffitt, 2008; Witt, Proffitt, & Epstein, 2004). These results are consistent with evidence that motor simulation selectively activates areas of the premotor cortex associated with the muscle groups that would be used to perform the target action and can contribute to the debate between participant and observer accounts of narrative understanding. There is a discrepancy between the global perspectives of spectators/readers and the local perspectives of characters within a narrative. Characters lack the global perspective of spectators and readers. Characters, unlike spectators and readers are as a result neither apprehensive nor hopeful about the probabilities of future events in the story. These phenomenal differences between the depicted experiences of characters and the real experiences of spectators and readers challenge the claim that we simulate the behavior of characters from a first person perspective (Carroll, 1997; Kieran, 2003). However, the effector specificity of motor simulation suggests that we need not project ourselves into a character in order to simulate their perspective. Rather, we hypothesize that we use our own cognitive processes to model discrete aspects of a characters’ depicted behavior when this information would contribute our understanding of their behavior. This would enable us to continue to maintain a strong sense of self while simulating the psychological perspectives of characters, as we do in ordinary contexts while simulating the behavior of others. Future directions involve generating a range of image sets and energetic costs stories to enable us to use the copying protocol described above to test this hypothesis.

c. Motor skill as a mediating variable in the effects of energetic costs on apparent egocentric distance. We predicted that, if effector specific motor simulation is the controlling mechanism, variations in motor skill, which influence task difficulty, should also affect apparent egocentric distance. We are currently conducting two experiments to test this hypothesis. In the first experiment (Ofeldt & Seeley) participants use perceptual matching measures and verbal assessments to make distance judgments about a target in three conditions: a) prior to reaching with scissors to make a small precision cut across the target with their dominant or non-dominant hands while holding the scissors, b) prior to reaching with scissors to make a small precision cut across the target with their dominant and non-dominant hands with the scissors beside them on the table, and c) while reaching and imagining using scissors to make a small precision cut across the target with their dominant and non-dominant hands (with the scissors on the table beside them). The procedures for the second experiment (Kahan & Seeley) are the same as the first except that participants anticipate or imagine reaching with a long pencil, held at the top near the eraser, to write a short word across the target with either their dominant or non-dominant hands. We predict that distances will appear longer when participants imagine or anticipate reaching to perform the task with their non-dominant hands.

d. What happens in a book stays in a book? Motor Simulation and Narrative Understanding in Literature.

It has been argued that, whereas motor simulation might play a role in the perception and understanding of visual narratives, it is unnecessary in literary contexts. Why? In these cases dialogue and narration are used as formal devices to explicitly tell the reader what he or she needs to know about a character’s motivations and dispositions. However, recent research in cognitive neuroscience has demonstrated that the premotor areas involved in motor preparation, action observation, and motor simulation play a role in the comprehension of action sentences (Fisher & Zwaan, 2008), and that these processes generalize to our understanding of actions depicted in literary contexts (Speer, Reynolds, Swallow, & Zacks, 2009). In this paper we review this research and argue it entails that our participant account model for energetic and emotional costs effects in picture perception generalizes to narrative contexts in literature.





Adolphs, R. (2004). Emotional vision. Nature Reviews Neuroscience, 7(11), 1167-1168.

Adolphs, R. and L. Pessoa (2010). Emotion Processing and the Amygdala: from a low road to many roads of evaluating biological significance. Nature Reviews Neuroscience 11: 773-783.

Barrett, L. F. and Bar, M. (2009). See It with Feelings: Affective Predictions during Object Perception. Philoosphical Transactions of the Royal Society B 364: 1325-1334.

Beardsley, M. (1982). What is going on in dance? Dance Research Journal, 15(1), 31-36.

Bonnar, L., Gosselin, F., and Schyns, P G. (2002). Understanding Dali’s Slave Market with Disappearing Bust of Voltaire: a case study in the scale information driving perception. Perception, 31, 683- 691.

Calvo-Merino B. Jola C, Glaser D, Haggard P (2008) Towards a sensorimotor aesthetics of performing art. Consciousness and Cognition, 17(3), 911-922.

Calvo-Merino, B., Grèzes, J., Glaser, D. E., Passingham, R. E., Haggard, P. (2006). Seeing or doing? Influence of visual and motor familiarity in action observation. Current Biology, 16(19), 1905-1910.

Calvo-Merino, B., Glaser, D. E., Grèzes, J, Passingham R. E, & Haggard, P. (2005). Action observation and acquired motor skills: an fMRI study with expert dancers. Cerebral Cortex,15, 1245-1249.

Carr, L., Iacoboni, M., Dubeau, M. C., Mazziotta, J. C., & Lenzi, G. L. (2003). Neural mechanisms of   empathy in humans: a relay from neural systems for imitation to limbic areas. Proceedings of the National Academy of Science, 100, 5497-5502.

Carroll, N. (1999). The Philosophy of Art (new York: Routledge).

Carroll, N. (1997). Simulation, emotions, and morality. Beyond Aesthetics (pp. 306-317). New York: Cambridge University Press, 2001.

Carroll, N., Moore, M. & Seeley, W. P. (2010). The philosophy of art and aesthetics, psychology, and neuroscience: studies in literature, visual arts, and music. In Arthur P. Shimamura & Stephen E. Palmer (Eds.), Aesthetic Science: Connecting Minds, Brains, and Experience. New York: Oxford            University Press.

Chun, M. & Marois, R. (2002). The dark side of visual attention. Current Opinion in Neurobiology, 12(22), 1-6.

Cross, E. S., Hamilton, A. F. de C., Kraemer, D. J. M, Kelley, W. M. & Grafton, S. T. (2009). Dissociable substrates for body motion and physical experience in the human action observation network. European Journal of Neuroscience, 30, 1383-1392.

Cross, E. S., Hamilton, A. F. de C., & Grafton, S. T. (2006). Building a motor simulation de novo: observation of dance by dancers. NeuroImage, 31(3), 1257-1267.

Fisher, M. H. & Zwaan, R. A. (2008). Embodied language: a review of the role of the motor system in language comprehension. The Quarterly Journal of Experimental Psychology, 61(6), 825-850.

Gallese, V., Goldman, A. (1998). Mirror neurons and the simulation theory of mind reading. Trends in Cognition Science, 2, 493–501.

Hamilton, A.F. & Grafton, S.T. (2006) Goal representation in human anterior intraparietal sulcus. The Journal of Neuroscience, 26, 1133–1137.

Hasson, U. Landsman, O., Knappmeyer, B., Vallines, I., Rubin, N.,  Heeger. D. J. (2008). Neurocinematics: The neuroscience of making movies. Projections: The Journal for Movies and Mind, 1(2): 2-26.

Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., and Malach, R. (2004). Intersubjective synchronization of cortical activity during natural vision. Science 303, 1634-1640.

Iacoboni, M. & Dapretto, M. (2006) The mirror neuron system and the consequences of its dysfunction. Nature Reviews Neuroscience, 7, 942–951.

Kastner, S. (2004). Attentional response modulation in the human visual system. In ed. Michael Posner, The Cognitive Neuroscience of Attention (New York: The Guilford Press), 144-156.

Kieran, M. (2003). In search of a narrative. Kieran, M, and Lopes, D. M. (eds.) Imagination, Philosophy, and the Arts (pp. 69-87). New York: Routledge.

Koivisto, M. & Revonsuo, A. (2007). How meaning shapes seeing. Psychological Science 18(10), 845-849.

Land, P. J., Greenwald, M. K., Bradley, M. M., and Hamm, A. O. (1993). Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30: 261-273.

Langer, S. (1953). Feeling and Form. New York: Charles Scribner’s Sons.

Livingston, M. S. (2002) Vision and Art: The Biology of Seeing. New York, New York: Harry N. Abrams.

Montero, B. (2006). Proprioception as an aesthetic sense. Journal of Aesthetics and Art Criticism, 64(2), 231-142.

Pessoa, L. S. Kastner, and L. G. Ungerleider (2002). Attentional Control of the Processing of Neutral and Emotional Stimuli. Cognitive Brain Research 15: 31-45

Prinz, J. (2004). Gut Reactions (New York: Oxford University Press).

Proffitt, D. (2006). Embodied perception and the economy of action. Perspectives in Psychological Science, 1(2), 110-122.

Rollins, R. (2004). What Monet meant: intention and attention in understanding art. Journal of Aesthetics and Art Criticism, 62(2), 2004, 175-188.

Schyns, P. (1998). Diagnostic recognition. Cognition, 67(1-2), 147-179
Seeley, W. P. (2010). Imagining crawling home: a case study in cognitive science and aesthetics. Review of Philosophy: Psychology and Experimental Philosophy.

Seeley, W. P. & Kozbelt, A. (2008). Art, artists, & perception: a model for premotor contributions to perceptual analysis and form recognition. Philosophical Psychology, 21(2), 149-171.

Shiffrar, M. & Pinto, J. (2002). The visual analysis of bodily motion. In eds. W. Prinz & B. Hommel, Common Mechanisms in Perception & Action (New York: Oxford University Press), 2002.

Speer, N. K., Reynolds, J. R., Swallow, K. M., & Zacks, J. M. (2209). Reading stories activates neural representations of visual and motor experiences. Psychological Science, 20(8), 989-999.

Witt, J. K. & Proffitt, D. R. (2008).Action-specific influences on distance perception: a role for motor simulation. Journal of Experimental Psychology: Human Perception and Performance, 34(6),             1479-1492.

Witt, J. K., Proffitt, D. R., & Epstein, W. (2004). Perceiving distance: a role of effort and intent. Perception 33(5), 570-590.

Urgesi, C., Calvo-Merino, B., Haggard, P., Aglioti, S. M. (2007). Transcranial magnetic stimulation reveals two cortical pathways for visual body processing. Journal of Neuroscience, 27(30), 8023-8030.

Vuilleumier, P. (2005). How brains beware: neural mechanisms of emotional attention. TRENDS in Cognitive Sciences, 9(12), 585-594.

Zatorre, R. J., Evans, A. C. & Meyer, E. (1994). Neural mechanisms underlying melody perception and memory for pitch. The Journal of Neuroscience, 14(4), 1908-1919.

Zwaan, R. (2004). The immersed experiencer: towards an embodied theory of language comprehension. In ed. B. H. Ross, The Psychology of Learning and Motivation (volume 44, pp. 35-63). New York: Academic Press.


(back to homepage)