This week, we are discussing specialized research designs that we often see in behavioral studies.? Watch the following excerpts from psychology studies and read the a

Perception, 1999, volume 28, pages 1059 ̂ 1074

Gorillas in our midst: sustained inattentional blindness for dynamic events

Daniel J Simons, Christopher F Chabris Department of Psychology, Harvard University, 33 Kirkland Street, Cambridge, MA 02138, USA; e-mail: [email protected] Received 9 May 1999, in revised form 20 June 1999

Abstract. With each eye fixation, we experience a richly detailed visual world. Yet recent work on visual integration and change direction reveals that we are surprisingly unaware of the details of our environment from one view to the next: we often do not detect large changes to objects and scenes ( c̀hange blindness’). Furthermore, without attention, we may not even perceive objects (`inattentional blindness’). Taken together, these findings suggest that we perceive and remember only those objects and details that receive focused attention. In this paper, we briefly review and discuss evidence for these cognitive forms of `blindness’. We then present a new study that builds on classic studies of divided visual attention to examine inattentional blindness for complex objects and events in dynamic scenes. Our results suggest that the likelihood of noticing an unexpected object depends on the similarity of that object to other objects in the display and on how difficult the priming monitoring task is. Interestingly, spatial proximity of the critical unattended object to attended locations does not appear to affect detection, suggesting that observers attend to objects and events, not spatial positions. We discuss the implications of these results for visual representations and awareness of our visual environment.

1 Introduction

`̀ It is a well-known phenomenon that we do not notice anything happening in our surround- ings while being absorbed in the inspection of something; focusing our attention on a certain object may happen to such an extent that we cannot perceive other objects placed in the peripheral parts of our visual field, although the light rays they emit arrive completely at the visual sphere of the cerebral cortex.”

Rezso« Bä lint 1907 (translated in Husain and Stein 1988, page 91)

Perhaps you have had the following experience: you are searching for an open seat in a crowded movie theater. After scanning for several minutes, you eventually spot one and sit down. The next day, your friends ask why you ignored them at the theater. They were waving at you, and you looked right at them but did not see them. Just as we sometimes overlook our friends in a crowded room, we occasionally fail to notice changes to the appearance of those around us. We have all had the embarrassing experience of failing to notice when a friend or colleague shaves off a beard, gets a haircut, or starts wearing contact lenses. We feel that we perceive and remember everything around us, and we take the occasional blindness to visual details to be an unusual exception. The richness of our visual experience leads us to believe that our visual representations will include and preserve the same amount of detail (Levin et al 2000).

The disparity between the richness of our experience and the details of our repre- sentation, though `well known’ to Bä lint in 1907, has been studied only sporadically in the psychological literature since then, and many of the most striking results appear to have been neglected by contemporary researchers. Although the past 20 years have seen increasing interest in the issue of the precision of visual representations, a series of studies from the 1970s and 1980s using dynamic visual displays provides some of the most dramatic demonstrations of the importance of attention in perception (see Neisser 1979 for an overview). In these studies, observers engage in a continuous task that requires them to focus on one aspect of a dynamic scene while ignoring others.

1060 D J Simons, C F Chabris

At some point during the task an unexpected event occurs, but the majority of observers do not report seeing it even though it is clearly visible to observers not engaged in the concurrent task (Becklen and Cervone 1983; Littman and Becklen 1976; Neisser 1979; Neisser and Becklen 1975; Rooney et al 1981; Stoffregen et al 1993; Stoffregen and Becklen 1989). Although these studies have profound implications for our understand- ing of perception with and without attention, and despite their obvious connection to more recent work on visual attention (eg change blindness, attentional blink, repetition blindness, inattentional blindness), the empirical approach has fallen into disuse. One goal of our research is to revive the approach used in these original studies of `selective looking’ in the context of more recent work on visual attention.

Over the past few years, several researchers have demonstrated that conscious per- ception seems to require attention. When attention is diverted to another object or task, observers often fail to perceive an unexpected object, even if it appears at fixation ö a phenomenon termed `inattentional blindness’ (eg Mack and Rock 1998).(1) These findings are reminiscent of another set of findings falling under the rubric of c̀hange blindness’. Observers often fail to notice large changes to objects or scenes from one view to the next, particularly if those objects are not the center of interest in the scene (Rensink et al 1997). For example, observers often do not notice when two people in a photograph exchange heads, provided that the change occurs during an eye movement (Grimes 1996; see Simons and Levin 1997 for a review). Such studies suggest that attention is necessary for change detection (see also Scholl 2000), but not sufficient, as even changes to attended objects are often not noticed (Levin and Simons 1997; Simons and Levin 1997, 1998; Williams and Simons 2000). For example, observers who were giving directions to an experimenter often did not notice that the experimenter was replaced by a different person during an interruption caused by a door being carried between them (Simons and Levin 1998).

Both areas of research focus on two fundamental questions. (i) To what degree are the details of our visual world perceived and represented? (ii) What role does attention play in this process? We will review recent evidence for inattentional blindness to provide a current context for a discussion of earlier research on the perception of unexpected events. We then present a new study examining the variables that affect inattentional blindness in naturalistic, dynamic events, and consider the results within the broader framework or recent attention research, including change blindness.

1.1 Inattentional blindness Studies of change blindness assume that, with attention, features can be encoded (abstractly or otherwise) and retained in memory. That is, all of the information in the visual environment is potentially available for attentive processing. Yet, without atten- tion, not much of this information is retained across views. Studies of inattentional blindness have made an even stronger claim: that, without attention, visual features of our environment are not perceived at all (or at least not consciously perceived)ö observers may fail not just at change detection, but at perception as well.

Recent work on the role of attention in perception has explored what happens to unattended parts of simple visual displays (Mack and Rock 1998; Mack et al 1992; Moore and Egeth 1997; Newby and Rock 1998; Rock et al 1992; Rubin and Hua 1998; Silverman and Mack 1997). In traditional models of visual search, features are often assumed to be processed preattentively if search speeds are unaffected by the number (1) Mack and Rock (1998) draw a distinction between conscious perception and implicit perception. Consistently with this distinction, when we use the term `perceive’ (or `notice’ or `see’) in this paper, we mean that observers have at some point had a conscious experience of an object or event. How- ever, it is important to note that even when observers do not perceive an object, it may still have an implicit influence on their subsequent decisions and performance (eg Chun and Jiang 1998; Moore and Egeth 1997).

Gorillas in our midst 1061

of distracter items in the display (ie the feature `pops out’ effortlessly). Preattentive processing of some features would allow for rapid perception of more complex objects that are built by combining such sensory primitives. However, visual search tasks may not truly assess the processing of unattended stimuli because observers have the expect- ation that a target may appearöobservers know that they will have to search the display for a particular stimulus. Hence, they may expect to perceive these features, which would allow their visual/cognitive system to anticipate the features. The inatten- tional-blindness paradigm developed by Mack, Rock, and colleagues avoids this potential confound of knowledge of the task (eg Mack and Rock 1998), allowing a more direct assessment of the perception of unattended stimuli. In a typical version of their task, observers judge which of two arms of a briefly displayed large cross is longer. On the fourth trial of this task, an unexpected object appears at the same time as the cross. After this trial, observers are asked to report if they saw anything other than the cross. After answering this question, observers view another trial, now with the suggestion that something might appear. This allows an assessment of perception under conditions of divided attention. Last, subjects complete a final, full-attention trial in which they look for and report the critical object but ignore the cross. Performance on the critical, unattended trial is compared with that on the divided-attention and full-attention trials to estimate the degree to which attention influences perception. The difference in the proportion of subjects noticing on the full-attention and critical trials is the amount of inattentional blindness.

Several clear patterns emerge from this body of research (see Mack and Rock 1998 for an overview). (i) About 25% of subjects are inattentionally blind when the cross is presented at fixation and the unexpected object is presented parafoveally (subjects typically detect the critical stimulus on divided-attention and full-attention trials). (ii) About 75% of subjects are inattentionally blind when the cross is presented para- foveally and the unexpected object is presented at fixation, suggesting an effortful shift of attention away from fixation to the cross and possible inhibition of processing at the ignored fixation location. (iii) These levels of detection are no different for features thought to be preattentively processed (eg color, orientation, motion) and those thought to require effort. (iv) Although objects composed of simple visual features are not easily detected, some meaningful stimuli are. Observers typically notice their own name or a smiley face even when they did not expect it. Note, however, they do not tend to notice their own name if one letter is changed (see also Rubin and Hua 1998). Observers do not consciously perceive the visual features, but they do perceive the meaning. (v) Observers seem to focus attention on particular locations on the screen. Objects that appear inside this zone of attention are more likely to be detected than those appearing out- side (Mack and Rock 1998; Newby and Rock 1998), suggesting that attention is focused not on the object or event itself, but on the area around that object.

1.2 `Selective looking’ These recent studies of inattentional blindness used simple, brief visual displays under precisely controlled timing conditions, in the vein of work on visual search and related attention paradigms that were largely designed to examine how we select and process features and objects. The paradigm was designed to be a visual analogue of dichotic- listening studies conducted during the 1950s and 1960s (Cherry 1953; Moray 1959; Treisman 1964), and largely succeeded in replicating the classic auditory effects with visual stimuli. Although relatively little unattended information reaches awareness, some particularly meaningful stimuli do. Despite the similarity of these theoretical conclusions, they are fundamentally different in an important way. Almost by necessity, dichotic-listening tasks involve dynamic rather than static events. Listening studies reveal a degree of `inattentional deafness’ that extends over time and over changes in the unattended stimulus.

1062 D J Simons, C F Chabris

In that sense, the computer-based inattention paradigm is not a true analogue of dichotic- listening tasks. Although the theoretical conclusions match our experience of not seeing friends in a crowded theater (and hearing our own name spoken at a noisy party), the experimental paradigm may not fully capture all aspects of that natural situation [see Neumann et al (1986) for a discussion of the difficulties of equating auditory and visual divided-attention tasks]. However, an earlier series of studies by Neisser and his colleagues did use dynamic events to address many of the same questions.

In an initial study (Neisser and Becklen 1975), observers viewed a display which presented two overlapping, simultaneous events. (The superimposition was achieved by showing both of the separately recorded events on an angled, half-silvered mirror.) One of the events was a hand-slapping game in which one player extended his hands with palms up and the other player placed his hands on his opponents hands with palms down. The player with palms up tries to slap the back of the other player’s hands, and the other player tries to avoid the slap. The second event depicted three people moving in irregular patterns and passing a basketball. Subjects were asked to closely monitor one of the two events. If they monitored the hand game, they pressed a button with each attempted slap. If they monitored the ball game, they pressed the button for each pass. Each subject viewed a total of ten trials. The first two trials showed each of the games alone. On the 3rd and 4th trials, both events were presented simultaneously, but subjects were asked to follow only one of them. On the 5th and 6th trials, subjects attempted to respond to both events, using one hand to respond to each (only twenty actions per minute rather than forty occurred in these two and subsequent trials). On the last four trials, subjects responded to only one of the events, but an additional unexpected event occurred as well. In trial 7, the two hand-game players stopped and shook hands. On trial 8, one of the ball-game players threw the ball out of the game and the players continued to pretend to be passing the ball. The ball was returned after 20 s of fake throws. On trial 9, the hand-game players briefly stopped their game and passed a small ball back and forth. On trial 10, each of the ball-game players stepped off camera and was replaced by a woman and, after 20 s, the original men returned in the same fashion.

The results of this study are largely consistent with the findings of computer-based inattention studies. In the initial trials, subjects could easily follow one event while ignoring another event occupying the same spatial position. [This was true even when subjects were not allowed to move their eyes; see Littman and Becklen (1976).] Not surprisingly, they had much greater difficulty simultaneously monitoring both events. More importantly, in the initial trial with an unexpected event, only one of twenty-four people spontaneously reported the hand shake, and three others mentioned it in post- experiment questioning. None of the subjects spontaneously reported the disappearance of the ball, three spontaneously reported the ball pass in the hand game, and three reported the exchange of women for men on the final trial. Subjects who noticed one of the unusual events were more likely to notice subsequent unusual events, much as subjects in the divided-attention conditions in inattentional-blindness studies typically reported the presence of the previously `unexpected’ object (Mack and Rock 1998). In total, 50% of Neisser and Becklen’s (1975) subjects showed no indication of having seen any of the unexpected events, and even subjects who did notice typically could not accurately report the details of them.

In a more recent version of this sort of divided-visual-attention task, observers viewed superimposed videotapes of two of the ball games described above (Becklen, Neisser, and Littman, discussed in Neisser 1979).(2) The players in one game wore

(2) Many of the `selective-looking’ studies conducted by Neisser and his colleagues were never published in complete empirical reports. In such cases, as here, we have cited unpublished or in-preparation manuscripts on the basis of their descriptions in other, published materials.

Gorillas in our midst 1063

black shirts and the players in the other game wore white shirts. This change made the attended and ignored events more similar, and therefore more difficult to discrim- inate. Nevertheless, observers could successfully follow one game while ignoring the other even when both teams wore the same clothing (in fact, the same three players appeared in each video stream).

In subsequent studies of selective looking, Neisser and his colleagues used this `basket- ball-game’ task [see Neisser (1979) for a description of several different versions]. In the most famous demonstration, observers attend to one team of players, pressing a key whenever one of them makes a pass, while ignoring the actions of the other team. After about 30 s, a woman carrying an open umbrella walks across the screen (this video was also superimposed on the others so all three events were partially transparent; see figure 1). She is visible for approximately 4 s before walking off the far end of the screen. The games then continue for another 25 s before the tape is stopped. Of twenty-eight naive observers, only six reported the presence of the umbrella woman, even when questioned directly after the task (Neisser and Dube, cited in Neisser 1979). Interestingly, when subjects had practice performing the task on two similar trials before the trial with the unexpected umbrella woman, 48% noticed her. When subjects just watched the screen and did not perform any task, they always noticed the umbrella woman, a result consistent with the inattentional-blindness findings reviewed earlier (and with work on saccade-contingent changes; see Grimes 1996; McConkie and Zola 1979).

Figure 1. A single frame captured from a late-generation video of the umbrella-woman sequence used by Neisser and colleagues (eg Neisser 1979). The woman is in the center of the image and her umbrella is white.

Interestingly, Neisser (1979) mentioned an additional study in which the umbrella woman wore the same-color shirt as either the attended or the unattended team. Apparently, this feature-similarity manipulation caused little difference in the rate of noticing. Also, when the unexpected character was a small boy rather than the umbrella woman, fewer subjects noticed him, and when the umbrella woman stopped her motion and performed a little dance, more subjects noticed. These latter two findings suggest that properties of an unattended stimulus can capture attention, even though similarity to attended stimuli seemed to make little difference. However, these findings must be evaluated tentatively, because the details of the experimental paradigm were not presented by Neisser (1979).

1064 D J Simons, C F Chabris

In one of the few published empirical reports with this paradigm, Becklen and Cervone (1983) examined the effect of eliminating the delay between the umbrella- woman’s appearance and the questioning of subjects. They found no difference in noticing rates when the entire video was shown (35%) and when the video ended immediately after the umbrella woman left the scene (33%). Furthermore, performance was substantially worse when the video ended as the umbrella woman was halfway across the court (7%), even though that meant that the last image subjects saw included the umbrella woman. These subjects provided accurate descriptions of the scene, including details of the player locations, but did not mention an umbrella woman. Neisser and Rooney (cited in Becklen and Cervone 1983) addressed the same question by interrupting the action as the umbrella woman was two-thirds of the way across the screen. Immediately after the interruption, a split screen appeared with the umbrella woman on one side and a boy holding a soda can on the other; subjects were asked to pick which they had seen. When the results were corrected for guessing, only 30% of observers had noticed the umbrella woman, a level comparable to that shown in postexperiment interviews.

These findings provide important evidence against the notion of `inattentional amnesia’, an alternative account of findings of inattentional blindness and change blindness. According to this view (Wolfe 1999), the unexpected event is consciously perceived, but immediately forgotten. Hence, the failure to report its appearance reflects a failure of memory rather than of perception. In this case, however, even though subjects are tested immediately after the event, they are no better at detecting it. Further- more, when people notice the unexpected event in this task, they sometimes smile or laugh; nonnoticers show no outward signs of detection. The forgetting would have to be so rapid as to be inseparable from the act of perception to allow any sort of amnesia to explain these findings.

This early work on selective looking raised a number of questions needing further study. What role does similarity between the unexpected and attended events play in detection? Are particularly unusual events more or less likely to be detected? Does task difficulty increase or decrease detection? Perhaps the most important question left unanswered in this early work is what role the unusual superimposition of the events played in causing inattentional blindness. Most cognitive psychologists we have talked to found these results interesting, but were somewhat less convinced of the importance of the failures to notice unexpected events. After all, the video superimposition gives the displays an odd appearance, one not typically experienced in the real world and one in which the players and the umbrella woman are not as easy to see as they would be without superimposition.

One more recent study has looked at performance when all of the actors and the umbrella woman are shot from a single video camera, with no superimposition (Stoffregen et al 1993). Under these conditions, the players and umbrella woman occluded each other and the balls. If failures to notice the umbrella woman in earlier studies resulted from the unnatural appearance of the superimposed version of the display, performance might be much better with a `live’ version. Subjects performed the task for approximately 30 s before the umbrella woman appeared and walked across the screen. The camera angle used for this film was much wider than in earlier studiesöit showed an entire regulation basketball court. Consequently, the umbrella woman was visible for a longer time (12 s) and the players and the umbrella woman were substantially smaller on screen than in earlier studies. Another notable difference is that only twelve passes occurred during the 60-s video (rather than 20 ̂ 40 as in earlier studies). Even in this live version of the study, only three of twenty normal subjects tested reported the presence of the umbrella woman. Although this finding does suggest that visual superimposition was not the cause of failures of noticing, it did

Gorillas in our midst 1065

not match the stimulus conditions of the other studies and did not directly compare performance with and without superimposition. The difference in camera angle (and consequent character size) alone may well have affected detection rates, so this study is not a well-controlled test of the generalizability of inattentional-blindness phenomena to more natural stimulus conditions.

Despite the importance of all the unanswered questions raised by these studies, to our knowledge the findings reviewed above are the only published reports using dynamic, naturalistic events to study the detection of unexpected objects.(3) Taken together, these studies lead to a number of striking conclusions, some consistent and others inconsis- tent with findings with simple displays. Unlike the computer-based studies (eg Mack and Rock 1998; Newby and Rock 1998), the video studies demonstrate that inatten- tional blindness does not result from attention being focused elsewhere in the display. In the superimposed version of the display, the umbrella woman occupied exactly the same spatial position as the attended players and balls. In fact, the balls even passed through the umbrella woman. This finding is inconsistent with the computer-based result that detection was better when the unexpected object appeared within the region defined by the attended object (Mack and Rock 1998). Several factors might account for this difference. First, there were simply more objects to attend to in the video displays, so attention may not have stayed on any one location for long. Second, the dynamic display may have captured and held attention more effectively than the cross task. Third, the video task may simply have been harder, leaving fewer attentional resources available to process unanticipated events. These video studies do show that a form of inattentional blindness can last much longer than the brief exposure times used in recent static-display studies. Subjects missed ongoing events that lasted for more than 4 s.

Although these differences between the computer-based and video studies are important, the general similarity of the conclusions is striking. In both cases, observers often do not see unanticipated objects and events. The video studies suggest that these findings can help explain real-world phenomena such as our inability to see our friends in a crowded movie theater or airplanes on an approaching runway when our attention is focused on a different goal. Both change blindness and inattentional blindness show that attention plays a critical role in perception and in representation. Without atten- tion, we often do not see unanticipated events, and even with attention, we cannot encode and retain all the details of what we see.

Although these video studies of inattentional blindness help to generalize findings from simple displays to more complex situations, the original reports do not fully examine all of the critical questions. For example, there is a hint that the visual similarity of the unexpected object to the attended ones makes no difference, but the details of that study were never published. Furthermore, the experiments did not systematically consider the role of task difficulty in detection. Perhaps most importantly, no direct comparisons were made between performance with the superimposed version of the display and with the `live’ version. In the studies reported here, we attempt to examine each of these factors. We also consider the nature of the unusual event. To combine all of these factors orthogonally within a single consistent paradigm, we filmed several video segments with the same set of actors in the same location on the same day. We then asked a large number of naive observers to watch the video recordings and later answer questions about the unexpected events.

(3) Haines (1989) did address this topic as part of a larger human-interface study. Pilots attempted to land a plane in a flight simulator while using a head-up display of critical flight information superimposed on the `windshield’. Under these conditions, some pilots failed to notice that a plane on the ground was blocking their path. In addition, Mack and Rock (1998) report several studies in which the unexpected object moved stroboscopically across part of the display, often without being detected during the 200 ms viewing period.

1066 D J Simons, C F Chabris

2 Method 2.1 Observers 228 observers, almost all undergraduate students, participated in the experiment. Each observer either volunteered to participate without compensation, received a large candy bar for participating, or was paid a single fee for participating in a larger testing session including another, unrelated experiment.

2.2 Materials Four videotapes, each 75 s in duration, were created. Each tape showed two teams of three players, one team wearing white shirts and the other wearing black shirts, who moved around in a relatively random fashion in an open area (approximately 3 m deep65.2 m wide) in front of a bank of three elevator doors. The members of each team passed a standard orange basketball to one another in a regular order: player 1 would pass to player 2, who would pass to player 3, who would pass to player 1, and so on. The passes were either bounce passes or aerial passes; players would also dribble the ball, wave their arms, and make other movements consistent with their overall pattern of action, only incidentally looking directly at the camera.

After 44 ̂ 48 s of this action, either of two unexpected events occurred: in the Umbrella-Woman condition, a tall woman holding an open umbrella walked from off camera on one side of the action to the other, left to right. The actions of the players, and this unexpected event, were designed to mimic the stimuli used by Neisser and colleagues. In the Gorilla condition, a shorter woman wearing a gorilla costume that fully covered her body walked through the action in the same way. In either case, the unexpected event lasted 5 s, and the players continued their actions during and after the event.

There were two styles of video: in the Transparent condition, the white team, black team, and unexpected event were all filmed separately, and the three video streams were rendered partially transparent and then superimposed by using digital video-editing software. (Neisser and colleagues achieved similar effects using analog equipm

The post This week, we are discussing specialized research designs that we often see in behavioral studies.? Watch the following excerpts from psychology studies and read the a first appeared on Nursing StudyMasters.