text:
Main Unmasking the Face: A Guide to Recognizing Emotions From Facial Expressions
Unmasking the Face: A Guide to Recognizing Emotions From Facial ExpressionsPaul Ekman, Wallace V. Friesen
This reprint edition of Ekman and Friesen's breakthrough research on the facial expression of emotion uses scores of photographs showing emotions of surprise, fear, disgust, anger, happiness, and sadness. The authors of UNMASKING THE FACE explain how to identify these basic emotions correctly and how to tell when people try to mask, simulate, or neutralize them. It features several practical exercises that help actors, teachers, salesmen, counselors, nurses, law-enforcement personnel and physicians -- and everyone else who deals with people -- to become adept, perceptive readers of the facial expressions of emotions.
You may be interested in
Most frequently terms
You can write a book review and share your experiences. Other readers will always be interested in your opinion of the books you've read. Whether you've loved the book or not, if you give your honest and detailed thoughts then people will find new books that are right for them.
unmasking the face Paul Ekman is a professor of psychology in the department of psychiatry at the University of California Medical School, San Francisco. An expert on expression, the physiology of emotion, and interpersonal deception, he has received many honors, most notably the Distinguished Scientific Contribution Award of the American Psychological Association, and is the author or editor of thirteen other books. He is a frequent consultant on emotional expression to government agencies such as the FBI, the CIA, and the ATF, to lawyers, judges, and police, and to corporations, including the animation studios Pixar and Industrial Light and Magic. He lives in northern California. Wallace V. Friesen was Lecturer in Psychology and Research Psychologist at the University of California at San Francisco, where he worked as a co-investigator with Paul Ekman for many years. He is now at the University of Kentucky where his research focuses on emotion in old age. He has co-authored numerous articles on emotions and longevity in periodicals and journals such as the American Psychologist, Psychiatry, and the Journal of Personality and Social Psychology. Other books by Paul Ekman: EMOTIONS REVEALED NATURE OF EMOTION TELLING LIES FACE OF MAN WHY KIDS LIE EMOTION IN THE HUMAN FACE: Guidelines for Research and an Integration of Findings DARWIN AND FACIAL EXPRESSION: A Century of Research in Review Copyright O 2003 by Paul Ekman All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storge or retrieval system, except as may be expressly permitted by the 1976 Copyright Act or in writing from the publisher. Requests for permission should be addressed in writing to ISHK, P. 0. Box 381069, Cambridge, MA 02238-1069, Library of Congress Cataloghg in Publication Data for p ~ v i o u sedition is recorded as follows: Ekman, Paul. Unmasking the face. Includes bibliographies. 1.Facial expression. I. Fnesen, Wallace V., joint author. 11. Title. BF637.C45E38 152.4'2 74-14544 ISBN 0-13-938183-X ISBN 0-13-938175-9 @bk.) This Malor edition ISBN 1-883536-36-7 Printed in the United States of America V 1 0 9 8 7 6 5 4 3 2 1 UNMASKING THE FACE A guide to recognizing emotions from facial clues PAUL EKMAN and WALLACE V. FRIESEN MALOR B O O K S Cambridge MA + 2003 For Patricia and Myriam contents acknowledgments introduction xi 1 why mistakes are made in understanding facial expressions of emotion 70 research on facial expressions of emotion 27 surprise fear disgust 34 47 66 x contents 7 anger 78 8 99 happiness 9 sadness 7 74 10 practice faces 11 facial deceit 735 12 checking your own facial expression 754 13 conclusion appendix I the facial blueprint photographs appendix II practice photos for chapter 10 appendix III log and judge sheets index 203 769 acknowledgments We are grateful to the National Institute of Mental Health (NIMH) for making it possible for us to study facial expression and body movement during the last eighteen years. Paul Ekman was able to start our research when he was awarded a predoctoral research fellowship from NIMH from 1955 to 1957. During military service from 1958 to 1960 Ekman and Friesen became research associates, a relationship which was later formalized when Friesen joined the project in 1965. A postdoctoral research fellowship from NIMH made it possible for Ekman to pursue the research from 1960 to 1963. Later, when the pressure of teaching seemed likely to curtail research, a Career Development Award from NIMH to Ekman allowed the team to continue the research from 1966 to 1972. During all these years, at each critical juncture, the late Bert Boothe, director of the Research Fellowship Branch, provided invaluable help, interest, and advice. The Clinical Research Branch of NIMH provided continuous support for our research on facial expression and body movement from 1963 until now (MH 11976-09). Its support allowed us to study mental patients and also has made it possible for us to work together since 1965. We are also grateful to the Advanced Research Project Agency (ARPA) of the Department of Defense for supporting our studies from 1966 to 1970. Lee Hough, former director of ARPA, convinced us of the importance of studying facial expression and gesture in different cultures. He helped us overcome our reluctance to try to resolve the argument over the possible universality of facial expression and gesture. When we launched our research in a remote area of New Guinea, Rowena Swanson, the monitor of the grant, found ways to solve the administrative and bureaucratic obstacles. We are grateful to Silvan S. Tomkins for his contagious excitement about facial expression of emotion. He encouraged us to learn how to read xii acknowledgments faces and to teach others to do so. For the past ten years, Patsy Garlan has been an invaluable help as we have approached that point in each experiment where the results of our work are conveyed to others. She has always had a keen understanding of our research, has worked to make our writing lucid, and has critically examined our ideas and searched for ambiguities and contradictions. We are also grateful to our friends, colleagues, and employees who have been enthusiastic about our studies of the face and our attempts to teach others what we have learned. Randall Harrison, John Bear, Allen Dittmann, and Stuart Miller gave many helpful suggestions about how to present this material in an understandable way. Harriett Lukes not only typed the manuscript, but was an enthusiastic first reader. Nina Honbo helped invaluably in keeping us organized and encouraging the completion of the materials. We cannot thank by name the many people who have worked on the research reported in this book; we are grateful for their fine work and for their extra efforts which gave us the time to write this book. Our special thanks go to our friends, students, and colleagues who let us show their faces in this book. We hope that you, our readers, get to know them well. introduction What This Book Is About This book is about faces and feelings-your own and those of the people around you. The first focus is on what the feelings look like, in other people's faces and in your own. Photographs show the facial blueprints of the major emotions-how surprise, fear, anger, disgust, sadness, and happiness are registered by changes in the forehead, eyebrows, eyelids, cheeks, nose, lips, and chin. Common confusions that plague the recognition of expressions of emotions are clarified by pictures highlighting the differences between surprise and fear, anger and disgust, sadness and fear. The subtleties of facial expressions of emotion are revealed in pictures that show the family of expressions for each feeling. Surprise, for example, is an emotion with a big family. There is not one surprise facial expression, but many-questioning surprise, dumbfounded surprise, dazed surprise, slight, moderate, and extreme surprise. The complexities of facial expressions are shown in photographs of how different emotions can blend into a single facial expression to show sad-angry expressions, angry-afraid expressions, surprise-fearful expressions, and so forth. You can use this information about the blueprints of facial expression to better understand the feelings of others, even when they are trying not to reveal their feelings (Chapter 11 on "Facial Deceit"). Or you can use the knowledge of the blueprints of facial expression to learn about your own face, to become more aware of what your face is telling you about how you feel and what your face is telling others. Chapter 12, "Checking Your Own Facial Expressions," describes how you can determine whether your facial expressions are characterized by a particular style. For example, are you a facial withholder (never showing anything on your face), an unwitting ex- 2 introduction pressor (not knowing you are showing a feeling when you do so), or a substitute expressor (thinking you are showing an angry face when, in fact, you look sad)? The blueprints of facial expression, whether used to understand others or yourself, is then the first focus of this book. The second focus is the feelings themselves. Although everyone uses the terms anger, fear, sadness, etc., few people really understand fully their own experience of these emotions. For example, what is it really like to be afraid? What does it feel like in your body? What situations make you afraid? Can you always anticipate when you will be afraid? Can you be both afraid and angry at the same time? When you are afraid, do you get aggressive, withdrawn, or thoughtful? Do you laugh fear off, or do you break out in hives? Do you ever enjoy being afraid -watching a horror movie, for example? Do other people react the same way you do when they are afraid? Does the same thing happen to their breathing? Do the same situations that make you afraid make others afraid? Or do you sometimes think or say, "I can't understand why he was afraid; that wouldn't bother me" or "I can't understand why she wasn't afraid; I was terrified. " Most people would have a difficult time answering such questions, at least about some of the emotions. This may happen when you are, to quote the clichb, out of touch with your feelings. This can also happen when you are in touch with your feelings but don't realize the extent to which your experience of an emotion is unique. There is usually one emotion, and maybe more, that you do not share openly, describing the feeling to others. It might be an emotion you in some way dread experiencing, or can't control, or habitually never think about. Or it may be an emotion you experience keenly but completely privately. You may discover, for example, after extended contact with an intimate, that what distresses you does not distress him. After the courtship is over, a marriage may founder when the mates discover they not only feel and express anger differently but that the one who explodes can't tolerate the one who holds it in, or the one with the long fuse can't accept the one with the short fuse. The second focus of the book is upon just these issues, describing in detail the experience of each of the emotions. It provides as much as we know about how these emotions can be experienced in daily life. You can use this information to learn about your own emotional experiences, the things you share with others and the way in which you differ. You may find out about experiences you are missing. And you may discover the basis for some of your feelings which you have never fully understood. You can use this information also to understand other people's feelings, to have a basis other than your own emotional experience for comprehending what an emotion may be like for another human being. Who This Book Is For this book is for psychotherapists, ministers, physicians and nurses, trial lawyers, personnel managers, salesmen, teachers, actors. ... The psychotherapist must know how people experience emotions. He must be alert to what the face may tell him about his patient's feelings. He can't rely solely upon the patient's words, for sometimes the patient can't describe his feelings. The patient's face may show the emotion being experienced even when it is too upsetting to put into words, when he doesn't have the words to describe his feelings, or when he doesn't know how he feels. The minister, or indeed anyone who counsels people, has the same needs as the psychotherapist. The physician and the nurse also need to understand emotions and facial expressions. People have an emotional reaction to their illness or the threat of illness which may be crucial in its outcome. The physician and the nurse must understand the different ways people experience fear, for this is a common emotional reaction to the possibility of illness and treatment, and may heighten pain, prevent early detection of illness, interfere with treatment plans, etc. And not everyone who is afraid of cancer, or surgery, will experience that fear in the same way. Recognizing sadness, which is often a reaction to loss, and helping a patient to deal with it can be an important factor in the patient's recovery when there is lingering or permanent disability. If many of the theories about psychosomatic disorder are correct, then the experience of anger also should be understood. Patients with a physical illness are often reluctant or embarrassed to mention their feelings about being sick, their fear or sadness, their feelings of self-disgust, and so forth. Physicians and nurses must, therefore, learn to interpret accurately facial expressions and facial signs that emotions are being controlled. The trial lawyer often can't trust the words of a witness or client. He needs another source, such as the face, to tell him how the person really feels. Trial lawyers emphasize the importance of reading emotional reactions in picking jurors and in assessing how a jury, once selected, reacts to different lines of argument. The personnel manager and the salesman may be in the same spot as the trial lawyer. In conducting an employment interview, the personnel manager wants to recognize signs that the applicant is controlling his feelings. For example, is his apparent confidence genuine, or is it covering nervousness about his capability. h he really as interested in this job as he says he is. The face can allow him to check on how the applicant really feels about the job. The salesman knows that the feelings which motivate a decision to buy may never be put into words, or the words may not be trustworthy. The teacher needs to know whether or not the students understand what 4 introduction he is saying. Interest, concentration, and perplexity are shown on the face The actor must understand the complexity of emotional experience in attempting to convey an emotion in performance. The discussion of deception may help him prevent his personal feelings from leaking through in his performance. When the actor is feeling his way into the emotional experience of a character, he needs to be sure that his expression of that emotion is commonly understood. And he should find the facial blueprints useful in understanding and perfecting his own ability to show emotions. All these professionals-psychotherapists, physicians, nurses, trial lawyers, personnel managers, salesmen, teachers-need also to become aware, like the actor, of the impressions they give in their own facial expressions to their various "audiences ." this book is also for job applicants, loan seekers, customers, voters, jurymen. ... The job applicant and the loan seeker need to know what impression they give by their own facial expressions to the personnel manager or bank officer, for certainly they are under scrutiny. They may want also to watch the face of the personnel manager or bank officer to gauge his reaction to them. Customers may be concerned about the honesty of the salesman-has that car really been driven by just one little old lady? The voter is often concerned with the man as well as his politics-how trustworthy he is in what he says, whether or not his campaign promises can be believed. In these days of television politics, this becomes even more of an issue. The juryman can't assume that the witness or defendant is telling the truth, or knows the truth. Understanding facial expressions may help him distinguish what the person testifying actually feels from what he wants people to think he feels. The juryman must also understand the experience of emotions if he is to comprehend fully the motivations behind certain criminal acts. Whether or not there were mitigating circumstances may depend upon the emotional state of the defendant. The reliability of a witness's account may depend upon understanding his emotional experience at the time of the crime as well as at the time he is giving his account. this book is for friends, spouses, parents, lovers, relatives. ... The information in this book is relevant as well to any relationships that are not mechanical, perfunctory, and businesslike. Everyone has relationships in which there is little or no emotional investment. Feelings are not shared; neither person makes an attempt to know the other's feelings, and introduction 5 to do so would be an affront. Everyone also has relationships in which the intimate sharing of feelings is the main core. It is no accident that in intimacy faces move closer together. Intimates also look more into each others' faces. People keep in sight pictures of the faces of those with whom they feel intimate. Though a telephone call is better than a letter, if you expect an important emotional experience or want to describe one that has just occurred (wedding, divorce, death, job promotion, etc.), you want to see the look on the other person's face and you want your face to be seen. Wanting to share feelings does not necessarily make this an easy thing to do. Intimates may find it hard to understand or accept the differences in how they experience an emotion. Intimacy may not survive such differences precisely because they are not understood. "You can't be angry about that; I don't believe it!" "If you were afraid, why didn't you tell me?" It is very hard to understand that people you care about, people you love, don't experience feelings the same way you do. Facial expressions that show feelings may be misinterpreted or missed entirely. If you don't fully understand the different ways a feeling may be experienced, how another's way may differ from your way of experiencing that feeling-if you don't know the various ways the face may show that same feeling-the chances for misunderstanding, for seeming inconsideration, multiply. This book is no panacea for the problems of intimacy, for not all those problems are due to misunderstanding, and misunderstandings can't be resolved by reading a book. But the descriptions of the varieties of emotional experience and the blueprints of facial expressions should help. this book is for you alone Understanding emotional experience applies not just to your relationships with others but also to your relationship with yourself. It can help you understand the most private, personal, unique part of your self. This is a part of your self which has enormous power over your life. Your work, your life, and even your death can be determined by your feelings. Sexual needs may not be satisfied, hunger not met, work not completed because of feelings that interfere. Feelings can motivate the taking of your own life or the life of another. Struggles of an extraordinary nature may be endured, awesome feats accomplished because of feelings. Yet we know less about our feelings than we do about our teeth, our car, or our neighbor's escapades. This is not a self-help book, but it may lead you to a better understanding of emotions and of your own emotional life. Although learning the facial blueprints will chiefly improve your ability to spot emotion in others, it can also teach you to be keenly aware of what your own facial muscles are telling you-about you. 6 introduction What You Can Get From This Book You can read this book, or you can study it. It depends upon your goal. Do you want simply to increase your knowledge of emotion? Or do you also want to master a skill? Both are possible, but the skills take longer to acquire than the knowledge. You can read this book in a matter of hours, concentrating on the words and looking at the pictures. You will learn a good deal about emotional experiences, your own and other people's-information that should be helpful in all the ways described above. But this approach won't teach you how to spot emotion in people's faces much better than you are already able to do. You won't be any better in spotting slight signs of emotion, blends of emotions, signs of emotional control, the leakage of emotions, and so forth. To be able to do these things requires the extra investment of learning the facial blueprints so well that you can use the information without thinking about it. It must become a skill. You may feel you don't need much improvement in interpreting faces, or you may need it for certain emotions but not for others, or you may find you are pretty far off in telling how people feel from their faces. In the sections on facial appearance in Chapters 4, 5, 6, 7, 8, and 9 are practice instructions. Each of those chapters also tells you how to make certain faces, constructing changes in expression so you can better learn how the face works. When you can accurately recognize the faces shown in Chapters 4 through 9, Chapter 10 shows you how to use a new set of faces to practice further and improve your skills. How This Book Relates to Popular Books about Body Movement and Nonverbal Communication There is little overlap in information between this book and the popular books about body movement or nonverbal communication which have appeared in the last few years. Those books said almost nothing about facial expression, chiefly because the information about the face has not been available before. Our two previous books on facial expression (Emotion in the Hrrman Face and Darwin and Facial Expression) are intended for the researcher who is seeking information on how to study facial expression, and for the scholar or student who wants to know what research has been done on facial expression. They address such questions as whether facial expressions of emotion are universal, and whether people can accurately tell an emotion from the face. But they don't show what universal expressions of emotion look like or show the reader how to make accurate judgments of emo- introduction 7 tion. This is our first attempt to provide this information, and to our knowledge it is the first attempt anyone has made to do this. This book differs from the popular nonverbal-communication books, too, in that it is about the face, and therefore about emotions, while those books are primarily about the body. Our studies of the body, published in professional journals, have explored the differences in what the face and the body tell us. Emotions are shown primarily in the face, not in the body. The body instead shows how people are coping with emotion. There is no specific body movement pattern that always signals anger or fear, but there are facial patterns specific to each emotion. If someone is angry, his body may show how he is coping with the anger. He may be tense and constrained (tight muscular tension in arms and legs, stiff posture). He may be withdrawing (a particular retreated position). There may be verbal attack (certain types of hand movements with his words), or the likelihood of physical attack (posture, orientation, hand movements). However, any of these movements can occur just as well when a person is afraid as when he is angry. Body movement reveals, of course, not just how someone copes with emotion, but also what a person's attitudes, interpersonal orientations, etc., might be. Some of this has been explained in the popular "body" books, some has regrettably been explained badly or erroneously, and much has just not been discussed. In a few years we plan to publish another book showing how body movement, facial expression, voice tone, and words fit together. They are all important in understanding people. But the face is the key for understanding people's emotional expression, and it is sufficiently important, complicated, and subtle to require a book to itself. Why There Is Need for a Book about Facial Expression Although everyone agrees that understanding emotions is crucial to personal well-being, to intimate relationships, and to success in many professions, no one teaches you how to do this unless you get into serious trouble. And then, the various techniques of psychotherapy usually focus only on those few emotions which their theories consider. Though there is strong evidence now that the face is the primary signal system for showing the emotions, no one taught you how to read those signals. And there is every reason to believe that you were not born with the knowledge. You have to pick it up. Much of what you know about emotions and facial expressions was shaped by your parents and other members of your family. The faces of your 8 introduction parents, of your siblings, and of other people who took care of you were the first you saw. Your family members may have been very expressive or very guarded in their facial expression. They may have shown you the full repertoire of emotional expressions, or only a few. Perhaps you never saw an angry face, or a fearful face. People in your family may have shown emotion in their faces as most other people do, or one of them might have developed a strange or peculiar way of looking disgusted or afraid, and that may still influence your recognition of that emotion. As a child, you may have been specifically instructed not to look at the facial expressions of others, or at least certain facial expressions. For example, some children are told never to look at someone who is crying. As an adult, you may be more sensitive to some emotions than to others. What you learned about reading emotions in your own family might have great applicability to understanding others or relatively little, at least for some emotions. Through watching television, or movies, or a close friend, you may have improved upon and added to your knowledge of facial expressions. Although almost everyone correctly reads some facial expressions, few people realize when they make mistakes or why they make them. The rules for translating a particular set of facial wrinkles into the judgment that a person is angry, afraid, etc. would be very hard for most people to describe. When you follow these rules you do so automatically, on the basis of habits established so long ago that usually you don't know how they operate, or even when they operate. In this sense, understanding facial expressions of emotion is like driving a car. You don't think about what you are doing when you do it. Unlike driving a car, with facial expression there never was an earlier period in which you were specifically taught the skills. There is no manual in which you can check how to correct mistakes. There are no equivalents to the traffic cop telling you when you missed or misinterpreted a signal. Often you do not know that a facial expression was the basis for your hunch or intuition about someone. You may only sense something about a person without being able to trace the source of your impression. And if you don't know what led to your judgment you can't correct yourself if it turns out to have been wrong. Sometimes you are puzzled by someone's facial expression; you can't figure out what he meant. Or you can figure out what he meant by the look on his face, but you can't decide whether or not to trust it. It is hard to check impressions with others, because there just isn't much of a vocabulary for describing the face itself. There are a lot of words for the messages you get from the face (afraid, terrified, horrified, apprehensive, worried, to mention a few of those related to fear), but few to describe the source of those messages. We do have the terms smile, grin, frown, squint, but there are relatively few such words that identify particular facial introduction 9 configurations, distinctive wrinkle patterns, or temporary shapes of the facial features. Without terms to refer to the face, we are handicapped in comparing or correcting our interpretations of facial expression. Comments such as the following, though accurate, are maddeningly cumbersome. "I know why you thought he was afraid. It was because the inner corners of his eyebrows were pulled together and raised. But you failed to see the omega wrinkle in his forehead. If you had noted that, or the fact that the outer eyebrow corners were down, not up, you would have known he was sad." At best, it is not easy to describe facial expression. Pictures are needed, because it is a visual phenomenon. In this book we are able to bring you hundreds of carefully selected photographs which show you how the face registers emotion. We think such a book is needed to accomplish the following: -Bring attention to what you may already be doing without knowing it -Show what you may be missing entirely -Correct what you may be misinterpreting --Show the subtleties (the families of facial expressions) and the complexities (the blends of two emotions in one facial expression) -Alert you to signs of facial control and teach you how to discover when a facial qualifier is used, or when an expression is modulated or falsified -Provide techniques for learning whether you show emotion in your own face in an unusual fashion The next chapter explains a number of the potential problems encountered in understanding facial expression, suggesting why mistakes are made and how to avoid them. You may have been brought up not to look at faces, or at certain faces. You may be distracted by other competing bids for your attention, the words someone speaks, the sound of his voice, the general appearance and movements of his body. You may not recognize the differences between controlled and uncontrolled expressions. You may not know what to look for in the face, or where to look, to tell if someone is angry, or afraid. You may not know exactly what an emotion is, how one emotion differs from another, or how an emotion differs from a mood, attitude, or character trait. why mistakes are made in understanding facial expressions of emotion The Face a s a Multisignal, Multimessage System The face provides more than one kind of signal to convey more than one kind of message. In trying to follow the emotion messages, you may look at the wrong signal. Or perhaps you don't clearly differentiate the emotion messages from the other messages conveyed by the face. A very familiar example of a multisignal, multimessage system is the system of road signs. It employs three types of signals: shape (triangular, square, circular, rectangular, octagonal), color (red, yellow, blue, green), and inscription (words, drawings, numbers). Road signs utilize these three types of signals to transmit three types of messages: regulations (stop, no U-turn, yield, etc.), warnings (school crossing, two-way traffic, etc.), and information (service area, bike route, camping, etc.). With road signs, as with facial expressions, you must focus on a particular type of signal if you wish to learn a particular type of message. If you want to know if you are approaching a rest area or a place to camp, you can search for the blue or green signs, because it is these colors that give this type of information (yellow signs give warnings, red signs give regulations). The parallel in reading faces is that if you want to know what emotion someone is feeling, you must watch the temporary changes in the face, because it is these rapid facial signals which give information about emotions. (If you were trying to determine someone's age, you would pay attention to more enduring aspects of the face, such as muscle tone or permanent wrinkles. ) The face provides three types of signals: static (such as skin color), slow (such as permanent wrinkles), and rapid (such as raising the eyebrows). The static signals include many more or less permanent aspects of why mistakes are made in understanding facial expressions of emotion 11 the face-skin pigmentation, the shape of the face, bone structure, cartilage, fatty deposits, and the size, shape, and location of the facial features (brows, eyes, nose, mouth). The s b w signals include changes in the facial appearance which occur gradually with time. In addition to the development of permanent wrinkles, changes in muscle tone, skin texture, and even skin coloration occur with age, primarily in the later years of adulthood. The rapid signals are produced by the movements of the facial muscles, resulting in temporary changes in facial appearance, shifts in the location and shape of the facial features, and temporary wrinkles. These changes flash on the face for a matter of seconds or fractions of a second. All three types of facial signals can be modified or disguised by personal choice, although it is hardest to modify the static and slow signals. Hair styles are used almost universally to modify these signals. Bangs, for example, can change the apparent size of the forehead, or conceal permanently etched wrinkles. Cosmetics, such facial ornaments as sunglasses and, at the most extreme, plastic surgery can modify the static and slow facial signals so that they broadcast a different set of messages. The rapid facial signals can be modified or disguised by inhibiting the muscle movements that produce them or by masking one expression with another, or by hiding the face with beards or sunglasses. Thus, one can be misled, purposefully or accidentally, by rapid, slow, or static signals. The face is not just a multisignal system (rapid, slow, static) but also a multimessage system. The face broadcasts messages about emotion, mood, attitudes, character, intelligence, attractiveness, age, sex, race, and probably other matters as well. This book is focused primarily on one type of message and one type of signal---emotion messages transmitted by the rapid signals. When we speak of emotions, we are referring to transitory feelings, such as fear, anger, surprise, etc. When these feelings occur, the facial muscles contract and there are visible changes in the appearance of the face. Wrinkles appear and disappear, the location and/or shape of the eyebrows, eyes, eyelids, nostrils, lips, cheeks, and chin temporarily change. Research has shown that accurate judgments of emotion can be made from the rapid facial signals, and has recently uncovered the particular facial signals-the blueprintswhich distinguish each of the primary emotions. Photographs were specially made for this book to reveal and contrast the facial blueprints that distinguish each of the primary emotions and the blends of these emotions. It is important to note that the emotion messages are not transmitted by either the slow or the static facial signals; however, these may affect the implications of an emotion message. If a person has a thin or fat face, a wrinkled or smooth face, a thin- or thick-lipped face, an old or young face, a male or female face, a Black, Oriental, or Caucasian face, that does not tell you whether the person is happy or angry or sad. But it may affect your 12 why mistakes are made in understanding facial expressions of emotion impression. For example, if the rapid facial signals tell you that a person is angry, your impression of why the person is angry and what he is likely to do while angry may depend in part upon the information you glean from the slow and static facial signals about the person's age, sex, race, personality, temperament, and character. These further interpretations are not discussed in this book because too little is yet known about them, and because there is a prior question with which we are primarily concerned here. Before considering the matter of what you may expect of different types of people once you do know how they feel, there is the question which is the focus of this book: How do you tell from the face how someone feels, and how can you tell if his facial expression is genuine or phony? This book also provides some information about the mood messages. Moods are closely related to emotions, and some are shown in the rapid facial signals. Moods differ from emotions in that the feelings involved last over a longer period. For example, a feeling of anger lasting for just a few minutes, or even for an hour, is called an emotion. But if the person remains angry all day, or becomes angry a dozen times during that day, or is angry for days, then it is a mood. "Irritable" is a word used to describe someone in such a mood, although one could just as well say that the person is in an angry mood. It is possible, although not likely, that a person in such a mood will show a complete anger facial expression, with the signals of anger registered over the entire face, throughout the period that the mood endures. More often just a trace of that angry facial expression may be held over a considerable period-a tightened jaw or tensed lower eyelid, or lip pressed against lip, or brows drawn down and together. Another way a mood may be registered in facial expression is by the frequency with which the total facial expression flashes on and off within some time period. It may be obvious that a person is in an irritable mood because he became angry so often this afternoon. Other moods that are shown in rapid facial expressions are depression (where the face shows evidence of sadness, fear, or a blend of both), anxiety (where the face shows evidence of fear), and euphoria (where the face shows evidence of happiness and excitement). The rapid facial signals also send emblematic messages. In our research we utilize the term emblem to describe signals the meaning of which is very specific, the nonverbal equivalent of a common word or phrase. The eyewink signal for the message of agreement ("right on," "sure") or flirtation ("will you?") is an example. Facial emblems are like hand movements for waving "hello" or "goodbye," and like head nods for "yes" and "no." The movement is always specific and easy to distinguish from other movements. The meaning is understood by everyone in a culture or subculture. We will not discuss the full array of facial emblems, but only those which are closely related, either in movement or message, to the emotional expressions. Rais- why mistakes are made in understanding facial expressions of emotion 13 ing the brows and holding them while keeping the rest of the face blank is an example of such an emblem. That brow movement is part of the rapid facial signal for surprise, but when it is not joined by a movement in the eyelids and lower face as well, it signals questioning. There are still other emblems that might be called emotion emblems, because the message they convey is about an emotion. These emotion emblems look like a facial expression of emotion, but they are different enough for the person seeing them to know that the person making them doesn't feel that way at the moment; he is just mentioning the emotion. For example, one of the emotion emblems for disgust is to wrinkle the nose, which is part of the disgust facial expression. When it is used as an emotion emblem it occurs alone, with little raising of the upper lip, and it flashes on and off the face quickly, and it is therefore not confused with an actual disgust facial expression. The message is "disgust (I don't feel that way now)." The rapid facial signals are used, then, to convey emotion messages and emblematic messages. They are also used as conversational punctuators. Everyone knows people who use their hands to accent or italicize a word or phrase as they speak. People can do the same thing with the rapid facial signals, punctuating what is being said in words with facial accents, commas and periods. The facial punctuators will be discussed and shown in later chapters. The face sends many rapid signals in addition to the ones that register emotion or are used in emotion-related emblems or as punctuators. Facial grimacing, contorting, and pantomiming; movements required by speech; such facial acts as lip-biting, lip-wiping, etc., will not be considered, because they are not related to or often confused with the facial expressions of emotion. Similarly, the face sends many messages in addition to those about emotions and moods. People believe they can read attitudes, personality, moral character, and intelligence from the face. We will not discuss these, because it is not known whether people even agree about such judgments, or, if so, whether their judgments are correct. Further, it is not known whether any such judgments are based on the rapid, the slow, or the static facial signals. People probably do agree and are correct, at least some of the time, in judging sex, age, and race from the face. Even with these more obvious messages, not much is known about the precise facial blueprints. Though we assume that they are transmitted primarily by the slow and static facial signals, we do not know precisely what signals transmit these types of information. For example, in reading whether someone is male or female, is this apparent in the shape of the upper or lower lip, or in the size of the lips, or the relative size of the two lips, or the pigmentation of the lips? Or is it not the lips at all, but the amount of hair in the eyebrows, or their shape; or is it the shape of the chin; etc.? 14 why mistakes are made in understanding facial expressions of emotion In regard to facial expressions of emotion, however, we know a good deal. We know that people can make correct judgments. We know the specific signals in each part of the face that convey the messages of fear, surprise, sadness, happiness, anger, disgust, and combinations thereof. If your understanding of facial expressions is to be improved so that you can interpret those shown by others and more adequately sense your own, then your attention must be focused upon the rapid facial signals and their distinctive messages. It is futile to look to the slow and static facial signals for information about emotion. You will need to learn the subtle and obvious differences and similarities among the emotions in their appearance on the face and in how they are experienced. And you will need to learn to distinguish the facial expressions of emotion from the facial emotion emblems, other facial emblems, and facial punctuators. Not Watching Faces Problems in understanding facial expressions arise because most of the time people do not watch each other's faces. Because most facial expressions of emotion are brief, you may often miss an important message. Some facial expressions are extremely rapid, lasting only a fraction of a second. We call these micro-expressions. Most people fail to see them or fail to recognize their importance. Later (Chapter 11) we will explain how these very quick micro-expressions can reveal emotions the person is attempting to conceal. Even the more usual macro-expressions frequently last only a few seconds. It is rare for a facial expression of emotion to last as long as five or ten seconds. If it does, the feeling must be intense, so intense that the feeling is likely to be simultaneously shown in the voice through a cry, laugh, roar, or in words. Even if you were not looking at a person's face, you would be likely not to miss these intense emotions, because you would hear them. More often, however, the very long facial expressions are not genuine expressions of emotion, but mock expressions, in which the person is playing at showing an emotion in an exaggerated fashion. It is obvious when you see it that the person is playing. Sometimes he may not be playing, but using the mock expression as a way to show the emotion without taking responsibility for doing so. For example, suppose you have agreed to join someone in an escapade that now, as you approach it, looks more risky than you had anticipated. You might show a mock fear expression. This would let you show how you feel, get the message across to your companion, so that he too could express hesitation; but if he is unsympathetic, he can't ridicule you because you were showing only mock fear. It is easy to understand why the micro-expressions might be missed, why mistakes are made in understanding facial expressions of emotion 15 because they are so quick. Later we will give some exercises for recognizing them. But even the longer macro-expressions, which last two or three seconds, are often missed, because the face is often not watched. In a sense this is paradoxical, because the face is very commanding of attention. As the site for the major sensory inputs-sight, hearing, smell, and taste-and for the major communicative output-speech-it has great importance in social life. Moreover, most people identify their very selves, in part, with their facial appearance. Industries exist around enhancing the appearance of the face. A person is most easily identified by his face, more so than by his body. His privacy is exposed much more if a picture taken of his face at some embarrassing moment appears in a newspaper, than if a similar picture of his body were to be published. And there is curiosity about other people's faces. When told about a person you haven't met, you will want to see a picture of his face; you don't usually think about wanting to see a picture that would clearly show his body. When you see a person's full face and body, you tend to look more at the face than at the body, unless perhaps there is some sexual interest. When you converse, you seldom look continuously at the other person's face. You probably spend more time looking away from the person you are talking with than you spend looking at him. Think about how the chairs are arranged, or how you rearrange them, when you are about to sit down and have a conversation. Usually you don't sit facing each other, head-on, but adjacent to or at angles to each other. When your head is aligned with your body, you and the person you are talking with are each looking off into space, which is what you probably do most of the time. In order to see the face of the other person you must turn your head, which you will do occasionally. Research is presently being carried on to explore just when two people conversing do look at each other. It seems that when you speak you look to see if the listener agrees, or is amused, angry, interested, bored, etc. You look to determine if the other person needs a chance to get a word in. You look if the other person has not made any audible listener responses for a while ("mm-hmm, '6 yes," "good," "uh-huh," 'is that so," etc.) to check him out. You look to give the other person the chance to take the floor. If you don't intend to give up the floor, you have to be careful not to look at the other person when you are pausing. When you are the listener, you may look at the speaker when he stresses a word, or at the end of his phrase, if his inflection rises and he seems to be inviting a verbal or nonverbal response from you. There are some situations in which at least one person looks at the other person's face much more of the time. If you are a member of an audience, you feel no inhibition about gazing continuously at the performer's face. He 99 16 why mistakes are made in understanding facial expressions of emotion has given you permission by virtue of his and your role. If you are an interrogator, you may continuously look at the person you are questioning; that's your job. Probably the same is true in any formal conversation in which one person has some clearly defined and acknowledged authority over another. Continuous face-watching occurs also in just the opposite type of situation. Everyone can recognize lovers, at least those who are still courting, by how much of the time they unabashedly gaze into each other's faces. Why is it that most of the time you don't look at the face of the person you are conversing with? Why may you actually look away at just the moment you sense he may show an emotional facial expression? Part of the answer is politeness-being brought up not to stare (at least in the U.S. and some other cultures). You don't want to be rude and intrusive, taking information that it is not clear the other person is giving to you. And you don't want to embarrass the person you are conversing with, or yourself. If he wants you to know how he feels, then he can say it in words; otherwise you would be overstepping the bounds of convention. Watching someone's face is intimate. You take such a liberty only if the other person gives it to you by being a public performer, or if your social role bestows it upon you (as if you avowedly seek to interrogator, employer, juryman, parent, etc.)-or share intimacy, looking and inviting the look of the other person. There is another side to it, however. It is not just being polite and not wanting to embarrass or be embarrassed. Often not looking at the other person's face is motivated by not wanting to be burdened with that knowledge or obligated to do something about how the person feels. By not looking at his face you don't know, or can pretend not to know. Unless he says it in words, you are not socially obligated to care for or respond to his feelings. If he is showing annoyance or anger on his face and you see it, and he knows you see it, then you may have to find out whether you are the cause, or if not, why he is angry. If he is showing sadness, you may have to comfort or aid him; the same is true if he is showing fear. In many social interactions the last thing either person wants is to acknowledge or take any responsibility for having to deal with the feelings of the other person. In addition to these fairly common reasons for not looking at faces, some people may have had experiences in their childhood which trained them not to look at certain people's facial expressions, or at certain expressions, no matter who shows them. A child may learn that it is dangerous to look at Daddy's face when he is angry, or dangerous to look at anyone's angry face. Such learning may occur so early that the grown person avoids seeing certain emotions, or certain types of people showing certain emotions, without knowing he is engaged in such avoidance. If your understanding of facial expressions is to improve, then you must reexamine whether or when you want to know how the other person feels. You may have to counteract habits you have learned and followed without why mistakes are made in understanding facial expressions of emotion 77 knowing it, which lead you to ignore and miss many facial expressions of emotion. The Communication Barrage The face is but one source of information available during a conversation, and the problem is that messages may be missed because of the distraction of the other, more commanding sources bidding for attention. In most conversations it is sight and sound, what you see and hear, that you rely upon. In most conversations among people from Western industrialized settings, there is not much tactile stimulation, and precautions are taken to camouflage odors. When listening, you gather information from at least three sources in the auditory channel: the actual words used; the sound of the voice; and such things as how rapidly the words are spoken, how many pauses there are, how much the speech is disrupted by words like "aah" or "ummh." When looking, you gather information from at least four sources in the visual channel: the face; the tilts of the head; the total body posture; and the skeletal muscle movements of the arms, hands, legs, and feet. Every one of these sources in both the auditory and visual channel can tell you something about emotion. The auditory and visual channels offer different advantages and disadvantages in communication to the sender and receiver of messages. The auditory channel can be completely turned off by the sender; he can simply shut up and thereby cease providing any information to the other person he was conversing with. This is an advantage for the sender, a disadvantage to the receiver. The visual channel is in some sense always turned on. The sender must maintain some body posture, some hand position, some set to the face; thus, some impression is always given to the other person. If the sender can assume that the other person is interested in the conversation, the use of the auditory channel provides greater likelihood that a message sent will get through and be received. The listener does not need to point his ears at the sender to receive a message; the listener's auditory channel is open. Not so with the visual channel. The other person might be interested, but fail to glance over at the moment a message is sent. The sender may take advantage of this to "slot" a message in the visual channel to comment to an onlooker without the knowledge of the person with whom he is conversing. For example, one person may signal visually to another person to rescue him from a boring conversation, without the boring conversant noticing the call for help. On the other hand, if the sender does want his message to get through to the other person with whom he is conversing, then he must wait until the other person is visually fixed on him before sending a message in the visual channel. Thus, when a sender and receiver share 18 why mistakes are made in understanding facial expressions of emotion an interest in communicating, the auditory channel has the advantage of more certain receipt of messages sent. On the other hand, the receiver can more easily pretend auditory than visual interest. Listening can be simulated, with occasional nods and "umhmms" thrown in, when in fact the receiver is listening to another conversation or to his own thoughts. There are no tell-tale signs of such auditory disloyalty which parallel the out-of-focus look to the gaze, if the viewer pretends to be looking at the sender but is in fact looking at someone behind or to the side of him; nothing to equal the tracking movements which show that the viewer's eyes are following someone else's activity. Incidentally, because it is more obvious who is looking than who is listening, both sender and receiver have more protection from eavesdropping by unwanted lookers than by unwanted listeners. Although there is some repetition or overlap in the type of information transmitted in the two channels, there is a natural division of labor, so that each channel better conveys certain messages. Words are best for most messages, particularly factual ones. If you are trying to tell someone where the museum is, who played the lead in that movie, whether you are hungry, or how much the meal costs, you use words. You don't resort to the visual channel as the primary transmitter of such messages unless you can't speak or be heard, or unless the message involves showing where something is in space (how you get to the post office from here). The voice tone and the visual channel sources add qualifying messages to what is transmitted in words. Nuances are added, points emphasized, instructions given as to how seriously to take what is said in words, etc. The visual channel also can be used for factual messages, by Indian sign language or the language of the deaf, but the advantage is clearly with words. Words can also be used to describe feelings or explain them, and typically are used in concert with other sources to do so. Here, however, the advantage is with the visual channel, because the rapid facial signals are the primary system for expression of emotions. It is the face you search to know whether someone is angry, disgusted, afraid, sad, etc. Words cannot always describe the feelings people have; often words are not adequate to express what you see in the look on someone's face at an emotional moment. And words are not trusted as much as faces, when it comes to emotions. If someone tells you he is angry and his face shows it as well, nothing is amiss. But if he says he is angry and shows no evidence facially, you are suspicious. If the reverse occurs and he looks angry but doesn't mention anger feelings in his words, you doubt the words but not the anger; you wonder why he doesn't admit how he feels. Emotion messages can also be transmitted by the sound of the voice, body posture, hand/arm movements, and leg/feet movements. But it is not why mistakes are made in understanding facial expressions of emotion 79 certain whether they transmit information about emotion with as much precision as does the face. They may more simply tell you that someone is upset, but not whether the upset feeling is anger, fear, disgust, or sadness. The clarity and precision of these other sources of emotion messages remains to be studied. We do know that the face is a primary, clear, and precise signal system for the expression of the specific emotions. With two channels-the auditory and visual-transmitting information from seven sources, communication is a barrage. The speaker sends out a barrage of signals and may not carefully attend to or employ all of them in an optimal fashion, and the listener is barraged and may not attend to all the signals available. With the visual channel, people focus more on the face than on the other sources; facial expressions are tuned more than body movements, and people look at faces more than body movements. But the auditory channel, in particular the words, usually receives the most attention, both from speaker and listener, because here can be transmitted the richest and most varied information about everything but emotion. You can miss important information about emotions in the face, then, because it is competing with these other information sources, and it may be somewhat handicapped in its bid for attention. Even if you focused as much on facial expressions as on words, the words would receive more attention because they always reach us, while one has to look to see a facial expression. The face is, nevertheless, of more importance than words for transmitting information about emotion. Controlling the Face Facial expression may be controlled or uncontrolled. One. expression may be voluntary, another involuntary; one may be truthful and another false. The problem is to tell which is which. Suppose someone has an emotional experience-something happens, let us say, to make him afraid. If an expression of fear comes over his face, it will have done so automatically. The person does not think about how to move his facial muscles to look afraid; the fear expression is involuntary. But there may be interference, dictated either by longstanding well-ingrained habit or by deliberate, self-conscious choice of the moment. The interference may be minor, only qualifying or modulating the expression, or major, interrupting or totally inhibiting the expression. The facial expression of fear would be qualified if the person added to the fear expression a bit of a smile, showing that he can "grin and bear it." The facial expression would be modulated if the person changed the apparent intensity of the expression, trying to show only slight worry when experiencing strong fear. The facial expres- 20 why mistakes are made in understanding facial expressions of emotion sion of fear could be interrupted, so that only a trace appeared; or the expression might be totally inhibited so that nothing showed on the face, or there was only a stiff appearance. Not only can facial expressions be interfered with, but people can simulate emotion expressions, attempting to create the impression that they feel an emotion when they are not experiencing it at all. A person may show an expression that looks like fear when in fact he feels nothing, or feels sadness or some other emotion. Facial expressions of emotion are controlled for various reasons. There are social conventions about what you can show on your face, cultural display rules that govern how people manage the appearance of their faces in public. For example, in the United States many little boys learn the cultural display rule, "little men do not cry or look afraid." There are also more personal display rules, not learned by most people within a culture, but the product of the idiosyncrasies of a particular family. A child may be taught never to look angrily at his father, or never to show sadness when disappointed, or whatever. These display rules, whether cultural ones shared by most people or personal, individual ones, are usually so well-learned, and learned so early, that the control of the facial expression they dictate is done automatically without thinking or awareness. Facial expressions are also controlled because of vocational need or practice. Some jobs seem to require or select people who are expert facial controllers. Anyone who is successful in such a job may need to be able to put on convincing simulations (the actor, or even the salesman). Or the requirement may be never to reveal how you actually feel (the diplomat). People also control their facial expressions of emotion because it is to their advantage at a particular moment. If a pupil cheats on an exam, he may conceal his apprehension when the proctor walks by because he doesn't want to get caught. Some of the confusion about facial expression arises, then, because the face conveys both true and false emotion messages. There are uncontrolled, involuntary, true expressions and also qualified, modulated, or false expressions, with lies of omission through inhibition and lies of commission through simulation. In order to improve your understanding of facial expressions of emotion, you will have to distinguish which is which. The first step is to learn how the actually felt emotions appear on the face (Chapters 4 to 9 ) , for without that knowledge you can't spot the clues to facial control. Later, in Chapter 11 on "Facial Deceit," the display rules and the various management techniques will be further explained. A number of suggestions will be given about how to distinguish felt from modulated or false facial expressions. Some of the clues are in the shape of the facial expression, some in its timing, others in its location in the conversational stream, and still others in how the facial expression relates to everything else the person is doing. research on facial expressions emotion There have been hundreds of experiments on facial expressions of emotion. Elsewhere we have analyzed these studies in detail (see Reference 5 at the end of this chapter). Here we will more briefly describe only those studies which are directly relevant to the information presented in this book. This chapter should help resolve the doubts of the skeptic about the scientific basis for what is said and shown in subsequent chapters. It is also provided for those who are curious about how facial expressions of emotion are studied. Which Emotions Does the Face Show? Does the face tell us only whether someone feels pleasant or unpleasant, or does it provide more precise information, conveying which unpleasant emotion is experienced? If the latter, how many of these specific emotions does the face show-six, eight, twelve, or what number? The typical method used to determine just which emotions can be read from the face has been to show photographs of facial expressions to observers, who are asked to say what emotion they see in each face. The observers may be given a predetermined list of emotion words to choose from, or left to their own resources to reply with whatever emotion word comes to mind. The investigator analyzes the answers of the different observers to determine what emotions they agree about in describing particular faces. He might find, for example, that 80 percent of the observers agree in describing a particular face with the word "afraid." They might not agree about a word to describe some other face; for example, a face called "disinterest" by some observers might be 22 research on facial expressions of emotion labeled with other emotions by other observers. On the basis of such results, the investigator reaches a conclusion about which emotions the face can convey. The six emotions that are the subject of this book-happiness, sadness, surprise, fear, anger, and disgust-were found by every investigator in the last thirty years who sought to determine the vocabulary of emotion terms associated with facial expressions. There are probably other emotions conveyed by the face-shame and excitement, for example; but these have not yet been as firmly established. Because we will be showing not only how these six emotions appear on the face, but also how thirty-three different blends of these six emotions appear, quite a large portion of the emotional repertoire of the face will be represented. Are Judgments of Emotion Accurate? It is not enough to determine what emotions are read from facial expressions. It is also crucial to discover whether the interpretations of the observers are correct or not. When people look at someone's face and think that person is afraid, are they right or wrong? Are facial expressions an accurate reflection of emotional experience? Or are the impressions gained from facial expression merely stereotypes-all agree about them, but they are wrong? To study this question the investigator must find some people whom he knows to be having a particular emotional experience. He must take some photographs, films, or videotapes of these people and then show them to observers. If the observers' judgments of the facial expressions fit with the investigator's knowledge of the emotional experience of the persons being judged, then accuracy is established. Most of the studies of accuracy in judging facial expression failed to provide conclusive evidence one way or another, usually because the investigator's knowledge of the emotional experience of the people being judged was faulty. In our analysis of the experiments conducted over the last fifty years, we did find consistent and conclusive evidence that accurate judgments of facial expression can be made. Some of these studies were conducted in our own laboratory. In one experiment, photographs were taken of psychiatric patients when they were admitted to a mental hospital, and again when they were less upset and ready for discharge. Untrained observers were shown these photographs and asked whether each facial expression was shown at time of admission or at time of discharge. The judgments were accurate. These same photographs were shown to another group of observers who were not told they were seeing photographs of psychiatric patients, but instead were asked to judge whether the emotion shown was pleasant or unpleasant. Again accuracy was proven, because the facial expressions shown research on facial expressions of emotion 23 at admission were judged as more unpleasant than those shown at discharge from the hospital. In another study, other observers were asked to judge how pleasant or unpleasant the facial expressions were, but the faces shown to them were of psychiatric trainees undergoing a stress interview. Without knowing which was which, the observers judged the facial expressions during stress as more unpleasant than the facial expressions drawn from a nonstressf ul part of the interview. In still another experiment, observers were shown two films of college students, one taken when they had been watching a very unpleasant film of surgery, and one when they had been watching a pleasant travelogue film. The observers accurately judged which film the college students were watching from their facial expressions. All these studies were concerned with spontaneous facial expressions which naturally occur when a person does not deliberately try to show an emotion in his face. But what of those situations in which a person deliberately tries to show an emotion, to look happy or angry, etc.? Many studies have indicated that observers can accurately judge which emotion is intended when a person deliberately tries to convey an emotion through facial expression. Are There Universal Facial Expressions of Emotion? Are facial expressions of emotion the same for people everywhere, no matter what their background? When someone is angry, will we see the same expression on his face regardless of his race, culture, or language? Or are facial expressions a language, the meaning of which we must learn anew for each culture, just as we need to learn the verbal language? A little more than one hundred years ago, Charles Darwin (see Reference 1 at the end of this chapter) wrote that facial expressions of emotion are universal, not learned differently in each culture; that they are biologically determined, the product of man's evolution. Since Darwin's time many writers have emphatically disagreed. Just recently, however, scientific investigations have conclusively settled this question, showing that the facial appearance of at least some emotions, those covered in this book, is indeed universal, although there are cultural differences in when these expressions are shown. Research conducted in our laboratory played a central role in settling the dispute over whether facial expressions are universal or specific to each culture. In one experiment, stress-inducing films were shown to college students in the United States and to college students in Japan. Part of the time, each person watched the film alone and part of the time the person watched while talking about the experience with a research assistant from the person's own culture. Measurements of the actual facial muscle movements, captured 24 research on facial expressions of emotion on videotapes, showed that when they were alone, the Japanese and Americans had virtually identical facial expressions (see Figure 1). When in the presence of another person, however, where cultural rules about the management of facial appearance (display rules) would be applied, there was little correspondence between Japanese and American facial expressions. The Japanese masked their facial expressions of unpleasant feelings more than did the Americans. This study was particularly important in demonstrating what about facial expression is universal and what differs for each culture. The universal feature is the distinctive appearance of the face for each of the primary emotions. But people in various cultures differ in what they have been taught about managing or controlling their facial expressions of emotion. In another experiment we showed photographs of the different emotion expressions to observers in the United States, Japan, Chile, Argentina, and Brazil. The ohservers in these different cultures had to choose one of the six primary emotion words for each photograph they saw. If facial expressions were a language that differs from culture to culture, then a facial expression said to be angry by Americans might be called disgust or fear by people in Brazil, or might not mean anything to them. Just the opposite was found. The same facial expressions were judged as showing the same emotions in all these countries, regardless of language or culture (see Figure 2 ) . Essentially the same experiment was carried out independently at the same Fig. 1 Example of spontaneous facial expression of a Japanese (on the left) and an American (on the right) shown when watching a stress film. Fig. 2 Exomples of the photographs utilized in a study of how emotions are judged across literate cultures Percentage Agreement in How Photogroph W a s Judged Across Cultures - UNITED STATES Fear Disgust Anger BRAZIL CHILE - p~~ ~ ARGENTINA JAPAN 26 research on facial expressions of emotion time by Carroll Izard (Reference 8) with observers in eight different cultures, and the same evidence of universality was found. Although we wished to interpret our findings as evidence that some facial expressions are universal, one loophole remained. All the people studied had some shared visual contact, usually not directly but through the mass media. It was still possible that facial expressions might really differ in all the cultures studied, but the people might have learned, through movies, television, and picture magazines, what each other's facial expressions of emotion looked like. Or facial expressions of emotion might be similar in all the cultures we studied precisely because the people had all learned how to show emotion on their face by watching the same actors in the movies or television and imitating their facial expressions. We had not eliminated the possibility that, among people who did not have the opportunity to view mass-media portrayals of facial expressions of emotion, emotions would be shown by entirely different facial muscular movements. The only way to settle this question was to study visually isolated people who had no contact with the massmedia, and little if any contact with the outside world. We conducted a series of experiments in the Southeast Highlands of New Guinea, where we were able to find people who met these criteria. Because these people were in no way accustomed to taking psychological tests or participating in experiments, and because we did not know their language but had to work through translators, we had to modify our experimental procedure. In other countries we had shown a single photograph of one or another of the emotion expressions and given the observer a choice among a list of emotion words. In New Guinea, we showed the person three photographs at once, had a translator read an emotion story, such as "A person's mother died," and asked the observer to point to the photograph that fit the story. We found that these people selected the same face for the same emotion as did people in all the other cultures we had studied. There was but one exception: the New Guineans failed to distinguish between the fear and surprise facial expressions. In a related experiment, other New Guineans were told an emotion story and each was asked to show the emotion on his own face. Videotapes were taken of these intended emotion expressions, some examples of which are shown in Figure 3. Analysis of these New Guineans' facial expressions showed again that the same facial expressions were produced for the same emotions as had been found in other cultures, with the exception of fear and surprise, which were confused with each other. Further confirmation of the universality of facial expressions was obtained by a study of another culture in West Irian, the western portion of the island of New Guinea. Karl and Eleanor Heider, who were skeptical of our evidence of universality, conducted the same experiments with people even more visually isolated than those we had studied, and they also obtained evidence of universality. r e s e a ~ hon facial expressions of emotion 27 Taken together, our studies, those of Izard, the Heiders' study, and evidence from Eibl-Eibesfeldt (an ethologist using very different methods), showed quite conclusively that Darwin was correct in claiming that there are universal facial expressions of emotion. Although the appearance of the face for each of the primary emotions is common to all peoples, facial expressions do vary across cultures in at least two respects. What elicits or calls forth an emotion will usually differ; people Fig. 3 Video frames of attempts to pore emotion by subjects from the Fore of New Guinea. The instruction for the top left photograph was "your friend has come and you ore happy"; for the top right "your child has died"; for the bottom left "you are angry and about to fight"; for the bottom right "you see a dead pig that has been lying there for a long time." 28 research on facial expressions of emotion may become disgusted or afraid in response to different things in different cultures. Also, cultures differ in the conventions people follow in attempting to control or manage the appearance of their faces in given social situations. People in two different cultures may feel sadness at the death of a loved one, but one culture may prescribe that the chief mourners mask their facial expression with a mildly happy countenance. How Does Each Emotion Appear on the Face? As we began to find evidence that there are some facial expressions of emotion which are universal, and before all the studies were completed, we began to investigate just what these universal facial expressions of emotion look like. We sought to construct an Atlas of the face, which would depict photographically each of the universal facial expressions of emotion. It is this Atlas (Reference 4) which forms the basis for the photographs shown in the subsequent chapters of this book. Our first step in developing the Facial Atlas was to study what others had said about the appearance of the face for each of the primary emotions. Some writers had described which muscles were contracted in particular emotions, while others concerned themselves only with the appearance of the surface of the face. None had systematically considered all the muscles or all the consequent changes in the surface appearance of the face for the six primary emotions. Putting together what was written by Darwin, Duchenne, a French anatomist whom Darwin had quoted extensively, Huber (Reference 7), an American anatomist writing thirty years ago, and Plutchik (Reference 9), an American psychologist concerned with emotion, we saw part of the picture emerge. We constructed a table which listed all the facial muscles and the six emotions, entering into the table what these men had written about which muscles were involved in what way for each emotion. There were many gaps, however, where no one had said anything about the involvement of a particular muscle in a particular emotion. Working with Silvan Tomkins (Reference lo), we filled in those gaps with information from our cross-cultural studies and our shared impressions. The next step was to photograph models, who were instructed to move particular facial muscles listed in the table. We separately photographed the three areas of the face which are capable of independent movement-the brow/forehead; the eyes/lids and root of the nose; and the lower face, including the cheeks, mouth, most of the nose, and chin. The completed Atlas consists of a series of photographs of these three different areas of the face, each photograph keyed to one of the six emotions. As might well be expected, for each of the emotions there is more than one Atlas photograph for at least research on facial expressions of emotion 29 one facial area. For example, for surprise there is one brow/forehead, one eyes/lids/root of nose, but four different Atlas photographs of the lower face. The next obvious question was whether the Atlas is correct. Are the six emotions-happiness, sadness, anger, fear, disgust, and surprise-in actuality composed of the facial appearances listed in the Atlas? Or does the Atlas appearance of disgust actually occur with anger, and so forth? We have conducted four experiments on the validity of the Atlas. Two of the experiments attempted to prove its validity by showing that measurements of the face with the Atlas corresponded with other evidence of the subjective emotional experience of the persons whose faces were measured. These experiments investigated the experiential validity of the Atlas. The other two experiments investigated the social validity of the Atlas. Rather than attempting to prove that the Atlas measurements correspond to the person's experience, these studies investigated whether the Atlas measurements can predict what observers think a person is feeling when they look at his face. Although experiential and social validity should be related, they need not necessarily be so. We may not look to others how we actually feel, at least all the time. Thus, it was necessary to study both experiential and social validity. The studies of experiential validity drew from materials gathered in one of the cross-cultural studies of facial expressions described earlier. College students in Japan and in the United States had individually watched pleasant and unpleasant movies while we videotaped their facial expressions. From their answers to questionnaires after the experiment, it was clear that they experienced very different emotions while watching the two films. In describing their reactions to the travelogue, the subjects had said it was interesting and pleasant, and caused them to feel moderate happiness. In describing their reactions to the surgical film, the subjects said they had unpleasant, disgusted, pained, fearful, sad, and surprised feelings. If the Facial Atlas is valid, then measurements based on it should be able to distinguish between the facial expressions shown when these two different sets of emotions were experienced. All the facial muscular movements visible on the videotapes were isolated, their duration was measured, and they were classified in terms of the Atlas. This measurement procedure was done in slow motion, with the measurements made separately for the three areas of the face, by three separate technicians. Such precise measurement required about five hours for each minute of videotaped facial behavior. The results were very clear-cut. Measurements with the Facial Atlas clearly distinguished the two emotional conditions-whether subjects had been watching a stressful film or a travelogue. And the Atlas was equally successful with the facial expressions of Japanese subjects and with Americans, as it should be, because it was built to show the universal facial expressions of emotion. One limitation of this experiment, however, is that it didn't determine whether the Atlas correctly depicts the 30 research on facial expressions of emotion facial appearances for each of the six emotions. It only shows that the Atlas is correct in distinguishing between unpleasant and pleasant experiences. The second experiential validity study provided a partial remedy to this limitation. Recent research on the physiology of emotions suggests that there are markedly different patterns of heart rate acceleration and deceleration with the emotions of surprise and disgust. Measures of heart rate and skin conductance had been gathered on the Japanese and American subjects when they were watching the pleasant and stressful films. If the Atlas is correct in what it says a surprise face and a disgust face look like, then when the Atlas says such facial expressions occur, there should be a different pattern of heart rate for each. When we examined the changes in heart rate which coincided with facial expressions the Atlas had designated as either surprise or disgust, the results showed the predicted difference. Although this second study does provide evidence of the validity of the Atlas for surprise and disgust, it doesn't show that the Atlas is necessarily valid in what it says about the other emotions-anger, happiness, sadness, and fear. Logically, if it is shown to be valid for surprise and disgust, the Atlas should be equally valid for the other emotions, because it was derived by the same method for all six emotions. But evidence is still required, and for that we turn to the third study, which examined the Atlas in terms of social validity. Could the Atlas predict how observers will interpret facial expressions? Photographs that had been taken by many different investigators of facial expression were obtained. These pictures were shown to observers, who were asked to judge which of the six emotions was shown in each picture. Only those on which the observers had agreed about the emotion expressed in the face were further considered. If the Atlas correctly depicts the appearance of each of the six emotions, then measurements based on the Atlas ought to be able to predict the emotion seen by the observers in each of these photographs. The Atlas measurements were made separately for the three areas of the face by three separate technicians, and predictions were made. With great success the Atlas predicted the emotions seen by observers when they looked at each of these photographs of facial expression. The fourth study was much like the one just described, except that here the facial expressions examined were those produced by dental and nursing students who had been instructed to attempt to show each of the six emotions by their facial expression. The problem was for the Atlas to predict for each photograph what emotion the student had been intending to show. The measurements made with the Atlas succeeded. While we were working on these experiments, independently and unknown to us a Swedish anatomist, Carl-Herman Hjortsjo (Reference 6 ) , was working with very different methods on the same problem. He photographed his own face as he contracted each of the facial muscles. Hjortsjo then looked research on facial expressions of emotion 31 at each of these photographs and asked what emotion these pictures seemed to show. On the basis of his own judgment, he then described in his Atlas the appearance of the facial expressions for each emotion. When we recently met with Hjortsjo, we were all pleased to discover that there was almost complete overlap between his Atlas and ours. Although no one of our four experiments alone would validate the Atlas, taken together and in conjunction with the independent discovery of the same Facial Atlas by Hjortsjo, the evidence for the validity of our Atlas is much more than tentative. Much research remains to be done to further validate and refine the Atlas; but the evidence is sufficient to share what we have already found. The information presented in the subsequent chapters about the appearance of the facial expressions is based on the Atlas and has been supported by one or more of our own experiments or by Hjortsjo's work. The photographs show only part of what is in our Atlas. We have selected those parts which are best supported by evidence and which are most important for learning the practical skill of reading facial expressions of emotion. It is unlikely that anything shown is in error. But it is likely that we are only telling part of the story, that further research will determine additional ways in which the facial expressions of emotion can be shown. How Are Facial Expressions Controlled? How can we tell a real facial expression of emotion from a simulated one? When a person doesn't feel the way he looks, but is attempting to mislead us about his feelings, is there any way to detect his real feelings in his facial expression? In short, does the face "leak"? We have been studying this problem for a number of years. We started with films of the facial expressions of psychiatric patients during interviews. In certain interviews we knew from subsequent events that the patients had been misleading the interviewer about their feelings. Study of these films provided the basis for a theory of nonverbal leakage-ways to tell, from facial expression or body movement, feelings the person was attempting to conceal. We have been testing this theory during the last few years by studying interviews in which one person purposefully conceals from another the negative emotions experienced as a result of watching very unpleasaz~t,stress movies. The subjects in this experiment try to convince the interviewer that the film they have seen was actually pleasant, and that they enjoyed it. Our study of these interviews is far from completed. We have yet to test many of our hypotheses about facial leakage. However, our findings to date are consistent with our hypotheses and with our earlier findings from the interviews with psychiatric patients. The information presented in Chapter 32 research on facial expressions of emotion 11 on "Facial Deceit" must be regarded as tentative. It is our best judgment of what the research will prove, based on our theory and cur many hours of inspecting facial expressions of people when they are lying. How Are the Emotions Experienced? We have not directly studied this question, and had thought in planning this book that we would be able to rely upon the scientific literature. We were disappointed to find that despite many studies of emotion and a variety of theories of emotion, relatively little attention has been paid to certain fundamental questions. What, for instance, are the different events which call forth each emotion? What are the variations in the intensity of each emotion? How do the sensations for each emotion differ? And what are the likely activities people will engage in when they feel angry, disgusted, afraid, etc.? There were some answers, or ideas about some of these matters, for some of the emotions. Of most help were the writings of Darwin and of Tomkins. Much of what we have written had to be extrapolated from our own experiences and the many years of thinking about the six emotions whose facial expressions we have been studying. In each of the next chapters is a section on "The Experience of . . ." which presents most of what has been scientifically established about each emotion, and much that has not been established scientifically, although it should be familar to everyone. A number of our friends and colleagues have checked these parts of the book and found them to fit their own lives and their knowledge of the lives of others. You will be able to determine the value of these discussions by comparing them with your own experience and that of your friends. If something that is said about anger, for example, doesn't fit your experience or that of your friends, we may be wrong. If it doesn't fit your experience, but makes sense to friends, you will have discovered a way in which your experience (or your friends') of that emotion is special. References For those interested in learning more about facial expression, a list of some of the most important works on this topic follows. 1. CHARLESDARWIN,The Expression of the Emotions in Man and Animals. London: John Murray, 1872. (Current edition: University of Chicago Press, 1965.) This is the seminal book on facial expressions, in which Darwin presented his observations on facial expressions in animals, human children, blind humans, and members of different cultures. research on facial expressions of emotion 33 2. IRANEUS EIBL-EIBESFELDT, Ethology, the Biology of Behavior. New York: Holt, Rinehart & Winston, 1970. A leading ethologist describes his work, primarily on animals. One chapter deals with his study of facial expressions in humans in various visually isolated cultures, and presents his conclusion that facial expressions of emotion are universal. 3. PAULEKMAN,Ed., Darwin and Facial Expression: A Century of Research in Review. New York: Academic Press, 1973. Separate chapters review and integrate the studies conducted over the last hundred years on facial expressions in infants and children, in nonhuman primates, and in people from different cultures. The book concludes that many of Darwin's ideas about facial expression have been supported. 4. PAULEKMAN and WALLACE V. FRIESEN, Facial Action Coding System. Palo Alto: Consulting Psychologists Press, 1978. Complete atlas, including photographs and film strips, for learning to measure and score facial behavior. This publication is designed for the serious research worker. 5. PAULEKMAN,WALLACE V. FRIESEN,and PHOEBEELLSWORTH, Emotion in the Human Face. Elmsford, New York: Pergamon Press, 1972. This critical review and integration of the quantitative research on human facial expression conducted over a fifty-five-year period presents answers to seven major research questions, suggestions for further research, and guidelines about how to conduct such research. 6. CARL-HERMAN HJORTSJO,Man's Face and Mimic Language. Studentliterature, Lund, Sweden, 1970. An anatomist describes the facial musculature and the particular muscular movements associated with the emotions. Drawings of the face are used to show both the muscle movements and the changed appearance of the surface of the face. 7. ERNST HUBER,Evolution of Facial Musculature and Facial Expression. Baltimore: The Johns Hopkins Press, 1931. An anatomist discusses the evolution of the facial musculature from the lower vertebrates through the primates. A major emphasis is on differences in the facial musculature among different genetic human groups, with some attention to emotional expression. 8. CARROLLE. IZARD, The Face of Emotion. New York: AppletonCentury-Crofts, 1971. This historical review of theories of emotion is a presentation of the author's own theory, based largely on the work of Silvan Tomkins, and on research across cultures and among children. Only slight attention is paid to how the emotions appear on the face. 9. ROBERTPLUTCHIK, The Emotions. New York: Random House, 1962. Another historical review of theories of emotion, this book presents the author's own theory and research in support of i t Only slight attention is paid to how the emotions appear on the face. 10. SILVANS. TOMKINS,Aflect, Imagery, Consciousness, vols. I and 11. New York: Springer Publishing Co., 1962, 1963. An extraordinary, complicated, and fascinating theory that emphasizes the importance of emotion in the psychology of human behavior. Broad in its scope, this work consistently emphasizes the role of the face in human life. surprise The Experience of Surprise Surprise is the briefest emotion. It is sudden in its onset. If you have time to think about the event and consider whether or not you are surprised, then you are not. You can never be surprised for long, unless the surprising event unfolds new surprising elements. It doesn't linger. When you cease being surprised, its disappearahce is often as sudden as was it onset. Surprise is triggered both by the unexpected and by what might be called the "misexpected" event. Suppose a man's wife appears in his office. If she customarily comes at that time to drop off his lunch he will not be surprised. The appearance of his wife is neither unexpected nor misexpected. If his wife rarely comes to his office, but her impending visit is announced by his secretary saying, "I can see your wife coming up the street," he will not be surprised when she arrives, because there will have been time to anticipate and perhaps figure out an explanation for this out-of-the-ordinary event. If his wife walks in unannounced and it is unusual for her to appear at her husband's office, then the event is an unexpected surprise-an unusual event which was unanticipated. It is called unexpected rather than misexpected because at that moment the surprised person was not expecting anything in particular to happen. If, however, the coffee vendor always comes by at that time and always gives a characteristic knock on the door, and it is the man's wife, not the coffee vendor who comes in, the event is a misexpected surprise. There was an aroused specific anticipation for something different to happen at that moment. In misexpected surprise the event need not be unusual to be surprising; it is the contrast with what is expected at that moment that is surprising. If, at the moment that the coffee vendor was surprise 35 expected, the secretary came in, there still might be surprise, probably slight surprise. If, in addition, the event that is contrary to what is expected is also unusual, then the surprise will be greater. The wife at that moment would be more surprising than the secretary. Almost anything can be surprising, provided it is either unexpected or misexpected. A sight, sound, smell, taste, or touch can be surprising. When one bites into a pastry that looks as if it is filled with chocolate cream, the taste of savory pork and mushrooms can be a surprise. The taste was misexpected. Not only physical sensations trigger surprise. An unanticipated or misanticipated idea, comment, or suggestion of another person can be surprising. So can a novel thought or feeling of your own. The technique of many mystery stories is not just to make the reader afraid (that is the ghost or horror-story genre), but also to surprise the reader with whodunnit. Many jokes rely upon misexpectation to achieve their humor. Your amusement depends upon being taken in by the plot of the joke, and surprised by the twist ending. If you have time to anticipate an event and do so correctly, then you cannot be surprised. If the man in our example could have seen his wife approach his office, he might have been surprised at his first sight of her, but by the time his wife knocked on his door-no surprise. Or if he had known his wife was going to be near his office shopping that morning, no surprise. Surprise lasts only until you have evaluated just what it was that occurred. Once you have determined the nature of the surprising event, you are no longer surprised. Usually there is a ready explanation-"I was shopping, ran out of checks, came by to get some blank checks from you, met the coffee vendor in the hallway, and got your coffee from him." If the event resists interpretation, the surprise lingers; you may become disoriented, fearful, or mystified. Suppose a woman is greeted on the doorstep by the husband whom she had thought was killed in the war. Surprise. But it ends when he explains, "I'm your husband's twin brother." or "I was missing in action, and had amnesia." If the interpretation is even more unlikely than the event, she may be surprised all over again, fearful or mystified-"I'm the spirit of your husband returned to talk to you." Once you have evaluated the unexpected or misexpected event, you move quickly from surprise into another emotion. "What a happy surprise,'' you say, not realizing that surprise itself is neutral in hedonic tone. It is, rather, the following emotion that gives a positive or negative tone to the experience, depending upon the nature of the event. Surprise turns to pleasure or happiness, if the event is or foretells something you like. Disgust greets the noxious or distasteful event. If the event is provocative of aggression, surprise yields to anger. And if the event poses a threat which you cannot 36 surprise obviously mitigate, you feel fear. Fear is the most common sequel to surprise, perhaps because unexpected events are often dangerous, and many people come to associate anything unexpected with danger. In a later chapter we will show how the recognition of fear and surprise can be confused because of similarities in their facial manifestations. Because the experience of surprise is brief, with another emotion quickly following it, the face often shows a blend of surprise and the subsequent emotion. Similarly, if one is already experiencing an emotion when a surprising event occurs, a blend is evident on the face. A keen observer who pays attention to fleeting facial expressions might catch a pure surprise expression. Most of us, however, are more familiar with the appearance of surprise in combination with elements of a second emotion. Thus, the wideopen eyes of surprise may endure for an instant, as a grin spreads over the lower face. Or the raised brow of surprise may momentarily appear with the stretched-back mouth of fear. In the next chapter we will show fearsurprise blends; in subsequent chapters we will show surprise-disgust, surprise-anger, and surprise-happy blends. Surprise varies in intensity from mild to extreme, depending upon the event itself. The unexpected appearance of a wife in her husband's office would presumably be less surprising than the unexpected appearance of an old childhood friend not seen for years. The startle reaction has been considered to be the most extreme form of surprise, but it has special characteristics which distinguish it from surprise. The startle reaction differs from surprise in facial appearance. The eyes blink, the head moves back, the lips retract, and there is a "jumping" or "starting" movement. A sudden and extreme change in stimulation, best exemplified by the sound of a gun or a car backfiring, produces the startle reaction. Unlike surprise, where correct anticipation of an event prevents the experience, the startle reaction can be elicited by events you anticipate. Successive loud reports from a gun continue to call forth the startle reaction, although the feeling and appearance diminish. Unlike the surprise experience, which is neither pleasant nor unpleasant, the startle experience is usually unpleasant. No one likes the way it feels. Although people speak of a startling idea, or being startled by what someone said, this may be a figure of speech. It is not clear whether a person can be startled by anything other than a sudden extreme sound, sight, or touch. You may be extremely surprised by what someone says, show an extreme surprise facial expression, and describe the occurrence as startling. Thus, the term startle is used to describe the most extreme surprise reaction, and also for a reaction related to but different from surprise. The startle reaction is also closely related to fear, and in the next chapter when we dis- surprise 37 tinguish fear from surprise, we will also further explain the relationship of the startle to both surprise and fear. The experience of each of the emotions we will discuss can be enjoyable. It is obvious that happiness is an enjoyable emotion, but surprise, fear, anger, disgust, and even sadness also can be enjoyed though obviously they usually are not. Some people rarely enjoy being happy, but instead feel guilty or ashamed of their pleasure. Enjoyment of the emotions, or the inability ever to enjoy them, is probably the result of learning during childhood, but little is known about just how this occurs. Certainly, there are people who love being surprised. A surprise party, a surprise gift, a surprise visit delights them. They organize their lives, in part at least, so that there is a high likelihood of being surprised. They seek the novel. Taken to the extreme, a person "addicted" to surprise, who thrives on it more than any other emotion, would have to sacrifice planning. His life might well tend to be disorganized so that he did not through anticipation eliminate the chance of surprise. There are also people who can't stand being surprised. They will tell you, "Please don't ever surprise me," even if the surprise is going to be a pleasant one. They don't wish to be subject to the unexpected. They organize their lives to reduce novelty and to avoid experiences in which they will not know wha