|This site maintained by: Aomar Boum. Site last updated on October, 2001.||
Journal of Political Ecology:
Case Studies in History and Society
VOLUME 2 (1995)
Claims-making as Social Science: a Review of Environmental Values in American Culture, by Willett Kempton, James S. Boster, and Jennifer A. Hartley; Cambridge, Mass: The MIT Press 1995. xii, 226 pp., 4 appendices.
Reviewed by Thomas Greider, Associate Professor of Community and Environmental Sociology University of Kentucky.
This book is easy to read. It will likely be popular among environmentalists globalist-oriented politicians, and social scientists who have adopted the now taken-for-granted mantra about "global environmental change" and the anthropogenic "causes" of such. The premises of this book fall easily in line with a number of "global environmental change" books written by people who are funded by NSF, NOAA, and NASA. Indeed, the first paragraphs of many of these books are interchangeable and are familiar to many. In the words of Kempton, Boster, and Hartley:
With this book, Kempton and colleagues aim to explicate the belief systems and values that they contend are located at the foundation of popular environmentalism in the U.S. To do so, they undertook a two-stage investigation, beginning with open-ended interviews that are intended to have people explain their insights and beliefs in their own words. The second stage of data collection involved a structured interview format, with composite statements developed from an analysis of transcripts from the initial open-ended interviews. Data-collection methods are extensively described. For doing so, the authors are to be applauded. The scrutiny this description invites, however lays bare a footing that is far too shaky to support the claims that Kempton and his colleagues seek to make either about the components and causes of popular environmental values in the U.S., or about the cultural models that exist across the American public.
At the outset I want it understood that I do not share the authors' unquivering acceptance of the "global environmental change" premises. I have absolutely no idea whether global warming, for example, will occur during the next decade or sometime during the next century (although I always wonder what those other scientists--S. Fred Smith or Robert C. Balling for example--might have to say as they are seldom [never] noted in the first paragraphs of books such as this). As a sociologist, however, it is absolutely clear to me that some groups of people, including (unfortunately) many social scientists, are making claims about catastrophic, human-caused global environmental change, and are calling for all sorts of political and social action in the name of those claims. The authors of this book have incorporated such claims into their research design and, with the inclusion of other equally questionable research techniques, have thereby produced a tract, rather than a social science book.
I focus this review on the authors' methods, their findings, and their interpretations of their findings. After reading the book several times, I am forced to conclude that the authors stretch the limits of their data by confusing facts with interpretations and by simply confusing their readers. Uncontrolled systematic bias runs throughout their qualitative and quantitative methods and interpretations. The following is a main conclusion that cannot be supported by their methods, data, inferences, and interpretations:
The primary data collected by the authors came from semistructured interviews and a "survey." Semistructured interviews were conducted with 46 informants which included "twenty lay informants as well as approximately five each from four specialist groups: grassroots environmentalists, coal industry workers, congressional staff working on environmental legislation, and automotive engineers" (p. 20). Turning to Appendices A and B, we learn more about the informants. There were 43 usable interviews. Of the 20 lay informants, 14 came from a "near-urban community" in New Jersey; the remainder came from "rural central Maine. The Maine interviewer... grew up in the area and selected informants from acquaintances with a diversity of backgrounds" (p. 228). We also learn that "approximately two-thirds of those approached [in the New Jersey community] declined to be interviewed" (p. 228). We are left with the unstated assumptions that we should simply trust the interviewer's selection of "acquaintances," that those who declined to be interviewed are similar to those who participated, and that residents of the Northeast are similar to residents of the Intermountain West, for example. Given the forceful new social movements gaining support in communities throughout the West, such as the property-rights movement and the "Take Back the West" movement, (which the authors cavalierly dismiss in a footnote) there is little basis for confidence in these assumptions.
Respondents for the "fixed-form survey" were selected according to an unconventional strategy that does not permit any type of generalization beyond the respondents themselves. They were chosen from environmental groups, residents of northern and southern California suburbs, dry cleaning store owners, and sawmill workers enrolled in retraining classes. The Earth First! group consisted of 31 respondents (out of an unknown number of prospects) who attended either a national meeting in Vermont or a regional meeting in Wisconsin. The authors report (p. 233) that the fieldworker who attended the meetings judged "that overall the sample is pretty representative of Earth First!." Without additional information, one is hard-pressed to determine the population to which generalizations might be inferred from the survey results. The Sierra Club group consisted of 40 individuals selected randomly from the membership list (containing an unreported number of names) of the Orange County, California, Chapter; a total of 28 completed questionnaires were returned. The "lay sample" consisted of 25 people who "were interviewed in three California cities.... Seven people were asked to complete the survey during the 40+ minute BART (subway) ride between Walnut Creek and San Francisco. In Southern California, ten were interviewed during a door-to-door canvass of homes in Carlsbad (a middle-class neighborhood); eight more were interviewed on the beach near Huntington Beach" (p. 233- 4). This type of "lay sample" hardly merits much confidence. Dry cleaners in the Los Angeles area constitute another group. The authors received completed questionnaires from 30 people out of 80 who originally agreed to participate. The fifth group consisted of laid-off sawmill workers enrolled in retraining classes at a community college in Oregon. "Of the forty-eight surveys passed out to those who volunteered to participate, 27 were completed and returned" (p. 234). No explanation is offered for why so many of the dry cleaners and sawmill workers who originally agreed to participate opted out of the study.
The questionnaire consisted of composite statements written to reflect ideas drawn from interviews with informants. Responses to the statements consisted of a six-point Likert scale indicating strongly agree, agree, slightly agree, slightly disagree, disagree, and strongly disagree. For the analysis, the authors intentionally reduced the variability across respondents by standardizing responses "so that all respondents' responses... have a mean of zero and a standard deviation of one. This reduces differences among the respondents in their different interpretations of the Likert scale both in response bias (agreeing with more of the items than other respondents) and in variance (using more extreme responses than other respondents)" (p. 234-235). The authors claim that "very little information was lost in this recoding, as it correlated with the original six-point scale with a Pearson r of .997" (p. 232). Here, the authors are confused about the nature of "information." Information on response variability across their five "samples" cannot be obtained from anything presented in the book. Perhaps more importantly, the authors assume that there is no qualitative difference between a response indicating "slightly disagree" and one indicating "strongly disagree." The appendix in which the "survey" data are presented is unenlightened. Categories remain collapsed and presented as "the percentage agreeing with the statement.... The second line is the strength of the answers on a scale from 0 to 2, with 0 being either 'slightly agree' or 'slightly disagree,' 1 being the middle strength, and 2 being 'strongly' agree or disagree.... [T]he agreement number alone contains most of the information in the data set (p. 254). No, it doesn't. The decision to analyze the collapsed categories undermines the strength of conclusions from their factor analysis in Appendix A "that there is a single belief system present, with approximately 50 percent of the beliefs and values shared. There is a single consensus, although there is still substantial variation among individuals" (p. 236). Eliminate variation and diversity is precluded. The survey itself is constructed in a questionable manner, filled with double-barreled questions or statements. Indeed, multiple components are intentionally introduced into a given statement: "our survey's statements often try to build an argument, to see how many informants will subscribe to the whole argument.... A disadvantage [of this technique] is that in multipart statements we do not know whether informants are agreeing with every fact included, or more generally with the gist of the argument" (p. 232). The limits of this approach by far outweigh any benefits it might produce.
Here are two examples: Statement 29 (p. 231) reads "The environment may have been abused, but it has tremendous recuperative powers. The radical measures being taken to protect the environment are not necessary and will cause too much economic harm." Statement 141 (p. 134) reads "We should return to more traditional values and a less materialistic way of life to help the environment." No one can know what any response to these statements mean.
The semistructured interviews were designed to elicit the environmental beliefs and attitudes of the informants and, amazingly, to teach informants the "Truth" about global warming and then elicit their policy preferences. The authors state that their reasons for conducting the study were that "it was of academic interest to us, that research and publication were part of our university jobs, and that we hoped to improve public understanding of global warming" (p. 168, emphasis added). "[T]he interviewer gave a short briefing on global warming to provide background information. The briefing was designed to be similar in length and detail to an in-depth article in a weekly news magazine" (p. 227). "Our briefing summarizes current scientific knowledge on global warming. We show a pie chart of causes... "(p. 81). The briefing is reproduced in Appendix C. Statements of "Truth" and "Fact" abound: "I'm going to briefly describe what some scientists are saying about the greenhouse effect. The greenhouse effect makes it hotter.... Many scientists expect.... [I]f the scientific studies are correct.... [T]his chart shows what contributes most to the greenhouse effect. The biggest part, over half, is burning coal, oil, and natural gas for energy.... Scientists estimate that if we cut the amount of these gases we produce in half, we would slow down the process a lot. They estimate that cutting by three-quarters would stop it" (p. 250-1). The authors presented to informants one particular definition of Truth--the one in which they are True Believers--and, as is normally the case in politicized "research," called on the authority of Scientists for legitimation of their truth-claims, first "some scientists," then "many scientists," and finally "Scientists." In law, such a tactic would be labeled "leading the witness;" in social science, it is the introduction of systematic bias and claims-making into the research design.
In addition to methodological issues, I believe there are many ethical issues involved in presenting the kind of "briefing" the authors gave to informants. Let me give an example by way of an analogy. Suppose we conducted a survey among anthropologists (Scientists) who are familiar with the culture of Southern Paiute people. We ask the anthropologists if they believe in the reality of water babies (the spirits who Southern Paiute people believe live in springs). The Scientists would probably respond that they do not believe in the reality of the spirits --the spirits do not really exist. In subsequent interviews with Southern Paiute people, should we then "brief" them on water babies, inform them that according to Scientists water babies do not exist, and help "improve public understanding of" physical reality? Of course not, and most sociologists and anthropologists would probably agree that this would constitute bad research. It would be bad research and borders on the unethical. So, too, is the "briefing" given to informants by Kempton and colleagues.
I am even more disappointed in the authors' analysis of the qualitative data gathered from their informants. The primary goal was to identify the cultural models ("models that are shared within a culture or social group" [p. 11]) by which Americans define and interpret nature and the environment. Recall that their informants included 20 lay people and 23 informants ("specialists") who were grassroots environmentalists, coal industry workers, congressional staff working on environmental legislation, and automotive engineers. Normally, textual analysis would be conducted within each group and then across the five groups to determine similarities of "models in order to draw the conclusion that the models represent cultural models that are shared social constructions of reality. The authors failed to analyze or present their data in this manner, thus leaving the reader unable to determine independently whether there is a "coherent shared set of environmental beliefs and values," or whether "American environmentalism represents a consensus view."
Chapter 3 is titled Cultural Models of Nature: "We begin our examination of how laypeople view environmental problems by describing the cultural models Americans use to understand nature and humanity's interaction with it" (p. 39). I expected quotes from the "laypeople" only, but the authors used quotes from a few of the "specialist" groups while omitting other "specialist" groups. The following nine models, subsumed into "three sets of general environmental models," (p. 39) are described: Human Reliance on a Limited World (quotes from nine lay people, no specialists)
1. Humans are part of the environment and depend on it.
2. The planet is limited in size.
3. Our wastes do not disappear but enter cycles and return to us. Nature as Interdependent, Balanced, and Unpredictable (quotes from five laypeople, three coal industry workers, one environmentalist, three congressional staffers, no automotive engineers)
4. Different parts of nature are so interdependent that changing one part can have chain reactions on a series of others.
5. Interdependencies in nature are so complex that the interdependencies cannot be predicted in advance.
6. Because interactions are impossible to predict, humans should not interfere with nature. Causes of Environmental Concern (quotes from four laypeople, four environmentalists, one congressional staffer, no coal industry workers, no automotive engineers)
7. Nature has been devalued by modern economic and social systems.
8. Lack of contact with nature leads to a lack of concern.
9. Primitive people placed high value on the environment. One example will be sufficient to indicate the zealous and overly eager labeling of a few quotes from a "cultural model." Model 8 "was raised by only four of our six environmentalists, not by other informants" (p. 55). Nevertheless, the authors assert that this is a "cultural model" (p. 56), apparently because "majorities of most groups" in their "survey" agreed with double-barreled statements the authors assert reflected "such matters." is titled Cultural Models of Weather and the Atmosphere and "covers cultural models that laypeople apply to the specific problem of global warming" (p. 63). Quotes from 14 of the 20 laypeople and 1 coal industry worker are included in this chapter. The following "cultural models" are asserted to exist by the authors.
10. A cultural model of pollution: one quote is given, which "illustrates the interview data from which the pollution model is inferred.... From statements such as this from many different informants [reviewer's comment: How many? From which groups?], we infer a cultural model of pollution" (P. 64).
11. "The American view of ozone depletion being caused by spray cans is probably a simple cultural model" (p. 68) (quotes from three laypeople and one coal industry worker).
12. "...a simplified model of plant photosynthesis and respiration" (p. 68) (quotes from two laypeople).
13. "In sum, many people already have a sense of how much temperature fluctuates and what effects hot weather has upon humans. This mixture of simple concepts and cultural models about temperature is used to interpret global warming" (p. 75, emphasis added) (quotes from five laypeople). The authors assert more cultural models in Chapter 4, titled Environmental Values.
14. "It appears that a cultural model of retribution or punishment is being invoked here" (p. 114) (one environmentalist is quoted).
The authors use their "survey" data to support their contention that this is a cultural model. Statement 78 is: "If any species has to become extinct as a result of human activities, it should be the human species." Roughly 24 of 30 Earth Firsters, 6 of 27 Sierra Club members, 6 of 29 members of the "public," 10 of 30 dry cleaners, and 2 of 26 out-of-work sawmill workers agreed with this statement. Even assuming the reliability of their data (an inappropriate assumption), it is a stretch to suggest that these responses represent a "cultural model." The authors' assertion that "a cultural model of retribution and punishment" exists may represent little more than wishful thinking on their part; the assertion is certainly not warranted by their "data." Still more cultural models are asserted to exist in Chapter 6: Cultural Models and Policy Reasoning.
15. "[Survey respondents] strongly prefer a preventive strategy, applying a cultural model something like that invoked by the traditional saying 'a stitch in time saves nine'" (p. 128).
16. "How do people imagine 'doing without some things'? An austere past is the best cultural model available to conceptualize a low-energy future" (p. 138) (one layperson is quoted).
17. "... cultural models relating to automobile fuel efficiency merit more careful analysis than our present data allow" (p. 145) (one environmentalist and one automotive engineer are quoted).
18. "National polling also supports our conclusion that energy taxes are opposed due to specific cultural models rather than resistance to taxes in general" (p. 152) (no specific quotes are given).
19. "The cultural model explaining poor corporate environmentalism was not that corporate officers were bad, but that industry is entirely directed by the profit incentive" (p. 156)(one layperson is quoted).
20. "We have only scratched the surface of cultural models of institutions related to global environmental change" (p. 158).
"Purposive sampling" is an accepted technique in social science research, but it is recognized as nonrepresentative and cannot be the basis for broad generalization. The authors' data and analysis simply do not warrant the generalizations they make to "Americans" or the American "public." Examples of misuses of their data follow: "The following chapters in this book present our findings on the environmental beliefs and values of Americans" (p. 15). "Our survey suggests that this model is held by a substantial majority of the public" (p. 73). "Two practical results flow from Americans' use of the oversimplified model of photosynthesis and respiration..." (p. 73). "Some of the quotations from our interviews, and the complex pattern of survey responses, suggest that there is much more to the question of life-style change than we have captured here. American opinion is divided as to the potential contribution of changing technology versus 'the way we live,'... Majorities believe that..." (p. 135). The authors' choice of words to generalize from their data reflects an inadequate and nonscientific understanding of research methods and sampling. Given their dismissal of probability sampling early in the book ("Although, it could have been done for some subgroups, such as the general public, we judged the gains to not warrant the delay" [p. 22, emphasis added]), the authors might be accused of misleading readers in the name of a globalist political agenda based on the claim of global environmental change.
Although I believe the research design to be fundamentally flawed, some examples of anthropological insight can be found in this book. There is, for example, a brief discussion devoted to the notion that changing language (words) may reflect more fundamental cultural change (swamps become wetlands, jungles become rainforests). The idea of, and empirical search for, cultural models associated with the environment and nature is to be applauded and continued, albeit with considerably greater attention to research and analytic techniques.
Unfortunately, the problems with this book far overshadow any contribution to the social scientific understanding of such cultural models. Just as unfortunately, the book is likely to erroneously provide additional encouragement to environmentalists, globalist politicians, funding agencies, and social scientists who share the now taken-for-granted claims of "global environmental change" and the political agenda incorporated in those claims. The political agenda invariably includes the development of "policies" to force "significant sacrifices" on everyone. I end this review with such an agenda from the authors (p. 212): "The strong endorsement of environmental values by the diverse groups studied in our survey may well reflect a general willingness for the American public to make significant sacrifices for the sake of the environment.... Policies must be crafted and leadership provided o overcome divergent individual and group self- interests."