Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your

If you are looking for affordable, custom-written, high-quality, and non-plagiarized papers, your student life just became easier with us. We are the ideal place for all your writing needs.


Order a Similar Paper Order a Different Paper

Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene.

Describe the steps you and your fellow investigators would follow when processing the crime scene, collecting and packaging physical evidence, and maintaining the chain of custody.

After the physical evidence is dropped off at the lab, describe the process that is used to analyze the evidence and the process that would be used to reconstruct the crime scene.

Your assignment must be at least two pages in length, and you should at least use your textbook as a reference. Adhere to APA Style when creating citations and references for this assignment.

Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your
Looking at the image above, taken from your textbook, imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your fellow investigators would follow when processing the crime scene, collecting and packaging physical evidence, and maintaining the chain of custody. After the physical evidence is dropped off at the lab, describe the process that is used to analyze the evidence and the process that would be used to reconstruct the crime scene. Your assignment must be at least two pages in length, and you should at least use your textbook as a reference. Adhere to APA Style when creating citations and references for this assignment.
Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your
Unit II: Crime Scene Investigation 1 Introduction to Unit II: In this unit, we examine the crime scene and the various measures that are taken to preserve and secure a crime scene, record and search for physical evidence, as well as reconstruct a crime scene. There are various tools and techniques used to package the types of physical evidence that are found at a crime scene, including how to determine the significance of physical evidence and the steps involved in the analysis of the evidence. Furthermore, we’ll discuss the concept of chain of custody, including the steps that are necessary to maintain the chain of custody and how to secure, protect, and preserve a crime scene. We’ll also discuss the general features of bloodstain formations, the information that can be gained from pattern analysis, and what that information tells us about the events involved in the crime. Lastly, the methods of documenting patterns of bloodstains will be discussed. 2 Critical thinking skills are essential for any analyst in crime scene investigations or crime scene reconstructions. Deductive and inductive reasoning is used by the analysts to find and exploit explicit knowledge of the series of events surrounding a criminal act or an event. Crime scene reconstruction is a discipline of forensic science. This involves the context evaluation of physical evidence in an effort to identify what happened and the chronological order of events leading up to the act. The holistic approach is the perspective in which an analyst views the development or the circumstances surrounding an event based on the physical evidence discovered, examined, and analyzed. The elements of evidence are interdependent, and each are significant as a piece to each action and event. 3 Like any investigation, crime scene reconstruction is like putting a puzzle together — although it is much more difficult in that you don’t have access to the box top, so you can’t have a reference of the shapes, colors, and the textures to help you put that puzzle together. The analyst does not know how the overall picture of the puzzle is to look in order to help solve it. This takes critical thinking skills. Understanding what is necessary to assess evidence is one of the first steps that an analyst must know in order to render better judgment or reasoning. Additionally, not all the puzzle pieces may be present and should be identified as gaps in intelligence. But analysis of the pieces that are present must continue to help fill in the holes. Once the analysts have enough pieces strung together in the correct order, the puzzle becomes clear enough to be able to view and even recognize the chain of events that occurred in the commission of the crime. Also, some of the most critical questions about the events may be answered. 4 Areas of importance: Within the discipline of forensic science, investigators will use the following to help them solve the crime or at least determine what happened at the crime scene. Specific incident reconstruction, which deals with homicides, bombings, traffic accidents, event reconstruction, which examines connections between evidence, sequences of events, and the identity of persons, and the physical evidence reconstruction, which focuses on blood, firearms, trace evidence, and other items for DNA analysis. In order to become a competent reconstructionist, you should possess technical knowledge and a thorough understanding of forensic investigations. While there is no current educational requirements, many analysts and technicians possess undergraduate or even graduate degrees in forensic science. The degrees, though, can range from criminal justice and forensic science, to chemistry, biology, and even in cases engineering. 5 Securing and protecting the crime scene: The first person who arrives on the scene of a crime is responsible for the preservation and the protection of the scene. This is important not only to the investigation but also the reconstruction. Usually, this individual is a patrol officer who may have to deal with a still violent circumstance. Second, the first person on the scene must attend to any victims who require medical attention. Therefore, he or she will also be part of documenting his or her observations as well as their movements for the crime scene investigators. After the scene has been preserved and protected, the investigators will begin the process of evaluation and recording of the crime scene using video and photography. The systematic search for evidence then begins with the collection of physical evidence for processing and analysis. 6 When setting up a search for evidence, search patterns are established to avoid confusion and contamination of a crime scene in collecting the evidence. Search patterns will include any of the following, and it’s dependent upon the crime scene itself as well as the technicians. These patterns include the strip or line search, a grid search, a wheel or ray search, a quadrant or zone search, a spiral search method. Each one of these methods is dependent, as I said earlier, on the crime scene itself as well as the technicians. Depending on the number of technicians available and the crime scene, if it’s too large, will give a better indication of which kind of search method to use. 7 Collection of physical evidence: During an investigation, all evidence is valued. But each type of evidence provides its own specific value to the investigation. This value should be kept in mind by the investigator. For example, collection of good fingerprints should be given more time and attention than, say, searching for evidence of fibers. Positive identification of an individual is crucial in investigation. Therefore, having fingerprints that can determine who was at the scene is significant to the overall investigation. In the event that no fingerprints are found, then searching for fibers found on, say, the body of a victim or in the victim’s proximity is the next best thing. Collection of as much evidence as possible is a best practice because not collecting enough evidence can either hurt or hinder the investigation. 8 Physical evidence continued: Here are a few physical evidence types that may be collected as well as how to handle each of these. Fingerprints, the collection of fingerprints includes not only fingers but also handprints, palm prints, and even bare footprints. We’ll go over shoe tracks later. This type of evidence is the best type to place an individual at the crime scene and should be given top priority. The materials for collecting this type of evidence are very minimal. From the standpoint of cost, this makes fingerprint collection ideal. Collecting fingerprints from any non -movable item at the scene should be processed in place using gray powder, black powder, or even magnetic powder. Transporting smaller items for processing in the lab is always a best practice. This can be done by placing each item separately in a paper bag or evidence envelopes to go back to the lab for processing. 9 Bite marks: A physical evidence element that is usually found in sexual assaults is a bite mark. Bite marks may be left by the perpetrator and can be matched just as easily as fingerprints. If bite marks are found, they should be photographed in a normal lighting condition. Alternative lighting should also be utilized such as ultraviolet or UV lights or even side lighting. If an impression was left on the skin, then a casting should be made. For comparison, a cast or a photograph of the suspect’s teeth should also be made. 10 Broken fingernails are found at crime scenes all the time and carry with them very unique characteristics. Much like fingerprints, teeth, bite marks — fingernails carry striations and other markings that make them unique to one individual. When we talk about questioned documents, these are just documents that are either can be handwritten or typed, and they can be matched to an individual based on such things as how they are handwritten — the ink from a printer or a typewriter and even the paper stock. Left on the paper, oils from the hands can also be traced. 11 Firearms and tool marks: Firearms used in a commission of a crime leave behind a significant amount of evidence. From the bullets that they discharge to the casings found, bullets can be matched to the firearm used. Casings can also be matched to the firearm used through the strike of the firing pin on the casing. The firearm itself is also part of the evidence as the bullet and casing can be matched directly to the barrel, which leaves striations and marks on both the bullet and the casing. The bullet and casing will tell investigators what make and model the firearm was, the caliber of the firearm as well. Furthermore, the firearm may have itself other trace evidence such as fingerprints, which should be processed. Remember that all evidence including bullets, casings, magazines, and the firearm itself should be packaged separately. Then, the firearm should be rendered safe prior to any packaging and transportation back to the laboratory. Each item in evidence should be packaged separately. Tool marks are impressions that are left on surfaces of material. This is referring to the top layer or the outside of an object. Examples such as, say, a basketball, or even the body, the skin surface. 12 Shoeprints and tire tracks: As referenced earlier, shoe prints are just like fingerprints. Beyond fingerprints and footprints, other evidence of transportation of the subject or the victim can be found in shoe prints and vehicle tire tracks. Let’s tackle shoeprints first. Shoeprints can be compared easily due to the specific tread patters manufactured by companies in the make, style, and the model of the shoe. Furthermore, the way an individual walks provides wear of the shoe tread. When collecting shoeprint evidence, shoeprints can be lifted just as fingerprints with electrostatic dust print powder. When not possible, photographs, of course, should be taken at various angles and in various lighting exposures. When taking these photos, a ruler and a level should be used. For tire tracks, cast impressions should be taken when possible. 13 Fracture matches: Fragments of materials, such as wood, glass, plastics, or other materials, should be processed as evidence as they can be used to match items in the suspect’s possession. This can be used to identify a suspect’s connection to a crime or a crime scene. Fibers can be found in various types of material and can be of natural or synthetic substance. Paint, like other materials, has the ability to transfer from one place to another by means of contact. Whether the paint is chipped or scraped, contact strikes will transfer and, therefore, allow the evidence to be collected from, say, tools, suspects, or other materials for comparison to paint found at the crime scene. Other trace evidence like glass and plastic can be transferred from the perpetrator to either the victim or to a surface. 14 Hair, blood, and bodily fluids are other elements or components of physical evidence. Hair found at the scene on a victim or even a suspect can tell investigators essential information. If the root sheath is attached to the hair follicle, then DNA can be extracted in order provide information about where the hair came from, specifically providing a certain percentage of the population from which the possible suspect maybe a part of. This is ethnicity, race, color, even age. The hair found may also be compared to the suspect’s own hair sample. If there’s no root attached, the analyst may be able to compare the hair samples with the evidenced hair under a microscope for similar characteristics. When discussing bodily fluids and blood, these are significant elements of evidence. Bodily fluid refers to and can include seminal fluid, urine, and even saliva. Blood and bodily fluids provide the highest degree of probability to be matched to an individual. These types of fluids, whether wet or dried, stained, can be collected. 15 Bloodstain patterns and analysis: When an individual is struck and blood is released into the atmosphere, where it lands and how it lands can tell investigators a lot of information. The examination of the location of blood includes the shapes and distributions of patterns, which can help to determine the physical events that occurred. Some of the analysis may take place right there at the crime scene, while other analyses may take place in a laboratory setting. This can include taking samples and photographs and then writing a detailed and comprehensive report of the analysis. This is why it is so important to have critical thinking skills as well as the ability to write clearly and concisely to convey the precise interpretations, findings, and ultimately the results. Written analysis by technicians can explain how the blood patterns were made. Technicians may also provide their scientific opinions, which are supported by tests, examples from previous cases, as well as legal precedents. 16 When presenting the analysis, the technicians will provide such information as the distance from the blood source, the direction the blood traveled, and the impact angles. This aids in ascertaining the nature of the force used that caused the victim to bleed and, in some cases, even the tool or instrument used to draw the blood. Due to research and development of tools and techniques, they can and have been able to provide the possible object used in blunt force trauma. Lastly, technicians can help determine the crime’s sequence of events as well as the interpretation of contact or transfer of blood stain or spatter patterns. By providing the documentation, which is a detailed and usually comprehensive report and preparing for legal proceedings, this process helps confirm or even refute the position of the victim, the suspect or the defendant, or a witness. Whether the event occurred as a result of an assault of a single individual, or the struggle that occurred between two or more people can also be determined by this process. In addition, this process can result in the confirmation or the repudiation of statements made by individuals in the case. 17 Let’s watch a short video on the main categories of bloodstain patterns. 18 Chain of custody: The process of evidence collection can only begin once the scene has been documented thoroughly and the location of evidence noted. When collecting evidence, most items will be placed in envelopes, bags, and even containers that are usually non -breakable and leak proof. Air -tight, clean metal cans that resemble paint containers are used for evidence collection in arson investigations. Once the packages, envelopes, bags, and containers are sealed they should be then labeled with the collector’s initials, the agency’s name, case file number, date and time of the collection, and a completed and detailed description to include where it was found. Whenever the evidence is handed to someone else, regardless of whether the container seal is broken or not, the name of the individual now in custody must sign the chain of custody form to acknowledge that he or she has received it. 19 20
Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your
CMJ 6303 , Forensics Criminal Intel Analysis 1 Cou rse Learning Outcomes for Unit II Upon completion of this unit, students should be able to: 2. Describe current trends in the professional field of forensics. 2.1 Describe the steps involved in processing a crime scene, collecting and packaging physical evidence, and maintaining the chain of custody. 2.2 Describe the process of crime scene reconstruction and the analysis of physical evidence. Course/Unit Learning Outcomes Learning Activity 2.1 Unit Lesson Chapter 2, pp. 29 –51 Unit II Assignment 2.2 Unit Lesson Chapter 3, pp. 61 –76 Chapter 4, pp. 79 –98 Unit II Assignment Required Unit Resources Chapter 2: The Crime Scene, pp. 29 –51 Chapter 3: Physical Evidence, pp. 61 –76 Chapter 4: Crime -Scene Reconstruction: Bloodstain Pattern Analysis, pp. 79 –98 Unit Lesson The content for this unit is presented in the Unit II Lesson Presentation . You will see text, and you can click the sound icon on each screen to hear audio. Transcript of Unit II Presentation UNIT II STUDY GUIDE Crime Scene Investigation CMJ 6303, Forensics Criminal Intel Analysis 2 UNIT x STUDY GUIDE Title Suggested Unit Resources In order to access the following resource, click the link below. In the paper listed below, David Moore presents essential information that analysts must possess; critical thinking skills are crucial in order to better cope with the complexities of the world. This paper provides tools and techniques to help the learner understand best practices in crit ical thinking and analyzing information. Moore, D. T. (2007). Critical thinking and intelligence analysis : Occasional paper number fourteen . National Defense Intelligence College. http:// ni-u.edu/ni_press/pdf/Critical_Thinking.pdf
Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your
Managing the private spies PCN 5193 The National Defense Intelligence College supports and encourages research on intelligence issues that distills lessons and improves Intelligence Community capabilities for policy-level and operational consumers Critical Thinking and Intelligence Analysis, Second Printing (with revisions) David T. Moore This series of Occasional Papers presents the work of faculty, students and others whose research on intelligence issues is supported or otherwise encouraged by the National Defense Intelligence College ( NDIC ) through its Center for Strategic Intelligence Research. Occasional Papers are distributed to Department of Defense schools and to the Intelligence Community, and unclassified papers are available to the public through the National Technical Information Service (www.ntis.gov). Selected papers are also available through the U.S. Government Printing Office (www.gpo.gov). This work builds on earlier publications in this series, particularly Occasional Papers Two, Six, and Seven; Getting Intelligence Right: The Power of Logical Procedure, Intelligence Essentials for Everyone , and Intelligence Analysis in Theater Joint Intelligence Centers: An Experiment in Applying Structured Methods . The author’s insights have been developed through years of creative interaction with the Community’s leaders in analytic methododology, and tested for practicality by successive waves of students in training and education courses throughout the Community. Proposed manuscripts for these papers are submitted for consideration to the NDIC Press Editorial Board. Papers undergo review by senior officials in Defense, Intelligence and civilian academic or business communities. Manuscripts or requests for additional copies of Occasional Papers should be addressed to Defense Intelligence Agency, National Defense Intelligence College, MC-X, Bolling AFB , Washington, DC 20340-5100. This publication has been approved for unrestricted distribution by the Office of Security Review, Department of Defense. Russell [email protected], Editor and Director Center for Strategic Intelligence Research C ritiCal t hinking and i ntelligenCe a nalysis Occasional Paper Number Fourteen d avid t . M oore National Defense Intelligence College Washington, DC March 2007 The opinions expressed in this paper are those of the author. They do not represent the official position of the National Security Agency, the Department of Defense, or any other agency of the U.S. Government. – iii – C ontents Figures and Tables ……………………………………………………………… v Preface ……………………………………………………………… …………….. vii Foreword Mark M. Lowenthal ……………………………………………………… ix Commentary Jeffrey R. Cooper ……………………………………………………….. xiii Francis J. Hughes ………………………………………………………… xvi Gregory F. Treverton ………………………………………………….. xvii Acknowledgments …………………………………………………………….. xix Abstract ……………………………………………………………… …………….. 1 Definitions ……………………………………………………………… …………. 2 Introduction: How Do People Reason? …………………………………. 3 What Is Critical Thinking? …………………………………………………… 8 Defining Critical Thinking ………………………………………… 8 Standards for Critical Thinking ……………………………….. 10 Skill-Based Definitions ……………………………………………. 13 A Disposition to Think Critically ……………………………… 15 The Role of Questions ……………………………………………. 17 Pseudo-Critical Thinking ………………………………………… 18 What Can Be Learned from the Past? Thinking Critically about Cuba …………………………………….. 20 Deploying the Missiles …………………………………………….. 20 Assessing the Implications ……………………………………….. 24 Between Dogmatism and Refutation ………………………… 26 Lacking: Disconfirmation ………………………………………… 29 The Roles of Critical Thinking in the Cuban Crisis …… 33 Winners and Losers: The Crisis in Context ……………….. 37 Ten Years Later, They Meet Again …………………………… 45 Judgment: Critical Thinking Would Have Made a Difference ……………………………………. 46 How Can Intelligence Analysts Employ Critical Thinking? …….. 48 The Poor Record …………………………………………………….. 48 Assessing Evidence ………………………………………………….. 51 Facilitating Evidentiary Assessment ………………………….. 56 Embracing a Methodology ……………………………………… 57 Creating Better Inferences ………………………………………. 58 Producing Intelligence ……………………………………………. 59 How can Analysts be Taught to Think Critically? …………………. 61 Critical Thinking Education Outside the Intelligence Community ……………………. 61 Critical Thinking Education Inside the Intelligence Community ………………………. 64 Implications of Teaching Critical Thinking ………………. 66 Evaluating Teaching Models ……………………………………. 68 Encouraging Analysts to Think Critically ………………….. 69 Persuading to Improve Analysis ……………………………….. 70 How Does Critical Thinking Transform? …………………………….. 74 Transforming Intelligence Corporations ……………………. 74 Learning from Early Adopters …………………………………. 75 The Costs and Benefits of Thinking Critically …………… 76 Validation ……………………………………………………………… 84 What Other Points of View Exist? ………………………………………. 86 What Does the Future Hold? ……………………………………………… 90 Conclusion ……………………………………………………………… ………. 95 Appendix: NSA ’s Critical Thinking and Structured Analysis Class Syllabus …………………………… 97 Bibliography ……………………………………………………………… …… 117 About the Author ……………………………………………………………. 133 Colophon ……………………………………………………………… ………. 134 – iv – Figures and t ables Figure 1: A Comparison of Inductive, Deductive, and Abductive Reasoning ………………………………. 5 Figure 2: Elements of Thought. ……………………………………………. 9 Figure 3: A Continuum of Intellectual Standards. ………………… 10 Figure 4: Levels of Questioning: Critical Thinking in a Social Setting. ……………………………… 17 Figure 5: Respective Ranges of Soviet SS -4 MRBM and SS -5 IRBM Missiles. ………………………………………………… 22 Figure 6: A Place for Analysis between Dogmatism and Criticism …………………………………………….. 28 Figure 7: Detail of a U-2 Photograph of an SS -4 MRBM Launch Site, San Cristobal, Cuba, 14 October 1962 ………. 30 Figure 8: U-2 Tracks over Cuba, 4–14 October 1962 ……………. 32 Table 1: A Comparison of Different Sets of Critical Thinker’s Competencies …………………………………… 14 Table 2: Applying Paul and Elder’s Critical Thinking Model to the Situation in Cuba, August–September 1962 ………….. 27 Table 3: A Comparative Assessment of Rival Motivations ………….. 38 Table 4: Goals and Outcomes in the Cuban Missile Crisis ……… 44 Table 5: How Analysts Decide ……………………………………………. 50 Table 6: Cooper’s Analytic Pathologies ……………………………….. 82 Table 7: Analysis: Past, Present and Future ………………………….. 92 – v – PreFaCe The world in which intelligence analysts work has changed dramatically over the 67 years since the beginning of the Second World War. Adversaries have shifted from large armies arrayed on battlefields to individuals lurking in the shadows or in plain sight. Further, plagues and pandemics, as well as floods and famines, pose threats not only to national stability but even to human existence. To paraphrase a Chinese curse, we certainly live in interesting times. Our times demand fresh, critical reasoning on the part of those tasked to assess and warn about threats as well as those tasked to act on those threats . Education in the bases and practices of intelligence foraging and sensemaking – often called intelligence collection and analysis – is a means by which this can be accomplished. Indeed, the Intelligence Reform and Terrorism Prevention Act of 2004 legislates improved education for intelligence analysis. But, that education is not specifically defined. This volume provides a framework for one area of the act’s educational requirement: improving how analysts think – and by extension, how policymakers act. It asserts that people who are skilled critical thinkers are better able to cope with interesting times than those who are not. The model for thinking developed here also provides specific tools for coping with accelerating disruptive technologies. Such technologies routinely appear in the hands of adversaries. They also offer intelligence professionals capabilities to counter adversaries in novel ways. The key is knowing which technologies are truly disruptive in advance, which pose threats, and which can be harnessed to mitigate threats. Critical thinking – as it is here defined and developed – provides part of the solution as it encourages careful consideration of the available evidence, close examination of presuppositions and assumptions, review of the alternate implications of decisions, and finally, discussion of alternative solutions and possibilities. In short, it equips intelligence professionals with an essential tool for their work. – vii – – viii – Foreword Mark M. Lowenthal Former Assistant Director of Central Intelligence for Analysis and Production President, Intelligence & Security Academy, LLC Some years ago, when I worked at the Congressional Research Service, my colleague next door had a sign in his office: “Thinking is hard work. That’s why so few people do it.” Indeed. As sentient beings we think all the time. Our waking, conscious hours are full of thoughts: what to wear; what to eat; how to respond to someone. These all take thought. One might venture to say that we think almost unconsciously through most of our day. And then there is that other thinking: Thinking about the conundrums that we face, the alternatives and choices we have to make on larger issues, the dilemmas we wish to resolve. This is hard work. And if the work you do is largely intellectual in nature, then you are thinking that much more often about these more difficult problems. This is not to suggest that those who work with their hands or in crafts or industry do not think. Of course they do. But these people, in the end, have something physical to show for their thought and its execution. Intellectuals only have their thoughts to show, presumably expressed for others in some way. Intelligence officers are engaged in an intellectual pursuit. They are trying to solve puzzles, resolve uncertainties, discover the nature and meaning of things that others would keep secret. They must have an entire intellectual apparatus to help them identify the problem, assess the parts they know and the parts they do not, come up with an explanation of what is going on and then express it in a way that others – including an audience not steeped in their own techniques – can understand. – ix – – x – As one would expect, the Intelligence Community has spent a fair amount of time thinking about how it does analysis. But most of this has been spent on techniques, presentation and outcomes; much less has been spent on what happens intellectually while you are thinking. That, indeed, is what critical thinking is about: the ability to step out of one’s thoughts, if you will, to examine them and the process that brought them about even while you are thinking them. It does not require some higher level of cosmic consciousness or a split personality. It does require training and thought! So now we are thinking about our thinking while we are thinking – a mental triple play, if you will. This is no mean feat and it is a very crucial skill for intelligence analysts. John McLaughlin, a former Deputy Director of Central Intelligence, has observed that one of the major perceptions that separates intelligence officers from the policy makers is that policy makers tend to be optimists, believing they can get things done, while intelligence officers tend to be skeptics, knowing that little is known with certainty. Like everyone else, however, intelligence analysts can be adamant about their conclusions, understanding the path they took to reach them, and well-satisfied that they have done the right work and come to the right end. That is where critical thinking comes in. The ability to examine how you came to your conclusion is an important adjunct to your other intellectual work. You may be certain or you may not – which is also acceptable. But you owe it to yourself and to your readers to have examined how you got there. This is not as easy as it sounds for intelligence analysts for the simple reason that they are usually very pressed for time. There are not a lot of lulls in the analyst’s work week, which makes it very difficult to be introspective. This means that critical thinking ought to be something that is ingrained in analysts as part of their training, so that it becomes reflexive. But to do that, one must first understand critical thinking – what it is, how to do it, how to teach or learn it. That is the outstanding value of this monograph by David Moore. The Intelligence Community is filled with people happily working away under a cloak of anonymity, satisfied in making their contribution to part of a larger activity about which very few beyond their immediate circle of co-workers will know. David Moore has long been one of these contributors. In addition to his duties as an intelligence officer at the National Security Agency, David Moore has devoted many fruitful hours to the intellectual underpinnings of intelligence, especially to what makes analysts and what makes better analysts. His work on analytic core competencies, written with his colleague Lisa Krizan, is a fascinating dissection of the intellectual attributes that make up an intelligence analyst. It was also a very influential work, becoming a prime motivator in the creation of the Analytic Resources Catalog created under Director of Central Intelligence George Tenet and now much touted by Director of National Intelligence John Negroponte. This new monograph on critical thinking will serve a similar purpose. Much has been written in the last five years about intelligence and about the errors that it can make. Some of this has been diagnostic; much of it has been blithely condemnatory. Prescriptions about connecting dots better (an absurd metaphor), about the need to share information better and alarms about groupthink do not get us closer to improvements in the art, craft and profession of intelligence. New ways of thinking about our profession, new ways of doing our work and new ways of assessing it will be useful. Critical thinking needs to be at the top of this list of desiderata. I would like to believe that we are on the verge of a renaissance in intelligence studies, and more specifically in the intellectual basis of intelligence analysis. There are many straws in the wind: discussions about a national intelligence university and a lessons learned center, more schools offering serious courses about intelligence and books like this. Critical thinking is and should be critical for all analysts and they will profit from this book. – xi – – xii – C oMMentary : Jeffrey R. Cooper SAIC Technical Fellow Science Applications International Corporation David Moore has written an elegant and largely persuasive argument for the Intelligence Community to move forthrightly and adopt critical thinking in order to improve the quality both of its analytic processes and its intelligence judgments. Since I have recently written about the need for the Community to identify and cure deep-seated and systemic “analytic pathologies,” I am obviously sympathetic to the general case for incorporating more rigorous analytic processes—among which “critical thinking” is one attractive approach. 1 Along with Moore, I also believe strongly that the traits often associated with critical thinking need to become fundamental characteristics of every analyst and to be practiced consistently in their work habits. Moore discusses these aspects at some length and highlights their value to making sounder judgments. In doing so, he makes a compelling case that critical thinking can help analysts reshape their methods in ways that help to avoid damaging errors and enhance judgment. He and I agree with Rieber and Thomason that we should validate the potential improvement before widespread deployment. 2 The need for validation raises some related points concerning the criteria for evaluation and measurement. As Philip Tetlock has noted, on the one hand, one can do evaluation by assessing whether a process (assumed to be good) has been followed; one the other hand, one can also assess 1 Jeffrey R. Cooper, “ Curing Analytic Pathologies: Pathways to Improved Intelligence Analysis ” (Washington, DC : Central Intelligence Agency, Center for the Study of Intelligence, 2005). Cited hereafter as Cooper, Pathologies . 2 Steven Rieber and Neil Thomason, “Toward Improving Intelligence Analysis: Creation of a National Institute for Analytic Methods,” Studies in Intelligence 49, no. 4 (Winter 2006), 71. – xiii – – xiv – quality by looking at the products themselves. 3 However, following a good process does not guarantee good products, nor does a good judgment necessarily validate the quality of the analysis or the analyst. Both approaches have flaws as methods of validation. I urge the leadership of the Intelligence Community to place far more emphasis on structured analytic methods. In my view, the transformation of the intelligence enterprise demands a more curious, more agile, and more deeply thoughtful cadre of intelligence analysts—but it should also require the same traits among its intelligence organizations and the intelligence enterprise as a whole. Moore notes that “Investment in critical thinking as part of the analysis process minimizes the likelihood of specific failures” (page 81). However, from my perspective, critical thinking (and other structured methods) are more important for changing the organization’s overall approach to analysis, rather than in improving specific judgments or preventing particular failures. I believe such methods are crucial in preventing systemic analytic pathologies from developing, exactly because the Community lacks many of the desirable self-corrective mechanisms found in science. Second, while Moore focuses on the role of critical thinking in improving an individual analyst’s ability to make good judgments, my view emphasizes the importance of more rigorous processes for the organization as a whole if it is to improve its capacity to meet user needs. Indeed, given my emphasis on the systemic nature of the pathologies that afflict intelligence analysis, structured analytic methods become a first line of defense in preventing networks of errors—they are like “ripstops” that keep problems from propagating into wider “error-inducing systems,” in Perrow’s terms. The quality of “mindfulness” and a more self-reflective process are essential if the intelligence organizations are to acquire some of the desirable characteristics of high-reliability organizations. While critical thinking can clearly assist individual analysts, and these “memes” can be spread through viral dissemination, I would place far more emphasis on fomenting social and organizational changes as remedies. 3 Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press, 2005). Finally, exactly because Moore focuses so tightly on the value of critical thinking to improve how individual analysts reason, he largely ignores the important role such habits should, and could, play in the “social construction” of intelligence analysis and its communication to users of that intelligence . The use of more structured techniques “to present more effective assessments” should go beyond simply convincing policymakers “to question their own assumptions on the issues” (page 80). By restructuring both the analytic process and the modalities of interactions with policy users, critical thinking techniques can draw policy users into “co-creating” important judgments and, by doing so, get them to adopt some of the important attributes of critical thinking. This internalization would be more powerful than hoping the exquisite brilliance of the improved analytic judgments might make them more palatable. These minor quibbles aside, David Moore has highlighted for the Intelligence Community and its users some tangible methods for improving intelligence analysis and addressing systemic analytic pathologies. – xv – – xvi – C oMMentary Francis J. Hughes Professor, National Defense Intelligence College The truly difficult problems of the information age are not technological; rather, they concern ourselves – what it is to think, to reason, and to engage in conversation [using] … new analytic techniques, new conceptual tools with which to analyze and understand the workings of the human mind. —Keith Devlin, Goodbye Descartes: The End of Logic and the Search for a New Cosmolog y of the Mind, 1997 David Moore has undertaken a valuable initiative on behalf of the Intelligence Community by producing this monograph on critical thinking. In doing so, he has addressed the concerns of Keith Devlin and many other scholarly writers on the subject of human reasoning. By contextualizing the concept and practices of critical thinking within intelligence analysis, he has set forth a much-needed cognitive linkage between intelligence analysis and the human thought process, which, as he states, remains otherwise “poorly understood.” Mr. Moore has developed a methods-of-thinking course, outlined in the Appendix to this paper, which should guide the collection of evidence, the reasoning from evidence to argument, and the application of objective decision-making judgment. Critical thinking in intelligence depends primarily on a conscious application of suitable habits of thought, and the course certainly promises to advance the education of intelligence professionals. C oMMentary Gregory F. Treverton RAND Corporation Former Assistant Director of Central Intelligence for Analysis and Production and Vice-Chairman, National Intelligence Council David Moore has added his powerful voice to those calling for America’s intelligence analysts to be more self-conscious about their methods and more venturesome in applying more formal methods. His starting point is critical thinking, but his strong case embraces a range of techniques from Paul and Elder’s checklist, to analysis of competing hypotheses ( ACH ), to Factions analysis, which is really a way to be more systematic in aggregating subjective judgments, to techniques such as those developed by the Novel Intelligence from Massive Data ( NIMD ) project. These help analysts search for patterns and test hypotheses. Moore’s discussion pays particular attention to critical thinking about evidence – that is perhaps natural given that the National Security Agency is his home agency, but it is also a welcome emphasis. My own explorations of the analytic agencies over the last few years confirm my own experience managing the National Intelligence Council ( NIC ): In general, U.S. intelligence analysis has been neither very self-conscious about its methods, nor made much use of machines and formal analytic tools. That is Moore’s point of departure, and it is a case made graphically by Rob Johnston, on whose work Moore draws. The state of affairs Moore and others describe is changing, and his work should contribute to accelerating that change, in a variety of experiments and pilot projects. Moore’s paper raises at least two intriguing questions at its edges. He rightly takes issue with Malcolm Gladwell’s book about the power of unconscious thought or “deliberation without attention.” – xvii – – xviii – That said, clever defense lawyers often discredit ballistic experts by asking them to be explicit about how they reached their conclusions. Likewise, chess masters can be reduced to middle-weights by having to explain the logic of their moves. In both cases, the decisions result from patterns as much sensed as seen, the result of thousands of previous cases. I sometimes worried at the NIC that the more we required analysts to be explicit about their methods, the more we risked turning them into middle-weights. Finding ways to incorporate not just expertise, but also hunch and sense into intelligence analysis is a challenge that lies ahead. The Defense Advanced Research Projects Agency’s ill-fated experiment a few years ago seeking to establish a terrorism futures market was testimony to how hard the challenge will be. When the idea became public, every politician – right, left and center – quickly denounced it as borderline immoral. Over the ensuing days, however, a wide range of analysis commented on what a good idea it might have been when information is in short supply and there is a premium on sensing patterns before they become facts. The second issue is consumers of intelligence, who figure only at the margins of Moore’s discussion. We can only hope he is right that better analysis will find, and perhaps help create, a better audience. But the unboundedness of the terrorist threat we now face means that intelligence and policy no longer share a “story” comparable to that about state threats, like the Soviet Union. In principle, that would suggest that policy should accord more time and attention to intelligence, but that is not likely to happen in Washington’s whirligig. Thus, intelligence will need to be as creative in finding new ways to work and communicate with policy officials as it is in improving its analytic techniques. In the process, it will need to rethink what it regards as its product. I came to think at the NIC that our product was not really National Intelligence Estimates but, rather, National Intelligence Officers – people, not paper, and people in a position to come to know their policy counterparts and to make their expertise available informally, in meetings or at lunch. a CknowledgMents The first notes for this paper were begun a year prior to the 11 September 2001 terrorist attacks in New York and Virginia as part of research conducted with a peer, Lisa Krizan, on a set of core competencies – knowledge, skills, abilities, and an associated set of personal characteristics – for successful intelligence analysis. At the time, the notion of critical thinking was at best vaguely understood by members of the Intelligence Community, and the National Security Agency ( NSA ) was no exception. Six years later critical thinking is a recognized key skill by which intelligence readiness is defined at NSA . Similar growth in its importance is recognized elsewhere in the Community. This recognition is due in part to the work of Dr. Linda Elder, Dr. Gerald Nosich and Dr. Richard Paul, all of the Foundation for Critical Thinking. They have created a practical model for critical thinking that is both effective and easy to teach. That the Paul and Elder model is known within the Intelligence Community is due in no small part to Professor Francis J. Hughes of the National Defense Intelligence College ( NDIC ). It was he who introduced the author to the model and encouraged attendance at Paul and Elder’s annual conference on critical thinking. He kindly also provided a commentary on the paper. Dax Norman, of the NSA Associate Directorate for Education and Training, a number of NSA colleagues, and approximately 250 participants in the NSA critical thinking and structured analysis course helped make that course a reality and additionally provided valuable insights into the nuances of critical thinking. Comments from LT Robert D. Folker, USAF , also provided insight into the teaching of critical thinking. LT Folker developed a similar course while a student at the NDIC that unfortunately did not remain in the curriculum. As the paper progressed, a number of colleagues and peers provided feedback, comments, and challenges to the arguments presented. They include, in alphabetical order, Solveig Brownfeld, NDIC ; James Bruce, SAIC ; Jan Goldman, NDIC ; Mary Rose Grossman, the John F. Kennedy Presidential Library and Museum; Alexandra Hamlet; Noel Hendrickson, James Madison University; – xix – Robert Heibel, Mercyhurst College; Richards J. Heuer, Jr., Central Intelligence Agency (retired); Norval J. Hilmer, Defense Intelligence Agency; LTCOL James Holden-Rhodes, USMC (retired); Morgan D. Jones, Central Intelligence Agency (retired); Hugo Keesing, Defense Intelligence Agency (retired); Mark Marshall, NDIC ; Stephen Marrin, University of Virginia; Montgomery McFate, Institute for Defense Analysis; William Reynolds, Least Squares Software; the staff of the Central Intelligence Agency’s Sherman Kent Center; and the staff of the John F. Kennedy Presidential Library and Museum . Several other colleagues at the National Security Agency – who wish to remain anonymous – also offered valuable insight and suggestions. Three colleagues, Mark Lowenthal, president of the Intelligence & Security Academy, LLC ; Jeffrey Cooper, Science Applications International Corporation; and Gregory Treverton, Rand Corporation, also deserve mention. Mark kindly agreed to write the foreword and offered valuable suggestions. His quip about really good analysts set up much of the arguments presented herein. Both Jeff and Greg provided critical comments and suggestions late in the game. They also kindly agreed to write commentaries on the paper. Russell Swenson, Director of NDIC’s Center for Strategic Intelligence Research and NDIC Press editor, did what a really good editor is supposed to do: His input and recommendations ensured that the arguments that follow were focused and strong. The Center’s William Spracher also provided valuable technical editing services. Colleague Lisa Krizan provided a final review of the text that proved extremely valuable. This paper would not have been published if not for their dedicated assistance to the author. Acknowledgment is due to four other people. Peter and Mary Moore encouraged their son to be inquisitive and to always look beyond the obvious for answers. Anthropologist John McDaniel first introduced the author – then a student at Washington and Lee University – to an analogy for intelligence sensemaking: science-based field archaeology. Finally, Elizabeth Moore provided tremendous encouragement. She also graciously tolerated the many nights and weekends spent writing, listening to, and commenting on, ideas as – xx – they occurred, sharpening the resulting arguments. It is to these people that this work is dedicated. While the author was privileged to receive assistance of others, this remains his work and any errors are his. The author may be reached via electronic mail at [email protected] – xxi – – – CRITICAL THINKING AND INTELLIGENCE ANALYSIS D AviD T. M oore Examples included in this paper were selected to illustrate points raised by the author. No interest by the National Security Agency, the Department of Defense, or any other agency of the U.S. Government should be inferred from their inclusion. A bsTr AcT Analysts and analysts alone create intelligence. Although technological marvels assist analysts by cataloguing and presenting data, information and evidence in new ways, they do not do analysis. To be most effective, analysts need an overarching, reflective framework to add structured reasoning to sound, intuitive thinking. “Critical thinking” provides such a framework and goes further, positively influencing the entire intelligence analysis process. Analysts who adopt critical thinking stand to improve their analyses. This paper defines critical thinking in the context of intelligence analysis, explains how it influences the entire intelligence process, explores how it toughens the art of intelligence analysis, suggests how it may be taught, and deduces how analysts can be persuaded to adopt this habit. – – D efiNiTioNs Thinking – or reasoning – involves objectively connecting present beliefs with evidence in order to believe something else. Critical Thinking is a deliberate meta-cognitive (thinking about thinking) and cognitive (thinking) act whereby a person reflects on the quality of the reasoning process simultaneously while reasoning to a conclusion. The thinker has two equally important goals: coming to a solution and improving the way she or he reasons. Intelligence is a specialized form of knowledge, an activity, and an organization. As knowledge, intelligence informs leaders, uniquely aiding their judgment and decision-making. As an activity, it is the means by which data and information are collected, their relevance to an issue established, interpreted to determine likely outcomes, and disseminated to individuals and organizations who can make use of it, otherwise known as “consumers of intelligence.” An intelligence organization directs and manages these activities to create such knowledge as effectively as possible. – – CRITICAL THINKING AND INTELLIGENCE ANALYSIS iNTroDucTioN : H ow D o P eoPle r eAsoN ? The best analytical tool remains a really good analyst. —Mark Lowenthal, Former Assistant Director of Central Intelligence for Analysis and Production To create intelligence requires transformations resulting from an intellectual endeavor that sorts the “significant from [the] insignificant, assessing them severally and jointly, and arriving at a conclusion by the exercise of judgment: part induction, part deduction,” and part abduction. That endeavor is known as thinking, or “that operation in which present facts suggest other facts (or truths).” Thinking – or as it is sometimes known, reasoning – creates an “objective connection” between our present beliefs and “the ground, warrant, [or] evidence, for believing something else.” These three reasoning processes trace the development of analytic beliefs along different paths. Whereas inductive reasoning reveals “that something is probably true,” deductive reasoning demonstrates “that something is necessarily true.” However, both are limited: William Millward, “Life in and out of Hut ,” in F. H. Hinsley and Alan Stripp, Codebreakers: The Inside Story of Bletchley Park (Oxford, UK : Oxford University Press, 99), . The author adds “abduction” for reasons that will shortly become evident. John Dewey, How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process (New York, NY : D.C. Heath and Company, 90), . Cited hereafter as Dewey, How We Think. Dewey, How We Think, . David A. Schum, “Species of Abductive Reasoning in Fact Investigation in Law,” Cardozo Law Review , nos. –, July 00 , , emphasis added. Cited – – inductive reasoning leads to multiple, equally likely solutions and deductive reasoning is subject to deception. Therefore, a third aid to judgment, abductive reasoning, showing “that something is plausibly true,” can offset the limitations of the others. While analysts who employ all three guides to sound judgment stand to be the most persuasive, fallacious reasoning or mischaracterization of rules, cases, or results in any of the three can affect reasoning using the others. Inductive reasoning, moving from the specific to the general, suggests many possible outcomes, or the range of what adversaries may do in the future. However, inductive reasoning lacks a means to distinguish among each outcome – all are possible. An analyst has no way of knowing whether a solution is correct. Deductive reasoning on the other hand, moving from the general to the specific, addresses questions about adversarial behavior and inten – tions. Deductive reasoning becomes essential for warning. Based on past perceptions, certain facts indicate specific outcomes. If, for exam – ple, troops are deployed to the border, communications are increased, and leadership is in defensive bunkers, then war is imminent. However, if leadership remains in the public eye then these preparations indicate that an exercise is imminent. Abductive reasoning reveals plausible outcomes to the intelligence analyst. When an adversary’s actions defy accurate interpretation through existing paradigms, abductive reasoning generates novel means of explanation. In the case of intelligence warning, an abductive process presents policy–making intelligence consumers with an “assessment of probabilities.” Although abduction provides no guarantee that the analyst has chosen the correct hypothesis, the probative force of the ac – companying argument indicates that the most likely hypothesis is known and that elusive, actionable intelligence is on tap. hereafter as Schum, “Species.” American mathematician, logician, semiotician, and philosopher Charles Sanders Peirce (9–9) developed the concept of abductive reasoning. See Joseph Brent, Charles Sanders Peirce: A Life (Bloomington, IN: Indiana University Press, 99). Schum, “Species,” . – – Figure models analysts’ thinking about HUMINT received during the summer of 9 regarding reports of foreigners in Cuba. An initial (fallacious) conclusion derived from inductive reasoning apparently was reapplied through deductive reasoning as explained in the case study incorporated in this paper. 9 Something else was needed – but not considered – to challenge the initial conclusion that “All HUMINT from Cuba was false.” Figure 1: A Comparison of Inductive, Deductive, and Abductive Reasoning Sources: Adapted from Thomas A. Sebeok, “One, Two, Three Spells UBER – TY ,” in Umberto Eco and Thomas A. Sebeok, The Sign of Three: Dupin, Hol – mes, Pierce (Bloomington, IN: Indiana University Press, 9), ; and James H. Hansen, “Soviet Deception in the Cuban Missile Crisis,” Studies in Intelligence, , no. (00), . If analysts can model an adversary’s intentions based on observable actions, adversaries also can deduce how those analysts will have 9 While there is no evidence it was applied in the Cuban case, abductive reasoning is included in the figure for purposes of illustration. – – moved toward these conclusions. Under these circumstances, the adversary then can engage in deceptive practices that can lead an intelligence service to misinterpret that adversary’s intentions. 0 For example, the Indian government deceived the U.S. Intelligence Community before the former’s 99 nuclear test. Through a 99 demarche from the U.S. government, the Indian government knew what indicators the U.S. sought and was able to obscure them until after the test. Then, the Indian government boasted of its success. Abduction forces a close consideration of the evidence at hand. Given competing estimates – such as “war is imminent” or “an exercise will occur” – the body of facts and inferences must be examined to determine “whether there are facts which falsify one of the estimates.” Only then can an assessment of accuracy be determined. Even then, the selected estimate remains only the “most likely.” Despite their individual limitations, induction, deduction, and abduction taken together offer a means of thoroughly examining evidence in order to arrive at accurate intelligence conclusions. However, as becomes obvious from studying figure 1, in order to be successful, intelligence analysis requires something more: an overarching framework is needed to ensure reasoning relies on valid assertions, is not fallacious, and is self–consciously objective. Critical thinking provides that framework by ensuring that each form of reasoning is appropriately used. Critical thinking extends to the entire intelligence analysis process. The claim here is that analysts who 10 Isaac Ben-Israel explores the duality of estimation in considerable detail. See “Philosophy and Methodology of Intelligence: The Logic of the Estimate Process, Intelligence and National Security , no. (October 99), 0–. Cited hereafter as Ben-Israel, “Logic of the Estimate Process.” Paul J. Raasa, “The Denial and Deception Challenge to Intelligence,” in Roy Godson and James J. Wirtz, Strategic Denial and Deception (New Brunswick, NJ: Transaction Publishers, 00), , –. Ben-Israel, “Logic of the Estimate Process,” . Cynthia M. Grabo, Anticipating Surprise: Analysis for Strategic Warning (Washington, DC : National Defense Intelligence College, 00), . – – become better critical thinkers will improve their analyses, helping to lessen the likelihood of intelligence failures. In the example above, critically thinking analysts would have questioned the generalizing of their initial conclusion by asking, “Was all the HUMINT from Cuba really false?” To avoid the tautological problem of equating quality analysis with the concept of “critical thinking” requires a careful assessment of cognition in the intelligence environment. The underlying question is “How can the intelligence analysts to whom Lowenthal’s epigraph refers be ‘really good’?” – – w HAT i s c riTicAl T HiNkiNg ? Defining Critical Thinking Defining critical thinking is a first step to understanding how it contributes to intelligence analysis. Richard Paul and Linda Elder, of the Foundation for Critical Thinking, consider it to be that mode of thinking – about any subject, content, or problem – in which the [solitary] thinker improves the quality of his or her thinking by skillfully taking charge of the structures inherent in thinking and imposing intellectual standards upon them. In other words, critical thinking is both a deliberate meta-cognitive (thinking about thinking) and cognitive (thinking) act whereby a person reflects on the quality of the reasoning process simultaneously while reasoning to a conclusion. The thinker has two equally important goals: improving the way she or he reasons and coming to a correct solution. To accomplish these goals, the reasoner requires assisting structures. Paul and Elder define eight elements of reasoning, shown in figure 2. These elements lead thinkers to ask focused questions about the topic being considered and the thinking process itself. Paul and Elder assert that whenever people reason, they do so for a purpose. This reasoning exists within a point of view and is shaped by both conscious and unconscious assumptions. Reasoning involves the creation of inferences based on conceptual frameworks about reality. These inferences are generated as people consider evidence necessary to answer questions or solve problems. Further, reasoning leads to decision points with implications – things that might happen Richard Paul and Linda Elder, The Miniature Guide to Critical Thinking Concepts and Tools , th Edition (Dillon Beach, CA : The Foundation for Critical Thinking, 00), . Cited hereafter as Paul and Elder, Concepts and Tools. 15 Exactly what is the “correct solution” is issue and context dependent. What is important here is that the process of reasoning is enhanced. – 9 – – and consequences – things that do happen once the decision has been made. Finally, Paul and Elder insist that “learning to analyze thinking requires practice in identifying the structures in use.” Figure 2: Elements of Thought Source: Derived from Richard Paul and Linda Elder, The Miniature Guide to Critical Thinking Concepts and Tools, th Edition (Dillon Beach, CA : The Foun – dation for Critical Thinking, 00), . Linda Elder and Richard Paul, The Foundations of Analytic Thinking: How to Take Thinking Apart and What to Look for When You Do (Dillon Beach, CA : The Foundation for Critical Thinking, 00), . Cited hereafter as Elder and Paul, Analytic Thinking . Paul and Elder, Concepts and Tools , . – 0 – Standards for Critical Thinking Figure 3: A Continuum of Intellectual Standards Source: Derived from Paul and Elder, Concepts and Tools, 9. Paul and Elder also establish a set of intellectual standards that offer criteria for assessing the level and quality of thinking. These form a continuum as shown in figure 3. Active – or Socratic – questioning in a social setting becomes the means of critically exploring a topic or discipline as well as assessing the thinking on that topic. Questions assessing the issue delve into purpose, assumptions, inferences, as well as points of view on a topic. Paul and Elder’s Elder and Paul, Analytic Thinking. –. Paul and Elder, Concepts and Tools, –9. – – approach ensures that a topic or issue is thoroughly, logically, and objectively developed. Their continuum provides qualitative metrics to inform the thinker and others who may evaluate the object of the thinking along with the process employed. Each standard is related to but independent of the others. The object of consideration might be vaguely or clearly presented. But it also might be inaccurate. An example of this would be “the world is flat.” This clear statement of reality is false. The reasoning or its object might be clearly and accurately presented but remain imprecise, the level of detail required depending on the issue itself. Precision becomes important in assessing the location of an adversary or a threat as in “terrorists placed the improvised explosive device ( IED ) near the highway.” It is possible to create clear, accurate, and precise answers that are irrelevant. If one is considering the IED along the highway but the issue is how best to fly the troops in question from one location to another then the answer is irrelevant. Similarly, topics and their accompanying reasoning need to consider both the complexities involved (depth) and other points of view (breadth). 9 Such reasoning becomes essential when systems must be assessed because in systems “chains of consequences extend over time and many areas: The effects of action are always multiple.” 0 Intelligence often must assess such systems – terrorist networks are but one example. Additionally, as sociologist Emile Durkheim observes, the combinations of elements related to an issue “combine and thereby produce, by the fact of their combination, new phenomena.” When the correct phenomena or questions – and related alternative explanations – are not fully considered, intelligence failures occur. For instance, the Senate Select Committee on Intelligence 9 Elder and Paul, Analytic Thinking, . 0 Robert Jervis, System Effects: Complexity in Political and Social Life (Princeton, NJ : Princeton University Press, 99 ), 0. Cited hereafter as Jervis, System Effects. Emile Durkheim, The Rules of Sociological Method (Glencoe, IL: Free Press, 1938), xlvii. – – found such reasoning to be among the causes of the Intelligence Community’s failure to assess accurately Iraq’s alleged programs to develop weapons of mass destruction ( WMD ). The senators noted that “Intelligence Community managers…did not encourage analysts to challenge their assumptions, fully consider alternative arguments, accurately characterize the intelligence reporting, or counsel analysts who lost their objectivity.” Logic and significance also come into play. If the evidence presented does not imply the conclusions then the results are illogical. Focusing on appropriate evidence maintains significance. For example, evidence that terrorists who place an IED also have Stinger surface–to–air missiles bears on how troops can move across hostile territory. Fairness deals with the agendas – hidden and expressed – of the thinker, her collaborative peers, and her customers. Knowing who has a stake in an issue as well as the stakes themselves helps ensure that issues are fairly reasoned and presented. Such considerations also reflect the analyst’s own biases or opinions. In considering this point, Paul and Elder offer the evaluative question, “Am I misrepresenting a view with which I disagree?” Finally, while getting to the right-hand side of the spectrum on each of these standards in figure 3 is highly desirable, the standard of thinking on an issue will vary based on the skills of the thinkers and the issues under scrutiny. Assessing the resulting shortfalls in thinking reveals gaps that can be corrected. The continuum therefore provides a detailed assessment by which thinking on issues can be improved. Senate Select Committee On Intelligence, Report on the U.S. Intelligence Community’s Prewar Intelligence Assessments on Iraq, United States Senate, 0 th Congress, July 00, . Cited hereafter as SSCI, Iraq. Elder and Paul, Analytic Thinking,, . – – Skill-Based Definitions Other approaches to defining critical thinking focus on the specific skills. For example, Diane Halpern considers that [critical] thinking is the use of those cognitive skills or strategies that increase the probability of a desirable outcome. It is… thinking that is purposeful, reasoned, and goal directed – the kind of thinking involved in solving problems, formulating inferences, calculating likelihoods, and making decisions, when the thinker is using skills that are thoughtful and effective to the particular context and type of thinking task. Additional advocates of skills-based critical thinking include Edward Glaser and Alec Fisher, and their sets of critical thinking skills include a number of elements common to those identified by Halpern. The overlapping competencies of critical thinkers as advanced by Paul and Elder and these three other proponents are summarized in table . The comparison reveals the completeness of the Paul and Elder model. These competencies assist intelligence analysts who contribute to the solution of threats to national security by ensuring the formulation of sound inferences about adversarial capabilities and intentions. Resulting findings are disseminated in reports that are often referred to as “assessments.” At their best, they offer choices to decision- making consumers, as well as a clear outline of the implications of those choices. Diane Halpern, Thought and Knowledge: An Introduction to Critical Thinking, th Edition (Mahwah, NJ: Lawrence Erlbaum Associates, Publishers, 00), . Cited hereafter as Halpern, Thought and Knowledge. The term “assessment” describes the result of an intelligence production process but is not universally used within the Intelligence Community. Assessments are created at the Central Intelligence Agency and at the Defense Intelligence Agency. Other names for the reports that analysts produce include “products” at the National Security Agency ( NSA) and “estimates” at the National Intelligence Council (NIC). Regardless of how they are named, they record and disseminate the results of analysts’ work. – – Competencies of Critical Thinkers Paul and Elder Fisher Glaser Halpern Recognize problems or questions and find effective means of solution 3 3 3 Engage in meta-cognitive activities that identify assump – tions, biases, and performance as solutions are developed 3 3 3 3 Interpret data, appraise evidence, and evaluate state – ments in order to recognize logical relationships between propositions 3 3 3 Infer warranted conclusions and generalizations from evidence 3 3 3 3 Test generalizations and conclusions by seeking out contradictory evidence that enables them to judge the credibility of claims 3 3 3 3 Convey sound, well-reasoned arguments 3 3 3 3 Focus on the process of rea – soning with the intention of improving the process 3 Table 1: A Comparison of Different Sets of Critical Thinker’s Competencies Sources: Compiled by author from Linda Elder and Richard Paul, The Founda – tions of Analytic Thinking: How to Take Thinking Apart and What to Look for When You Do (Dillon Beach, CA: The Foundation for Critical Thinking, 00); Diane Halpern, Thought and Knowledge: An Introduction to Critical Thinking, th Edition (Mahwah, NJ: Lawrence Erlbaum Associates, Publishers, 00), ; Edward M. Glaser, An Experiment in the Development of Critical Thinking (New York, NY: AMS Press, 9), ; and Alec Fisher, Critical Thinking: An Introduction (Cambridge, UK: Cambridge University Press, 00), . – – A Disposition to Think Critically It is not enough merely to know the skills needed for critical thinking. To be successful, analysts as critical thinkers also need “certain attitudes, dispositions, passions, [and] traits of mind.” Actively thinking critically hones the skills; practice yields proficiency. But in order to gain mastery, willingness to reason in this manner becomes essential. The importance of this disposition to critically think cannot be over-emphasized. According to Peter A. Facione, Noreen C. Facione, and Carol A. Giancarlo, “Empirical studies…at multiple sites indicate that for all practical purposes” both critical thinking skills and the disposition to critically think are essential. There are a number of characteristics associated with the disposition to think critically. The Faciones and Giancarlo identify “seven characterological attributes or habits of mind… truth-seeking, open-mindedness, analyticity, systematicity, critical thinking, self confidence, inquisitiveness, and maturity of judgment.” According to Richard Paul and Gerald Nosich, the characteristics of critical thinkers include think ing independently, exercising fair-mindedness, developing insight into egocentricity and sociocentricity, developing intellectual humility and suspending judgment, developing intellectual courage, developing intellectual good faith and integrity, developing intellectual perseverance, Richard W. Paul and Gerald Nosich, A Model for the National Assessment of Higher Order Thinking, (Dillon Beach, CA : Foundation for Critical Thinking, n.d.), 0. Cited hereafter as Paul and Nosich, National Assessment. Peter A. Facione, Noreen C. Facione, and Carol A. Giancarlo, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill,” Infor mal Logic 0, no. (000): –. Reprinted by Insight Assessment, URL: < http://www.insightassessment.com/pdf_ files/J_Infrml_Ppr%20_2000%20–%20Disp%20&%20Skls.PDF>, last accessed March 00. Cited hereafter as Facione, “Disposition.” Peter A. Facione, Noreen C. Facione, Carol A. F. Giancarlo, Professional Judgment and the Disposition Toward Critical Thinking (Milbrae, CA : California Academic Press, 2002), URL: < http://www.calpress.com/pdf_files/Prof_jmn.pdf>, last accessed March 00. – – developing confidence in reason, exploring thoughts underlying feelings and feelings underlying thoughts, developing intellectual curiosity. 9 Both sets closely match the characteristics of successful intelligence analysts identified by Lisa Krizan and the author in their work on intelligence analysts’ core competencies. Krizan and the author observe that successful intelligence analysts are insatiably curious. Fascinated by puzzles, their high levels of self-motivation lead them to observe and read voraciously, and to take fair-minded and varied perspectives. This helps them to make the creative connections necessary for solving the hardest intelligence problems. Finally, the emotional tensions created by problems, and the cathartic release at their solution, powerfully motivate analysts. 0 In addition, emotions play a significant part in the process of critical thinking. As adult learning expert Stephen Brookfield observes, “[emotive] aspects – feelings, emotional responses, intuitions, sensing – are central to critical thinking.” He argues that the consideration of alternatives “to one’s current ways of thinking” characterizes the approach. Indeed, the consideration of these alternatives requires creativity, which he considers a non–rational form of thought. Creative reasoning generates hypotheses; it has contributed significantly to the development of intellectual, scientific, and technological knowledge. 9 Paul and Nosich, National Assessment . 0 David Moore and Lisa Krizan, “Intelligence Analysis: Does NSA Have What it Takes,” reprint NSA Center for Cryptologic History, Cryptologic Quarterly 0, nos. / (Summer/Fall 00), –. Cited hereafter as Moore and Krizan, “NSA ,” 31 Stephen D. Brookfield, Developing Critical Thinkers: Challenging Adults to Explore Alternative Ways of Thinking and Acting (San Francisco, CA : Jossey–Bass Publishers, 1987), 12. Cited hereafter as Brookfield, Developing Critical Thinkers. 32 Brookfield, Developing Critical Thinkers, . 33 Brookfield, Developing Critical Thinkers, . – – The Role of Questions Greg Treverton observes that in intelligence the right questions rarely get asked. Yet, critical thinking offers a means by which at least appropriate questions may be raised. The combination of critical thinking skills and the disposition to think critically focuses and directs inductive, deductive, and abductive reasoning to solve problems. This is an interrogative paradigm. Critical thinking involves questioning that forces broader consideration of issues and problems as well as the various means of resolving or answering them. Such questioning happens at both the individual and collective levels as shown in figure 4. Figure 4: Levels of Questioning: Critical Thinking in a Social Setting Source: Created by the author with contributions by Russell G. Swenson. Questioning is either personal or impersonal and ranges from the musings of the individual critical thinker to that of a global discourse. As an individual ponders problems and issues, she abduces the most likely explanations. Moving up a level, questioning becomes a Greg Treverton, conversation with the author, April 00. – – hierarchical dialogue within dyads or larger assemblies. A critically thinking questioner ferrets out answers from others within the group. The process generates new questions that are answered in turn – and in turn, raise further questions. At the level of a community debate, critical thinkers explore what is possible politically and within the community’s purview. Ideally, the relationship between thinkers is peer-to-peer. Finally, questions addressed within the context of a global discourse take into account the entire biological landscape (noösphere). Such questioning produces new and creative thinking, analogous to switching from Newtonian to quantum physics. In this analogy, an atomistic approach to analysis gives way to the associative impulse of synthesis as structured thoughts facilitate a leap to new questions, ideas, and possibilities. This overarching framework of structured reasoning frames questions to help analysts decide the best means or best combination of means that are suited to solving specific intelligence problems. The questions serve to alert an analyst to the notion that she might be deceived, or that she needs to employ some other means of reasoning to determine which of several alternative outcomes is most likely. These questions provide a formal means by which an analyst confronts her biases about likely outcomes. Therefore, in the context of intelligence analysis, critical thinking becomes one of the most – if not the most – important skills of the analyst. Facing symmetric and asymmetric threats, analysts have a potent and powerful tool to facilitate their asking the right questions in the process of improving their understanding of the problem at hand as well as their own reasoning about that problem. Pseudo-Critical Thinking It is important to understand what critical thinking is not . Critical thinking, as has been noted, focuses on both the process and the results of reasoning. However, the term is also used in reference to reasoning that is not reflective. The application of formal logic is sometimes (incorrectly) equated to critical thinking. So too are 35 Wikipedia defines the noösphere as the domain of human thought. Wikipedia, entry under “noösphere,” accessed February 00. – 9 – problem solving and structured methods of analysis. Developers of school curricula and other exponents of “sound thinking” often lay claim to the mantel of critical thinking but are really leveraging their coverage of logic or problem solving to capitalize on an uncritical understanding of what critical thinking is. Problem solving, for example, focuses on answers and not on the process by which an answer is obtained. Additionally, logic or problem solving, being goal oriented, offer little means by which a person can improve the process of her thinking. The following problem is a typical example of pseudo–critical thinking: Two cyclists race along a straight course. The faster of the pair maintains an average speed of 0 mph. The slower cyclist averages miles per hour. When the race ends, the judges announce that the faster cyclist crossed the finish line one hour before the slower racer. How many miles long was the racing course? This example (and others like it) focuses on the answer, provides no guidance on the process, and ignores any improvement in reasoning skills. Solvers either figure it out through trial and error or by following a rule-based strategy learned by rote. Therefore it fails to be critical thinking. Unless the process, and the means, to improve a person’s reasoning are emphasized, then at best such examples teach structured problem solving. At worst, they deal merely in criticism; they fail to assist people to learn to reason better. Central to critical thinking are the twin foci on the way in which a person is reasoning and the goal of improving that process. One could easily infer that misconceptions about critical thinking could be associated with what is taught as critical thinking in American educational institutions. Michael A. DiSpezio, Classic Critical Thinking Puzzles (New York, NY: Main Street, 00), . The answer, by the way, is 0 miles (page ). Michael R. LeGault reaches this same conclusion in Think: Why Crucial Decisions Can’t Be Made in the Blink of an Eye (New York, NY : Threshold Editions, 00), . Cited hereafter as LeGault, Think. – 0 – w HAT c AN b e l eAr NeD froM THe P AsT ? T HiNkiNg c riTic Ally AbouT c ubA Examining past intelligence successes and failures in light of new developments provides a means to reassess what happened and how outcomes could have been different. The case of the Soviet missiles in Cuba in the summer and fall of 1962 provides a means of examining how critical thinking and structured methodologies could have made a difference. Deploying the Missiles During the summer of 9, CIA analysts received a spate of potentially alarming reports about Russians being seen in Cuba. The reports, however, were only part of a stream of similar, “farfetched tales of African troops with rings in their noses, lurking Mongolians, and even Chinese troops” on the island. Most or all of these reports were discounted by analysts who were inured to spurious reports of Soviet equipment secreted away in caves. 9 James Hansen – who worked in both the Central Intelligence Agency ( CIA ) and Defense Intelligence Agency ( DIA ) – posits that the U.S. Intelligence and Policy Communities were the victims of a concerted Soviet campaign of denial and deception that masked the deployment of Soviet forces and missiles into Cuba. 0 The deception campaign even included “accurate information about the deployment [leaked] so as to mask it.” As Raymond Garthoff relates, “there were literally thousands of reports of missiles in Cuba in the period before any missiles were actually brought there.” James H. Hansen, “Soviet Deception in the Cuban Missile Crisis,” Studies in Intelligence , no. (00), . Cited hereafter as Hansen, “Soviet Deception.” 9 Hansen, “Soviet Deception,” . 0 Hansen, “Soviet Deception,” 9–. Domingo Amuchastegui, “Cuban Intelligence and the October Crisis,” in James G. Blight and David A. Welch, Eds., Intelligence and the Cuban Missile Crisis (London, UK : Frank Cass, 99), 0. Raymond L. Garthoff, “US Intelligence in the Cuban Missile Crisis,” in – – In fact, the Soviets were able to deploy more than the offensive nuclear missiles that became the centerpiece of the subsequent crisis with the United States. While U.S. analysts and policymakers knew of the conventional weapons, they were blind to the presence of SS- Medium Range Ballistic Missiles ( MRBM ) and SS- Intermediate Range Ballistic Missiles prior to the U-2 overflights of 14 October. Additionally, it seems they never discovered the presence of approximately 100 tactical nuclear weapons deployed on the island. There is debate whether U.S. Intelligence also underestimated the number of Soviet troops deployed to Cuba. Garthoff reports that one CIA unit concluded there were between ,000 and 0,000 Soviet troops in Cuba (the actual number was about 42,000) but the official estimate was between ,00 and ,000 prior to the crisis. The Soviets employed an elaborate campaign of denial and deception that took advantage of American points of view about the likelihood of Soviet weapons being located in Cuba. As Robert Jervis makes clear, the U.S. did not expect the Russians to put missiles into Cuba or Japan to attack Pearl Harbor because American officials knew that the U.S. would thwart these measures if they were taken. These judgments were correct, but because other countries saw the world and the U.S. less accurately, the American predictions were inaccurate. It could have been worse. As Gil Merom writes, the Soviets might James G. Blight and David A. Welch, Eds., Intelligence and the Cuban Missile Crisis (London, UK : Frank Cass, 99), . Emphasis in original. Cited hereafter as Garthoff, “US Intelligence.” Garthoff, “US Intelligence,” 9. Garthoff, “US Intelligence,” , . U.S. Intelligence never did reach a correct conclusion. The closest they got was an estimate of ,000 troops in early 9 (Garthoff, “U.S. Intelligence,” ). Robert Jervis, System Effects: Complexity in Political and Social Life (Princeton, NJ: Princeton University Press, 99 ), . Jervis draws on the work of Klaus Knorr. See Klaus Knorr, “Failures in National Intelligence Estimates: The Case of the Cuban Missiles,” World Politics (April 9): –. – – have completed the bases and “threatened major U.S. cities with extinction.” The threat is brought home by figure 5. What caused U.S. analysts to get it wrong? Apparently the U.S. analysts concluded that because some of the HUMINT evidence was ludicrous, all of it was. This inductive conclusion then led them to discount further evidence. For example, having concluded that all the previously considered Cuban HUMINT was false, each new piece, since it came from Cuba, also had to be false. Thus, a false inductive conclusion led subsequently to false deductive conclusions. It does not appear that abductive reasoning strategies were ever employed. Figure 5: Respective Ranges of Soviet SS -4 MRBM and SS -5 IRBM Missiles Source: Derived from Hansen, “Soviet Deception,” 9. Gil Merom, “The 9 Cuban Intelligence Estimate: A Methodological Perspective,” Intelligence and National Security , no. (Autumn 999), 9. Cited hereafter as Merom, “Estimate.” 47 Inductive and deductive reasoning were illustrated in figure 1. – – It should be noted that the U.S. Intelligence Community was not blind to the possibility that the Soviets might conduct a military buildup in Cuba. In fact, there were two theories being debated: One, that the Soviets would emplace defensive weapons, and the other that they would emplace offensive weapons. Senior analysts in the Intelligence Community held the former theory while John McCone, then Director of Central Intelligence, favored the latter. Part of McCone’s reasoning had to do with the cost and utility of the installation of SA- air defense missiles. McCone apparently reasoned that the purpose of installing such expensive missiles had to be greater than merely denying the United States overflight capabilities (the SA- could shoot down – as Francis Gary Powers discovered – a U- ). This led McCone to come “up with an answer that no one wanted to hear: the SA-s were on the island to deny the United States the capability to see the construction of offensive missile installations.” 9 Unfortunately, McCone was unable to dissuade the majority from their point of view. This may have stemmed, as James Blight and David Welch write, from the realization that while McCone’s inference seemed reasonable when viewed in hindsight, in foresight it was faulty because it failed to sufficiently cover the alternatives. 0 Blight and Welch observe that the Soviets had also deployed SA- missiles to Egypt, Syria, and Indonesia, and in none of those cases had they also deployed strategic nuclear weapons. Indeed, the US intelligence community [sic] expected that the Soviet Union Thomas R. Johnson and David A. Hatch, NSA and the Cuban Missile Crisis (Fort Meade, MD : National Security Agency Center for Cryptologic History, 1998), URL: , last accessed April 00. Cited hereafter as Johnson and Hatch, NSA. 9 Johnson and Hatch, NSA . 0 James G. Blight and David A. Welch, Eds., Intelligence and the Cuban Missile Crisis (London, UK : Frank Cass, 99), . Cited hereafter as Blight and Welch, Intelligence and the Cuban Missile Crisis. – – would deploy SA- missiles to Cuba, precisely because it had done so elsewhere. Even though history proved McCone to be correct (for the wrong reasons), the defensive weapons theory predominated. The Soviets took advantage of the American beliefs and faulty reasoning. Capitalizing on the idea that it is easier to lead a target astray than to try to change his mind, they successfully placed the nuclear missiles in Cuba. Heuer observes that [deceptions] that follow this principle seldom fail, for the odds are then strongly in favor of the deceiver. The human capacity to rationalize contradictory evidence is easily sufficient to outweigh the pernicious effects of security leaks and uncontrolled channels of information that planners of deception…might compromise their efforts. Assessing the Implications Exactly what happened in the Cuban case and does it apply to contemporary issues? Roberta Wohlstetter notes in retrospect, “We would like to know not only how we felt, but what we did and what we might have done, and in particular what we knew or what we could have known.” Wohlstetter’s musings lead to two key questions for analysts: How could this successful deception campaign have been Blight and Welch, Intelligence and the Cuban Missile Crisis , . Merom, “Estimate,” . Richards J. Heuer, Jr., “Strategic Deception and Counterdeception: A Cognitive Process Approach,” International Studies Quarterly , no. (June 9), 00. Cited hereafter as Heuer, “Strategic Deception.” It is interesting to speculate whether the Soviets had a feedback channel that informed them of the predominant theory. Heuer, “Strategic Deception,” 00. Roberta Wohlstetter, “Cuba and Pearl Harbor: Hindsight and Foresight,” Foreign Affairs , no. (July 9), 9. Cited hereafter as Wohlstetter, “Cuba.” – – thwarted? What can be learned to advise analysts about current adversarial denial and deception? Simply looking at additional evidence is sometimes promoted as a means of detecting denial and deception by adversaries. In the Cuban case, however, analysts had already processed a superabundance of evidence. Wohlstetter suggests that such riches can be “embarrassing.” This is because even as “signals” point to the action or to an adversary’s intention to undertake it, “noise” or a background of irrelevant or inconsistent signals, [and] signs pointing in the wrong directions…tend always to obscure the signs pointing the right way. HUMINT assets overwhelmed analysts’ abilities to distinguish signals from noise. Heuer and Hansen agree that once “locked in,” analysts resist changing their minds. More evidence alone fails to change an analyst’s mind because “[new] information is assimilated to existing images.” Yet, analysts stand to benefit from changing their opinion in the face of disconfirming evidence. The limitations of hindsight analysis notwithstanding, if analysts had employed critical reasoning in 9 they could have detected the Soviet maskirovka in Cuba . For example, by applying Paul and Elder’s model of critical thinking, analysts would have had a means to question their assumptions and points of view, and subsequently might have questioned the purpose of the apparent “noise” they were discounting as well as their assumptions about the prevalent point of view that the Soviets would not place missiles on the island because we would react. Gil Merom suggests that some appropriate questions would have included: “How aware were the analysts of their own assumptions? Were these assumptions sensible and was it sensible to adhere to them?” 9 Other considerations that critical Wohlstetter, “Cuba,” 9. Wohlstetter, “Cuba,” 9. Heuer, Psycholog y, 0–. 9 Merom, “Estimate,” . – – thinking about the crisis might have raised are included in table . Applying Paul and Elder’s model reveals other considerations that conceivably would have led to an earlier detection of the likelihood of the deployed weapons being offensive. Would this approach have allowed detection of what the Soviets were doing in Cuba before mid-October 9? One could argue counterfactually that it might have enhanced analysis of collection from other sources. 0 SIGINT assets capable of collecting Russian communications were in place near the island. Imagery assets might have been tasked differently prior to October. Sources less subject to generating noise would have introduced inconsistencies to an analysis of competing hypotheses that was solely HUMINT -based. Between Dogmatism and Refutation A systematic approach to directed collection also would have moved analysts toward a disconfirmatory analytic approach. By contrast, a confirmatory approach, often characterized as confirmation bias, leads analysts to accept what they set out to confirm. A disconfirmatory approach aims to refute alternative 60 Counterfactual reasoning explores differing outcomes arising from alternative causes. Counterfactuals are commonly used in whenever alternate hypotheses are explored. They are essential for both post-mortem reviews and futures scenario exercises. See Philip E. Tetlock, and Aaron Belkin, Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives (Princeton, NJ: Princeton University Press, 99), –. In the same volume, Richard Ned Lebow and Janice Gross Stein conduct a counterfactual review of the Cuban missile crisis. See Richard Ned Lebow, and Janice Gross Stein, “Back to the Past: Counterfactuals and the Cuban Missile Crisis,” in Philip E. Tetlock, and Aaron Belkin, Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives (Princeton, NJ: Princeton University Press, 99), 9–. Johnson and Hatch, NSA . According to the authors, SIGINT played a key role in determining the operational status of the SA – air defense missiles. However, SIGINT failed to detect the delivery and installation of the nuclear missiles. 62 Wikipedia defines confirmation bias as “a type of cognitive bias toward confirmation of the hypothesis under study.” (Wikipedia, entry under “confirmation bias”). – – Purpose · Determine what is going on in Cuba as a part of the U.S. “war” against Castro and the Cuban communists. Key Questions · If this really is a Soviet military buildup, what kinds of weapons are being deployed: offensive weapons or defen – sive weapons? · Why is an expensive missile system like the SA–2 being installed in Cuba? · What is the SA–2 really protecting? · Does a crisis exist? Evidentiary Considerations · If defensive weapons are being deployed, what evidence should be observed? · If offensive weapons are being deployed, what evidence should be observed? · What is not being seen? Inferences · What is inferred from the observed and collected evidence? Assumptions · What is being assumed about the evidence? · What is being assumed about the sources of the evidence? · What is being assumed about why the Soviets would deploy weapons to Cuba? Concepts · How does human perception affect the analysis? · How reliable are the sources of evidence? (Could also be an assumption.) Implications and Consequences · If conclusions are incorrect about the Soviet build up, what might occur next? · If conclusions are correct about the Soviet build up, what can be expected? · If conclusions are incorrect about offensive weapons be – ing deployed, what happens next? · If conclusions are incorrect about defensive weapons be – ing deployed, what happens next? Points of View · What other points of view about what is going on in Cuba exist? Table 2: Applying Paul and Elder’s Critical Thinking Model to the Situation in Cuba, August–September 1962. Source: Developed by author. – – hypotheses. Whatever was left at the end of the process would be the most likely explanation. Would this guarantee a correct “answer”? Ben-Israel concludes that although the answer has to be “no,” the approach does allow for narrowing “the margin of error.” It does so by moving the analyst away from a “pole of dogmatic tenacity” toward a “pole of refutation” (summarized in figure 6). Figure 6: A Place for Analysis between Dogmatism and Criticism Source: Derived from Ben-Israel, “Methodology of Intelligence,” 9. It should be noted that operating too closely to a pole of refutation can also create problems. For example, analysts might be unable to distinguish between real threats and false positives; they might react to each one regardless of its validity. In the Cuban case, this problem would have manifested itself in goading continual U.S. military reactions to every perceived Soviet and Cuban threat. Here again, Ben-Israel, “Logic of the Estimate Process,” 9 Ben-Israel, “Logic of the Estimate Process,” 9. – 9 – critical thinking could moderate this Cassandrian approach. The questioning of assumptions and points of view in critical thinking allows analysts to discount certain evidence. Based on his own Israeli intelligence analyst experiences, Ben-Israel asserts that anchoring oneself some distance from a pole of refutation, and not far from the center, is probably best. In other words, enlightened skepticism probably works as a method for distinguishing between real and perceived threats. Critical thinking leads one to assess the processes one is using and the point on the continuum from which to work. Lacking: Disconfirmation Merom confirms that a disconfirmatory process did not occur on the Cuban missile problem: “Information that was inconsistent with the prevailing conservative theory was not considered as alarming and necessitating revision, but rather was ‘rehabilitated’ and rendered ‘harmless’ on the basis of ad hoc statements.” Instead, inductive reasoning generally led analysts to “prove” their theory and subsequently to adhere to it “ until it was proven wrong by conclusive hard evidence” – the U–2 photos, to be exact (of which a detail is reproduced in figure 7). Cassandra was the daughter of Priam, King of Troy. She was blessed and cursed by the god Apollo to accurately predict the future yet never be believed. In intelligence, a Cassandrian approach is characterized as one that emphasizes dire outcomes and worst–case scenarios. Analysts who produce such assessments are often discounted and rarely thanked when they are correct. For a real account of how such warnings are perceived (at least at the time they are made), see Charles E. Allen, “Intelligence: Cult, Craft, or Business?” in Seminar on Intelligence, Command, and Control, Guest Presentations, Spring 000 (Cambridge, MA : Harvard University Program on Information Resources Policy, I–0–, April 000), URL: , last accessed 11 January 2006. Ben-Israel, “Logic of the Estimate Process,” 9. Merom, “Estimate,” 9. Merom, “Estimate,” 9. Italics in original. – 0 – Figure 7: Detail of a U-2 Photograph of an SS-4 MRBM Launch Site, San Cristobal, Cuba, 14 October 1962. This evidence confirmed that Soviet missiles were being installed in Cuba. Source: U.S. Department of Defense, photograph in the John Fitzgerald Kennedy Library, Boston, MA, PX –0: October 9. What tipped off the overhead surveillance were two HUMINT reports of “a Soviet truck convoy that appeared to be towing ballistic missiles toward the San Cristobal area.” 9 That these reports were taken seriously is one of the curious serendipities that occur from time to time in intelligence analysis (and other research-based domains). What prompted the CIA and DIA analysts to take these reports seriously while earlier accounts had been dismissed remains a mystery. Garthoff asserts that it was new information taken in context with the observed “pattern of SA- surface-to-air missile sites in Western 9 Graham Allison and Philip Zelikow, Essence of Decision: Explaining the Cuban Missile Crisis, nd Edition (New York, NY: Longman, 999), 0. Cited hereafter as Allison and Zelikow, Essence of Decision. Raymond Garthoff also notes this to be the case. See Garthoff, “US Intelligence,” . – – Cuba” that led to the tasking of the U-2 flight on 14 October (the track of which is shown in figure 8). 0 Also contributing to the determination that the deployed missiles were SS- MRBM s was information transmitted to the British and American intelligence services by Oleg Penkovsky. While Penkovsky’s espionage apparently did not warn the Americans that the Soviets were about to deploy offensive missiles in Cuba, he is credited with – among other things – providing technical information about the missiles. Len Scott observes that Penkovsky provided the technical information that allowed the determination that the missiles were SS- s instead of SS- s, as well as information that allowed accurate assessment by the Americans of their readiness. Scott writes that this was important because the SS- had a range which US intelligence estimated at 0 nautical miles (nm), enabling coverage of seven Strategic Air Command ( SAC ) bomber/tanker bases; the 00nm SS – could target 18 bomber/tanker bases (plus one ICBM complex), and some cities with a population of over 00,000, including Washington, accounting for 9 million people. Shortly thereafter Penkovsky was detected, arrested, tried, and executed by the Soviets for his espionage. 0 Garthoff, “US Intelligence,” . Len Scott, “Espionage and the Cold War: Oleg Penkovsky and the Cuban Missile Crisis,” Intelligence and National Security , no. (Autumn 999), . Cited hereafter as Scott, “Penkovsky.” Scott, “Penkovsky,” . In making this assertion Scott draws on work previously published by Allison and Zelikow in Essence of Decision and by Mary S. McAuliffe, ed., CIA Documents on the Cuban Missile Crisis, 1962 (Washington DC : Central Intelligence Agency, 99). – – Figure 8: U–2 Tracks over Cuba, 4–14 October 1962. Sources: Map, Central Intelligence Agency. Tracks derived from a map in Mary S. McAuliffe, ed., CIA Documents on the Cuban Missile Crisis, 1962 (Washington DC: Central Intelligence Agency, 99), . Hereafter McAu – liffe. Missile locations derived from Arthur C. Lundahl, “Additional Informa – tion – Mission 0,” Memorandum for Director of Central Intelligence and Director, Defense Intelligence Agency, October 9, in McAuliffe, –; and Arthur C. Lundahl, “Additional Information – Mission 0,” Memorandum for Director of Central Intelligence and Director, Defense Intelligence Agency, 9 October 9, in McAuliffe, 09. Track and missile locations are approximations. Lundahl was the Director, National Pho – tographic Interpretation Center. The 9 October memo reports that a September U–2 overflight did not detect the latter two missile sites. How – ever, a review of the September track (as presented in McAuliffe) shows the aircraft only came near the Sagua La Grande site. It should also be noted that the track of the U-2 – as flown on 14 October – deviated from the origi – nal planned route (which was farther to the west). The track as flown took the aircraft closer to the San Cristobal site. What might have happened had the original track been flown? Despite their efforts at denial and deception, there was an evident lack of concealment of the missiles by the Soviets prior to their “discovery” on October. General Anatoli Gribkov, a member of the Soviet General Staff in Cuba at the time relates, – – A missile–launching complex is not easily disguised…[Such] an installation…could be hidden from ground-level view. But from above, however, it could and did stick out like a sore thumb. Allison and Zelikow note that the Soviets only began to camouflage the sites after “the U.S. announced the discovery of the missiles and announced the blockade.” They conclude that the Soviet forces building the bases lacked personnel and resources to conceal them – a situation that apparently changed only after the missiles had been discovered. The Roles of Critical Thinking in the Cuban Crisis Merom’s critique of the Intelligence Community’s estimate of Soviet intentions vi s-à -vis Soviet weapons in Cuba reveals areas that would be well-served by critical thinking. First, when a critical thinking paradigm is in control, intelligence foraging and gathering are efficiently oriented. Questions are raised about the existing evidence – both anomalous and consistent – as well as where new disconfirming and confirming evidence might be discovered. Second, alternative – what in the context of Cuba, Merom calls revolutionary – theories are considered. A structured process also speeds up the process of analytic sensemaking: estimates are crafted earlier. Third, a methodological process opens analysis to “guiding principles of Anatoli Gribkov, “The View from Moscow and Havana,” in Anatoli Gribkov and William Y. Smith, Operation ANADYR: U.S. and Soviet Generals Recount the Cuban Missile Crisis (Chicago, IL: Edition Q , 99 ), 9. Cited hereafter as Gribkov, “The View.” Allison and Zelikow, Essence of Decision , 0. Allison and Zelikow, Essence of Decision, . Merom, “Estimate,” . 77 Structured processes provide a framework that reduces the flailing around as analysts seek to find a starting point for their analyses. Morgan Jones discusses this at some length. See Jones, Thinker’s Toolkit, xi–xvi. – – research,” enriching both the process and the results. However, there are dangers to such a paradigm, especially as “science” commonly is understood. As Jeffrey Cooper notes, “a ‘science of analysis’ is a conceit, partly engendered by Sherman Kent’s dominating view of intelligence as a counterpart of the scientific method.” 9 Mark and Barbara Stefik argue that “the working methods of science and invention leaves [sic] out imagination. This makes it both boring and misleading.” 0 Analysts must focus on both evidence and inferences; otherwise they can “get the details right at the cost of ignoring important inferential judgments that need to be conveyed in order to provide a true sense of the uncertainties of both evidence and judgment.” Here again, critical thinking, with its emphasis on creative questioning, moderates the process. Such questioning opens the way for imaginative thinking. “[Intuition], curiosity, and a thirst for discovery – all essential elements of good science” could have alerted analysts working in 9 (or at any other time) to the possibility that they were being denied or deceived. An interesting intersection exists between critical thinking and analogy, or drawing comparisons to derive new patterns and explanations. This intersection lies at the heart of creativity. According to Keith Holyoak and Paul Thagard, there are commonly four steps to analogical problem solving: Often a problem solver will select a source analog by retrieving information about it from memory ( selection ), map the source to the target and thereby generate inferences about the target Merom, “Estimate,” . 9 Cooper, Pathologies, .. 80 Mark Stefik, and Barbara Stefik, Breakthrough: Stories and Strategies of Radical Innovation (Cambridge, MA : MIT Press, 00), 0. Cooper, Pathologies, . Cooper, Pathologies, . This statement illustrates just how pervasive and powerful the use of analogy is in human reasoning and discourse. Taken literally, creativity has no heart (nor any other organs) since it is a notion or a concept, not a living animal. Yet the analogy to a living being aids understanding. – – (mapping ), evaluate and adapt the inferences to take account of unique aspects of the target ( evaluation ), and finally learn something more general from the success or failure of the analogy ( learning ). Since analogy is such a powerful element in human reasoning, how can critical thinking outwit or control it? As developed in this paper, it does so by imposing a structure on the thinking. By examining inferences and implications as well as alternative points of view, critical thinking calls into question the appropriateness of the analogies in use. Recent work by Paul and Elder reveals how this works. They request, for example, that reasoners state and restate explanations, add examples, and then include an analogy. While initial statements about a phenomenon do not imply understanding, restatement, examples, and analogies do and further, provide measures by which comprehension can be fixed and assessed; they make knowledge explicit – something that can be thought about. When knowledge and reasoning are explicit, assumptions are revealed. According to Richard Neustadt and Ernest May, it is known that analogy is a tool commonly invoked by policymakers. Critically thinking analysts can add substantively to the policymakers’ options by constructively challenging the tendency to rely upon analogy as a way of addressing situations. For example, Neustadt and May observe that in coping with the missile crisis in Cuba, President Kennedy and his advisers relied on – among other things – an analogy to Pearl Harbor to justify why a “surprise” bombing of the Soviet missile Keith J. Holyoak and Paul Thagard, Mental Leaps: Analog y in Creative Thought (Cambridge, MA : The MIT Press, 99 ), . Emphasis in original. Cited hereafter as Holyoak and Thagard, Mental Leaps. Richard Paul and Linda Elder, “Instructions to Participants,” th Annual Conference on Critical Thinking, Berkeley, California, 9 July 2005. The first three elements were developed previously; analogy was new in 00. Holyoak and Thagard, Mental Leaps, 20–22. Richard E. Neustadt and Ernest R. May, Thinking in Time: The Uses of History for Decision Makers (New York, NY: The Free Press, 9). Cited hereafter as Neustadt and May, Thinking in Time. – – bases was not a satisfactory option. Kennedy’s advisers, acting as analysts of the situation, critically evaluated the appropriateness of the analogy, pointing out its weaknesses and strengths. Kennedy concluded that sneak attacks were not a tactic lying within the U.S. tradition. 9 In the Cuban case, getting analysts to restate their conclusions, provide examples, and make analogies would have revealed the strengths and weaknesses of their arguments about what they observed from the HUMINT sources. It would have shown their comprehension of what they wanted to conclude, revealed their assumptions and, in so doing, opened an avenue for “alternative” assessments of the issue. It would have done so by causing the analysts to question how it was that they failed to notice or ignore things. 90 The stage would have been set for an earlier, less risky, defusing of the impending crisis. Another analogy employed in the Cuban missile crisis was the comparison to the Soviet position vis-à-vis the presence of American MRBM s in Turkey. A critical review reveals that the Soviet Union had tolerated the presence of these missiles – which had a longer range than those placed in Cuba – since 9. 9 If the U.S. demanded the removal of the Soviet missiles from Cuba, was not a similar withdrawal of the American missiles from Turkey appropriate? Ultimately, this analogy provided a face-saving solution for the Soviet Union in the negotiations that followed. As a “secret” part of the agreement, the American Jupiter MRBM s were removed from Turkey five months after the Soviet Union removed its missiles from Cuba. 9 Neustadt and May, Thinking in Time, . 9 Neustadt and May, Thinking in Time, . 90 Margaret A. Boden, The Creative Mind: Myths and Mechanisms (New York, NY: Basic Books, 990), 25. 9 Neustadt and May, Thinking in Time, 9. 9 Neustadt and May, Thinking in Time, . – – Winners and Losers: The Crisis in Context A Critical Assessment of the Leaders. In any confrontation there are winners and there are losers. Thinking critically about the confrontation over the Soviet missiles also involves determining who actually won as well as what the other outcomes might have been. Each of the three actors, Castro, Khrushchev, and Kennedy, had a number of interests at stake. In a number of circumstances each stood to gain at the expense of the others. What solutions were the most advantageous to each of the three actors? What were the motivating factors? A speculative – and admittedly simplified – critical comparison, again using Paul and Elder’s model (summarized in table ) illustrates how critical thinking also reveals much about who stood to gain and who stood to lose in the crisis. Castro. Castro found his country under attack from the United States – both directly through economic sanctions and indirectly by Batista loyalists living in Florida. 9 The previous year the United States had launched an invasion attempt at the Bay of Pigs – which was repulsed. However the attempts by the United States to destabilize or overthrow the Cuban government did not end with their defeat. As Raymond Garthoff observes, “by the spring of 9 the United States had embarked on a concerted campaign of overt and covert political, economic, psychological, and clandestine operations to weaken the Castro regime…including attempts to assassinate Castro.” 9 In short, the United States was doing everything except conventional warfare to destroy Castro and his regime. Castro’s point of view was that the United States was a real enemy. He could reasonably assume that in light of its other activities and having attempted an invasion once, the United States would repeat its actions and this time might succeed (an implication) . Another possible outcome that probably was considered was that war, conventional or nuclear, was likely. Castro must have possessed evidence of the military capabilities of the United States and probably inferred that 9 Raymond L. Garthoff, Reflections on the Cuban Missile Crisis (Washington, DC : The Brookings Institution, 99), . Cited hereafter as Garthoff, Reflections. 9 Garthoff, Reflections, 9. – – Element of Thinking Considerations Considerations as seen from Castro’s Point of View as seen from Khrushchev’s Point of View as seen from Kennedy’s Point of View Purpose · Preserve Regime · Protect Cuba from United States · Force concessions from United States · Remove U.S. and NATO from Berlin · Spread Communism · Preserve Regime · Get the missiles out of Cuba · Defend U.S. and allies against Com – munism · Preserve Regime Point of View · United States threatens regime · United States a threat to Commu – nism · Cuba threatened with invasion · Communism threatens United States · Missiles are a threat Assumptions · A second invasion will be attempted · Missiles in Cuba bargaining points · Missiles in Cuba protect Castro · Effort will be successful. · Small Soviet Force · Strategy of removal is possible Implications · Invasion might succeed · War: conventional or nuclear · U.S. concessions · War: conventional or nuclear · Successful removal · Escalation of measures · War: conventional or nuclear Evidence · U.S. armed forces formidable · U.S. armed forces formidable · Administration politically vulnerable over crisis · Khrushchev is a formidable adversary · Khrushchev is bluffing Inferences · U.S. can overcome unaided Cuban forces. · U.S. can overcome unaided Cuban forces. · Missiles provide necessary aid · United States will concede. · Khrushchev will back down · Military option not required Concepts · Military strategy and doctrine, commu – nist and capitalist theories · Military strategy and doctrine, com – munist and capitalist theories · Military strategy and doctrine, com – munist and capitalist theories · Monroe Doctrine Key Questions · How to deter United States from invading island · How to deter United States from invading Cuba · How to dominate the United States · How to get the Soviets and their missiles off of Cuba Table 3: A Comparative Assessment of Rival Motivations Source: Developed by author. – 9 – Element of Thinking Considerations Considerations as seen from Castro’s Point of View as seen from Khrushchev’s Point of View as seen from Kennedy’s Point of View Purpose · Preserve Regime · Protect Cuba from United States · Force concessions from United States · Remove U.S. and NATO from Berlin · Spread Communism · Preserve Regime · Get the missiles out of Cuba · Defend U.S. and allies against Com – munism · Preserve Regime Point of View · United States threatens regime · United States a threat to Commu – nism · Cuba threatened with invasion · Communism threatens United States · Missiles are a threat Assumptions · A second invasion will be attempted · Missiles in Cuba bargaining points · Missiles in Cuba protect Castro · Effort will be successful. · Small Soviet Force · Strategy of removal is possible Implications · Invasion might succeed · War: conventional or nuclear · U.S. concessions · War: conventional or nuclear · Successful removal · Escalation of measures · War: conventional or nuclear Evidence · U.S. armed forces formidable · U.S. armed forces formidable · Administration politically vulnerable over crisis · Khrushchev is a formidable adversary · Khrushchev is bluffing Inferences · U.S. can overcome unaided Cuban forces. · U.S. can overcome unaided Cuban forces. · Missiles provide necessary aid · United States will concede. · Khrushchev will back down · Military option not required Concepts · Military strategy and doctrine, commu – nist and capitalist theories · Military strategy and doctrine, com – munist and capitalist theories · Military strategy and doctrine, com – munist and capitalist theories · Monroe Doctrine Key Questions · How to deter United States from invading island · How to deter United States from invading Cuba · How to dominate the United States · How to get the Soviets and their missiles off of Cuba Table 3: A Comparative Assessment of Rival Motivations Source: Developed by author. – 0 – unaided, his own forces were no match for the U.S. forces. Underlying Castro’s considerations were a number of ideas about military strategy and doctrine, as well as communist and capitalist theories. A corollary consideration commonly believed by many Americans (at the time and still today) might have been how Communism could have been spread across the Western Hemisphere, especially the United States. Castro’s key question was, “How can I prevent the United States from invading again?” Khrushchev. What motivated Khrushchev remains controversial. His motives certainly were more complex, as various explanatory hypotheses reflect. First, there is the argument that Khrushchev had an ally “in the faith” to protect. He too believed that the United States would invade Cuba a second time. 9 However, as Allison and Zelikow counsel, the troop buildup itself and not the nuclear missiles was what Khrushchev did to offset the perceived threat to Cuba from the United States. 9 The deployment of the nuclear missiles was related to some other issue. Next, the situation posed an opportunity to force concessions from the United States – perhaps about Berlin, or the United States’ own offensive missiles located in Europe and Turkey. Allison and Zelikow find sufficient evidence to lead them to believe that removing the U.S. and NATO troops from Berlin was a key factor motivating Khrushchev to deploy the missiles to Cuba. 9 Securing his borders from the American’s Jupiter missiles may have been an additional motivating factor. Additionally, there was Khrushchev’s avowed purpose of spreading Communism across the globe. In this context the deployment of weapons lies within the context of “a great power rivalry… between the U.S. and the values and interests it represented…and 9 James J. Wirtz, “Organizing for Crisis Intelligence: Lessons form the Cuban Missile Crisis,” in Blight and Welch, Intelligence and the Cuban Missile Crisis, 0. Cited hereafter as Wirtz, “Organizing.” 9 Allison and Zelikow, Essence of Decision , . 9 Allison and Zelikow, Essence of Decision , 99–09. – – the Soviet Union’s communist agenda.” 9 At a reception for foreign ambassadors in Moscow in 9 , Khrushchev threatened, “Whether you like it or not, history is on our side. We [the communists] will bury you!” 99 This remark (among others) also reveals an additional purpose: Khrushchev, like all dictators, had to appear stronger than his adversaries – his political survival depended on it. Khrushchev’s point of view was that the United States (and its allies) were a threat to Communism everywhere and needed to be contained. He was presented with a confluence of opportunities and responded with the military buildup in Cuba and the deployment of the missiles. He apparently assumed that if he could get nuclear missiles into Cuba he’d have bargaining points useful in such a containment strategy. Later, Khrushchev believed he could effect a change in the balance of power between the two nations. 00 Possible outcomes included the prospect of further U.S. concessions, protection of Cuba from invasion, and either conventional or nuclear war. Khrushchev had direct evidence drawn from the Korean War and possibly other evidence from spies operating in the United States about the capability of the U.S. military. Given the nuclear missiles he actually deployed as well as his assignment (temporary) of operational control of those missiles to the Soviet Group of Forces commander, General Issa Pliyev, Khrushchev seems to have inferred the U.S. forces were formidable. 0 He also apparently concluded that the missiles – and the other forces – would be adequate to the task of successfully satisfying his purposes; the United States, faced with 9 Allison and Zelikow, Essence of Decision , . 99 Nikita Khrushchev, “Speech to Ambassadors at Reception, November 9 ,” in James Beasley Simpson, compiler, Simpson’s Contemporary Quotations: The most Notable Quotes since 1950 (Boston, MA : Houghton Mifflin Company, 1988), online edition, URL: , last accessed 0 April 00. Ironically, Khrushchev was wrong. History sided with his adversaries. 00 Cited in Garthoff, Reflections, . 00 Gribkov, “The View,” . Control reverted back to Moscow in September. – – operational missiles in Cuba, would concede. 0 The concepts on which Khrushchev relied probably were the same as those on which Castro relied: military strategy and doctrine as well as communist and capitalist theories. Again, Khrushchev’s key question mirrored that of Castro: How to keep the Americans from invading Cuba. Additionally, there could have been a second question on how to dominate, or at least gain concessions from, the United States. Kennedy. Finally, there was Kennedy. While Kennedy’s exact purpose vis-à-vis Cuba prior to the discovery of the missiles also remains unclear, it is known that he did not plan to invade the island during the summer and fall of 9. 0 However, 9 was an election year and the Kennedy administration was vulnerable with respect to Cuba. 0 Kennedy could not afford to appear soft on Communism. Kennedy also had an interest in keeping the U.S. and NATO forces in Berlin. All this changed once the missiles were discovered. Kennedy’s deliberations revolved around removing the missiles as expeditiously as possible. He no doubt realized that if his administration bungled the crisis they might lose at the polls in November. 0 Therefore, it is reasonable to conclude that Kennedy’s goals included retaining control of Congress – he too had a regime to preserve. Kennedy’s point of view was clear: the deployed Soviet strategic nuclear missiles were a threat to the United States and must be removed. He assumed that such a strategy could be developed. He apparently also assumed that the Soviets had placed a small force on the island. Possible outcomes included successful removal of the missiles and their 0 Wirtz, “Organizing,” . Wirtz argues that the Soviets were not only wrong about the U.S. reaction, they were surprised! 0 Wirtz, “Organizing,” 0. 0 Fen Osler Hampson, “The Divided Decision-Maker: American Domestic Politics and the Cuban Crises,” International Security 9, no (Winter 9 / ), . Cited hereafter as Hampson, “Divided.” 0 Hampson, “Divided,” . – – supporting forces, an escalation of measures to prompt that removal, and the possibility of conventional or nuclear war. 0 Kennedy had evidence of his political vulnerability He also had evidence – Khrushchev’s own actions included – that the Soviet leader was a formidable adversary. But based on Penkovsky’s evidence he inferred that Khrushchev might be persuaded to back down. As Scott concludes, “various writers contend that Penkovsky’s intelligence…[guided] Kennedy’s handling of the crisis from a position of strength.” 0 This occurred in part because “the KGB’s discovery of Penkovsky’s espionage alerted Khrushchev to the fact that Kennedy now knew he was bluffing.” 0 As has been noted, Kennedy lacked accurate evidence about what was really on the island. Nevertheless as has been noted, he inferred that military strikes were not – at least at that time – an option. Kennedy probably relied on the same concepts on which Castro and Khrushchev relied. In addition, the concepts embodied in the Monroe Doctrine were also probably a factor. Kennedy’s key question was, “How to get the Soviets and their missiles out of Cuba?” Foresight. Viewed with foresight, there were a number of possible outcomes to the crisis based on the inferrable goals that could be associated with any of the three leaders. A win for one of the leaders was not necessarily a loss for the others as table shows. In some cases, achieving one’s goal was a win for the leader, but a failure to achieve it was not necessarily a loss. “Winning” the crisis should have involved attaining all or most of one’s goals. However, since not all goals were of equal importance, failing to achieve one could mean that one lost in the larger crisis. Similarly, achieving one’s goals did not guarantee winning the larger crisis. This becomes evident as the actual winners and losers are considered. 0 Neustadt and May, Thinking in Time, -. Given the presence of the Soviet tactical nuclear weapons on Cuba it is probable that any invasion that began with conventional forces would have escalated to nuclear scenarios. 0 Scott, “Penkovsky,” . 0 Scott, “Penkovsky,” . – – And the Winner is? Who actually won the Missile Crisis? Critical assessments of the three leaders involved and the attainment of their goals reveals the answers. First, Castro certainly got what he wanted: The United States never again invaded Cuba. What did Khrushchev get out of the resolution of the crisis? From the standpoint of his first goal, to protect Cuba from the United States, he was successful. The subsequent removal of the Jupiter Goal Outcome Leader Castro Khrushchev Kennedy Prevent U.S. Inva – sion of Cuba Achieve Win Win Neutral Fail Lose Lose Win Get the Missiles out of Cuba Achieve Neutral Lose Win Fail Win Win Lose Spread Commu – nism Achieve Win Win Lose Fail Neutral Lose Win Get U.S. and Allies out of Berlin Achieve Neutral Win Lose Fail Neutral Lose Win Force Concessions from United States Achieve Win Win Lose Fail Lose Lose Win Force Concessions from Soviet Union Achieve Neutral Lose Win Fail Neutral Win Lose Force Concessions from Cuba Achieve Lose Neutral Win Fail Win Win Neutral Preserve Regime Achieve Win Win Win Fail Lose Lose Lose Table 4: Goals and Outcomes in the Cuban Missile Crisis Source: Developed by author. missiles in Turkey appeared to increase the security of his borders – an apparent concession from the United States. 09 However, Khrushchev failed to spread Communism further. And Berlin remained partially 109 The issue of the Jupiter missiles is complex. They were not included in the formal agreement but were offered up in a secret unilateral assurance from Kennedy (Garthoff, Reflections, ). – – in the hands of the West. If, as Allison and Zelikow postulate, this was the real motivation behind the nuclear deployment then Khrushchev’s failure was highly significant. Additionally, his regime was embarrassed by the revelation in the United Nations of the presence of the missiles. Ultimately, he was removed from power two years later and placed under house arrest until his death in 9. So, Khrushchev, while he won a number of his goals, wound up the overall loser. Kennedy is the other winner in the crisis. He gets the Soviets missiles removed from Cuba while avoiding war. He is later able to retire an obsolete missile system (the Jupiters). He prevents the Soviet Union from seizing control of Berlin. Finally, his popularity increases and the Democrats retained power in the 9 elections. Ten Years Later, They Meet Again It is worth noting that Soviet denial and deception in support of military deployments and operations in Cuba did not end in the early winter of 9. In 90 Henry Kissinger stormed into the office of H. R. Haldeman and demanded to see President Nixon. Aerial reconnaissance had brought news. “The Cubans are building soccer fields,” Kissinger said. “Cubans play baseball . Russians play soccer.” 0 The Soviets it seems had been discovered building a submarine base at Cienfuegos. Prior to a particular overflight, that fact had been denied to the United States. The soccer fields were for the recreation of Soviet troops. 0 H.R. Haldeman, with Joseph DiMona, T he Ends of Power (New York, NY: Times Books, 9), –, in Patrick J. Haney, “Soccer Fields and Submarines in Cuba: The Politics of Problem Definition,” Naval War College Review L, no , sequence 0 (Autumn 99 ), URL: , last accessed 6 April 2006. Emphasis in original. – – Judgment: Critical Thinking Would Have Made a Difference As this case study makes clear, there are a number of junctures where critical thinking or structured analytic methods could have made a difference in analyzing the Soviet missile deployment to Cuba. While it is true that Sherman Kent argued for a scientific approach to analysis, it does not seem to have been widely practiced at the time. As of 9, most studies that we now have in hand on human reasoning in intelligence had not been completed. The champions of structured intelligence analysis methods had not yet developed their techniques. Nevertheless, a study of the crisis is germane because the same kinds of errors repeat themselves again and again. The errors seen in the Cuban case – failure to question assumptions, to take seriously the evidence and the patterns they imply, to counterfactually examine analytic implication and consequences, in short to make reflective judgments – also were cited by the Senate in its critique of the 00 Iraqi WMD estimate. In both cases, deceptions confounded analysts, leading them to wrong conclusions. In the Cuban case the analysts eventually figured it out. This did not occur in the Sherman Kent, Strategic Intelligence for American World Policy (Princeton, PA : Princeton University Press, 99), –0. In the Cuban situation, the debate between McCone and analysts over the nature of the weapons the Soviets might emplace came close to overcoming these errors. Regarding the Iraqi WMD estimate, the debate between the CIA and the Department of Energy over the use of the aluminum tubes also came close. Unfortunately, the incipient debate focused on the results, not the process. In both cases, the side that was eventually proved right failed to make the now- accepted case. Sherman Kent challenged this assertion in his defense of the original estimate and the process by which it was created. He believed that an earlier revelation of the Soviet actions would have led to an intractable situation. However, an earlier revelation – say in August – would have offered President Kennedy and his advisers more options, particularly more diplomatic options. Kent also failed to observe that denial and deception occurred – at least for the record. See Sherman Kent, “A Crucial Estimate Relived,” Reprint, Studies in Intelligence , no. (99): –9. – – most recent failure. Critical thinking skills went unused among both generations of analysts. – – H ow c AN i NTelligeNce A NAlysTs eMPloy c riTicAl T HiNkiNg ? The Poor Record Critical thinking is what consumers of intelligence appear to expect when they task producers to examine issues. Corporate consumers require analysts creating intelligence to “[evaluate] a situation, problem, or argument and [choose] a path of investigation that leads to finding the best possible answers.” In the national security field, strategic intelligence pioneer Washington Platt notes that “[intelligence] is a meaningful statement derived from information which has been selected, evaluated, interpreted, and finally expressed so that its significance to a current national policy problem is clear.” Derived from strategic intelligence, “best answers” should clearly express what is significant to national policy problems. They may also support warfighters with essential operational and tactical intelligence. Critical thinking leads to the best answers for the specific context at hand. Unfortunately, analysts’ biases and mindsets repeatedly obscure best questions and answers. From well before the Japanese surprise attack on Pearl Harbor to the 00 estimate on Iraqi weapons of mass destruction, a failure to think critically about potential crises contributed to repeated intelligence failures. “Expert analysis” Daniel Feldman, Critical Thinking: Strategies for Decision Making (Menlo Park, CA : Crisp Publications, Inc, 00), . Washington Platt, Strategic Intelligence Production: Basic Principles (New York, NY: Frederick A. Praeger, 9), . George S. Pettee, The Future of American Secret Intelligence (Washington, DC : Infantry Journal Press, 9). Chapter provides a summary of U.S. intelligence failures during the first half of the 20 th Century. While hindsight is an imperfect mirror for reviewing the past, one conclusion to be drawn from a review of the evidence is that critical thinking could have minimized many of the ensuing crises. – 9 – was not enough. Biases and mindsets too often converted subject- matter confidence into arrogance; false assumptions blinded analysts to their target’s true intentions. For example, at least one CIA Iraq analyst acknowledged that the 990 invasion of Kuwait, was an intelligence failure…We were guilty of a kind of mindset…The idea that a country [Iraq] would march up to the border, put 00,000 troops there, go in and do what they’ve done; I don’t think anybody thought they’d do it. In 2002, Intelligence Community analysts failed to definitively assess whether the Iraqi government still possessed weapons of mass destruction. The Senate noted in its review of the failure that [rather] than thinking imaginatively, and considering seemingly unlikely and unpopular possibilities, the Intelligence Community instead found itself wedded to a set of assumptions about Iraq, focusing on intelligence reporting that appeared to confirm those assumptions. 9 The mistakes of 00 also occurred in 990 – and for that matter in 9. Analysts failed to question assumptions widely held at the time. Instead they chose the first answer that satisfied the situation, a phenomenon known as “satisficing.” 0 Other means by which intelligence analysts and policymaking customers reasoned poorly are listed in table . According to Ephraim Kam, the problem is so great that the intelligence analysis process is “consistently biased, and…bias 117 Anthropologist Rob Johnston explores this paradox. See Dr. Rob Johnston, Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study (Washington, DC : Center for the Study of Intelligence, 00 ), – . Cited hereafter as Johnston, Analytic Culture. Anonymous CIA Analyst, 990, in Don Oberdorfer, “Missed Signals in the Middle East,” Washington Post Magazine , March 99, 0. 9 The Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction, Report to the President of the United States, March , 00 , URL: < http://www.wmd.gov/report/ index.html >, last accessed July 00, . Cited hereafter as WMD Commission, Report. 0 Morgan D. Jones, conversation with the author, December 00. – 0 – is the cornerstone of intelligence failures.” Paul and Elder claim that much thinking is “biased, distorted, partial, uninformed, or down-right [sic] prejudiced.” There are repeated failures to think critically; but could critical thinking about these situations prevent the failures? Means by Which Decisions are Made Select first answer that appears “good enough.” Focus on narrow range of alternatives, ignoring need for dramatic change from existing position. Opt for answer that elicits the greatest agreement and support. Choose the answer that appears most likely to avoid some previous error or duplicate a previous success. Rely on a set of principles that distinguish “good” alternatives from “bad” alternatives. Table 5: How Analysts Decide Source: Excerpted from Alexander George, Presidential Decisionmaking in Foreign Policy: The Effective Use of Infor mation and Advice (Boulder, CO: Westview Press, 90), Chapter . Critical thinking helps mitigate the effects of mindsets and biases by invoking skillful examination of evidence both for and against an issue, as well as consideration of obvious and less obvious alternative explanations. In the 1990 example, thinking critically would have raised other possible explanations for why Hussein’s troops were on the Kuwaiti border and what he intended for them to do. That the Iraqi troops were trained by the former Soviet Union and followed its tactics might have been an indicator of future intentions. In the case of the weapons of mass destruction ( WMD ), analysts might have Ephraim Kam, Surprise Attack: The Victim’s Perspective (Cambridge, MA : Harvard University Press, 990), . Paul and Elder, Concepts and Tools, . A consideration of who trained Hussein’s troops – in this case, the Soviet Union – might have led to an examination of military doctrine. Thus, it quickly would have become clear that troops mobilized on a border were going to cross that border. Analysts would then have known that an invasion of Kuwait was imminent. – – asked, as they realized they had no new evidence of the weapons, “How can we prove that the weapons of mass destruction definitely are not there?” or “What would we expect to see if Saddam Hussein had gotten rid of his weapons of mass destruction?” We must assume that analysts can be encouraged to think in this manner. Modern strategic intelligence pioneer Sherman Kent believed this to be the case when he wrote nearly 0 years ago that intelligence analysts “are supposed to have had more training in the techniques of guarding against their own intellectual frailties” than the larger populace. Assessing Evidence Understanding how evidence is assessed is a necessary first step in understanding how analysts can better reason with and about it. The incomplete and ambiguous information and data with which intelligence professionals work compounds their reasoning process. Sources are unreliable and contradictory; adversarial denial and deception increase doubt. When information and data are questionable, derived evidence is even more uncertain, and inferences arising from that evidence may be highly suspect. One way that analysts can reduce this uncertainty is by building defensible inferences These analyses are those of the author. The last question was jointly arrived at in a conversation between the author and Mark Lowenthal, July 00. Sherman Kent, Strategic Intelligence for American World Policy (Princeton, NJ: Princeton University Press, 99), 99. This section is based (with additions) on materials developed by Francis J. Hughes and the author for a structured analytic methods course taught at the National Defense Intelligence College. For a deeper examination of evidence assessment see Francis J. Hughes and David A. Schum, Credibility Assessment: A First Step in Intelligence Analysis, unpublished tutorial, National Defense Intelligence College, April 00. Information, data, and evidence are different kinds of knowledge. Pieces of information comprise data, which in turn comprise evidence. Evidence is further distinguished from data by the fact that it contributes to discrimination among alternative end states. Data, while relevant, do not necessarily allow discrimination. – – from the evidence. At their most basic level, these inferences depend on credibility (can it be believed), relevance (does the evidence bear on the issue), and inferential or probative force (how compelling is it in answering the questions raised by the issue). Unfortunately, no mass or body of evidence – in intelligence or anywhere else – comes with these three properties already established. Establishment of these properties to abet uncertainty reduction through inference occurs only through a process of argument, creative hypothesis generation, and the development of chains of reasoning. Authenticity, accuracy, and reliability represent criteria for establishing the credibility of tangible evidence. An analyst striving “Tangible evidence” is a technical term describing things that bear relevance to an issue under scrutiny. It is contrasted with “testimonial evidence” TESTIMONIAL: APPLYING CRITICAL THINKING This is a true story. All the characters are real and really naive. As a fresh graduate of NSA’s critical thinking and structured analysis class, I attended an Intelligence Community class on counterintelligence. My fellow students included folks from all over the community, including CIA and FBI case officers. During the class, the instructor put forth a case study for students to think about and decide how best to “analyze and investigate” to find the mole. Differing opinions surfaced, but a common thread appeared among the case officers: follow your “gut” feeling and collect evidence to support that assumption, no matter how long it took. The instructor, enjoying the collective mindset and the opportunity to shatter paradigms, concentrated on how information was collected and analyzed. Again, the case officers agreed the best solution was to continue gathering information until you “proved your case.” I, the lone NSA token student, raised my objection, “How long is long enough? Until you’ve ruined the career of innocent, hardworking persons in the IC, or until you find another answer you like?” Although it did not earn me friends, it did open up a window to inform these case officers and other attendees about the intricacies of anal ysis. Critical thinking is not just about putting information together, finding a pattern, then choosing an answer, it is about reducing bias, considering all options available, and presenting options to a decision–maker. And critical thinking is about paying attention to what and how you are doing it. I reasoned that, since the investigators in the case study concentrated on only one Source: A mid–level intelligence professional at the National Security Agency – – to determine authenticity may ask, “Is the evidence what it seems to be?” In determining accuracy, the key question is whether the piece of evidence or the system that produced it has the resolution to reveal what the analyst believes the event or record should reveal. Assessing reliability involves determining whether different means of collection produce the same results. If, on the other hand, the evidence is testimonial, different criteria apply. The first thing to be established is whether the source is being truthful. Truthfulness is not absolute. Rather, it is time- and context- dependent. A source may believe he is being truthful about an issue or may have legitimate reasons to be untruthful about that issue. In – what people say about the issue. person, the analysis was faulty because somewhere along the line they limited their suspects (introduced bias) regardless of reason (access, family connections, angel–like qualities, etc.). Although it would take time, the investigators should conduct quantitative and qualitative analysis, make a decision tree (options open to a mole in a heavy security environment), and then play Devil’s Advocate. In addition, analysis never reveals one “solution” or “smoking gun”; it leads to two or three “options” which can be investigated in detail. The instructor, pleased that someone in the class knew what the “Analysis of Competing Hypotheses” was, revealed the answers of the real–life case study: after three years of investigating, the case officers had the wrong person due to incorrect information in old reports and limiting their suspect list. Instead of going back to the original source information, the officers read old reports that were unfortunately biased by Agency politics and external societal events (1950–60s). The real mole was discovered over a decade later; he was the son of a former Agency chief, well educated and well liked, but did not have access to the information. The individual “borrowed” interesting reports from friends who had access, covered his tracks, and continued spying against the U.S. for several years. Just as the authorities were about to arrest him, he was found dead in a hotel room from a reported suicide, but the cops could not figure out how he had two bullets in his chest and one in the back of his head. Miracle suicide. What is the moral of the story? Conduct a thorough analysis right the first time and you can catch a mole anytime. I only hope the attending case officers got the message, especially from a non–gun toting NSA analyst. who wishes to remain anonymous, email to the author, 9 March 2006. – – another time and about another issue, these impediments to veracity may not exist for that source. Therefore, establishing the truthfulness of a source can pose a significant challenge to the analyst. In an ideal situation, a review of evidentiary relevance causes an analyst to examine the likelihood of any potential answer to the problem or question – a hypothesis – with an eye to the modification of existing hypotheses or even the invention of new ones. In other words, an analyst might theorize that a certain bit of evidence will indicate that an individual or a group of individuals will engage in nefarious activity. Yet, if none of the evidence at hand bears on the issue, then the analyst may need to reconsider – or even reject – this evidence based on its lack of relevance. The analyst also should consider that the individual may not engage in the activity. What is true about the future is also true about the present and the past. A lack of evidence relevant to an issue should prompt analysts to reassess their theories about the issue at hand. For example, an unnamed FBI investigator in the 2001 anthrax case noted, “[Reasoning] says that if you think a person is your guy, but you can’t find anything to put him in the game, you’ve got to keep looking at others.” 9 The failure of the FBI to implicate its principal suspect forced it to consider other explanations as to who sent the anthrax-laden letters in September 2001 to political leaders and media figures. The analyst also is concerned with how strongly the evidence 129 Allan Lengel and Guy Gugliotta, “Md. Pond Produces No Anthrax Microbes: FBI Sought Clues In Deadly Attacks,” Washington Post , August 00, A0. Cited hereafter as Washington Post, “No Anthrax.” While this is a law- enforcement example, it illustrates good critical questioning about an apparent lack of evidence. Criminal investigation and intelligence analysis similarly assess events and evidence with the goal of description, explanation, and interpretation. The principal difference is that intelligence analysis endeavors to do so before the event occurs – in other words, to estimate. It is worth noting that over two years after the pond was drained the case remains unsolved; the individual under suspicion at the time was never charged with the crime. The FBI was unable to link the evidence to the individual. See for example, Allan Lengel, “Little Progress In FBI Probe of Anthrax Attacks: Internal Report Compiled As Agents Hope for a Break,” Washington Post, September 00, A0. – – undermines or supports the particular hypotheses under examination – the probative force of the evidence. Certain evidence, coming from certain kinds of sources, persuades more strongly than does other evidence drawn from other sources. Tangible evidence might have greater probative force than testimonial evidence. Consider, for example, a hypothetical biological weapons issue. Traces of certain toxic substances found near an alleged biological weapons lab carry greater probative force than the testimonial denials of the government of the country in which the lab and the samples were found. The “ideal” analyst also assesses the objectivity, observational sensitivity, and competence of sources. One question suitable for this assessment is how biases may have corrupted a source’s objectivity. Did the source see or hear the evidence directly, and under what conditions did this occur? A reality check also is made at this point. Does it makes sense that a particular source claims to have been in a position to make the observation or have access to the source of information? Another consideration involves denial and deception. In assessing the evidence, the analyst should ask, “What is the likelihood that I am being deceived? This question is surprisingly difficult to answer accurately. Given that anchoring biases and confirmation heuristics cause analysts to find what they seek to find, a denial and deception hypothesis is often easily proved. 0 Critical thinking challenges this by forcing an examination of alternative points of view. For intelligence analysts, proactive, focused and surreptitious collection of information about the often minimal disconfirming evidence pays rich dividends. In this case, the question becomes, “What is the likelihood that I am not being deceived?” Thus engaged, really good intelligence analysts create valuable knowledge. 0 Author’s notes, National Defense Intelligence College, Denial and Deception Advanced Studies Program, Spring 00. The author is a National Intelligence Council-sponsored participant in the Foreign Denial and Deception Committee’s Denial and Deception Advanced Study Program at the NDIC . Ben-Israel, “Logic of the Estimate Process,” 0–09. – – Facilitating Evidentiary Assessment The questions that need to be asked about each piece of evidence are the same as those employed in critical thinking. In making her evaluation of what constitutes evidence, the analyst repeatedly asks “why”: “Why do I believe this information is relevant to the question at hand (either against or for) and therefore exists as evidence?” “Why do I believe the source to be credible?” Additional questions about the analyst’s own thinking processes might arise, such as: “What are my biases and why do they lead me to think this way?” For example, in assessing the intentions of former Iraqi dictator Saddam Hussein in light of the U.S. destruction of his intelligence headquarters in 99 , an analyst might ask whether Hussein’s declarations not to retaliate against the United States were credible. A corollary question might be, “What does Saddam Hussein gain by denial and deception?” Based on Hussein’s statement, a determination that additional evidence was needed would stimulate collection of similar statements from other episodes. By analyzing what Hussein did in those instances, the analyst might determine that he was usually untruthful. Therefore, this piece of evidence would be deemed to be of low credibility despite its probative force, and relevance to the determination whether Hussein would or would not retaliate. In other words, Hussein’s statement might reflect an ongoing deception campaign. To explore this hypothesis further, the analyst might seek to determine whether increases in Iraqi agent communications were relevant to the issue and whether additional collection of such broadcasts was warranted. In practice, these and similar questions can be answered quickly. Analysts often answer some of them unconsciously as they struggle 132 Heuer develops this scenario and evidence in an example of the analysis of competing hypotheses. See Heuer, Psycholog y , 0–0. Heuer, Psychology, 0–0. For a recent detailed look at Saddam Hussein and deception see Kevin Woods, James Lacey, and Williamson Murray, “Saddam’s Delusions: The View from the Inside,” Foreign Affairs , no. (May/June 00). URL: , last accessed March 00. – – to meet short deadlines. However, as a function of not employing scientific methods or other structured and critical thinking techniques, their thinking is largely intuitive. Analysts choose to ignore how mindsets and assumptions impede their judgments. Conversely, critical thinking ensures that the reasoning process is a self-conscious one. By making the unconscious conscious, analysts reveal where they may be biased, helping ensure that questions they address are thoroughly and fairly considered. An analyst committed to critical thinking continually asks questions while developing the mass of evidence needed to assess an issue. Evidence arises from questions answered satisfactorily. Information for which the questions cannot satisfactorily be answered is excluded – but only after the analyst reflects on the process of evidence development. Consideration is owed to the question of how biases, and possibly active denial and deception by an adversary, have influenced the selection of both questions and answers. One way of revealing bias is by asking questions such as, “If the opposite outcome is actually true, what other evidence would I expect to see?” The marshaling of evidence refers to a questioning process by which data and information are assessed and evidence is created. It may be done in solitary fashion, Socratically with a teammate, or collegially among Intelligence Community focus groups. It is a vital ingredient of productively imaginative intelligence analysis. Each question an analyst asks not only becomes a device for attracting existing evidence, but also generates new evidence not yet visible. Identifying new evidence increases the thoroughness by which the issue is evaluated and increases the probability that the correct solution has been discovered. Embracing a Methodology Once a relevant mass of evidence is established, the analyst evaluates which method or methods of analysis may best develop Each method of analysis uniquely interacts with the intelligence question; therefore, the method (or methods) selected can pro – duce profoundly different results – affecting the analyst’s arrival at the most accurate answer. – – a solution to the issue. Differences among various analytic methodologies, techniques, and tools are not trivial. In fact, comparing the results of different analysis methods can be a valid means of establishing the accuracy of the answer. If various means of analysis yield multiple results, a review by different analysts makes for the ideal environment for critical thinking to discuss and debate those results. Therefore, assessment of the available evidence includes redirecting thinking, soliciting feedback from other sources, appraising the quality of possible answers, and comparing initial goals with results. In so doing an analyst employs interpretive and evaluative skills to select the best mass of evidence to analyze. Creating Better Inferences Critical thinking aims to ensure that inferences are reasonable and evidence-based. Inference creation begins at the same time that analysis starts. As each piece, or the whole mass of evidence, is considered, inferences are made. Resulting “chains” of inferences linking the evidence to the hypotheses under consideration are known as arguments. Chains of inferences converge – strengthening the argument – or diverge – weakening it. The self-reflective nature of critical thinking places a check on these inferences. The analyst asks, “Do my inferences flow from the evidence?” and, “Are my inferences logical given the evidence and other inferences I have made?” Inferences lead to a search for additional evidence. In other words, based on inferences drawn from the evidence at hand, an analyst may infer that there are other sources of evidence to consider. For example, such reasoning was cited in the explanation of why For a more detailed discussion of this and other means of countering biases, see Heuer, Psycholog y, 0– ; Irving Janis, Groupthink: Psychological Studies of Policy Decisions and Fiascoes, nd Edition (Boston, MA : Houghton Mifflin Company, 9); and Scott Plous, The Psycholog y of Judgment and Decision-making (New York, NY: McGraw Hill, Inc., 99). Indeed, the analyst’s acceptance – or rejection – of this evidence is a result of inferences: either the evidence is valid, credible, and relevant, or it is not. – 9 – the FBI investigated a pond in western Maryland during June 2003, searching for the source of the 2001 anthrax attacks. At the time, the FBI believed their primary suspect lived near the pond in question. In their search for additional evidence, they reasoned that the pond might be something in which evidence had been hidden. Thus, inference led the FBI to drain the pond to search for that evidence. The analyst self-consciously evaluates the thinking process and the biases that have affected it to reduce unproductive thinking and to consciously develop new ways of understanding the evidence at hand. This self-regulation also plays a role as the available evidence is considered. A means of accomplishing this within a collaborative setting is to seek the assistance of colleagues of diverse backgrounds. The underlying premise is that their biases differ sufficiently to enable productive and thorough analysis to occur. Producing Intelligence Analysts who produce intelligence assessments and judgments have the opportunity to employ what Peter Facione considers the core cognitive skills of critical thinking: interpretation, analysis, evaluation, inference, explanation, and self-regulation. Each of these competencies contributes to the integration of available evidence through a clear line of reasoning to the most probable answer. Facione asserts that good critical thinkers can explain “what they think and how they arrived at that judgment.” 9 By documenting the Washington Post, “No Anthrax.” See Robert Callum, “The Case for Cultural Diversity in the Intelligence Community,” International Journal of Intelligence and Counter Intelligence , no. , Spring 00: –. Peter A. Facione, Critical Thinking: What It Is and Why It Counts (Milbrae, CA : California Academic Press, 99, updated 00), URL, last accessed 22 July 2005, 4. Cited hereafter as Facione, Critical Thinking. 9 Facione, Critical Thinking, . – 0 – reasoning process used to arrive at an answer, analysts move beyond merely “stating results [to] justifying procedures, and presenting arguments.” 0 Questioning biases and mindsets encourages consideration of alternative possibilities that overcome what Josh Kerbel refers to as “single–outcome [analytic] trajectories.” Most importantly, analysts demonstrate a critical spirit – a reflection of character. Thus, critical thinking contributes to short-term analysis and assumes an essential role in longer-term analysis. Indeed, building a comprehensive picture of an issue or target requires critical thinking to determine which previous reports are included or excluded. “How do the parts contribute to the whole?” is one question the analyst asks. Another is, “How is the whole greater than the sum of its parts?” When previously published intelligence reports diverge, the critical thinking process helps the analyst ensure that the divergence is considered fairly and that the resulting intelligence does not merely satisfice. 0 Facione, Critical Thinking, . Josh Kerbel, “Thinking Straight: Cognitive Bias in the US Debate about China,” Studies in Intelligence no. (00), URL: , last accessed 22 February 2006. Facione, Critical Thinking, . See also Moore and Krizan, “Intelligence Analysis,” 9–. – – H ow cAN A NAlysTs be T A ugHT To T HiNk c riTic Ally ? Many people would rather die than think – in fact, they do. —Bertrand Russell Critical Thinking Education Outside the Intelligence Community Critical thinking offers a framework for structured problem solving. Yet, despite a corpus of associated literature, critical thinking remains in its infancy as a discipline. It is still “largely misunderstood…existing more in stereotype than in substance, more in image than in reality.” As Bertrand Russell’s humorous quip reminds us, critical thinking is not a habit acquired by just being alive. Ideally, valuable skills and dispositions should be developed among prospective analysts before they join intelligence-producing corporations. Yet, observations by the author of newly hired intelligence analysts suggest this happens rarely if at all. This raises two questions, “What are the opportunities for prospective analysts to become critical thinkers before they are hired?” and often “Why do these opportunities not exist?” Despite its importance, critical thinking is not widely taught in schools and universities. A mid-990s California study on the role of critical thinking in the curricula of public and private universities concluded that the skill is “clearly an honorific phrase in the minds of most educators.” The study concluded that university Richard W. Paul, “A Draft Statement of Principles,” The National Council for Excellence in Critical Thinking, URL : , last accessed March 18, 2003. The reasons why critical thinking remains an undeveloped discipline while important, go beyond the scope of this essay and are not addressed. 144 Richard W. Paul, Linda Elder, and Ted Bartell, “Executive Summary, – – faculty members “feel obliged to claim both familiarity with it and commitment to it in their teaching, despite the fact that…most have only a vague understanding of what it is and what is involved in bringing it successfully into instruction.” Indeed, the authors of the study found that while 89% of the faculty they interviewed “claimed critical thinking was the primary objective of their instruction,” only 19% could define the term and only 9% were evidently using it on a daily basis in their instruction. If the results of the California study are representative of the nation at large, they explain why prospective new hires – themselves college graduates – generally fail to exhibit skill in critical thinking at any level of proficiency. Informal conversations with recent hires at NSA support this premise. Although slightly fewer than half of these individuals indicate they have been exposed to critical thinking skills in college, most have been exposed only in one class and then only as an approach to learning the materials covered in that class. While not discouraged, respondents apparently were not encouraged to apply the skills to other subjects. Thus, a disposition to think critically is rarely fostered. Further, when asked to define critical thinking, most Study of Public Universities and Private Universities To Determine Faculty Emphasis on Critical Thinking In Instruction,” California Teacher Preparation for Instruction in Critical Thinking: Research Findings and Policy Recommendations, California Commission on Teacher Credentialing, Sacramento California, 99 (Dillon, CA : Foundation for Critical Thinking, 99 ), URL : , last accessed March 18, 2003. Cited hereafter as Paul, Elder, and Bartell, Executive Summary. Paul, Elder, and Bartell, Executive Summary. Richard W. Paul, Linda Elder, and Ted Bartell, California Teacher Preparation for Instruction in Critical Thinking: Research Findings and Policy Recommendations California Commission on Teacher Credentialing, Sacramento California, 99 (Dillon, CA : Foundation for Critical Thinking, 99), . 147 A recent but unscientific survey of several major eastern American universities’ online catalogs shows the term “critical thinking” to be used widely in course descriptions. Further examination of some of those courses suggests that East Coast academics may share a similar lack of understanding about critical thinking. Although beyond the purview of this paper, it appears that there is sufficient evidence to warrant repeating the California survey on a national basis. – – young analysts were unable to do so in any comprehensive fashion. Finally, few employ any form of critical thinking in their analytic (or other) reasoning unless they have been trained to do so. In fairness, it should be noted that critical thinking awareness, attitude, and skills varies from one academic discipline to another. For example, students of the physical sciences who employ the scientific method in a community setting probably have a greater inclination toward a basic form of critical thinking if only through osmosis. 9 There are also some primary, secondary, and university education programs that have adopted a meta-cognitive critical thinking paradigm. Although the results of the impact of such programs are largely anecdotal, at least one study, conducted by Jennifer Reed as part of her dissertation found that students’ critical thinking skills improved after just one course. 0 Faculty and administrators of these programs routinely attend the annual critical thinking conference sponsored by the Center for Critical Thinking where their results are shared. However, the approximately 500 people who attend this international event represent a small fraction of the educators in the United States. The elements of scientific method – the formulation of hypotheses, collection of relevant data, testing and evaluation of hypotheses, and the logical derivation of conclusions – are matched step-by-step by These conversations occur routinely as part of a training course the author teaches to newly hired intelligence and language analysts at NSA . When asked to complete the following statement, “In my opinion, critical thinking involves…” typical answers center on “thinking outside the box.” 9 Francis J. Hughes, conversation with the author, Washington, DC : National Defense Intelligence College, May 00. Mr. Hughes is one of the few proponents and teachers of evidence-based inferential intelligence analysis, a means of analysis requiring critical thinking at every stage in the process. 0 Jennifer H. Reed, Effect of a Model For Critical Thinking on Student Achievement In Primary Source Document Analysis And Interpretation, Argumentative Reasoning, Critical Thinking Dispositions, And History Content in a Community College History Course, PhD Dissertation, College of Education, University of South Florida, December 99 , vii. URL: , last accessed May 00. – – critical thinking. Given that most newly hired intelligence analysts – at least at NSA – are drawn from fields other than the physical sciences, one can expect that new hires lack adequate critical thinking skills. If NSA ’s newly hired intelligence analysts are representative of those being hired across the Intelligence Community then it is probable that few of the thousands of new hires arrive with adequate critical thinking skills. Critical Thinking Education Inside the Intelligence Community That analysts need to develop critical thinking skills is recognized within the Intelligence Community. Heuer wrote in 1999, “[Traditionally], analysts at all levels devote little attention to improving how they think.” As a direct response to Heuer’s criticism, the CIA ’s Sherman Kent School includes critical thinking as part of the curricula for training new analysts, and recently initiated a class in critical thinking. New employees are encouraged to develop a disposition to think critically as they are taught the skills of intelligence analysis. A similar approach is employed in courses on structured analysis methods at the National Defense Intelligence College ( NDIC ). Critical thinking is claimed as a feature of many NDIC courses. However, the emphasis in most courses is on topical or issue-related material, and only incidentally on the process of thinking. At present, the skill itself is largely not taught. Instead, students are expected to figure Steven D. Schafersman, “An Introduction to Critical Thinking,” January 1991, URL: , last accessed 9 March 00. Richards J. Heuer, Jr., Psycholog y of Intelligence Analysis (Washington, DC : CIA Center for the Study of Intelligence, 999), . Cited hereafter as Heuer, Psycholog y. For a detailed account of how the faculty of the CIA ’s Sherman Kent School are working to improve their analysts’ knowledge, skills, and abilities, see Stephen D. Marrin, “ CIA ’s Kent School: Improving Training for New Analysts,” International Journal of Intelligence and Counter Intelligence , no. (Winter 00 –00 ): 09–. – – it out on their own, encouraged by the thesis writing process. The college is aware of the importance of critical thinking and new core courses may include formal critical thinking instruction. New NSA analysts are provided with an introduction to the solitary part of the skill (as discussed in figure 4) as part of their orientation. The agency’s 40-hour follow-on program is the first Intelligence Community course to focus primarily on enhancing analysts’ solitary and communal critical thinking skills. In addition to critical thinking skills, participants learn and apply, and then assess, the appropriateness of fourteen structured methods of analysis. The course has been taught to both U.S. and Allied personnel drawn from intelligence, counterintelligence, information assurance, and law enforcement communities. Training in critical thinking is offered to Defense Intelligence Agency ( DIA ) analysts using a variation of the NSA-developed course. Also, in 00 , DIA tested the “critical thinking skills” of a sample of its employees using the Watson-Glaser Critical Thinking Appraisal . The instrument claims to “measure abilities involved in critical thinking, including the abilities to define problems, select important information for the solution to problems, recognize stated and unstated assumption, formulate and select relevant and promising hypotheses, [and] draw valid conclusions and judge the validity of inferences.” The appraisal appears to confuse abilities with skills, although both do belong within the domain of analysis. Finally, it appears that the test does not evaluate an individual’s meta-cognitive skills in assessing and correcting the process of reasoning. The increasing opportunities for enhancing critical thinking skills and dispositions reflect a recognition of the importance of critical 154 The syllabus from the course is in the Appendix. Harcourt Assessment, Inc., Local Nor ms Report, Watson-Glaser Critical Thinking Assessment, Report prepared for the Defense Intelligence Agency, 00, . Cited hereafter as Harcourt, Watson–Glaser. Abilities and skills are easily differentiated. Individuals are born with certain abilities (and can improve them through training) but they must learn a skill. See for example, Moore and Krizan, “ NSA .” – – thinking in intelligence analysis. Evidence becomes intelligence through an “ordered thinking process [involving] careful judgments or judicious evaluations leading to defensible conclusions” – through critical thinking. Former CIA analyst Morgan Jones asserts that methods for critical thinking and problem solving, if applied, can improve the quality of analysis and decisionmaking. 9 Implications of Teaching Critical Thinking Although Sherman Kent, Richards J. Heuer, Jr., and others have over the years addressed intelligence analysis and its relationship with critical thinking, recent presidential executive orders and legislative mandates are bringing a new emphasis to how the Intelligence Community can change analytic practices to achieve improved outcomes. Teaching analysts to be better critical thinkers may be seen as an easy way to satisfy these requirements. However, linkages between intelligence analysis and critical thinking remain poorly understood. Considerable confusion remains about what constitutes critical thinking and how it supports intelligence analysis. A common excuse among analysts to defend their “non-use of such self-conscious processes is a lack of time.” 0 Teaching critical thinking skills is of little value if analysts are not inclined to use them. For those who are willing to think critically, various The Director of National Intelligence, John D. Negroponte, in the Foreword of the initial National Intelligence Strateg y of the United States of America (Washington, DC : Office of the DNI , October 00), wrote that one of the three principal tasks for the reformed Intelligence Community is to “bring more depth and accuracy to intelligence analysis.” See http://www.dni.gov/NISOctober 00 . pdf. Moore and Krizan, “Intelligence Analysis,” . 9 Morgan D. Jones, The Thinker’s Toolkit: 14 Powerful Techniques for Problem Solving (New York, NY: Random House, Inc, 1995), xi. Cited hereafter as Jones, Thinker’s Toolkit. 0 Stephen Marrin, email to the author, December , 00. The author has heard similar complaints from analysts at NSA and DIA . Linda Elder, Alec Fisher, Diane Halpern, Gerald Nosich, Richard Paul, and others develop and publish the materials for – as well as teach – such courses. – – instructional models offer complementary means of expanding and enhancing analysts’ skills. For example, in an area studies class on Russia, students might be asked to evaluate and comment in depth on what different sources say about the influence of organized crime on the national government. Alternatively, critical thinking can be injected into the course as part of a problem-solving curriculum. Here students focus on how they think as they apply different strategies to each assignment, and then, ideally, transfer those enhanced skills to their day-to-day analysis. The use of realistic case studies makes classroom-acquired knowledge actionable. Self-reflective and group- reflective analyses of the process of reasoning keep the classes focused toward the working environment of analysts. Such transformations involve behavior modification and as such take time. Participants engaged in such instruction cannot be expected to become critical thinkers in a one-day or even a week-long course. The instructors of the NSA course attempt over a period of 0 weeks to transform their students into critical thinkers. Some other Community efforts also provide instruction over an extended period. Even so, students leaving the course are still novices in this practice and can slip back to their old methods of reasoning. Becoming critical thinkers requires a change in behavior that extends long past the end of the formal instruction. There is no substitute for continued practice. Finally, it serves critically thinking analysts poorly if their management and corporations are indisposed toward the application of the skills. It is well known that “engendering the desire to use [critical thinking] as a favored means of problem solving and decision making prepares the ground for teaching and learning the [critical thinking] skills.” Such encouragement can occur in the classroom but its effectiveness is limited unless the corporation encourages and welcomes strategies to employ critical thinking in the workplace. However, there are few metrics for determining the effectiveness of these materials and courses and so their value remains undetermined. The fact that the books developed for, and used in, such courses remain in print (in successive editions) is an indicator that at least this approach is popular, if not effective. Facione, “Disposition.” – – The best place for such encouragement is from senior and midlevel management. In order for this to occur, they too must be educated in the methods of, and reasons for, critical thinking. Evaluating Teaching Models Removing analysts from their work for education and training disrupts their primary mission of intelligence production. Spreading instruction over time provides a reasonable answer to this dilemma. Since improving critical thinking requires a high level of experiential, hands-on practice, an extended course offers students time to practice and apply what they are learning. This is the model used for the NSA course on critical thinking. Feedback from students indicates that once-a-week instruction works best. Mission is minimally disrupted and there is time to study and practice what is being taught. However, the long duration limits students’ work-related travel and leave. Experiential learning also requires that such classes be relatively small and that instructors be proficient. Without a large teaching staff, training a large workforce takes years. Since senior managers – and their strategic visions – change often, a long-term corporate commitment is crucial to success. Such a long-term commitment exists at the Sherman Kent School for Intelligence Analysis. The school itself grew out of a month-long course on analytic tradecraft developed by former Deputy Director for Intelligence Douglas J. MacEachin. Beginning in 99, the course was delivered to the entire analytic workforce. The school itself was established in 000 and continues to evolve as both a training center and a center of best analytic practices. Alternately, a large staff can be created, trained, and assigned to teach an entire workforce. This was accomplished in a knowledge domain at NSA in the 990s. If inducements and cultural change accompanied such a program, this approach might present certain advantages even if its costs are high. An analytic workforce could be transformed rapidly through such a “boot camp” approach. Marrin, “Kent School,” . Marrin, “Kent School,” 09. – 9 – Another means of critical thinking instruction is to create an interactive, computer-based or web-based course. This has the benefit of allowing as many analysts to attend as wish to do so. One or two instructors working full-time can answer students’ questions. Exercises allow student self-assessment. However, this is presently an unsatisfactory model for critical thinking instruction. Learning to think critically requires Socratic interaction and the impersonal nature of web-based instruction discourages the practice even for the population of dedicated solitary learners. In the future, multi- player simulations and other games could be used to reinforce the lessons learned. However, these are not yet available. Although computer- and web-based instruction can be accomplished at the analyst’s desk, there are other reasons why this may not be a good idea. Analysts who remain in their offices are subject to interruptions. Further, course work is often relegated to the end of the workday, when analysts are tired and learning is impaired. Little learning occurs when taking a class is a chore. The current limitations of both classroom and computer- or web- based education suggest that experimentation with new means of instruction for critical thinking is needed. Encouraging Analysts to Think Critically Incorporating critical thinking into both orientation and basic analyst education and training is one way to help newly hired analysts develop their skills. Subsequently placing those analysts in offices where critical thinking is practiced is another way to encourage an analytic culture that fosters thinking critically. Through direct exposure to successful, experienced analysts, junior analysts’ doubts about the employment of critical thinking techniques can be overcome. “Skills pay” can be an inducement for analysts to learn and then employ critical thinking. Rewarding the acquisition, maintenance, and use of other special skills is common in government and industry. For example, linguists at NSA earn significant bonuses for achieving specific levels of foreign-language proficiency – an effective way – 0 – to maintain critical language competencies within one intelligence agency’s workforce. Given the high-level concern for the health of the intelligence analysis process, direct monetary incentives should be available for analysts to master new tools and demonstrate constructive analytical behaviors. On the other hand, if an intelligence enterprise fails to recognize and reward the acquisition and application of analytic skills such as critical thinking, it sends a very clear message: the enterprise does not value those skills. Faced with such a message and a lack of inducements to excel, the best and brightest analysts may leave, especially from the more junior ranks – and midlevel employees with families and mortgages may “retire in place.” Either outcome hurts mission-critical functions, and can be avoided. As a previous Director of NSA , Air Force Lieutenant General Kenneth A. Minihan, noted, “If we don’t win the talent war, it doesn’t matter what we invest in the infrastructure.” Recognizing and rewarding critical thinkers is one way to win that talent war – with the assumption that really good analysts are more likely to remain active within their intelligence agency workforces. If an analyst adopts a congeries of skills that contributes to the mastery of critical thinking, and is compensated monetarily, that mastery needs to be certified. If a curriculum that drives the acquisition of those skills is in place, then an assessment of those skills can be administered in-house. Specific tests exist for the assessment of critical thinking, such as the “Thinking Skills Assessment Test.” Persuading to Improve Analysis Teaching critical thinking is but a first step toward improving analysis. Because analysts and managers have different needs and LtGen Kenneth A. Minihan, USAF, Director, NSA , in Robert K. Ackerman, “Security Agency Transitions from Backer to Participant,” Signal , no. (October 99), . 167 For more information, see http://tsa.ucles.org.uk/index.html, last accessed March 00. – – time constraints, multiple versions of a course are needed to meet the needs of each group. Analysts and first-line supervisors may take an entire course, midlevel managers an abbreviated course, and senior managers an overview. This strategy already is employed at the CIA ’s Sherman Kent School where new employees spend weeks learning analysis skills, techniques, and methods while supervisors, depending on their seniority, spend three days or one day becoming familiar with these same skills and methods. The analysts also have opportunities to apply new skills immediately as they perform four- week “internships” in various CIA offices. Other CIA analysts, already in place, do not have the same opportunities to receive this training. Thus a two-tiered analyst population with more skilled junior analysts and more knowledgeable senior analysts is being created. Ideally, each will transfer skills and knowledge to the others. However, an alternate possibility is that it will generate distrust and animosity between the two groups. Older, more experienced analysts may resent the opportunities given to their newer counterparts. This issue is not trivial. Offering the wrong curriculum to the wrong groups of analysts and managers can destroy its effectiveness. Managers in the CIA and elsewhere can influence how the new analytic methods being taught will be adopted, as agencies respond to new legislative and executive mandates. Will analysts embrace critical thinking as a means to improve their analysis? There are numerous observations of analysts’ reluctance to adopt new paradigms. This is true even when analysts are confronted with the fact that their conclusions were wrong. In referring to her work on the issue of Iraqi WMD , one CIA analyst told Senate investigators, “their ‘bottom line’ judgments would have remained the same.” 9 Rob Johnston also found a similar reluctance to change opinions among the analysts he studied. He noted that Stephen Marrin, “Homeland Security and the Analysis of Foreign Intelligence,” Markle Foundation Task Force on National Security in the Information Age, July 00, URL: , last accessed December 9, 2003, 9. Cited hereafter as Marrin, “Homeland Security.” 9 SSCI, Iraq, 99. – – although “analysts can change an opinion based on new information or by revisiting old information with a new hypothesis,” they perceive a loss of trust and respect and a subsequent loss of “social capital, or power, within [their] group.” 0 Yet, a case can be made that analysts, and especially experienced analysts, will benefit the most from enhanced critical thinking skills training. These are the analysts who are in positions of technical leadership, who work the most difficult aspects of complex targets. There may be significant consequences if they fail to notice and make sense of an issue. On the other hand, the fact that many of these senior personnel will soon be eligible for retirement raises an important question: Does the corporation get added value from teaching analysts who will soon retire to think more critically in their analysis if they are predisposed not to do so? Maybe so. Heuer, while the head of CIA ’s Methods and Forecasting Division found that analysts, once persuaded to use new analytic methods, found them “interesting and well worth doing.” Other intelligence analysts have adopted new analytic methods that add value to their analyses. In one example, NSA personnel involved in research and development adopted a means of matching target characteristics and their vulnerabilities with exploitation capabilities and their costs. Such analyses helped ensure that appropriate resources were dedicated to collection and that such collection was better tailored to production analysts’ needs. However, some research analysts initially refused to employ the model, claiming it took too much time even as it reduced the volumes of information 0 Johnston, Analytic Culture, . Richards J. Heuer, Jr., Adapting Academic Methods and Models to Governmental Needs: The CIA Experience (Carlisle, PA : Strategic Studies Institute, U.S. Army War College, July 9), . Referenced in Marrin, “Homeland Security,” 9. 172 In this context, a “target” refers to an entity – geographical, individual, or topical – in which the Intelligence Community has an interest. – – they were required to examine. Adoption of the analytic paradigm required appropriate persuasion. One interesting means of self-persuasion may be analysts’ frustration with the really hard problem. This frustration may lead analysts to try something new. Such frustration occurs more often among experienced analysts. Years of attempting to make sense of overwhelming masses of information with inadequate analytic paradigms and technologies leaves some experienced analysts willing to grasp at anything that will improve how they work. On the other hand, newly hired analysts who have not yet experienced the frustration of inadequate paradigms for analysis may be resistant to adopting rigorous analytic paradigms such as that afforded by critical thinking. Multiple midlevel NSA analysts, interviews with the author, 99 – 00. LT Robert D. Folker, USAF, email to the author, 9 December 00. Cited hereafter as Folker, email, 9 December 00. Folker, email, 9 December 00 . – – H ow Does c riTicAl T HiNkiNg T rANsforM ? Transforming Intelligence Corporations In order for an institution to change, all affected personnel, from the lowest to the highest, must first recognize that change is needed and is advantageous – both corporately and personally. As retired World Bank executive Stephen Denning argues, this is difficult to accomplish. Logically sound arguments do not sway employees. Instead, employees remain convinced that what they are doing is satisfactory. Further, outsiders who attempt to induce change face opposition because employees presume that external consultants are arrogant in suggesting that things are not right, and that change is needed. One way to effect change is through a “springboard story.” This approach contrasts with past conventional – and largely unsuccessful – transformation efforts that relied on fixing the systems involved and were characterized by an overabundance of buzzwords: Enhance quality. Streamline procedures. Reform and flatten the organizational structure. Analyze things in terms of grids and charts. Develop plans in which individuals are programmed to operate like so many obedient computers. Hone our interpersonal mechanics and build skill inventories. Bring to our difficulties a fix-it attitude, as though our past errors can be easily corrected with straightforward explanations. For a discussion of Denning’s philosophy about change, see Jeff de Cagna, “Making Change Happen: Steve Denning Tells the Story of Storytelling,” Information Outlook, January 00 , – , and Stephen Denning, The Springboard: How Storytelling Ignites Action in Knowledge-Era Organizations (Boston, MA : Butterworth- Heinemann, 00). Cited hereafter as Denning, Springboard. Denning, Springboard, xvii. In 23 years of Intelligence Community work, the author heard all of these phrases as slogans for various programs designed to “fix” analysis or the corporations involved. These associated corporate efforts to bring these goals about often had the opposite effect. In the process of “flattening – – As Denning points out, these strategies fail to account for the messy and chaotic reality in which real organizations live and work, especially intelligence agencies which have the charter to engage honestly with “the other,” or mission-related targets, outside of the bureaucracy in which they serve. The springboard story helps employees at all levels envision what is needed for the proposed transformation. Denning asserts that it “invites them to see analogies from their own backgrounds, their own contexts, their own fields of expertise.” 9 He cautions that transformational stories are not a panacea – there are situations and circumstances in which they are not effective, such as when the change being proposed is a bad idea. 0 The key is finding the appropriate stories within the corporate culture. One way to find these stories is to examine the best practices of intelligence analysis successes where critical thinking played a role. Just recounting a success is not enough. Listeners need to be able to identify and empathize with the scenario and its actors. The effective story is “about people who have lived [a] knowledge- sharing idea and how things happen in a real-life situation.” For example, conducting a trial or experiment on critical thinking within the intelligence analysis process, if it leads to certifiable, actionable success, does effectively persuade. Learning from Early Adopters Once a sustained number of early adopters openly apply systematic, critical thinking to hard analytic problems, the stage is set for a “tipping point” in the spread of structured methods across the organization” at one intelligence agency, more hierarchical managerial layers were actually created! Denning, Springboard, xvii. 9 Denning, Springboard, xix. 0 Denning, Springboard, xxi. Denning, Springboard, . – – analytic populations. Stanley Feder recounts how a tipping point began to apply to a political analysis method at the CIA in the 90s and 90s. The method involved two estimative tools known as Factions and Policon that were used by the “Intelligence Directorate and the National Intelligence Council’s Analysis Group to analyze scores of policy and instability issues in over 0 countries.” The reasons for the adoption of these tools remain the same: “Forecasts and analyses…have proved to be significantly more precise and detailed than traditional analyses.” Writing about the method in 1987, Feder predicted that its use would continue to expand. The method is still in use 9 years after Feder’s article was published. However, expanded use failed – perhaps because the tool was on a computer platform that ceased to be supported by the Agency. The recent transfer of the tool to a new suite of programs corresponds with observations that its use is once again expanding. Non-intelligence-related transformational stories can be applied in the Intelligence Community to facilitate the spread of new ways Malcolm Gladwell, The Tipping Point: How Little Things Can Make a Big Difference (Boston, MA : Little Brown and Company, 000), –9. Gladwell shows how “social epidemics” can infect a variety of domains. Stanley Feder, “Factions and Policon: New Ways to Analyze Politics,” in H. Bradford Westerfield, ed., Inside CIA ’s Private World: Declassified Articles from the Agency’s Internal Journal, 1955–1992 (New Haven, CT: Yale University Press, 99 ), . Cited hereafter as Feder, “Factions and Policon.” Feder, “Factions and Policon,” 9. Feder, “Factions and Policon,” 9. Feder was wrong about its sustained popular growth. 186 This story is not an isolated instance. In the author’s experience, initial implementation and popularization are often followed by a gradually reduced user-base. Certain organizations find the tools useful and they tend to continue to use them even though any underlying technology may be obsolete. In the case of Factions and Policon, the tools were maintained on an aged Macintosh computer. They were updated in 00 as part of the work of an intelligence research firm in New Mexico. The Factions tool was rewritten and included in Landscape Decision®, a suite of modeling and simulation tools developed under a research contract with the Department of Defense’s Advanced Research and Development Activity ( ARDA ). The updated technology was reinserted into the tasking organization. Other technology transfers are pending. – – of thinking. For example, two video commercial advertisements from the Electronic Data System ( EDS ) company resonate with the challenge of rebuilding intelligence analysis in the face of skeptics, and what it takes to be an intelligence analyst. The two related commercials depict farcical situations based on two well-known clichés: “You’ve got to build it before you can fly it!” and “It’s like herding cats; it can’t be done!” In the former, a group of mechanics is building an airplane in mid-flight; while in the other, a group of cowboys has accomplished the impossible – they are herding cats across the Great Plains. EDS’ depiction of its revolutionary design capabilities bears a message about corporate rebuilding and individual transformation. The message encourages managers and analysts to compare themselves and their situation to the people tasked with building an airplane at “0,000 feet,” or with herding cats across a river. Intelligence analysts and managers who have seen the videos have had little trouble drawing analogies with the corporate and personal transformations needed in the Intelligence Community. That the situations are humorous only adds to a viewer’s buy- in – they are “hooked” before they realize it. Developing other stories that specifically apply to corporate transformation within the Intelligence Community and its various elements is a logical next step in encouraging analysts and managers to use and support critical thinking in analysis. If these stories can be drawn from recent analytic successes, their value can only increase. The Costs and Benefits of Thinking Critically Are intelligence failures inevitable if analysts lack critical thinking skills? Yes. However, critical thinking is not a panacea. Intelligence failures still can be expected. If no measure “will guarantee that accurate conclusions will be drawn from the incomplete and ambiguous information that intelligence analysts See Building Airplanes in Flight , television advertisement produced for EDS by Fallon-McElligott, November 000, videocassette; and Herding Cats, television advertisement produced for EDS by Fallon-McElligott, January 000, videocassette. – – typically work with…[and] occasional failures must be expected,” why should the Intelligence Community invest in teaching its analytic workforce to think critically? A brief review of the causes of failure clarifies the need for such an investment. Intelligence failures occur in both systematic and functional domains. Systematic intelligence failures occur when producers fail to notice phenomena and the consumers of intelligence fail to heed warnings or even to notice that they are being warned. It may be that both consumers and producers are focused on other issues at the time. Alternatively, intelligence-based warning is not taken seriously at high official levels. In cases where collaboration among intelligence agencies might be critical, key evidence may not be recognized and shared. All of these failings are illustrated in the “intelligence failure” to warn of the attacks of September 00 on the World Trade Center in New York and the Pentagon near Washington, DC . In the first place, the policy issue that was at the forefront – according to press at the time – was a national missile defense system, not the likelihood of a terrorist attack on the United States. Richard Clarke, the Clinton and then Bush administrations’ terrorism “czar” observes that Bush administration officials were slow to recognize or consider the threats posed by Osama bin Laden. 9 Evidence passed by an FBI field agent about students learning to fly large aircraft with little regard for landing or takeoffs was not taken seriously at higher levels within the Bureau. 90 The Joint Congressional investigation into the attacks noted in its report that collaboration was lacking between agencies, especially between the law enforcement and strategic Heuer, Psycholog y, . 9 “Clarke: Bush Didn’t see Terrorism as ‘Urgent,’” CNN.com, 9 May 00 , URL: , last accessed 9 March 2006. 90 Michael Elliott, “How the U.S. Missed the Clues,” CNN.com, 0 May 00 , URL: < http://archives.cnn.com/ 00 /ALLPOLITICS/0 / 0/time. clues/index.html>, last accessed 9 March 2006. – 9 – intelligence agencies. 9 This was both a cultural and a physical phenomenon. Airport security breaches during the summer of 00 may have been related, but based on assessments of the catastrophe, were not connected to the events. 9 The same kinds of failings were identified by the Senate Select Committee on Intelligence in its assessment of the National Intelligence Estimate on Iraqi WMD. As has already been observed, analysts and their managers were focused on the results and not the process. 9 Critical thinking can mitigate some common causes of failure and provide means by which they can be avoided in the future. Specifically, an intelligence process based on critical thinking offsets the following failures: Analysts are Wrong. It is unrealistic to expect that analysts can always be correct. Regardless of the processes they employ, analysts make errors and fail. Anthropologist Rob Johnston defines errors as “factual inaccuracies in analysis resulting from poor or missing data.” Conversely, intelligence failures are “systemic organizational surprise resulting from incorrect, missing, discarded, or inadequate hypotheses.” 9 Critical thinking mitigates these by providing means to assess errors in reasoning as they occur and before they become 9 U.S. Congress, Joint Inquiry into Intelligence Community Activities Before and After the Terrorist Attacks of September 11, 2001, Report of the U.S. Senate Select Committee on Intelligence and U.S. House Permanent Select Committee On Intelligence, Together with Additional Views, Senate Report No. 0– /House Report. No. 0–9, 0th Congress, nd Session, December 2002, xvi. 9 The author recalls that media reports of persons hopping over or otherwise improperly passing airport security checkpoints in and around Boston, New York, and Washington seemed to be higher than usual during the summer of 00. 9 DNI Negroponte, in the National Intelligence Strateg y of the United States (October 00 , ) highlights the notion that the Community appropriately undertakes study of its internal processes, as well as of the quality of its products, by addressing both “enterprise” objectives as well as “mission” objectives. 9 Johnston, Analytic Culture, . – 0 – systemic failures. Such a meta-cognitive approach to the analytic process helps keep it under active review at the highest levels. Policymakers Ignore Intelligence. 9 A critically thinking analytic population cannot directly affect what a policymaker can or will do – neither in fact, can a non-critically thinking analytic population. What critically thinking analysts can do, however, is present more effective assessments, perhaps leading policymakers to question their own assumptions on the issues. Additionally, thinking critically about how analysts interact with policymakers can identify ways to restructure the analysis dissemination process to involve policymakers more effectively. Such a process might also encourage them to adopt some of the attributes of critical thinking leading to (it may be presumed) more effective policy. 9 Adversary Denies and Deceives. Critical thinking reduces the effects of adversarial denial and deception by leading analysts to consider alternative possibilities, to question biases and assumptions, to examine systematically the validity of evidence being considered, and to take seriously anomalies in the evidence. Adversary is More Capable. In any adversarial system, there are winners and there are losers. While analysts can do everything possible to ensure their work is correct, they rarely work with all the evidence, and indeed may still be deceived. In such cases, they may come to wrong conclusions. Critical 9 There is a “classic” argument as to whether this is or is not an “intelligence” failure. In summary, the two sides condense as follows: On the one hand, intelligence should have been so persuasively presented as to compel the policymaker to pay attention. On the other hand, intelligence should not be telling the policymaker what to do. The argument goes all the way to the roots of the post-World War II strategic intelligence system currently in place. Former CIA analyst Jack Davis summarizes the issue in “The Kent-Kendall Debate of 99,” Studies in Intelligence , no. (Summer 99): –0 . 9 Cooper, email to the author, March 00 . Cited hereafter as Cooper, email, March 00. – – thinking, however, by providing structure and oversight to their reasoning, provides an audit trail. In this case, the means by which the analytic conclusions were reached can be subsequently reviewed, errors and deceptions revealed, and steps taken to improve the process so that the failure is not repeated . Indeed, because of its focus on the process, critical thinking becomes a powerful tool for evaluating and enhancing analytical reasoning. Investment in critical thinking as part of the analysis process minimizes the likelihood of specific failures. With critical thinking essential to so many parts of the intelligence production process, enhancing it increases the likelihood of intelligence success. 9 With the cost of failure catastrophically high, the Intelligence Community is well advised to improve the likelihood of intelligence success, including improving critical thinking. As diplomat Richard Holbrooke opined in 00 , “intelligence is…indispensable. And its greatest successes are preventative.” 9 There are other reasons why systemic intelligence failures occur. Jeffrey Cooper considers ten pathologies, summarized in table , that impede successful analysis at both the individual and corporate levels. Cooper specifically believes that given the [emphasis] on the systemic nature of the pathologies that afflict intelligence analysis, structured analytic methods 9 One metric for intelligence is decreasing intelligence failures. In this case, fewer failures could be considered as “improving intelligence.” But the reasons for the prevention of the failures need to be considered. The reduced number of failures could be due to a critically thinking workforce. Or it could be due to coincidences. Although a pointed discussion of metrics for intelligence is beyond the scope of this paper, recent articles explore the issue of intelligence metrics. See, for example, David T. Moore, Lisa Krizan, and Elizabeth J. Moore, “Evaluating Intelligence: A Competency-Based Model,” International Journal of Intelligence and Counterintelligence , no. (Summer 00): 0–0. 9 Richard Holbrooke, in Judith Miller, “A Battle of Words Over War Intelligence,” New York Times, online edition, November 00, URL: , last accessed November 28, 00. – – become a first line of defense in preventing the networks of errors from developing – they are like “ripstops” that keep problems from propagating into wider “error -inducing systems,” in [Charles] Perrow’s terms. 99 99 Jeffrey Cooper, email to the author, March 00. Cooper refers to Charles Perrow, Nor mal Accidents: Living with High-Risk Technologies (Princeton, NJ: Princeton University Press, 999). Cooper’s Analytic Pathology Description Inefficient “Account” Structure Work is subdivided into accounts with ownership; provides basis for accountability; ownership inhibits sharing, cooperation, and collaboration; encourages “stovepiping” in collection disciplines. Evidence-Based Scientism Descriptive and explanatory intelligence support current policy and military operations; less support for anticipatory intelligence; analysis needs to be “augmented and leavened with intuition, curiosity, and a thirst for discovery – all essential elements of good science.” Tyranny of Current Intelligence Response to customers’ current concerns; support to current, ongoing military operations; little long- term research. Over-Emphasis on Production Databases need filling; scheduled reports and assess – ments must be produced on time; metrics for suc – cess measure data collected and processed; number of reports issued used to determine and justify re – sources allocated. Over-Reliance on Previous Judgments Previous reports presumed to be authoritative; pre – vious reports combined to form new judgments; agreed-upon positions retained despite newer con – tradictory evidence. Table 6: Cooper’s Analytic Pathologies Source: Summarized from Cooper, Pathologies , 0–9. – – Critical thinking combats this by enhancing the processes by which decisions are made and by which such processes are reviewed. This paper focuses primarily on individual experience , and of course, intelligence corporations are made up of individuals. If a sufficient number of intelligence professionals are thinking critically, it is likely they will transform their corporations, if only through the sheer attrition of non -critically thinking managers and leaders who are Cooper’s Analytic Pathology Description Neglect of Research Short–term taskings prevent longer–term research; reward structure favors current reporting, not lon – ger term work; stunts development of deep target expertise. Neglect of Anticipatory Intelligence “Information Revolution” leaves intelligence com – peting with journalism and Internet for policymak – ers’ attention; lack of predictive intelligence against new emerging threats ensures continuing failures to warn. Loss of “Keystone Species” and “Intellectual Middleware” Periodic reductions in force skew analyst demo – graphics; “Keystone Species” (journeymen analysts) lacking; journeymen do bulk of professional mainte – nance; essential for knowledge retention and trans – fer to apprentices; maintain “intellectual Middle – ware” or deep understanding of analytic domains and processes. Lack of Analytic Tools and Methods of Validation Available and proposed tools not formally validated for accuracy and usefulness; focus on results, not processes, to determine success. Hindrances of Security Mindset Security procedures impede multi–source analysis; protection of sources and methods more impor – tant than “effective exploitation and cross–fertiliza – tion;” artificial and outdated mission distinctions prevent collaboration. – – retiring from the work force. It should be observed that an Intelligence Community enterprise -wide emphasis on developing critical thinking skills at all levels would speed the process. Validation A critically thinking Intelligence Community remains essential for effective intelligence reform. Steven Rieber and Neil Thomason advance this argument in their recent article . The authors assert that “the opinions of experts regarding which methods [of analysis] work may be misleading or seriously wrong.” 00 Unfortunately, as the authors show, past (and present) efforts at intelligence reform rely on expert intuitive judgments. However, “[examples] from a wide range of fields show that expert opinion about which methods work are often [not only] dead wrong…but also are generally not self -correcting.” 0 Further support of Rieber’s thesis is seen in devil’s advocacy – not as theoretically applicable but as practiced. They cite Irving Janis who, quoting a Stanford political scientist, notes that “instead of stirring up much -needed turbulence among the members of a policymaking group, [devil’s advocacy] may create ‘the comforting feeling that they have considered all sides of the issue.’” 0 To mitigate this and similar analysis -improvement fallacies, Rieber and Thomason argue that improvements in analysis and any proposed methods of judgment and decision -making require validation through scientific study. They note, for example, that research reveals “[a] certain cognitive style, marked by an open – mindedness and skepticism toward grand theories, is associated with 00 Rieber and Thomason, “Creation of a National Institute for Analytic Methods,” . 0 Rieber and Thomason, “Creation of a National Institute for Analytic Methods,” . 0 Irving L. Janis, Groupthink: Psychological Studies of Policy Decisions and Fiascoes, nd edition. (Boston, MA: Houghton Mifflin and Company, 1982), 268. Referenced in Rieber and Thomason, “Creation of a National Institute for Analytic Methods,” . – – substantially better judgments about international affairs.” 0 As the present paper has argued, such attitudes are found in critical thinkers. Scientific study of this and other methods will determine when and how they are appropriate. Critical thinking also supports Rieber and Thomason’s call for a “National Institute for Analytic Methods” by providing an overarching structure to champion open -mindedness and skepticism in the study of which methods are appropriate for intelligence analysis. Only then, they argue, will real intelligence reform and improvement occur. Rieber and Thomason’s proposed institute could determine which strategies are most effective at disposing analysts and their corporations to employ critical thinking. Research evidence indicates that simply teaching critical thinking (or for that matter, structured analytic) skills is insufficient. People will not adopt the strategies unless motivated to do so. As noted here, springboard stories are one means of implanting a positive disposition toward critical thinking. Other means certainly also exist. A desirable objective of research in the proposed National Institute for Analytic Methods would be to discover and assess what might motivate analysts most effectively toward thinking critically. 0 Rieber and Thomason, “Creation of a National Institute for Analytic Methods,” . – – w HAT o THer P oiNTs of v iew e xisT ? A recent best -selling book advances the comfortable idea that conscious reasoning may not confer advantage to the reasoner. 0 Recent research suggests that whereas simple choices may benefit from conscious thought, complex issues are best left to unconscious thought, or “deliberation -without -attention.” 0 The explanation of this finding is that in conscious thinking, people face a severe limit on the number of factors that they can effectively consider simultaneously and, second, that in conscious thought people “inflate the importance of some attributes at the expense of others.” 0 The authors base this finding on four experiments with subjects who were asked to indicate their preference for various consumer items. The experiments involved differing levels of complexity in terms of factors to be taken into consideration. They ultimately suggest that: [there] is no a priori reason to assume that the deliberation – without -attention effect does not generalize to other types of choices – political, managerial or otherwise. In such cases, it should benefit the individual to think consciously about simple matters and to delegate thinking about more complex matters to the unconscious. 0 It appears true that the human capacity to weigh evidence consciously is limited to approximately seven factors. 0 But this seven – 0 Malcolm Gladwell, Blink: The Power of Thinking Without Thinking (New York, NY: Little Brown and Company, 00). Gladwell argues that thinking does not require detailed assessment of information. Instead, rapid cognitive responses are adequate for decisionmaking. 0 Ap Dijksterhuis, Martin W. Bos, Loran F. Nordgren, and Rick B. von Baren, “On Making the Right Choice: The Deliberation-Without-Attention Effect,” Science , no. ( February 00 ), 00 . Cited hereafter as Dijksterhuis, “Deliberation-Without-Attention.” 0 Dijksterhuis, “Deliberation-Without-Attention,” 00. 0 Dijksterhuis, “Deliberation-Without-Attention,” 00. 0 George A. Miller. “The Magical Number Seven, Plus or Minus Two,” The Psychological Review (9 ), . The paper is available online: URL: < http://psychclassics.yorku.ca/Miller/>, last accessed 14 March 2006. – – item limit is easily extended: people can hierarchically consider multiple sets of seven items. 09 Employing structured methods of reasoning also extends this capacity. In this context, critical thinking is at once both a structuring method and a means of assessing and monitoring the processes of selecting and using other structured methods. Additionally, “deliberation -without -attention” as applied to consumer choices in the study noted above was measuring preferences. Preferences change from person to person – different people conclude differently about oven mitts and cars – as the variety of these products on the market, in kitchens, or on the roads makes clear. 0 Intelligence analysis is not about preferences; it is about best answers in ambiguous situations with high -stakes implications and consequences. As recent intelligence failures make clear, without forced consideration of alternatives, results are biased. Structured reasoning methods such as Analysis of Competing Hypotheses (ACH) develop these alternatives, allow for multiple factors to be considered fairly, and extend analyst capacities to assess complex situations as accurately as possible. Intuition is what both Dijksterhuis and Gladwell consider in their respective works. However, exactly what is intuition? Rather than just appearing from nowhere, intuition is “almost always informed by experience and hard knowledge won by reasoning things out.” As Michael LeGault notes, good decisions [are] a nuanced and interwoven mental process involving bits of emotion, observation, intuition, and critical thinking….The essential background to all this is a solid base of knowledge…The broader the base, the more likely all the parts will fit together. 09 William Reynolds, conversation with the author, March 00. 0 Oven mitts and cars are two of the categories for which Dijksterhuis and his colleagues tested. LeGault, Think, . LeGault, Think, . – – Uninformed “intuitive thinking” contributes to intelligence failures because it fails to reflect on presuppositions. Such unexamined biases and mindsets contributed to the Cuban missile intelligence failure described above. Heuer’s outline of the specific biases and mindsets that impede effective analysis is reinforced by the findings of others. For example, Merom, writing about the Cuban crisis, concludes that intelligence failures occur due to a “lack of commitment to fundamental principles of investigation and analysis.” Although intuition or “thin slicing” may be appropriate in some domains, in intelligence analysis it appears to be associated with cognitive impediments that cause costly intelligence failures. However, as Rieber and Thomason note, testing needs to be conducted to determine which methods do work best in which situations in intelligence analysis. Their work ought to go further, developing and modifying methods to overcome the very limitations they identify. For example, investigators have found that ACH suffers from confirmation bias. The present author notes that this may be because non -confirmatory reasoning is so difficult for people to do. 213 See for example, Scott Plous, The Psycholog y of Judgment and Decision-making (New York, NY: McGraw Hill, Inc., 1993); Thomas Gilovich, Dale Griffin, and Daniel Kahneman, Eds. Heuristics and Biases: The Psycholog y of Intuitive Judgment (Cambridge, UK : Cambridge University Press, 00); and Daniel Kahneman, Paul Slovic, and Amos Tversky, eds. Judgment under Uncertainty: Heuristics and Biases (Cambridge, UK: Cambridge University Press, 9). Merom, “Estimate,” 9. Gladwell introduces the notion of “thin slicing” as a reasoning strategy in Blink . Rieber and Thomason, “Creation of a National Institute for Analytic Methods,” . Brant A. Cheikes, Mark J. Brown, Paul E. Lehner, and Leonard Adelman, Confir mation Bias in Complex Analyses, Mitre Technical Report MTR 0B00000 (Bedford, MA : Center for Integrated Intelligence Systems, October 00). The author teaches Analysis of Competing Hypotheses to both new and experienced analysts at the National Security Agency. He notes that holding a non-confirmatory attitude – which ACH , as developed by Heuer, requires – is – 9 – Finally, it may be a fallacy that conscious and unconscious reasoning are truly separate. Instead, Morgan Jones (citing work by Richard Restak) asserts that “the unconscious has a governing role in much that we consciously think and do.” 9 This means that analysts commit a variety of “analytic sins,” including focusing on the initially favored solution – which is also often the first satisfactory solution considered. 0 Part of the problem seems to arise from confusion about “‘discussing/thinking hard’ about a problem and ‘analyzing’ it, when the two activities are not at all the same.” Ultimately, the unconscious or instinctive approach to reasoning seems to “remain closed to alternatives.” This is an unsatisfactory model for intelligence analysis where alternatives must be considered. extremely difficult for people to do. Since the method has value, a means of mitigating this difficulty is worth developing. 9 Jones, Thinker’s Toolkit, 0. Jones bases his assertions and argument on the work of Richard Restak, The Brain has a Mind of Its Own (New York, NY: Harmony Books, 99). 0 Jones, Thinker’s Toolkit, . Jones, Thinker’s Toolkit, . Jones, Thinker’s Toolkit, . – 90 – w HAT D oes THe f uTure H olD ? Looking ahead, new paradigms for analysis become necessary in light of the many changes occurring across the globe. Rapidly emerging twenty-first century issues and challenges stress both infrastructures and sensemaking enterprises. The changes needed to maintain an edge against adversaries becomes clear as one considers what it is that analysts do, and how and when, as table illustrates. Gregory Treverton’s observations about the evolution of intelligence sensemaking and the organizations that comprise the national intelligence enterprise suggest that current “reform” efforts are merely the first steps in a much lager – and fundamental – transformation. A careful, considered examination of what intelligence must accomplish and how this can be best achieved may ensure that the goals Treverton identifies as necessary for the future are met – and met sooner rather than later. Here again critical thinking has much to offer. By encouraging discussion of the alternatives and their advantages and disadvantages, and by examining analogous cases, critical thinkers aid the transformation. Questioning of assumptions about the roles of intelligence, its activity, and organization will reveal which long-held beliefs need retirement. Further, technology efforts in support of analysis, such as those being developed through the efforts of the ARDA – and other – advanced technology efforts presume that analysts think critically. For instance, by focusing “analytic attention on the most critical information found within massive data,” some research projects seek to uncover otherwise unknown information that indicates the potential for strategic surprise. These projects seek to build Critical information is that which is essential for analysts to make sense of what their targets are going to do. Often it is buried in massive amounts of “noise.” Advanced Research and Development Activity, Broad Agency Announcement for the Novel Intelligence from Massive Data ( NIMD ) R&D Program, NMA 0–0– BAA – 000, JUNE 00, . – 9 – partnerships between analysts and technology applications. In one case, a research firm has developed technology that enables analysts to generate and validate novel hypotheses, distill complex systems into digestible elements, create effective narratives, and inspire creative thinking. Other examples are illustrated in the adjacent text box. Follow-on research efforts such as the Collaboration and Analyst/ System Effectiveness ( CASE ) effort will develop these efforts further. The intelligence workforce these ARDA -supported efforts presuppose is being hired and trained now. The future also holds opportunities for further developing models of reasoning in intelligence. Exactly how analogy and critical thinking interact, the roles of creative thinking, the roles of intuition, and specific strategies for bias mitigation are all areas warranting further research. Formal validation of how critical thinking improves intelligence reasoning is another research topic. Rieber and Thomason’s proposed National Institute of Analytic Methods is an obvious venue for such research. Additionally, the ARDA advanced questioning and answering research program, AQUAINT , as well as other ARDA programs dealing with predictive analysis, prior and tacit knowledge, and hypothesis generation offer other domains in which such research could occur. Such research is of both intrinsic value and necessary for developing new means 225 The five areas on which the NIMD effort focused were modeling analysts and how they work; analysts’ prior and tacit knowledge; creating, representing, and tracking multiple scenarios, hypotheses, and/or strategies; massive data management and analysis; and the means by which analysts interact with technology. The author represented his agency in evaluating and mentoring the efforts of the researchers. 226 The author is a technical mentor of this firm’s work. For more information, see “Collaboration and Analyst/System Effectiveness ( CASE ),” AFRL Directorate: IFED BAA 0–0– IFKA , October 2005, URL: , last accessed 11 March 00. Advanced Research and Development Activity ( ARDA ), “Preliminary Solicitation Information for Advanced QUestion & Answering for INTelligence (AQUAINT ),” URL: , last accessed 26 March 2006. – 9 – Issue 1970s and 1980s 1990s and 2000s Future Focus of Analysis Continuing large, well-defined issues and adversaries Emergence of complex, rapidly shifting issues and adversaries Complex, rapidly shifting issues and adversaries and large, well-defined issues and adversaries Space for longer-term thinking Bias toward current intelligence Both immediate question–answering and deeper analysis Organization and Workflow Large, centrally organized and managed Large, centrally organized and managed Tailored to rapidly adapt to shifting foci Hierarchical Still hierarchical, though problem- oriented “centers” added Flat, problem-centric networks Institutional and operational memory mostly in analysts heads Institutional and operational memory mostly in analysts heads Technology helps to notice what analysts are watching and asking Time pressure persistent but low intensity (mostly) Time pressure drives toward premature closure Technology allows memory even of hypotheses, data rejected by analysts Sources and Methods Dominated by secret sources Broader range of sources, but secrets still primary Draws on a wide variety of sources, open and secret Analysts are separated from collectors Analysts are also their own collectors Analysts are their own collectors Analysts mostly passive recipients of information Limited searching of available data Much more aggressive searching and reaching for data…both classified and unclassified Analysis focuses on previous patterns Same, though growing interest in new methods and tools for shaping, remembering, and examining hypotheses Formative pattern recognition and data mining capabilities searches for out of the ordinary Analysts operate on basis of own experience and biases Limited use of formal method and technology Wide use of method and technology – from aggregating expert views to searching, data mining, pattern recognition Analysts’ Characteristics Many analysts are deep specialists Many, perhaps most, analysts are generalists Mix of generalists and deep specialists, both technical and political Analysts mostly work alone or in small groups Analysts mostly work alone or in small groups Analysts work in larger virtual networks Key analytic choices with analysts Key analytic choices with analysts Key analytic choices with analysts Table 7: Analysis: Past, Present and Future Source: Developed by Gregory F. Treverton with input from the author. Email exchanges 17–18 May 2006. – 9 – Issue 1970s and 1980s 1990s and 2000s Future Focus of Analysis Continuing large, well-defined issues and adversaries Emergence of complex, rapidly shifting issues and adversaries Complex, rapidly shifting issues and adversaries and large, well-defined issues and adversaries Space for longer-term thinking Bias toward current intelligence Both immediate question–answering and deeper analysis Organization and Workflow Large, centrally organized and managed Large, centrally organized and managed Tailored to rapidly adapt to shifting foci Hierarchical Still hierarchical, though problem- oriented “centers” added Flat, problem-centric networks Institutional and operational memory mostly in analysts heads Institutional and operational memory mostly in analysts heads Technology helps to notice what analysts are watching and asking Time pressure persistent but low intensity (mostly) Time pressure drives toward premature closure Technology allows memory even of hypotheses, data rejected by analysts Sources and Methods Dominated by secret sources Broader range of sources, but secrets still primary Draws on a wide variety of sources, open and secret Analysts are separated from collectors Analysts are also their own collectors Analysts are their own collectors Analysts mostly passive recipients of information Limited searching of available data Much more aggressive searching and reaching for data…both classified and unclassified Analysis focuses on previous patterns Same, though growing interest in new methods and tools for shaping, remembering, and examining hypotheses Formative pattern recognition and data mining capabilities searches for out of the ordinary Analysts operate on basis of own experience and biases Limited use of formal method and technology Wide use of method and technology – from aggregating expert views to searching, data mining, pattern recognition Analysts’ Characteristics Many analysts are deep specialists Many, perhaps most, analysts are generalists Mix of generalists and deep specialists, both technical and political Analysts mostly work alone or in small groups Analysts mostly work alone or in small groups Analysts work in larger virtual networks Key analytic choices with analysts Key analytic choices with analysts Key analytic choices with analysts Table 7: Analysis: Past, Present and Future Source: Developed by Gregory F. Treverton with input from the author. Email exchanges 17–18 May 2006. – 9 – of improving intelligence analysis. Knowing the details of how analysts reason and how they might reason more effectively guides managers to understand where educational and training efforts can be most valuable. TECHNOLOGICAL ADVANCES IN CRITICAL THINKING One ARDA research team has created a technological system employing an automated critical thinking-based model that provides an overarching con – trol structure for analytic reasoning. It includes both model generation (for counter–examples) and more traditional argument building. The system’s creators assert – based on preliminary testing – that analysts employing the system are faster and more accurate problem solvers. Suppose an analyst develops an argument in the course of using this system, that confirms that Saddam Hussein has WMD and an argument that con – firms that Saddam does not have WMD . Working both sides of this issue, this analyst will make inferences from evidence and link them with other evidence-derived inferences. She will not imagine a situation in which all her evidence is true, but her hypothesis is not. For example, the evidence might include communications from which she infers that those communicating – including Saddam’s scientists – believe they have WMD . Using the system, she also would be led to consider that the scientists might have wanted to deceive Saddam into believing that they were further along than they were. The generation of this alternative model can only arise through a mechanism that conducts the analyst away from the comfortable, inductive, confirmation path associated with the initial hypothesis. – 9 – c oNclusioN Never before in our peacetime history have the stakes of foreign policy been higher. —Sherman Kent, 99 This paper posed the intriguing question: “How can intelligence analysts be ‘really good’”? Critical thinking, if conceived and employed by intelligence analysts as suggested here, appears capable of leading analysts to adopt personal habits of thought appropriate to the resolution of hard intelligence problems. The idea that intelligence analysts are expected to bring to the table a capability to draw reasoned and actionable conclusions from scant and conflicting information distinguishes their charge from that of their academic brethren. Thus, intelligence analysts in government or other applied work environments may deserve the lavish budgets and technological capabilities they often enjoy. To earn their high level of resource support, analysts can take advantage of a capability to redirect unique intelligence collection capabilities. In doing so, they are perfectly positioned to apply critical thinking methods to the hard problems. They can, for example, order up special collection against targets that exhibit even fleeting evidence that convincingly disconfirms one or more alternative hypotheses about an impending threat. This is the nub of Ben- Israel’s argument for a “logical” approach to intelligence analysis for national security, and is an approach no doubt under-used by those who do not systematically think about threats in the manner developed in this paper. Although most of the specific threats have changed since Sherman Kent first wrote in 1949, his epigraph remains as true today as it was then: the survival and prosperity of the United States remain at stake; they depend on effective, informed foreign policy. 9 Critical thinking brings with it an indispensable capability to inform a rational 9 Kent, Strategic Intelligence, ix. – 9 – foreign policy. “Really good analysts” are those who think critically about the issues they work and the problems they solve; they bring structured thought to the foreign policy realm. – 97 – A ppendix nSA’ S C ritiCAl t hinking A nd S truCtured A n Aly SiS C lASS S yll AbuS 230 Background Twenty-first Century intelligence issues involve uncertainty, mysteries, and risk. This differs from the 20 th Century paradigms of security, secrets, and prevention. Analysis of current complex issues requires of its practitioners novel approaches including a productively imaginative process of inquiry. Questions an analyst asks not only serve as devices for attracting existing evidence, but also as devices for generating new evidence not presently considered. In this way, analysts more thoroughly examine complex issues and, aided by technology, are more likely to create novel intelligence and prevent strategic surprise. However, such reasoning is at odds with how people – all people, including intelligence analysts – naturally think. Instead, people seek to confirm the first answer to a problem they discover, selectively using evidence to support that position even when there is compelling evidence that an alternative hypothesis may actually be the correct one. That people routinely fall prey to such poor thinking is well documented. Indeed, most commercial advertisers strive to take advantage of this. So do adversaries. One element of most intelligence failures includes poor thinking on the part of analysts— poor thinking of which adversaries have taken advantage. So how can analysts avoid such thinking? One solution is to teach intelligence analysts to think critically. 230 This syllabus has been developed and refined by the author through several years of teaching critical thinking at the National Security Agency’s National Cryptologic School. An early version of the course bore similarity to one developed by (then) MSGT Robert D. Folker, while a student at the (now) National Defense Intelligence College. Folker’s course focused on the methods of analysis, not on the overarching critical thinking. His course (as written) is not taught at the college. – 98 – Critical thinking provides structure to the reasoning process that identifies for analysts where they are likely to go astray. It offers a means for self-reflective reasoning that leads to improved thinking. If such thinking is aided by structured analytic techniques, then analysts will (and do) improve how they resolve issues with which they are confronted. The quality of their work improves. This critical thinking and analytic problem-solving course offers participants a chance to learn a paradigm for critical thinking and critically explore 14 different structured methods of analysis. Texts by critical thinking experts Richard Paul and Linda Elder, and structured analysis experts Morgan Jones and Richards Heuer, as well as materials developed by the instructor, teach the concepts and techniques. Classroom problems as well as operational examples (introduced and developed by the students from their own work) reinforce and help transfer what is learned into the operational environment. A final project developed by student teams completes the formal requirements. Learning to think critically and to solve problems in a structured manner requires active participation. The class requires 40 hours of classroom time, consisting of ten sessions of four hours each. The method of instruction is Socratic, demanding active classroom participation. Participants also can expect homework, requiring both office and non-office time. Participants will prepare reading summaries for each class session, and develop one (or more) operational examples of at least one structured analytic method. Finally, participants work together on teams to complete classroom assignments and a final project dealing with an operational issue (employing at least five structured analytic methods). – 99 – Administration: Enrollment: Up to 21 Students. Class Date/Time/Duration: One 4-hour class per day for 10 weeks. Class Location: ________________________. Homework: Yes, but hopefully not too odious. Operational examples are required. A team project is due at the end of the course. Texts: Elder, Linda, and Richard Paul. The Foundations of Analytic Thinking: How to Take Thinking Apart and What to Look for When You Do (Dillon Beach, CA : Foundation for Critical Thinking, 2003). Heuer, Richards J., Jr. The Psycholog y of Intelligence Analysis (Washington, DC : Center for the Study of Intelligence, 1999), URL : , last accessed 15 March 2006. Jones, Morgan D. The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in B usiness and in Life (New York, NY : Crown Publishing Group, 1997). Several handouts (TBD) Objectives: The overall objective of the class is to enable you to critically think about analysis and what analysts are tasked to analyze. A second objective is to provide you with a set of analytic tools that are useful to your analysis. At the end of the class you will be equipped with a set of analytic skills and will have honed your critical thinking skills, allowing you to better function in the workplace. Specifically, the course objectives are as follows: – 100 – Upon completion of this course you will be able to: • Use critical thinking techniques to provide structure to your analytic reasoning. • Identify, describe, and employ 14 methods for structured reasoning. • Demonstrate critical thinking proficiency through lecture, classroom participation, and weekly homework assignments. • Complete a final class assignment using a minimum of five structured analytic methods presented in this course. • Apply knowledge of critical thinking by using a set of analytic tools designed to hone your skills as an analyst. In other words, at the end of this 10-week-long class, you will have • Learned to critically analyze intelligence-associated data, information, and evidence. • Honed your critical thinking skills. • Built a “toolbox” of analytic and problem-solving methods. • Become better analysts. For example, when you approach a problem you will be able to • Discover the true problem by restating and considering alternative outcomes. • Have a variety of methods by which you can organize and make sense of the relevant evidence. Formal Requirements: Written summaries of readings. (No more than one page per chapter assigned.) The summaries should answer exactly the questions on page five of the syllabus. The summaries should also include answers to the exercises in Morgan Jones’ book. The summaries will be typed unless prior arrangements have been made with the instructor. In-class discussions will draw heavily on the readings. Problems from the work environment (Operational Exemplars). As we study the elements of reasoning and the methods of problem solving, we need operational examples – 101 – against which to apply/illustrate what we are learning. You will be responsible for providing at least one of those examples for the class to be presented during the week we discuss the method. The best exemplars may be saved for use in subsequent classes. Final project. Working in teams of three or four, and using any Problem Restatement and Divergent and Convergent Thinking plus at least three other methods developed in the course – for a total of of at least five methods – you will develop an operational project to be presented at the last class. You will apply the elements of critical thinking to the method chosen as well as the specific problem, apply the appropriate methods to solve of problem, report the results, and evaluate the process. The project will be presented during a 15–20 minute briefing. A list of the specific elements that must be included and the format by which the project is graded is on the last page of this syllabus. Grading: • Written Summaries (25%). Due weekly. The first summary will be graded. Subsequently, a random number generator will be used to select three (3) additional summaries for grading. Grading will be based on the “Universal Intellectual Standards” in The Foundations of Analytic Thinking and on whether instructions are followed. For example, if you are asked to identify what key questions an author is attempting to answer, it is expected that you will provide those questions in the reading summary. • Class Participation (25%). Since this is a discussion course, you are expected to engage in the process. • Operational Exemplars (25%). Assigned the first week. No longer than 5 minutes each. • Final Project (25%). – 102 – Class Descriptions and Weekly Assignments (complete prior to each class): Class 1 – How We Don’t Think and How We Might Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life, Chapters 1–2 (Part One). Psycholog y of Intelligence Analysis, 1–30. Critical Thinking: Concepts and Tools, entire work. Written Assignment: Reading summaries for chapters from Jones and Heuer. Exercises in text. Class Objectives: At the end of Class 1 you will be able to: • Define a bias and discuss the implications of biases in our decisions. • Identify sources of cognitive biases. • Describe the inherent dangers/benefits of biases and the difficulty of compensating for perceptual biases. • Describe the characteristics and three principles of perception. • Describe how analysts fall prey to absence-of-evidence biases. • Describe how anchoring impacts analytical decision- making. • Describe how a target can use assimilation biases to deceive. • Acknowledge how analysts unwittingly use confirmation bias to support early assessments. • Discuss how hindsight and reliability biases play a part in intelligence failures. • Describe how oversensitivity to consistency bias can lead to undesirable results. • Discuss how expert biases and the Pollyanna and Cassandra complexes distort our thinking. • Describe how cultural, personal, and organizational – 103 – mindsets impact analysis. • Define mindsets, discuss how they are derived, and describe how they influence predictions. • Identify how we think. • Describe critical thinking and the standards used for evaluating our thinking. • Identify the elemental structures of thought. • Describe the differences between inferences and assumptions in intelligence analysis. – 104 – Class 2 – Critical Problem Restatement and Alternative Thinking Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life, Chapters 3, 5. Written Assignment : Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 2 you will be able to: • Demonstrate knowledge of the critical thinking process by providing an example that meets the universal intellectual standards. • Describe the role of questioning in critical thinking. • Identify the three types of questions used in critical thinking. • Determine the sample domains involved in complex questions. • Define problem restatement and apply its use through a practical example. • Discuss the role that our biases play in problem restatement. • Demonstrate knowledge of the critical thinking process by providing an example that meets the universal intellectual standards. • Define divergent thinking and its benefits when performing a problem restatement. • Discuss the pitfall involved in problem definition and how it relates to problem restatement. • Describe some effective techniques for problem restatement. • Identify the types of problems that benefit from problem restatement. • Discuss how points of view influence the critical thinking process. • Discuss the logic, benefits, risks, and elements of divergent thinking. • Identify the four main ideas of divergent thinking. – 105 – • Describe what is needed to move from divergence to convergence. • Identify the types of problems that benefit from divergent thinking. – 106 – Class 3 – Pollyanna, Cassandra, and Marshaling Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life , Chapters 4, 6. Written Assignment : Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 3 you will be able to: • Discuss the pros-cons-fixes approach to critical thinking and the six-step method employed by successful analysts. • Identify the logic behind the pros-cons-fixes approach and discuss those critical thinking problems best suited to this method. • Apply the techniques of sorting, chronologies, and timelines to critical thinking and identify those critical thinking problems best suited for this approach. • Identify the two-step technique used for sorted lists, chronologies, and timelines. – 107 – Class 4 – Causes, Squares, and Shrubs Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life , Chapters 7, 8, 9. Written Assignment: Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 4 you will be able to: • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for causal flow diagramming. • Identify the major factors and cause-and-effect relationships influencing causal flow diagramming. • Characterize the differences between direct and inverse relationships and their impact on causal flow diagramming. • Define a feedback loop and discuss what makes it stable or unstable. • Apply your knowledge of causal diagramming by participating in a classroom exercise. • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for scenario matrices and trees. • Define and discuss the characteristics and differences between matrices and scenario trees. • Apply your knowledge of scenario matrices by participating in a classroom exercise. • Apply your knowledge of scenario trees by participating in a classroom exercise. – 108 – Class 5 – Weighing the Likelihood Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life, Chapters 10, 13. Written Assignment: Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 5 you will be able to: • Discuss how individuals view issues. • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for weighted ranking. • List the 9-step process involved in weighted ranking. • Demonstrate knowledge of weighted ranking through class and small group exercises. • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for probability. • Define and describe how and why we characterize and assign events, and how this influences intelligence analysis. • Discuss and explain the differences between mutually exclusive and conditionally dependent events relative to probability. • Identify the seven steps in creating a probability tree. • Apply your knowledge of probability by participating in a classroom exercise. – 109 – Class 6 – Testing the Devil’s and Other’s Advocacy Reading Assignment: The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life, Chapters 11–12. The Psycholog y of Intelligence Analysis, Chapter 8. Written Assignment: Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 6 you will be able to: • Define intelligence. • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for devil’s advocacy. • Define, describe, and characterize situations appropriate to employ methods of devil’s advocacy in intelligence analysis • Demonstrate prowess in devil’s advocacy through participation in a practical case study. • Discuss the purpose, logic, underlying question, evidence, concepts, and types of problems best suited for hypothesis testing. • Discuss the benefits of hypothesis testing. • Identify the 8-step process of hypothesis testing. • Demonstrate prowess in hypothesis testing through participation in a practical case study. – 110 – Class 7 – Analyzing Apples and Toyotas Reading Assignment : The Thinker’s Toolkit: Fourteen Skills for Making Smarter Decisions in Business and in Life, Chapters 14, 15, 16. Written Assignment: Reading summaries for each chapter. Exercises in text. Class Objectives: At the end of Class 7 you will be able to: • Describe how individual viewpoints influence utility analysis. • Define utility analysis and discuss the type of problems best suited for this method. • Identify the logic of utility analysis. • Employ utility analysis to arrive at the most cost-effective solutions. • List the steps involved in creating a utility tree and apply this knowledge by participating in classroom exercises. • List the steps involved in creating a utility matrix and apply this knowledge by participating in classroom exercises. • Examine the benefits of structuring as it applies to utility and probability analysis. • Describe the differences between utility and probability analysis. • Examine multi-perspective utility analysis and its use by participating in classroom exercises. • List the 13 steps of multi-perspective utility analysis. • Apply your knowledge of multi-perspective utility analysis by participating in a classroom exercise. – 111 – Class 8 – And Now for Something Completely Different Reading Assignment: Review all assigned readings. Written Assignment : None. Class Objectives: At the end of Class 8 you will be able to: • Describe which structured analysis methods work best in particular situations. • Demonstrate knowledge of critical thinking and structured analysis through a practicum. • Demonstrate your knowledge of critical thinking and structured analysis through a small group exercise. – 112 – Class 9 – Final Project Preparation Reading Assignment: Review all assigned readings. Written Assignment : None. Class Objectives: During Class 9 you will: • Learn through experience the pros and cons of research, analysis and reporting in small teams. – 113 – Class 10 – Final Project Presentation and Wrap-up Reading Assignment: Review all assigned readings. Written Assignment : None. Class Objectives: During Class 10: • Your small group will make a multi-media presentation on the analytic problem you chose. Methods: • A sample of the problem statement and restatement process. • A sample of the convergent and divergent thinking applied to the problem. • Concise demonstrations of the three structured analytic methods used to answer the analytic problem. Assessment: • You are evaluated on how clearly the group presents its project, and on whether the project requirements listed above are met as shown on the last page of the syllabus. – 114 – Reading Summary Questions: 231 For each chapter/work, answer the following questions (in writing) about the reading. 1. What is the author’s purpose? If there is more than one main point, what is the overarching purpose? 2. What are the key questions the author raises/addresses? 3. What evidence does the author provide to support his argument? 4. What inferences does the author make from the evidence? 5. On what underlying concepts does the author rely? 6. What does the author take for granted? What assumptions does the author make? 7. What are the implications of the author’s point of view? What are the implications if we adopt/do not adopt what the author recommends? 8. What is the author’s point of view? What other points of view are there? 231 Paul and Elder, Concepts and Tools, 13. – 115 – Final Project Evaluation Worksheet 232 Clarity Accuracy Precision Relevance Depth Breadth Logic Significance Fairness Purpose Key Questions Assumptions Evidence A: Methods B: Analysis Concepts Inferences Implications Other Points of View Lessons Learned Comments 232 Derived by author from Paul and Elder, Concepts and Tools, 2–9. – 116 – – 117 – B iBliography Ackerman, Robert K. “Information Age Poses New Challenges to Intelligence.” Signal 53, no. 2 (October 1998): 23–25. Advanced Research and Development Activity. Broad Agency Announcement for the Novel Intelligence from Massive Data ( NIMD ) R&D Program. NMA- 401-02- BAA -0004, 7 June 2002, 1. __________. “Preliminary Solicitation Information for Advanced QU estion & Answering for INT elligence ( AQUAINT ).” URL : . Last accessed 26 March 2006. Air Force Research Laboratory ( AFRL ). IFED Directorate. “Collaboration and Analyst/System Effectiveness ( CASE ).” BAA 06-02- IFKA , 28 October 2005. URL : . Last accessed 11 March 2006. Allen, Charles E. “Intelligence: Cult, Craft, or Business?” In Seminar on Intelligence, Command, and Control, Guest Presentations, Spring 2000. Cambridge, MA : Harvard University Program on Information Resources Policy, I-01-1, April 2000. URL : . Last accessed 11 January 2006. Allison, Graham, and Philip Zelikow. Essence of Decision: Explaining the Cuban Missile Crisis, 2nd Edition. New York, NY : Longman, 1999. Amuchastegui, Domingo “Cuban Intelligence and the October Crisis.” In James G. Blight and David A. Welch, Eds. Intelligence and the Cuban Missile Crisis. London, UK : Frank Cass, 1998: 88–119. – 118 – Anderson, Terence J., and William Twining. Analysis of Evidence: How to Do Things with Facts Based on Wigmore’s Science of Judicial Proof. Chicago, IL: Northwestern University Press, 1991. Anderson, Terence J., David A. Schum, and William Twining. Analysis of Evidence. Cambridge, UK : Cambridge University Press, 2005. Bathurst, Robert B. Intelligence and the Mirror: On Creating an Enemy. Oslo, Norway: International Peace Research Institute, 1993. Ben-Israel, Isaac. “Philosophy and Methodology of Intelligence: The Logic of the Estimate Process. Intelligence and National Security 4, no. 4 (October 1989): 660–718. Blight, James G., and David A. Welch, Eds. Intelligence and the Cuban Missile Crisis. London, UK : Frank Cass, 1998. Boden, Margaret A. The Creative Mind: Myths and Mechanisms. New York, NY : Basic Books, 1990. Brei, William S., CAPT, USAF. Getting Intelligence Right: The Power of Logical Procedure. Occasional Paper Number Two. Washington, DC : Joint Military Intelligence College, January 1996. Brent, Joseph. Charles Sanders Peirce: A Life. Bloomington, IN : Indiana University Press, 1998. Brookfield, Stephen D. Developing Critical Thinkers: Challenging Adults to Explore Alternative Ways of Thinking and Acting. San Francisco, CA : Jossey-Bass Publishers, 1987. Callum, Robert. “The Case for Cultural Diversity in the Intelligence Community.” International Journal of Intelligence and CounterIntelligence 14, no. 1, Spring 2001: 25–48. – 119 – Cheikes, Brant A., Mark J. Brown, Paul E. Lehner, and Leonard Adelman. Confirmation Bias in Complex Analyses, Mitre Technical Report MTR 04B0000017. Bedford, MA : Center for Integrated Intelligence Systems, October 2004. Christensen, Clayton M. The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail. Boston, MA : Harvard Business School Press, 1997. Christensen, Clayton M., and Michael E. Raynor. The Innovator’s Solution: Creating and Sustaining Successful Growth. Boston, MA : Harvard Business School Press, 2003. Christensen, Clayton M., Scott D. Anthony, and Erik A. Roth. Seeing What’s Next: Using the Theories of Innovation to Predict Industry Change. Boston, MA : Harvard Business School Press, 2004. “Clarke: Bush Didn’t See Terrorism as ‘Urgent’.” CNN.com, 19 May 2004. URL : . Last accessed 9 March 2006. Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction. Report to the President of the United States. Washington DC : Government Printing Office, March 31, 2005. URL : < http://www.wmd.gov/ report/ index.html >, last accessed 28 July 2005. Cooper, Jeffrey R. Curing Analytic Pathologies: Pathways to Improved Intelligence Analysis. Washington, DC : Center for the Study of Intelligence, 2005 . __________. Email to the author. 31 March 2006. Davis, Jack. “The Kent-Kendall Debate of 1949.” Studies in Intelligence 35, no. 2 (Summer 1991): 37–50 . De Bono, Edward. de Bono’s Thinking Course. New York, NY : Facts on File Publications, 1982. – 120 – De Cagna, Jeff. “Making Change Happen: Steve Denning Tells the Story of Storytelling.” Infor mation Outlook, January 2001: 28–35. De Rivera, Joseph. T he Psychological Dimension of Foreign Policy. Columbus, OH : Charles E. Merrill Publishing Company, 1968. Denning, Stephen. T he Springboard: How Storytelling Ignites Action in Knowledge-Era Organizations. Boston, MA : Butterworth- Heinemann, 2001. Devlin, Keith. Goodbye Descartes: The End of Logic and the Search for a New Cosmolog y of the Mind. New York, NY : John Wiley & Sons, Inc., 1997. Dewey, John. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. New York, NY : D.C. Heath and Company, 1910. Dijksterhuis, Ap, Martin W. Bos, Loran F. Nordgren, and Rick B. von Baren. “On Making the Right Choice: The Deliberation- Without-Attention Effect.” Science, 311, no. 5763 (17 February 2006): 1005–1007. DiSpezio, Michael A. Classic Critical Thinking Puzzles. New York, NY : Main Street, 2005. Durkheim, Emile. The Rules of Sociological Method. Glencoe, IL: Free Press, 1938. Eco, Umberto, and Thomas A. Sebeok. The Sign of Three: Dupin, Holmes, Pierce. Bloomington, IN : Indiana University Press, 1988 EDS. Herding Cats . Television advertisement produced for EDS by Fallon-McElligott. Videocassette, January 2000. – 121 – __________. Building Airplanes in Flight . Television advertisement produced for EDS by Fallon-McElligott. Videocassette, November 2000. Elder, Linda, and Richard Paul. The Foundations of Analytic Thinking: How to Take Thinking Apart and What to Look for When You Do. Dillon Beach, CA : The Foundation for Critical Thinking, 2003 . Elliott, Michael. “How the U.S. Missed the Clues.” CNN.com, 20 May 2002. URL : < http://archives.cnn.com/2002/ ALLPOLITICS /05/20/time.clues/index.html>. Last accessed 9 March 2006. Facione, Peter A. Critical Thinking: What It Is and Why It Counts . Milbrae, CA : California Academic Press, 1998, updated 2004. URL . Last accessed 22 July 2005. Facione, Peter A., Noreen C. Facione, and Carol A. Giancarlo. “The Disposition Towards Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill,” Infor mal Logic 20, no. 1 (2000): 61–84. __________. Professional Judgment and the Disposition Toward Critical Thinking (Milbrae, CA : California Academic Press, 2002). URL : < http://www.calpress.com/ pdf_files/Prof_jmn.pdf>. Last accessed 31 March 2006. Feder, Stanley. “Factions and Policon: New Ways to Analyze Politics.” In Westerfield, H. Bradford, ed. Inside CIA ’s Private World: Declassified Articles from the Agency’s Internal Journal, 1955–1992. New Haven, CT : Yale University Press, 1995. Feldman, Daniel. Critical Thinking: Strategies for Decision Making. Menlo Park, CA : Crisp Publications, Inc, 2002. Fisher, Alec. Critical T hinking: An Introduction. Cambridge, UK : Cambridge University Press, 2001. – 122 – Folker, Robert D., MSGT , USAF . Intelligence Analysis in Theater Joint Intelligence Centers: An Experiment in Applying Structured Methods. Occasional Paper Number Seven. Washington, DC : Joint Military Intelligence College, January 2000. Folker, Robert D., LT , USAF . Email to the author. 9 December 2003. Fursenko, Aleksandr and Timothy Naftali. “One Hell of a Gamble:” Khrushchev, Castro, and Kennedy, 1958–1964. New York, NY : W. W. Norton & Company, 1997. Garthoff, Raymond L. “US Intelligence in the Cuban Missile Crisis.” In James G. Blight and David A. Welch, Eds. Intelligence and the Cuban Missile Crisis. London, UK : Frank Cass, 1998: 18–63. George, Alexander. Presidential Decisionmaking in Foreign Policy: The Effective Use of Infor mation and Advice. Boulder, CO : Westview Press, 1980. Gilovich, Thomas. How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life. New York, NY : The Free Press, 1991. Gilovich, Thomas, Dale Griffin, and Daniel Kahneman, Eds. Heuristics and Biases: The Psycholog y of Intuitive Judgment. Cambridge, UK : Cambridge University Press, 2002. Gladwell, Malcolm. The Tipping Point: How Little Things Can Make a Big Difference . Boston, MA : Little Brown and Company, 2000. __________. Blink: The Power of Thinking without Thinking. Boston, MA : Little Brown and Company, 2005. Glaser, Edward M. An Experiment in the Development of Critical Thinking. New York, NY : AMS Press, 1941. – 123 – Grabo, Cynthia M. Anticipating Surprise: Analysis for Strategic Warning. Washington DC : Joint Military Intelligence College, 2002. Gribkov, Anatoli. “The View from Moscow and Havana.” In Gribkov, Anatoli, and William Y. Smith. Operation ANADYR : U.S. and Soviet Generals Recount the Cuban Missile Crisis. Chicago, IL: Edition q, 1994: 3–76. Gribkov, Anatoli, and William Y. Smith. Operation ANADYR : U.S. and Soviet Generals Recount the Cuban Missile Crisis. Chicago, IL: Edition q, 1994. Halpern, Diane. T hought and Knowledge: An Introduction to Critical T hinking, 4 th Edition. Mahwah, NJ : Lawrence Erlbaum Associates, Publishers, 2002. Hampson, Fen Osler. “The Divided Decision-Maker: American Domestic Politics and the Cuban Crises.” International Security, 9, no 3 (Winter 1984/85): 130-165. Harcourt Assessment, Inc. Local Nor ms Report, Watson-Glaser Critical T hinking Assessment. Report prepared for the Defense Intelligence Agency, 2005 . Heuer, Richards J., Jr. “Strategic Deception and Counterdeception: A Cognitive Process Approach.” International Studies Quarterly 25, no. 2 (June 1981): 294–327. __________. Adapting Academic Methods and Models to Governmental Needs: The CIA Experience. Strategic Studies Institute. U.S. Army War College. July 31, 1978. __________. Psycholog y of Intelligence Analysis. Washington, DC : CIA Center for the Study of Intelligence, 1999 . Hinsley, F. H., and Alan Stripp, Eds. Codebreakers: The Inside Story of Bletchley Park . Oxford, UK : Oxford University Press, 1993. Holyoak, Keith J., and Paul Thagard. Mental Leaps: Analog y in Creative Thought. Cambridge, MA : The MIT Press, 1995. – 124 – Hughes, Francis J. Conversation with the author. Washington, DC : Joint Military Intelligence College. 8 May 2003. Hughes, Francis J., and David A. Schum. Credibility Assessment: A First Step in Intelligence Analysis. Unpublished tutorial. Joint Military Intelligence College, April 2003. Janis, Irving. Groupthink: Psychological Studies of Policy Decisions and Fiascoes, 2nd Edition. Boston, MA : Houghton Mifflin Company, 1982. Jervis, Robert. System Effects: Complexity in Political and Social Life. Princeton, NJ : Princeton University Press, 1997. Johnson Thomas R., and David A. Hatch. NSA and the Cuban Missile Crisis. Fort Meade, MD : National Security Agency Center for Cryptologic History, 1998. URL : . Accessed 18 April 2006 Johnston, Rob. Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study. Washington, DC : Center for the Study of Intelligence, 2005. Jones, Morgan D. The Thinker’s Toolkit: 14 Powerful Techniques for Problem Solving. New York, NY : Random House, Inc, 1995 . __________. Conversation with the author. 15 December 2003. Kahneman, Daniel, Paul Slovic, and Amos Tversky, eds. Judgment under Uncertainty: Heuristics and Biases. Cambridge, UK : Cambridge University Press, 1982. Kam, Ephraim. Surprise Attack: The Victim’s Perspective . Cambridge, MA : Harvard University Press, 1990. Kent, Sherman. Strategic Intelligence for American World Policy. Princeton, NJ : Princeton University Press, 1949. __________. “A Crucial Estimate Relived.” Studies in Intelligence 36, no. 5 (1992): 111–119. – 125 – Kerbel, Josh. “Thinking Straight: Cognitive Bias in the US Debate about China.” Studies in Intelligence 48, no. 3 (2004), URL : . Last accessed 22 February 2006. Klein, Gary. Intuition at Work: Why Developing Your Gut Instincts Will Make You Better at What You Do. New York, NY : Doubleday, 2003. Krizan, Lisa. Intelligence Essentials for Everyone. Occasional Paper Number Six. Washington, DC : Joint Military Intelligence College, 1999. Lebow, Richard Ned, and Janice Gross Stein. “Back to the Past: Counterfactuals and the Cuban Missile Crisis.” In Philip E. Tetlock, and Aaron Belkin Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives . Princeton, NJ : Princeton University Press, 1996: 119–148. LeGault, Michael R. Think: Why Crucial Decisions Can’t Be Made in the Blink of an Eye. New York, NY : Threshold Editions, 2006. Lengel, Allan. “Little Progress in FBI Probe of Anthrax Attacks: Internal Report Compiled As Agents Hope for a Break.” Washington Post. 16 September 2005. A01. Lengel, Allan, and Guy Gugliotta. “Md. Pond Produces No Anthrax Microbes: FBI Sought Clues In Deadly Attacks.” Washington Post . 1 August 2003. A03. Lipman, Matthew. Thinking in Education, 2nd Edition. Cambridge, UK : Cambridge University Press, 2003. Lundahl, Arthur C. “Additional Information—Mission 3102.” Memorandum for Director of Central Intelligence and Director, Defense Intelligence Agency, 15 October 1962. In Mary S. McAuliffe, ed., CIA Documents on the Cuban Missile Crisis, 1962. Washington DC : Central Intelligence Agency, 1992: 181–182. – 126 – __________. “Additional Infor mation—Mission 3107.” Memorandum for Director of Central Intelligence and Director, Defense Intelligence Agency, 19 October 1962. In Mary S. McAuliffe, ed., CIA Documents on the Cuban Missile Crisis, 1962. Washington DC : Central Intelligence Agency, 1992: 209. McAuliffe, Mary S., ed. CIA Documents on the Cuban Missile Crisis, 1962 Washington DC : Central Intelligence Agency, 1992. Marrin, Stephen. “Homeland Security and the Analysis of Foreign Intelligence,” Markle Foundation Task Force on National Security in the Information Age, 15 July 2002, URL : . Last accessed December 9, 2003. __________. Email to the author, 8 December 2003. __________. “ CIA ’s Kent School: Improving Training for New Analysts.” International Journal of Intelligence and CounterIntelligence 16, no. 4 (Winter 2003–2004): 609–637. May, Ernest R. Lessons of the Past: The Use and Misuse of History in American Foreign Policy. Oxford, UK : Oxford University Press, 1973. Merom, Gil. “The 1962 Cuban Intelligence Estimate: A Methodological Perspective.” Intelligence and National Security, 14, no. 3 (Autumn 1999): 48–80. Mid-Level Intelligence Professional. National Security Agency. Email to the author, 9 March 2006. Miller, George A. “The Magical Number Seven, Plus or Minus Two.” The Psychological Review 63 (1956): 81–97. – 127 – Miller, Judith. “A Battle of Words over War Intelligence.” New York Times. Online edition, 22 November 2003. URL : . Last accessed November 28, 2003. Millward, William. “Life in and out of Hut 3.” In Hinsley, F. H., and Alan Stripp, Eds. Codebreakers: The Inside Story of Bletchley Park . Oxford, UK: Oxford University Press, 1993: 17–29. Moore, David T. Creating Intelligence: Evidence and Inference in the Analysis Process. MSSI Thesis. Washington, DC : Joint Military Intelligence College, July 2002. __________. “Species of Competencies for Intelligence Analysis.” Defense Intelligence Journal 11, no. 2 (Summer 2002): 97–119. __________. Personal Notes. Joint Military Intelligence College, Denial and Deception Advanced Studies Program, Spring 2005. Moore David T., and Lisa Krizan. “Intelligence Analysis: Does NSA Have What it Takes.” Reprint: NSA Center for Cryptologic History. Cryptologic Quarterly, 20, nos 1/2 (Summer/Fall 2001): 1–33. __________. “Core Competencies for Intelligence Analysis at the National Security Agency.” In Swenson, Russell G., ed. Bringing Intelligence About: Practitioners Reflect on Best Practices. Washington DC : Joint Intelligence Military College, 2003: 95–132. Moore, David T., Lisa Krizan, and Elizabeth J. Moore. “Evaluating Intelligence: A Competency-Based Model.” International Journal of Intelligence and Counterintelligence 18, no. 2 (Summer 2005): 204–220. Myers, David G. Intuition: Its Powers and Perils. New Haven, CT : Yale University Press, 2002. – 128 – Negroponte, John D., Director of National Intelligence. National Intelligence Strateg y of the United States of America. Washington, DC : Office of the Director of National Intelligence, October 2005. URL : . Last accessed 12 March 2006. Neustadt, Richard E., and Ernest R. May. Thinking in Time: The Uses of History for Decision Makers. New York, NY : The Free Press, 1986. Oberdorfer, Don. “Missed Signals in the Middle East.” Washington Post Magazine , 17 March 1991. Paul, Richard W. “A Draft Statement of Principles.” The National Council for Excellence in Critical Thinking. URL : . Last accessed March 18, 2003. Paul, Richard W., and Linda Elder. The Miniature Guide to Critical Thinking Concepts and Tools , 4 th Edition. Dillon Beach, CA : The Foundation for Critical Thinking, 2004. Paul, Richard W., and Gerald Nosich. A Model for the National Assessment of Higher Order Thinking. Dillon Beach, CA : Foundation for Critical Thinking, n.d. Paul, Richard W., Linda Elder, and Ted Bartell. “Executive Summary, Study of 38 Public Universities and 28 Private Universities To Determine Faculty Emphasis on Critical Thinking In Instruction.” California Teacher Preparation for Instruction in Critical Thinking: Research Findings and Policy Recommendations. California Commission on Teacher Credentialing, Sacramento, California, 1997. Dillon, CA : Foundation for Critical Thinking, 1997. URL : . Last accessed March 18, 2003. – 129 – Paul, Richard W., Linda Elder, and Ted Bartell. California Teacher Preparation for Instruction in Critical Thinking: Research Findings and Policy Recommendations. California Commission on Teacher Credentialing, Sacramento California, 1997. Dillon, CA : Foundation for Critical Thinking, 1997. Perrow, Charles. Nor mal Accidents: Living with High-Risk Technologies. Princeton, NJ : Princeton University Press, 1999. Pettee, George S. The Future of American Secret Intelligence. Washington, DC : Infantry Journal Press, 1946. Platt, Washington. Strategic Intelligence Production: Basic Principles. New York, NY : Frederick A. Praeger, 1957. Plous, Scott. The Psycholog y of Judgment and Decision-Making. New York, NY : McGraw Hill, Inc., 1993. Raasa, Paul J. “The Denial and Deception Challenge to Intelligence.” In Godson, Roy, and James J. Wirtz, Strategic Denial and Deception. New Brunswick, NJ : Transaction Publishers, 2002: 223–228. Reed, Jennifer H. Effect of a Model For Critical Thinking on Student Achievement In Primary Source Document Analysis And Interpretation, Argumentative Reasoning, Critical Thinking Dispositions, and History Content in a Community College History Course. PhD Dissertation. College of Education, University of South Florida, December 1998. URL : . Last accessed 6 May 2006. Restak, Richard, M.D. The Brain Has a Mind of Its Own: Insights from a Practicing Neurologist. New York, NY : Harmony Books, 1991. Reynolds, William. Conversation with the author, 14 March 2006. – 130 – Rieber, Steven, and Neil Thomason. “Toward Improving Intelligence Analysis: Creation of a National Institute for Analytic Methods.” Studies in Intelligence 49, no. 4 (Winter 2006): 71–80. Schafersman, Steven D. “An Introduction to Critical Thinking,” January 1991. URL : . Last accessed 9 March 2006. Schecter Jerrold L., and Peter S. Deriabin. The Spy Who Saved the World. New York, NY : Scribner’s, 1992. Schum, David A. Evidence and Inference for the Intelligence Analyst, 2 volumes. Lanham, MD : University Press of America, 1987. __________. Evidential Foundations of Probabilistic Reasoning. New York, NY : John Wiley & Sons, Inc., 1994. __________. “Species of Abductive Reasoning in Fact Investigation in Law” Cardozo Law Review 22, nos. 5/6 (July 2001): 1645– 1681. __________. “Evidence Marshaling for Imaginative Fact Investigation.” Artificial Intelligence and Law 9 (2001): 165– 188. Sebeok, Thomas A. “One, Two, Three Spells UBERTY .” In Umberto Eco and Thomas A. Sebeok. The Sign of Three: Dupin, Holmes, Pierce. Bloomington, IN : Indiana University Press, 1988: 1–10. Senate Select Committee On Intelligence. Report on the U.S. Intelligence Community’s Prewar Intelligence Assessments on Iraq. United States Senate. 108 th Congress, 7 July 2004 . Simon, Herbert A. “Invariants of Human Behavior.” Annual Review of Psycholog y 41 (1990):1–19. – 131 – Stefik, Mark, and Barbara Stefik. Breakthrough: Stories and Strategies of Radical Innovation. Cambridge, MA : MIT Press, 2004. Swenson, Russell G., ed. Research: Design and Methods. Washington, DC : Joint Intelligence Military College, 2000. __________. Bringing Intelligence About: Practitioners Reflect on Best Practices. Washington, DC : Joint Intelligence Military College, 2003. Tetlock, Philip E. Expert Political Judgment: How Good Is It? How Can We Know? Princeton, NJ : Princeton University Press, 2005. Tetlock, Philip E. and Aaron Belkin. Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives. Princeton, NJ : Princeton University Press, 1996. Thagard, Paul. Coherence in Thought and Action. Cambridge, MA : The MIT Press, 2000. Treverton, Gregory F. Conversation with the author, 18 May 2006. __________. Email exchanges with the author, 17–18 May 2006. U.S. Congress. Joint Inquiry into Intelligence Community Activities Before and After the Terrorist Attacks of September 11, 2001. Report of the U.S. Senate Select Committee on Intelligence and U.S. House Permanent Select Committee on Intelligence, Together with Additional Views. Senate Report No. 107- 351/House Report. No. 107-792. 107 th Congress, 2 nd Session. December 2002. U.S. Department of Defense. Photograph in the John Fitzgerald Kennedy Library. Boston, MA . PX 66-20:7 14 October 1962. White, Mark J. Missiles in Cuba: Kennedy, Khrushchev, Castro, and the 1962 Crisis. Chicago, IL: Ivan R. Dee, 1997. – 132 – Wikipedia. “The Free Encyclopedia.” URL : . Last accessed 12 March 2006. Wirtz, James J. “Organizing for Crisis Intelligence: Lessons form the Cuban Missile Crisis.” In James G. Blight and David A. Welch, Eds. Intelligence and the Cuban Missile Crisis. London, UK : Frank Cass, 1998: 120–149. Wohlstetter, Roberta. “Cuba and Pearl Harbor: Hindsight and Foresight.” Foreign Affairs 46, no. 3 (July 1965): 691–707. Woods, Kevin, James Lacey, and Williamson Murray. “Saddam’s Delusions: The View from the Inside,” Foreign Affairs , 85, no. 3 (May/June 2006). URL : . Accessed 31 March 2006. Woods, Kevin M., Michael R. Pease, Mark E. Stout, Williamson Murray, and James G. Lacey, Iraqi Perspectives Project: A View of Operation Iraqi Freedom from Saddam’s Senior Leadership . Norfolk, VA: Joint Center for Operational Analysis, 2006. URL : . Last accessed 31 March 2006. Zlotnick, Jack. “Bayes’ Theorem for Intelligence Analysis.” Studies in Intelligence 11, no. 4 (Spring 1972): 1–12. URL : . Last accessed 7 April 2006. – 133 – a Bout the a uthor D aviD t . M oore is a career senior intelligence analyst and technical director at the National Security Agency where he is an advocate for, and mentor of best practices in intelligence analysis. He is an adjunct faculty member of the National Cryptologic School; has taught at the National Defense Intelligence College, Washington, DC ; and at Trinity University, Washington, DC . Mr. Moore holds a B.A . in sociology from Washington and Lee University and a Master of Science of Strategic Intelligence from the National Defense Intelligence College (formerly the Joint Military Intelligence College). Mr. Moore’s other publications include: • “Species of Competencies for Intelligence Analysis,” Defense Intelligence Journal, 11, no. 2 (Summer 2002): 97-119. • “Species of Competencies for Intelligence Analysis,” American Intelligence Journal , 23 (2005): 29–43 (an expanded version of the original article). • With coauthor Lisa Krizan: o “Intelligence Analysis, Does NSA Have What it Takes,” Cryptologic Quarterly, 20, nos. 1/2 (Summer/Fall 2001): 1-32. o “Core Competencies for Intelligence Analysis at the National Security Agency,” in Bringing Intelligence About: Practitioners Reflect on Best Practices, Russell Swenson, ed. (2004): 95-131. • With coauthors Lisa Krizan and Elizabeth J. Moore o “Evaluating Intelligence: A Competency-Based Approach,” in the International Journal of Intelligence and CounterIntelligence , 18, no. 2 (Summer 2005): 204-220. • With coauthor William N. Reynolds: o “So Many Ways To Lie: The Complexity of Denial and Deception,” Defense Intelligence Journal, 15, no. 2 (Fall 2006): 95-116. – 134 – C olophon A colophon records the details of the layout and design of a publication. It acknowledges the craftsmanship and art required in creating a printed work. Whereas the acknowledgements record the textual assistance an author receives, a colophon records the equally important technical assistance received. This Occasional Paper was set in Baskerville, a transitional typeface created by Birmingham printer John Baskerville around 1752. George Jones redesigned it in 1930 for Linotype-Hell; the International Typeface Corporation licensed it in 1982. The long, elegant serifs and the subtle transfer of stroke weight from thick to very thin give it a delicacy and grace that makes it an excellent text typeface. Titling and primary headings were set in Castellar, a Monotype typeface created by John Peters in 1957. Named after a town in the Alps, Castellar also was designed for refined display work. Tables were set in Gill Sans, designed by artist and type designer Eric Gill in 1931 for Monotype. Unlike many sans serif typefaces it is highly legible and readable in text. The cover was set in Californian, a typeface originally designed by Frederic Goudy in 1938 and adapted by various designers for the Font Bureau. This publication was composed on an Apple iMac and a Macbook Pro using Adobe Systems InDesign. Masters were created in Adobe Acrobat. Illustrations were prepared in Omni Systems Omni Graffle and Adobe Photoshop. Baskerville, Californian, Castellar, Gill Sans, iMac, Macbook Pro, Acrobat, InDesign, Omni Graffle, and Photoshop are registered trademarks of their respective companies. Desktop composition by the author. The cover was designed by Bridgette White. Published by the National Defense Intelligence College Press, May 2006; and by the National Defense Intelligence College Foundation, March 2007. Printed at the Defense Intelligence Agency, Washington DC .
Looking at the image in the attached file labeled (Looking at the Image), imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your
Looking at the image above, taken from your textbook, imagine that you are a crime scene investigator and you have just arrived at the crime scene. Describe the steps you and your fellow investigators would follow when processing the crime scene, collecting and packaging physical evidence, and maintaining the chain of custody. After the physical evidence is dropped off at the lab, describe the process that is used to analyze the evidence and the process that would be used to reconstruct the crime scene. Your assignment must be at least two pages in length, and you should at least use your textbook as a reference. Adhere to APA Style when creating citations and references for this assignment.

Are you stuck with another assignment? Use our paper writing service to score better grades and meet your deadlines. We are here to help!


Order a Similar Paper Order a Different Paper
Writerbay.net