BCISS Conference Materials--Understanding and Improving Intelligence Analysis

Understanding and Improving Intelligence Analysis: Learning from other Disciplines 12-13 July, 2012 * London, England * RAF Club The purpose of this conference is to engage in a cross-disciplinary discussion about the value of learning from other fields to improve both the understanding and the practice of intelligence analysis. It will also create the network and infrastructure for an international research collaboration for the study of intelligence analysis. Intelligence, like journalism, involves the acquisition, evaluation, and dissemination of information. In 1949, Sherman Kent, described as the father of US intelligence analysis, said: “Intelligence organizations must also have many of the qualities of those of our greatest metropolitan newspapers. …They watch, report, summarize, and analyze. They have their foreign correspondents and home staff…. They have their responsibilities for completeness and accuracy—with commensurately greater penalties for omission and error. . . They even have the problem of editorial control…. Intelligence organizations (should) put more study upon newspaper organization and borrow those phases of it which they require.” But the similarities between intelligence analysis and journalism are not unique. Professionals in other fields—including medicine, the social and behavioural sciences, history and historiography, anthropology and other disciplines engaged in ethnographic research, econometric forecasting, and legal reasoning— also face many similar challenges to those that exist in intelligence analysis, including:

       

Difficulties acquiring information from a wide variety of sources Vetting and evaluating the information that is acquired Deriving understanding and meaning from that information Impact of deadlines, editing, and other production processes on accuracy of analysis and assessment Problems in dissemination and distribution to consumers or customers Managing relationship between producer and consumer (role, responsibility, independence & objectivity) Developing professional infrastructure (recruit, select, train, & develop personnel; code of ethics) Overcoming impact of changing technology and alternative information distribution systems

How do practitioners in various non-intelligence fields overcome these kinds of challenges? How are their challenges similar to or different from those that exist in the intelligence arena? What can be learned from the comparison? This event has been funded through a grant from the Brunel University Research and Innovation Fund. They are organized and hosted by Brunel University’s Centre for Intelligence and Security Studies in collaboration with University of Mississippi’s Center for Intelligence and Security Studies.

1

Conference agenda Day 1: Thursday 12 July 8:00 Registration and Coffee 9:30 Welcome Part 1: 

Stephen Marrin (Brunel University): Learning from Other Disciplines, including Medicine and Journalism

10:00 Comparisons to Other Fields Part 1    

Chair: Stephen Marrin (Brunel University) David Chuter (Center for Security Sector Management; Cranfield University): Intelligence Analysis as a Type of Information Processing Michael Robinson (Deloitte and Touche, LLP): Intelligence Analysis and Digital Forensics Carl Jensen (University of Mississippi): Intelligence Analysis and Futures Methodology

11:00 Break 11:30 Comparisons to Other Fields Part 2    

Chair: Carl Jensen (University of Mississippi) Anne Bishop (Snell & Wilmer, LLP, Phoenix AZ): Intelligence Analysis and Legal Reasoning Anthony C. Cain (Air Force Research Institute): Intelligence Analysis and Professional Journal Publishing William Lawhead (University of Mississippi): Seeing Is Not Believing: Insights for Intelligence Analysis from Professional Magicians.

12:30 Lunch 1:30 Intelligence Analysis and Social Science  

Chair: Stephen Marrin (Brunel University) Stefania Paladini (Coventry University): Bayesian analysis in intelligence and social sciences: a tool for all trades?"

2:30 Break 3:00 Intelligence Analysis and History    

Chair: Philip Davies (Brunel University) Melissa Graves (University of Mississippi): Intelligence Analysis and Historiography Howard Clarke (The Intelligence Solutions Group) Seekers after Truth: Cross-Disciplinary Insights for the Intelligence Profession from the Biblical and Theological Studies Discipline WJR Gardner (Naval Historical Branch, UK Ministry of Defence): The Unreliable Memoirs of an Applied Historian … and Operational Analyst and …

4:00 Break 4:15 Intelligence Analysis and Perception  

 

Chair: Melissa Graves (University of Mississippi) Patricia Rooney (National Ground Intelligence Center, US Army) and Peter Ronayne (Federal Executive Institute): Neuroscience in Intelligence Analysis: Understanding How the Brain Works and Its Impact on Analytic Decision Making Musa Tuzuner (Intelligence Studies Research Center; Turkish National Police Academy): MultiLevel Cognitive Biases in the Chain of Intelligence Flow—From Source to Consumers Mandeep Dhami (University of Surrey): Intelligence Analysis and Decision Science (Putting the Science into the Art of Intelligence Analysis)

5:30 Day One Adjourns

2

Conference agenda Day 2: Friday 13 July 8:00 Registration and Coffee 9:30 Welcome Part 2 9:45 Evaluating Intelligence  John Kringen (former CIA Director of Intelligence; Institute for Defence Analysis): Judging Intelligence Success and Failure 10:45 Evaluating Intelligence  Chair: Philip Davies (Brunel University)  Stephen Marrin (Brunel University): Evaluating the Quality of Intelligence Analysis: By What (Mis) Measure?  Rhiannon Gainor and France Bouthillier (McGill University): Measuring Intelligence Success: Conceptual and Methodological Challenges of Intelligence Measurement 11:15 Break 11:30 Intelligence Analysis and the Dismal Science  Chair: Julian Richards (University of Buckingham)  Michael Herman (Oxford University): Intelligence Analysis and Government Statistics  Nick Hare (UK Ministry of Defence): Intelligence Analysis and Economics 12:30 Lunch 1:30 Improving Communication  Chair: Julian Richards (University of Buckingham)  Sean Newman (Fire Department of New York): How Adherence to Journalistic Standards Positioned FDNY Watchline at the Forefront of Fire Service Analytical Intelligence  Douglas Bernhardt (South Africa: Wits Business School): Intelligence Reporting: What Works, What Doesn't and How to Fix It  Gabriel Sebe (Bucharest University): Political Marketing Intelligence 2:30 Break 3:00 Applying Knowledge from Other Fields to Increase Understanding  Peter Nardulli and Dan Roth (University of Illinois): Advancing our Understanding of Global Patterns of Civil Unrest: An Interdisciplinary Effort  Kathleen Vogel (Cornell University): Current Intelligence Reporting and the H5N1 Bird Flu Virus: Insights from the Field of Science and Technology Studies 4:00 Break 4:15 Practitioner views, summary, and next steps  TBD 5:30pm Conference Adjourns

3

ABOUT THE SPEAKERS Douglas Bernhardt lectures in the subject of Competitive Intelligence at Wits Business School, Johannesburg, the University of Stellenbosch Business School, and the Nelson Mandela Metropolitan University Business School, Port Elizabeth. He is also an adjunct instructor in Competitive Intelligence for Mercyhurst University. Previously, Douglas served as an adjunct professor of Business Intelligence & Corporate Security at the European campus of the Thunderbird School of Global Management. He has also taught at IMD, Lausanne and the Rotterdam School of Management. He is the author of three books, including Competitive Intelligence: Acquiring and using corporate intelligence and counterintelligence, published in 2003 by Financial Times Prentice-Hall. From 1993-2001 Douglas served as Managing Director for one of Europe’s leading competitive intelligence consultancies, Business Research Group SA, in Geneva and London, supporting the CI requirements for major firms in industries such as pharmaceuticals, telecommunications, and FMCGs. He was a member of the Board of Directors of the US-based Society of Competitive Intelligence Professionals (SCIP) from 1996-1999. His early commercial experience includes the defence industry and foreign trade. Anne W. Bishop is an attorney at Snell & Wilmer, LLP, in Phoenix, Arizona. Prior to attending law school, Ms. Bishop worked at the National Security Agency at Ft. Meade, Maryland where she held positions as a linguist and analyst, and taught intelligence analysis to other analysts. Following a fellowship working for a New York Congresswoman, she worked in NSA’s legislative affairs office, working with the House Permanent Select Committee on Intelligence Issues. Ms. Bishop’s legal practice focuses on health care compliance, health care litigation, and business disputes. She advises physicians and hospitals on various health-care related issues. Ms. Bishop is also a member of the privacy and data security group. Ms. Bishop is a frequent writer and speaker on health care compliance and legal ethics. Ms. Bishop graduated in 1986 from the University of Pennsylvania with a degree in international relations & in 2006 from the Sandra Day O’Connor College of Law at Arizona State University. Dr. Anthony C. (Chris) Cain is Chief of Academic Affairs at The Air University, Maxwell Air Force Base, Alabama. He received his M.A. and Ph.D. in Military History from The Ohio State University, and also holds Masters Degrees from the Air War College and Embry-Riddle Aeronautical University, as well as a B.S. from Georgia State University. Dr. Cain is a graduate of the Air Command and Staff College (ACSC), and the Air War College. Prior to assuming his present duties, he served as the Deputy Director of the Air Force Research Institute (AFRI), Dean of AFRI, Chief of AFRI’s Outreach Division, founding Editor-in-Chief of Strategic Studies Quarterly, Research Director and Dean of Education & Curriculum at ACSC, and as the Chief, Professional Journals Division & editor of the Air and Space Power Journal at the College of Aerospace Doctrine, Research and Education. Dr. Cain is the author of The Forgotten Air Force: The French Air Force and Air Doctrine in the 1930s (Smithsonian Institute Press, 2002), as well as book chapters and articles on issues dealing with airpower, professional military education, effectsbased operations, regional threats from weapons of mass destruction, & military history. Dr. David Chuter worked for more than thirty years for the UK government in the defence and security area. He was involved, among other subjects, in the negotiations leading to the Maastricht Political Union Treaty, and subsequent discussions about a European security and defence policy, as well as arms control, weapons of mass destruction, and war crimes and transitional justice. His last job was as Special Adviser to the Policy Director of the French Ministry of Defence in Paris. He took early retirement at the end of 2008 and is now an author, lecturer, translator and consultant based in Paris. Dr Chuter is a lecturer at Sciences Po in Paris, where he teaches courses in Intelligence and Policy-making, and in Crisis Management. He is also Senior Research Fellow at the Centre for Security Sector Management at Cranfield University, on whose behalf he teaches security-related subjects in various countries, and Chair of the Governing Board of the International Security Information Service Europe. He is the author of four books, and many articles, most recently Governing and Managing the Defence Sector (2011). His current research interest is the politics and practical problems of the Rule of Law in the Security Sector.

4

Howard Clarke, MCS, CCA is an internationally experienced law enforcement intelligence practitioner. During his thirty-year law enforcement career Howard held senior intelligence analysis positions with law enforcement agencies in Australia and Canada. For the past decade he has specialized in strategic intelligence analysis applications in law enforcement and homeland security and he remains active in intelligence analysis training and consulting work. He has taught extensively on strategic intelligence analysis and open source intelligence issues in the U.K., Europe, Asia and North America and is a published author on Insider Threat issues. He holds a Master of Christian Studies from Regent College International Graduate Theological School. Howard is currently an intelligence analysis instructor at the British Columbia Institute of Technology and the Justice Institute of British Columbia in Canada and he formerly held adjunct faculty status with Henley-Putnam University in the U.S. He is a Lifetime Certified Criminal Analyst, a former member of the Board of Governors of the Society of Certified Criminal Analysts (SCCA), a member of the International Association of Law Enforcement Intelligence Analysts (IALEIA) and the International Association for Intelligence Education (IAFIE). Dr. Philip H.J. Davies is a political sociologist specialising in the study of national intelligence institutions. He is Director of the Brunel Centre for Intelligence and Security Studies (BCISS). In 2010 BCISS was awarded University Interdisciplinary Research Centre status and now includes scholars from across the University from departments and schools as diverse as Engineering and Design, Economics and Law in addition to its core team based in Politics and History. During 2009-10, Dr. Davies headed BCISS work on the new UK military Joint Intelligence Doctrine. On this project, Brunel was selected by the Ministry of Defence as a full partner with its Development, Concepts and Doctrine Centre and Defence Intelligence (former the Defence Intelligence Staff). This work has involved both the fundamental rethinking and reformulation of intelligence doctrine (JDP 2-00 Understanding and Intelligence in the Joint Operating Environment) but also the articulation of an entirely new doctrine on ‘understanding’ (JDP 04 Understanding). Before that Dr. Davies ran an ESRC-funded international seminar series entitled ‘Intelligence and Government in the 21st Century’ working in collaboration with Dr. Robert Dover at Loughborough. He has also recently completed a major and intensive comparative study of British and American national intelligence has been published by Praeger Security International in 2012. In 2004 Dr. Davies designed and, until 2010, convened Brunel’s highly successful MA in Intelligence and Security Studies, including its ground-breaking Brunel Analytical Simulation Exercise. BASE is term-long practical in strategic intelligence assessment in which students are divided into drafting teams modelled on the UK Joint Intelligence Organisation analytic staff and produce JIO-style assessments on live, real-world topics using open sources. Dr. Mandeep K. Dhami is a Reader in Forensic Psychology at the University of Surrey. Previously she held academic appointments at the University of Cambridge (UK), University of Victoria (Canada), the University of Maryland (USA), and the Max Planck Institute for Human Development (Germany). She has worked as a Senior Scientist for the Defence Science and Technology Laboratory (MoD). Dr. Dhami has a PhD in Psychology and an MA in Criminology. Her research draws conceptual connections between Psychology (esp. decision science and social cognition) and Criminology. She has examined topics such as decision making, risk, and forecasting. She has applied these issues to the criminal justice system, and more recently to the defence and security sectors (with a focus on intelligence analysis and cyber issues). To-date, she has over 65 publications, and is lead editor of a book entitled “Judgment and Decision Making as a Skill: Learning, Development and Evolution” published by Cambridge University Press in 2011. Her research has won several awards including from the European Association of Decision Making and the Society for the Psychological Study of Social Issue (SPSSI, Division 9 of the American Psychological Society). Dr. Dhami is Fellow of SPSSI (Division 9 of the American Psychological Association), and is on the editorial board of several journals including Perspectives on Psychological Science. She has provided advice and consultancy to various government bodies including the Home Office, Ministry of Justice, Government Communications Headquarters, and the Metropolitan Police (New Scotland Yard).

5

Rhiannon Gainor is a PhD candidate at McGill University’s School of Information Studies, a McConnell Foundation Fellow in 2010/2011, and a recipient of Québec’s Fonds de recherche Société et culture (FQRSC) grant. Her research interests are knowledge management, competitive intelligence, and information metrics. She has a Master’s of Library and Information Studies, and Master’s of Arts in Humanities Computing from the University of Alberta. W J R (Jock) Gardner served in the Royal Navy for 30 years, specialising in anti-submarine warfare. He served on operational, analysis and intelligence staffs. In 1989 he was awarded an M Phil in International Relations at the University of Cambridge, writing a dissertation on Soviet nuclear ballistic missile submarines. He was also the Editor of The Cambridge Review of International Affairs. Leaving the Navy in 1994, he joined the Naval Historical Branch as a Historian and taking an especial interest in World War II, antisubmarine warfare and intelligence. His publications include Anti-submarine Warfare (Brassey’s, 1996) and Decoding History: the Battle of the Atlantic and Ultra (Naval Institute Press, 1999). He has lectured on naval historical and intelligence topics from Moscow to Monterey. He is the external examiner at Brunel University for the MA in Intelligence and Security Studies, a member of the Editorial Board of The Mariner’s Mirror and a Fellow of the Royal Historical Society. Melissa Graves, JD, MA serves as Project Coordinator and Instructor at the University of Mississippi’s Center for Intelligence and Security Studies. Among her accomplishments, she and her co-developer Walter Flaschka have designed and implemented the Days of Intrigue, a realistic practical exercise that is conducted yearly at UM and which involves numerous intelligence community agencies. Ms. Graves received her BA with a double major in English and Communications from Hardin-Simmons College, an MA in History from UM, and a JD from the UM School of Law. She is presently pursuing a Ph.D. in History. Her work will appear in upcoming publications, including Introduction to Intelligence Studies (Carl Jensen, David McElreath, and Melissa Graves) and Intelligence Studies in Britain and the US: Historiography since 1945 (ed. Christopher Moran). Nick Hare is a professional economist, and has worked in a number of roles in the UK Ministry of Defence (MOD) since graduating in 1999 with an MA in Philosophy and Economics from the University of Edinburgh. He is currently head of the Futures and Analytical Methods (FAM) team in the MOD. The FAM team works to promote the aims of the Professional Head of Defence Intelligence Analysis. This involves encouraging analysts to use structured and auditable methods to refine intelligence requirements, identify assumptions, generate hypotheses and scenarios, test hypotheses using data, collect information, and describe conclusions using appropriate probabilistic language. The FAM team also researches new and developing tools and approaches that are of potential use to intelligence analysts, and works with other departments to promote professionalisation of analysis across government. Michael Herman served from 1952 to 1987 in Britain's Government Communications Headquarters, with secondments to the Cabinet Office (as Secretary of the Joint Intelligence Committee) and to the Ministry of Defence.. His Intelligence Power in Peace and War was published in 1996 and has been regularly reprinted. His Intelligence Services in the Information Age was published in 2001, and he has co-edited and contributed to Intelligence in the Cold War: What Difference Did It Make (in publication). He is currently an Honorary Departmental Fellow at Aberystwyth University and an Associate Member of Nuffield College Oxford, as Founder Director of the Oxford Intelligence Group. He is an Honorary D.Litt of Nottingham University.

6

Dr. Carl J. Jensen III. is a 1978 graduate of the U. S. Naval Academy. He served in the Navy from 1978 until 1983, first aboard the nuclear fleet ballistic missile submarine USS George Washington Carver and then as an aide to the Commander of Submarine Group Five. Dr. Jensen graduated from FBI New Agents Training in 1984 and served as a field agent in Atlanta, Georgia, Monterey, California, and Youngstown, Ohio. In August 1992, Dr. Jensen reported to the FBI Laboratory where he received certification as a Racketeering Records Examiner. In June 1997, Dr. Jensen reported to the Behavioral Science Unit at the FBI Academy in Quantico, Virginia, where he instructed senior police officials at the FBI National Academy, conducted research, provided consultation, and served as Assistant Unit Chief. Upon his retirement from the FBI in 2006, Dr. Jensen joined the RAND Corporation as a Senior Behavioral Scientist. In 2007, he joined the Legal Studies faculty at the University of Mississippi, where he currently serves as director of the Center for Intelligence and Security Studies. He is the 2012 recipient of the Outstanding Instructor of the Year Award from the International Association for Intelligence Education, a 2008 recipient of the Thomas Crowe Outstanding Faculty Award from the University of Mississippi and a 2004 recipient of the Jefferson Award for Outstanding Research from the University of Virginia. Dr. Jensen holds a Master of Arts degree from Kent State University and a Doctor of Philosophy degree from the University of Maryland. He has instructed throughout the world and is the author of numerous articles, books, book chapters and technical reports. Dr. John A. Kringen is a Research Staff Member with the Institute for Defense Analyses in Alexandria Virginia. He recently retired from the Central Intelligence Agency after more than three decades of government service. During 2005-2008 he served as the Director of Intelligence at CIA where he managed and led the activities of several thousand CIA analysts and staff. Prior to becoming Director of Intelligence, he served in a variety of managerial assignments in that Directorate, including as Director of the CIA’s Crime and Narcotics Center. Beyond CIA, he served twice overseas in Europe, most recently as a senior intelligence advisor to United States European Command from July 2008 to August 2011. In the late 1990s, he was head of imagery analysis at the predecessor organization to the National Geospatial Intelligence Agency. He received a Ph.D. from the University of Minnesota in 1978. Dr. William Lawhead retired from the University of Mississippi in summer 2012 after having completed seven years as Chair of the Department of Philosophy and Religion. He also completed forty-one years of teaching philosophy, thirty-two of them at UM. Among the courses he has taught are history of philosophy, logic and critical thinking, and philosophy of science. He has served on the advisory board of the University of Mississippi’s Center for Intelligence and Security Studies and has participated in and given papers at 5-Eyes Conferences. He is a member of the International Association for Intelligence Education. He has published two books. These are a history of Western philosophy, The Voyage of Discovery, 3rd ed. (Wadsworth, 2007) and an introduction to philosophy, The Philosophical Journey, 5th ed. (McGraw-Hill, 2011). He is working on a third book, Taking Philosophy Seriously, for Oxford University Press. Dr. Lawhead received his Ph.D. in philosophy from the University of Texas, Austin. Dr. Stephen Marrin is a Lecturer in the Department of Politics and History at Brunel University in London, England. He received his PhD and MA in foreign affairs from the Department of Politics at the University of Virginia. He has written about many different aspects of intelligence analysis, including new analyst training at CIA’s Sherman Kent School, the similarities and differences between intelligence analysis and medical diagnosis as a source of ideas for improving the quality of future intelligence analysis, and the professionalization of intelligence analysis. In 2004 the National Journal profiled him as one of the ten leading experts on intelligence reform. He previously worked as an analyst at both the US Central Intelligence Agency and the US Government Accountability Office, and taught in the Intelligence Studies Department at Mercyhurst University.

7

Dr. Peter F. Nardulli is Professor of Political Science and Law at the University of Illinois at UrbanaChampaign, the founding Director of the Cline Center for Democracy, and the editor of a book series with the University of Illinois Press: Democracy, Free Enterprise and the Rule of Law. He has been on the faculty at UIUC since 1974 and served as department head in Political Science from 1992 until 2006. Nardulli is the author of six books on various aspects of the legal process and empirical democratic theory. He has authored a number of articles in journals such at the American Political Science Review, Public Choice, Political Communication, Political Behavior and a number of law reviews. Nardulli is currently directing a global study, the Societal Infrastructures and Development Project (SID). SID uses a number of technologically advanced, innovative methodologies to examine the impact of political, legal and economic institutions on a wide range of societal development indicators (economic growth, human rights, societal stability, environmental quality, educational attainment etc.). Current projects involve using data from the SID project’s Social, Political and Economic Event Database (SPEED) project to examine the impact on civil unrest of such things as climate change, natural resources, socio-cultural animosities and political institutions. Captain Sean S. Newman, a 16-year veteran of the Fire Department City of New York (FDNY), has worked most of his career in Manhattan, and is a charter member of FDNY Center for Terrorism and Disaster Preparedness (CTDP), opened in 2004. Captain Newman has been editor and contributing writer for FDNY Watchline since the intelligence product’s inception in 2008. Released in December 2011, he co-wrote the “FDNY Counterterrorism and Risk Management Strategy.” Captain Newman graduated with honors from the Center for Homeland Defense and Security (CHDS) at the U.S. Naval Postgraduate School with a master’s degree in security studies. Aside from his professional fire experience, Captain Newman is a former journalist on the Reuters equities' desk in NYC. Dr. Stefania Paladini is Associate Head of Department in Strategy and Management at Coventry University. Before joining academia she has spent 7 years appointed as trade commissioner in East Asia, where she has also obtained a PhD from City University of Hong Kong in security studies. An economist and a statistician by professional practice, she has also published extensively on environmental and economic security and given a number of policy briefings in Italy at ministerial level. She has a long term interest in intelligence studies, where she is now working on the application of Bayesian analysis and other quantitative methods. Dr. Julian Richards successfully completed a doctorate in political violence in Pakistan, at Cambridge University, in 1993. He then spent nearly 20 years working in intelligence and security for the British government. In 2008, he co-founded the Centre for Security and Intelligence Studies (BUCSIS) at the University of Buckingham, where he teaches on two MA programmes in Intelligence Studies and Global Security, and supervises a number of PhD students. His affiliations include Associate of the Pakistan Security Research Unit in Bradford University, a member of the European Ideas Network, and the editorial board of the Spanish journal Intelligencia y seguridad. Julian Richards is the publisher of two books: The Art and Science of Intelligence Analysis (Oxford University Press, 2010); and A Guide to National Security: Threats, Responses and Strategies (Oxford University Press, 2012); in addition to a number of papers on a range of security and intelligence issues. He is also a regular media commentator on security and intelligence issues for BBC television and radio, in addition to appearances on Al Jazeera and the Islam Channel. His current research interests include attitudes towards surveillance in contemporary society; the rise of cyber-threats and cyber-security; security issues in Pakistan and Afghanistan; and violent extremism in Europe.

8

Michael Robinson is a Specialist Leader with Deloitte and Touche, LLP, where he conducts computer and mobile device forensics. Prior to joining Deloitte and Touche, LLP, Robinson conducted forensic examinations for the Federal Bureau of Investigation. He also previously worked as the Chief Information Officer of the U.S. Department of Defense's Business Transformation Agency, where he was responsible for all Information Technology and Information Assurance activities, as well as forensic examinations. Robinson is an adjunct professor at Stevenson University in the School of Graduate and Professional Studies, where is the program coordinator of the Cyber Forensics degree. He is also an adjunct professor at George Mason University, where he teaches mobile device forensics. Dr. Peter V. Ronayne is a senior faculty member at The Federal Executive Institute (FEI), an executive leadership education center within the U.S. Office of Personnel Management. Pete formerly directed FEI’s flagship Leadership for a Democratic Society program, co-founded FEI’s Center for Global Leadership and FEI’s Leadership Horizons Series for the Senior Executive Service. He joined FEI after studying, teaching, and researching at the University of Virginia. Pete is a leading voice in the public sector on generational and demographic issues and their impact on leadership and organizations. He also writes, researches, and speaks widely on issues of global leadership, neuroscience and leadership, and the future of public service and governance. A former Presidential Management Fellow, Pete is an adjunct professor at the University of Virginia where he teaches undergraduate courses in world politics, diplomatic history, and leadership. Pete is currently at work on a biography of Secretary of State Madeleine Albright to be published in 2013. He is also the author of Never Again?The United States and the Prevention and Punishment of Genocide since the Holocaust, the first book to chronicle America’s complicated and conflicted response to the crime of genocide since 1945. Pete earned a PhD in International Affairs at the University of Virginia and a BS in Languages and Linguistics at Georgetown University. Prior to his executive education career, Pete worked in the fields of international emergency medical assistance and international development, where he worked on small-scale, sustainable and replicable technologies for developing nations. Dr. Patricia J. Rooney is currently the Director of Training at the U.S. Army’s National Ground Intelligence Center (NGIC). Prior to joining NGIC, PJ served as senior faculty at the Federal Executive Institute where she taught executive leadership courses with a focus on neuroscience. Before Joining FEI, PJ served in a number of leadership positions within the U.S. Department of Defense (DoD). While at the National Geospatial-Intelligence Agency (NGA), PJ was the Deputy Director of the field operations office co-located with NGIC. She led the National Geospatial-Intelligence College as Deputy Director, and has served as the Dean for NGA’s School of Leadership and Professional Studies. In addition to experience in the Federal government, PJ served as a Senior Associate with SENSA Solutions, working with government leaders on strategic planning, business process re-engineering, leadership change strategies, and senior executive coaching. Before joining the intelligence community, PJ served on the faculty at the United States Military Academy teaching in the Leadership and Behavioral Science Department; and served as a counselor for Victims of Violent Crime with the Metropolitan Nashville Police Department. PJ earned a PhD in Education from the University of Virginia, an MS in Strategic Intelligence from the Joint Military Intelligence College, a MEd from Vanderbilt University, and a BA in Political Science from the University of North Carolina at Chapel Hill.

9

Dr. Sorin-Gabriel Sebe teaches in the Department of Political Science at the University of Bucharest in Romania. He graduated from the Faculty of Mathematics, University of Bucharest in 1986 (the graduation thesis: „Non-Riemannian Geometries in Dislocation Theory”). From 1990 to 1995 he worked as a Research Associate at GPM2 Laboratory in Grenoble, France, in the realm of mathematical modeling of irreversible phenomena. In 1996 he joined the Faculty of Political Science, University of Bucharest (FSPUB) and currently, beyond teaching, serves as Vice-Dean. He received his PhD in sociology in 2004 (the final thesis: „Subjective Quality of Life Modeling”) from the Faculty of Sociology, University of Bucharest. He was a general manager at INSOMAR (2004-2006). His research as a political science academic has primarily been concerned with the processes associated to the democratization of former non-democratic regimes, such as the reform of the intelligence apparatus and the development of a political market. He favors a trans-disciplinary approach on research, having as current interest the development of a curriculum for the study of intelligence in academic settings, in a political science faculty, in a former communist country. Dr. Musa Tuzuner had a long and varied experience in the intelligence field in Turkey. PhD in Political Science (Kent State University, 2009 with Best Dissertation Award); Assistant Professor, Turkish National Police Academy (2009–); Founding Director, Intelligence Studies Research Center (2009–) & Intelligence Studies Master Program (2010–), TNP Academy; current main interests: national and international security behavior, theorizing Turkish foreign policy, and international event data developments. Most recent book: Intelligence Cooperation Practices in the 21st Century: Towards a Culture of Sharing (IOS Press, 2010). Most recent article: Quantifying intelligence cooperation: The United States International Intelligence Behavior (USIIB) dataset, with Ersel Aydinli (Journal of Peace Research 48(5) 673–682, 2011). Dr. Kathleen Vogel is an associate professor at Cornell, with a joint appointment in the Department of Science and Technology Studies and the Judith Reppy Institute for Peace and Conflict Studies. Vogel holds a Ph.D. in biological chemistry from Princeton University. Prior to joining the Cornell faculty, Vogel was appointed as a William C. Foster Fellow in the U.S. Department of State’s Office of Proliferation Threat Reduction in the Bureau of Nonproliferation. Vogel has also spent time as a visiting scholar at the Cooperative Monitoring Center, Sandia National Laboratories and the Center for Nonproliferation Studies, Monterey Institute of International Studies.

10

Understanding and Improving Intelligence Analysis: Learning from other Disciplines 12-13 July, 2012 * London, England * RAF Club

RELATED READINGS Benson, Sumner. "The Historian as Foreign Policy Analyst." The Public Historian 3, no. 1 (1981): 15-25. (HISTORY/HISTORIOGRAPHY) Converse, Ray & Randy Pherson. Intelligence and Medicine: Parallel Cognitive Traps. Unpublished paper. Pherson Associates. 2009. 1-6. (MEDICINE/PSYCHOLOGY/COGNITION) Fisher, Rebecca and Rob Johnston. Is Intelligence Analysis a Discipline? Roger George and James Bruce (Eds). Analyzing Intelligence. Georgetown University Press. Washington DC. 2008. 55-68. (LAW/LEGAL PROFESSION/LIBRARY SCIENCE/MEDICINE) Heuer, Richards J. “Adapting Academic Methods and Models to Governmental Needs.” Quantitative Approaches to Political Intelligence: The CIA Experience. (Ed. Richards J. Heuer). Westview Press. 1978. (1-10). (SOCIAL SCIENCE METHODS) Johnston, Rob. Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study. Washingon, DC: Central Intelligence Agency, Center for the Study of Intelligence, 2005. (ANTHROPOLOGY) Kerbel, Josh. “Lost for Words: The Intelligence Community’s Struggle to Find its Voice.” Parameters. 2008. 102-112. (MEDICINE) Knorr, Klaus. “Foreign Intelligence and the Social Sciences.” Research Monograph No. 17. Center of International Studies. Princeton University. June 1, 1964. (SOCIAL SCIENCE METHODS) Laqueur, Walter. "The Question of Judgment: Intelligence and Medicine." Journal of Contemporary History 18 (Oct. 1983): 533-548. (MEDICINE) Leslau, Ohad. "Intelligence and Economics: Two Disciplines with a Common Dilemma."

International Journal of Intelligence and Counterintelligence 20, no. 1 (Spring 2007): 106121.(ECONOMICS/ECONOMIC ADVISORS)

11

Marrin, Stephen, & Jonathan Clemente. "Improving Intelligence Analysis by Looking to the Medical Profession." International Journal of Intelligence & Counterintelligence 18/4 (Winter 2005-06): 707-729. (MEDICINE) Marrin, Stephen, and Jonathan Clemente. "Modeling an Intelligence Analysis Profession on Medicine." International Journal of Intelligence and Counterintelligence 19/4 (Winter 2006-2007): 642-665. (MEDICINE) Marrin, Stephen. “Intelligence Analysis: Structured Methods or Intuition?” American Intelligence Journal. Vol. 25.No. 1.(Summer 2007).7-16. (MEDICINE) Nolte, William. "Thinking about Rethinking: Examples of Reform in Other Professions." Studies in Intelligence 52, no. 2 (Jun. 2008): 19-25. (GENERAL) Pritchard, Matthew C. & Michael S. Goodman (2008): Intelligence: The Loss of Innocence, International Journal of Intelligence and CounterIntelligence, 22/1, 147-164. (ARCHAEOLOGY) Rieber, Steven, and Neil Thomason. "Toward Improving Intelligence Analysis: Creation of a National Institute for Analytic Methods." Studies in Intelligence 49, no. 4 (2005): 71-77. (MEDICINE) Rodgers, R. Scott. "Improving Analysis: Dealing with Information Processing Errors." International Journal of Intelligence and Counterintelligence 19/4 (Winter 2006-2007): 622-641. (PSYCHOLOGY) Spivey, Robin V. "The Devil Is in the Details: The Legal Profession as a Model for Authentic Dissent." International Journal of Intelligence and Counterintelligence 22/4 (Winter 2009): 632-651. (LAW/LEGAL REASONING) Weiss, Charles. "Communicating Uncertainty in Intelligence and Other Professions." International Journal of Intelligence and Counterintelligence 21, no 1 (Spring 2008): 57-85. (GENERAL/SCIENCE/LAW) Young, David. Scholarship, Intelligence, and Journalism. Reuters Fellows Lecture. March 2004. (JOURNALISM)

12

Paper and Presentation Abstracts Understanding and Improving Intelligence Analysis: Learning from other Disciplines 12-13 July, 2012 * London, England * RAF Club Douglas Bernhardt (South Africa: Wits Business School): Intelligence Reporting: What Works, What Doesn’t and How to Fix It: This paper argues that however whatever the quality of the findings and analysis of an intelligence deliverable, it will often fail to achieve its main objective; that is, to provide a decision-maker with product that he or she finds both compelling and relevant to his or her specific decision challenges. This occurs mainly for two reasons: (1) the ‘disconnect’ which often exists between intelligence consumers and producers and (2) the ‘packaging’ of finished intelligence as a set of dry, colourless news and facts. This author, a long-time practitioner and lecturer in the field of Competitive Intelligence, has previously described the phenomenon of the sub-optimal relationships which exist in many firms between Competitive Intelligence units and their executive customers. In this paper he not only revisits and updates earlier discussions, but adds a fresh dimension; one which centres around the process of persuasion. In short, if an intelligence product is not persuasive (think the PDB of 6 August 2001), it has no value; it’s just another report to digest. This paper will consider the factors which can and should make an intelligence briefing, or report, persuasive; borrowing, in part, from the fields of advertising and sales. It will also focus on the unique contribution that human source information (HUMINT) makes to intelligence analysis and reporting OSINT is important, but it’s not enough. The paper/presentation we propose will explore two principle topics: (1) what are the issues. involved in, and what needs to be done to overcome, the challenges inherent in the producer-consumer relationship? and (2) what tools can we apply to improve, and sometimes totally recast, the packaging and delivery of intelligence product that makes a difference, that serves as a springboard for its users to gain ‘competitive advantage’? Although this presentation will be discussed within a corporate sector context, the key lessons are equally relevant to the domains of national security and law enforcement.

Anne Bishop (Snell & Wilmer, LLP, Phoenix AZ): Intelligence Analysis and Legal Reasoning: This presentation will explore the similarities between legal reasoning and intelligence analysis. The presenter is a practicing attorney who spent about 15 years in the U.S. intelligence community, working as an analyst and linguist and training intelligence analysts. Ms. Bishop will focus on the similarities in acquiring relevant information, evaluating sources, and deriving meaning from that information. In both intelligence analysis and legal research, locating the relevant information is critically important. Ms. Bishop will discuss access, search strategies, and ways to validate search results. In addition, Ms. Bishop will discuss the relatively static hierarchy of sources that exists in the legal research realm. However, the bulk of the presentation will focus on analytic techniques that are central to legal analysis-- analogies, extrapolation, legal reasoning, and the importance of context. Lawyers are taught in law school to draw analogies between their current cases and existing case law precedent. By creating a compelling analogy, a lawyer is likely to prevail in a court. Similarly, by drawing analogies to prior events, an intelligence analyst can analogize what is transpiring today based on similar prior events. In addition, Ms. Bishop will discuss extrapolation and its value to both intelligence analysis and legal reasoning. That is, in legal reasoning, there is often no legal principal that is completely similar. Lawyers learn to find a case that is analytically similar and extrapolate the principles to the situation at hand. Similarly, intelligence analysts can look at an existing set of facts and extrapolate from known facts into the unknown. In legal reasoning, lawyers look at two existing cases and draw inferences regarding how those two cases fit together in order to determine how a court should rue on a current case. This technique could be used predicatively by intelligence analysts. Finally, Ms. Bishop will discuss the importance of context. Context is key in determining a legal outcome. The same is true for intelligence analysis. Determining the status quo is critical in both context in order to figure out the potential outcomes.

13

Anthony C. Cain (Air Force Research Institute): Intelligence Analysis and Professional Journal Publishing: Publishing professional military journals requires many of the same processes and decision capabilities found in intelligence analysis. Professional journals, by their nature, can narrowly focus on technical or tactical subjects, but the selection and editing of content requires broad knowledge in several areas. For example, journals that focus on operational issues may receive article submissions that deal with the full spectrum of service or joint capabilities. Journal editors will rarely be trained or equipped to evaluate the credibility and merits of such a broad spectrum of content. Therefore, creating mechanisms to ensure the credibility and quality of journal content form the essential tasks for professional journal editing. Those same mechanisms could be adapted to serve intelligence analysis processes. Specific cases from the author’s experience as the editor of Air and Space Power Journal and as founding editor-in-chief of Strategic Studies Quarterly will illustrate the challenge and solutions adopted to preserve the professional credibility of both journals.

David Chuter (Center for Security Sector Management; Cranfield University): Intelligence Analysis as a Type of Information Processing Although intelligence analysis is obviously a special case of the collection, processing and use of information, it is not entirely unique. Indeed, we carry out analogous tasks to intelligence analysis all the time, not only professionally, but in our personal lives as well. Intelligence analysis can be seen as a type of information processing which has a special combination of characteristics, generally not found together elsewhere. We can analyse all information processing under three types of heading. The first is the degree of applicability, which is to say how far the process is designed to lead to an operational outcome. At one extreme, practitioners of emergency medicine have a short period in which to gather, process and act on information in order to save a life. At the other extreme, scientists, archaeologists or sociologists may spend years on a research programme which simply validates prior assumptions, and does not have any operational output at all. Intelligence is in principle only collected in response to specific tasking to answer specific questions, and so in principle is a high-applicability process. The second is the degree of completeness of the information which can reasonably be desired. Surgeons contemplating major, nonacute surgery, will exhaustively research everything they can, to make sure that what they do is as effective as possible. At the other extreme, many decisions are successfully taken in everyday life on the basis of very incomplete information: if you arrive at an airport and realize that you have left your adaptor plug at home, you will decide to buy the first thing you see that functions correctly. There are many intermediate cases: historical research, for example, can never in principle be complete or definitive, no matter how long it continues. Criminal trials generally only consider enough evidence to establish guilt beyond a reasonable doubt, and some potential crimes may not even be charged. Intelligence is by definition always incomplete, and its reliability is always suspect. The third is the need for and expectation of accuracy. Most medical decisions have to be very accurate, since otherwise the patient will be harmed, and there are professional sanctions for those who make mistakes. At the other extreme, journalism and the productions of advocacy NGOs are seldom expected to be fully accurate, and there are few professional sanctions for inaccuracy or even downright untruths. There are intermediate cases such as investment advice, where there is an expectation of at least broad accuracy, but few sanctions in its absence. There are also cases (like the conclusions of juries in criminal trials) where the degree of accuracy is not knowable, and all one can ask is whether the verdict is reasonable in the circumstances. Intelligence is generally an area with high (and often excessive) expectations of accuracy, and a great requirement for it. The special combination of high applicability, lack of completeness and significant demand for accuracy makes intelligence analysis a special and interesting case.

14

Howard Clarke (The Intelligence Solutions Group) Seekers after Truth: Cross-Disciplinary Insights for the Intelligence Profession from the Biblical and Theological Studies Discipline Biblical hermeneutics, or exegesis, is an interpretative activity that contains some significant parallels with important facets of the intelligence analysis endeavor. The term exegesis refers to careful investigation of the original meaning of texts in their historical and literary contexts. The process involves asking analytical questions about various aspects of the source materials and their contexts, and it implies careful, detailed analysis. As an analytic discipline, effective biblical interpretation requires a capacity to deal with challenging data sets, complex source evaluation circumstances, and it seeks to provide insights which will shape and influence the beliefs and actions of a significant community (i.e. the consumers). In principle, a process not dissimilar to the Intelligence Cycle is employed and at various times and in various contexts there is a need to utilize techniques which can be construed as forms of HUMINT, OSINT, and TECHINT. As a result of the significant analytic challenge involved a growing body of interpretative techniques and methodologies has developed over many centuries. In the sense that intelligence analysis is essentially concerned with the search for truth and meaning (albeit in particular geopolitical, security and various specific contexts) and the communication of resultant insights to decision-makers, then there are correlations to be drawn with the sub-disciplines of biblical exegesis and hermeneutics and the overarching theological discipline they inform. Particular challenges that must be addressed in biblical interpretation include: (1) the challenge of distance of time – the key events and communications to be understood are separated from the contemporary interpreter by a vast gulf of time; (2) the challenge of cultural distance – key actors and events reside within an ancient agrarian, middle eastern context; (3) the challenge of geographical distance – requires an appreciation of the geography of the Middle East at various points in history; (4) the challenge of linguistic distance – communication occurs in the languages of Hebrew, Aramaic and Greek in the form those languages took thousands of years ago; and (5) • various challenges around authorship and sourcing. The successful exegetical practitioner, like the proficient intelligence analyst, must: (a) • understand the world behind the communication being researched – the context of the source and the subject the source is reporting on; (b) Understand the world within that communication – the language and structuring of the communication, what is the source saying and how is the source saying it, and what meaning does this convey?; and (c) understand the world before the communication – the context of the message recipient (the intelligence consumer if you like). This paper will explore and discuss various areas of similarity and dissimilarity between intelligence analysis practice and exegetical practice and will suggest some opportunities for productive crossdisciplinary learning for intelligence practitioners.

Mandeep Dhami (University of Surrey): Intelligence Analysis and Decision Science (Putting the Science into the Art of Intelligence Analysis) What is the probability that Iran will have nuclear weapons in 2012? What is the risk posed by Anonymous to UK cyber space? Which al-Qaeda operatives should be prioritized for further intelligence gathering? These are some of the types of questions that intelligence analysts must consider. The field of Decision Science has existed for over 60 years, and is devoted to understanding how people perform some of the sorts of tasks that intelligence analysts are faced with. For example, decision scientists ask: How do people judge probability, perceive risk, and make choices? How do people search and select relevant information, and use (weight and integrate) it appropriately when making decisions? Decision scientists with backgrounds in disciplines such as Psychology and fields such as Behavioral Economics develop and test correspondence- or coherence-based theories. They employ quantitative methods. And, they consider normative, descriptive and prescriptive questions. I shall discuss how past research from the field of Decision Science can be used to shed some light on how intelligence analysts might perform as well as how their performance can be enhanced. In addition, I explore how theories and methods from this field can be usefully applied to understanding the art of intelligence analysis.

15

Rhiannon Gainor and France Bouthillier (McGill University): Measuring Intelligence Success: Conceptual and Methodological Challenges of Intelligence Measurement The purpose of intelligence practices is to source and develop intelligence to inform a decision. While there is a significant body of literature dealing on how to implement and design intelligence processes, little is known regarding how intelligence makes a difference in organizations. Measurement allows organizations to identify deviations in standards, benchmark success, and whether objectives are met. When an expert, equipped with intelligence, advises a decision-maker, it can be difficult to measure not only the role the intelligence played in the decision, but the success of the decision. While intelligence processes can be and are measured with some effort, identification and valuation of intelligence outcomes and impacts can be elusive. The fields of competitive/business/strategic intelligence, intellectual capital, and knowledge management struggle with the challenge of how to measure what is intangible, subjective, and frequently delayed in manifestation, as does the larger intelligence field. Many organizations measure process, inputs, and satisfaction, and sidestep the considerable methodological challenges related to accurate measurement of intelligence outcomes and impacts. The literature of intangibles measurement provides some insights applicable to intelligence measurement, such as identifying standard practice, problems with conceptualization and current measurement models, and best practices in developing measures. The purpose of this paper is first, to clarify terms and concepts related to measurement, asking: what is measurement, why measurement is needed, and what are the characteristics of ‘good’ measurement, all within the context of measuring intangible outcomes. The second purpose of this paper is to identify from the literature of intangibles measurement best practices and frameworks for developing measures of intelligence success that account not only for costs and processes, but also outcomes and impacts. The intent is to identify conceptual and methodological issues related to measurement and to offer for consideration insights from various fields that should inform measurement strategies in intelligence.

WJR Gardner (Naval Historical Branch, UK Ministry of Defence): The Unreliable Memoirs of an Applied Historian … and Operational Analyst and … The sound practices of history and intelligence have much in common and, it can be argued, each discipline can learn from the other. But there may, in fact, be an even larger range of disciplines which may not be quite so obvious or indeed make quite such a clear contribution to proper intelligence method and analysis. Arguably these might range from art history to zoology; from paleontology to psychology. In the relatively restricted compass of a conference paper or presentation it is not possible to produce a full description far less an evaluation of all these potential relationships. However, this paper will attempt to take some consideration of a number of other disciplines whilst concentrating on the symbiotic relationships that exist between intelligence and history, especially as it has related to the experience of one historian and intelligence officer. These would range from consideration of sources, analytical methods, understanding the significance of derived reporting and – perhaps as importantly - its limitations and, lastly, the essential skills needed in growing and maintaining an effective relationship between producer and consumer. It will be demonstrated that not only are there theoretical advantages to be gained on both sides but there is also a considerable track record of practitioners, moving relatively seamlessly and successfully from one discipline to the other, and also how many of the theoretical and analytical practices of one field can be applied to the other.

Melissa Graves (University of Mississippi): Intelligence Analysis and Historiography Historians function similar to intelligence analysts. They consume huge amounts of disparate and oftentimes questionably reliable material. Both must piece together incomplete stories. With both intelligence and history, a source can make or break one’s ability to compose good analysis. Historians and analysts both construct their particular version of world based upon primary records. Likewise, they face inherent perils of analytic biases, particularly mirror imaging. Both understand that events and people do not exist within a vacuum and they must accurately and fully take into account many competing forces. This presentation will discuss how the use of historical methodology can assist analysts in looking at the broader picture and understanding trends and motivations. In each story they construct, historians examine how race, class, gender, and religion affect key players’ decisions. By looking at historical methodology, analysts can glean tactics helpful to their own work.

16

Nick Hare (UK Ministry of Defence): Intelligence Analysis and Economics At first glance, the overlap between 'economics' and 'intelligence analysis' should be significant. Both disciplines are concerned with using evidence to test hypotheses about behaviour, and with modelling of the actions of individuals and systems through examination of incentives at both micro- and macroscopic levels. Why, then, is there relatively little practical interaction and overlap between the two? Based on an informal survey of the few individuals who, in the UK, have worked in the intersection, I will present some key similarities and differences between the professions of economics and intelligence analysis. I will propose some tentative hypotheses to explain these similarities and differences, and identify some key insights that each has to offer the other.

Michael Herman (Oxford University): Intelligence and Government Statistics British intelligence and statistics have evolved as government institutions over a similar periods as part of the ‘knowledgeable state’, and useful comparisons can be drawn between their institutions. Despite being pulled between centralization and decentralization, both have moved towards systems with central output and influence on standards; but the moves are uneven. Statistics now has a national statistician and supervisory authority overseeing what is still a mixture of central and departmental units: by contrast intelligence’s professional authority has remained its central committee system. Arguably statistics has moved in a natural direction for government’s knowledge producers, while British intelligence retains the committee system formed as part of the tri-service military structure with which it fought the Second War and moved into peace.

Carl Jensen (University of Mississippi): Intelligence Analysis and Futures Methodology The National Commission on Terrorist Attacks Upon the United States (also known as the 9-11 Commission) concluded that the U.S. intelligence community exhibited a “lack of imagination” in failing to prevent the terrorist attacks of 2001. Like intelligence analysts, futurists are concerned with forecasting future events; likewise, they must exhibit considerable imagination in divining emerging or distant trends. Over the years, futures research has produced strategies for improving creative thinking. As well, professional futurists have had to learn how to adapt their creative forecasts so that they have utility for concrete and pragmatic customers. This presentation will discuss how the methods of futurists can be integrated into the intelligence world, both in terms of analysis and establishing and promoting dialogue.

John Kringen (former CIA Director of Intelligence; Institute for Defence Analysis): Judging Intelligence Success and Failure There has been little public dialogue about the appropriate standards for evaluating the performance of intelligence organizations. In the United States, for example, intelligence performance has typically been judged on a case-by-case basis in the wake of perceived intelligence “failures” -- with limited discussion or debate on the standards being applied. What are realistic standards for evaluating the performance of intelligence organizations in an uncertain world? At a minimum, they need to go beyond the notions of surprise and embarrassment. What responsibilities do customers and those entities responsible for oversight of intelligence have for the performance of intelligence organizations? In this regard, clear guidance on such matters as intelligence priorities is critical. Finally, what insights on appropriate performance standards can be gained from examining practices outside the intelligence enterprise? For example, if major defense contractors in Europe and the United States are often challenged in terms of their ability to project their own delivery schedules, what standards should be applied to weapons analysts in the intelligence community? This PowerPoint presentation will propose a framework for evaluating the performance of intelligence organizations – drawing upon debates in recent decades about “intelligence failure” and relevant practice in other professions

17

William Lawhead (University of Mississippi): Seeing Is Not Believing: Insights for Intelligence Analysis from Professional Magicians. To relate magic (conjuring) and intelligence analysis seems like quite a stretch. However, throughout history magicians have served their country by using their professional skills in warfare and espionage. I briefly mention some examples of how magicians have used their knowledge to make contributions to these two activities. But these two activities are not the same as analysis. What do the skills and knowledge of professional magicians have to offer to intelligence analysts? This paper will suggest that they have a lot to teach us in the areas of deception, perception, and cognition. Deception The most obvious application of magic to intelligence analysis is in the area of deception and counterdeception. Here, the goals of the magician and the analyst are opposite. The magician tries to deceive us and the analyst tries to avoid being deceived or to uncover attempts at deception. But, understanding the magician’s techniques and principles will be useful in understanding deception in the intelligence arena. Barton Whaley is a political scientist who is a leading authority and author on the topic of military deception. He is also a magician and author of several books on magic. Some of his insights, based on knowledge of magic, will be discussed. Perception A greater knowledge of how we experience the world will be useful to the analyst. Here, magicians can instruct us. After all, magicians are able to make us think we see things that aren’t there and make us not see things that are there. Recently, neuroscientists have started doing research on why magicians fool us. For example, scientists have used eye-tracking technology on subjects while they watched a magician perform in order to see what causes the illusion. They found that the magician’s use of misdirection did not divert the gaze of the audience but diverted their attention. This is related to the phenomenon of “inattentional blindness.” We do not see objects in our visual field (including the magician’s sneaky moves) even though photons from the object are reaching our retina, because of a manipulation of our attention. Certainly, for the analyst, staring at lots and lots of data is not enough. We have to be concerned with how the data is being viewed and how our attention is directed or misdirected. Cognition Related to the previous topics is the issue of how the magician controls our minds. While some people think that misdirection is the main tool of the magician, other theorists think it is the way that the magician plants assumptions in our minds or exploits the assumptions we already have. Magician Roberto Giobbi says “Magic should be very easy, since our spectators fool themselves. All you need to do is avoid any words, thoughts or actions that interrupt this tendency.” Since assumption checking is an important technique for the analyst, an examination of how magicians use our assumptions will be instructive.

Stephen Marrin (Brunel University): Learning from Other Disciplines, including Medicine and Journalism Revisits and extends call for research agenda articulated at the end of the 2005 article “Improving Intelligence Analysis by Looking to the Medical Profession”: Crossing Professional Lines: Finally, the lessons that intelligence can draw from an examination of the similarities and differences with the medical profession indicate the importance of looking to analogous professions for ideas that can be adapted to an intelligence context. Doing so might help improve finished intelligence production processes and the incorporation of intelligence into decisionmaking. Analogies serve a number of purposes, such as aiding communication about difficult topics by finding illustrative examples in other fields, or by more directly affecting existing ways of doing business through the incorporation of tools that exist to achieve similar purposes in other fields. Many of the challenges intelligence analysts face are not as unique as its practitioners believe, but the insularity of the field prevents them from being able to identify the lessons from other professions that could be useful as models to follow. As a result, the first task is to identify analogous professions, and examine them for the lessons they might provide. Any profession that encounters similar problems-such as medicine, journalism, law, or law enforcement- may provide fertile ground for deriving ideas to improve existing practices. Perhaps if intelligence analysts adopted methods from analogous professions-or adapted them to the unique requirements of intelligence analysis-some of the obstacles they currently face in accurately portraying their understandings of the international environment could be overcome.” (Source: Stephen Marrin and Jonathan Clemente. Improving Intelligence Analysis by Looking to the Medical Profession. International Journal of Intelligence and CounterIntelligence, 18:(707-729). 2005 (pps 726-727).

18

Stephen Marrin (Brunel University): Evaluating the Quality of Intelligence Analysis: By What (Mis) Measure?: Each of the criteria most frequently used to evaluate the quality of intelligence analysis has limitations and problems. When accuracy and surprise are employed as absolute standards, their use reflects unrealistic expectations of perfection and omniscience. Scholars have adjusted by exploring the use of a relative standard consisting of the ratio of success to failure, most frequently illustrated using the batting average analogy from baseball. Unfortunately even this relative standard is flawed in that there is no way to determine either what the batting average is or should be. Finally, a standard based on the decisionmakers’ perspective is sometimes used to evaluate the analytic product’s relevance and utility. But this metric, too, has significant limitations. In the end, there is no consensus as to which is the best criteria to use in evaluating analytic quality, reflecting the lack of consensus as to what the actual purpose of intelligence analysis is or should be.

Peter Nardulli and Dan Roth (University of Illinois): Advancing our Understanding of Global Patterns of Civil Unrest: An Interdisciplinary Effort This paper deals with an area of increasing concern to intelligence analysis: the study of civil unrest, including destabilizing acts initiated by private citizens and disruptive state actions. It is based on one component (the Societal Stability Protocol, or SSP) of a six-year multidisciplinary effort, the Social, Political and Economic Event Database project (SPEED), which was initiated by and housed in the Cline Center for Democracy. The paper will outline how SPEED has used diverse technologies to enhance human capacities to conduct core intelligence functions: the detection, documentation, summarization and analysis of destabilizing developments. The paper will consist of three parts. The first will briefly outline: (1) the creation of SPEED’s global news archive, which contains over 150 million digitized news reports from a diverse set of news sources for every country in the world for the post-1945 era; (2) the creation of its destabilizing event ontology, which serves to focus the project’s detection efforts; and (3) its use of automatic text classification and event extraction technologies to capture pertinent information within the news archive. The second main section will present preliminary results for a tool developed by Roth’s Cognitive Computation Group, the Event Annotation Tool (EAT+). EAT+ uses advanced NLP technologies in conjunction with large amounts of training data to annotate key textual passages (trigger words, actors, dates, places, etc.). The technology underlying EAT+ will be outlined and success rates for detecting references to a representative set of key civil unrest events (generic political attacks, kidnappings, assassinations, assembly of coercive forces, imposing a curfew, declaring a state of emergency, restricting movement of citizens, etc.) will be reported. The final section of the paper will report on SPEED’s efforts to harness the analytic power of SSP data through the use of various composite measures of event intensity and event origins. This section will outline post WWII trends in civil unrest and state repression.

Sean Newman (Fire Department of New York): How Adherence to Journalistic Standards Positioned FDNY Watchline at the Forefront of Fire Service Analytical Intelligence Captain Newman’s presentation will show why the FDNY*s weekly intelligence newsletter, Watchline, continues to add readers and garner praise from across the globe by following some basic rules of journalism, such as concise headlines and lead sentences, brevity, story selection, and the collective editing process. Other topics to be discussed are feature stories, deadlines and the importance of feedback. Also, attendees will see how Watchline compares to some of Sherman Kent’ principles.

19

Stefania Paladini (Coventry University): Bayesian analysis in intelligence and social sciences: a tool for all trades?” Bayesian analysis is one of the most versatile methods in social sciences, with its application spanning from probability to finance to psychology. Incidentally, it is also one which relevance in intelligence is well known and documented, at least since Sherman Kent’s times. In a now quite famous text, Psychology of Intelligence Analysis, Heuer also provided some interesting examples of its potential and its worthiness for the job. Also, he argued with reason, Bayesian reasoning can significantly reduce the effects of cognitive biases, one of the main problems in intelligence. Still, Bayesian analysis’ application, in intelligence as much as in social sciences, can be somehow problematic or difficult, for a series of reasons – first of all, challenges linked to its technical complexity. It has been proved that even people mathematically trained are generally poor in estimating conditional probabilities, which are the objects of Bayes’ Theorem. Furthermore, some of its difficulties are linked to the quality of the prior information available, as well as to the collection of suitable posterior ones. However, by reviewing the available literature and by conducting some experiments in this sense it is the author's impression that the instrument fails more than often due to the fact that it is used to address the wrong questions - and not because of its internal weaknesses. The aim of this paper is to explore the issues related to Bayesian analysis, to define when its use is appropriate and in which term, both in intelligence and in social sciences and to provide suitable evidence.

Michael Robinson (Deloitte and Touche, LLP): Intelligence Analysis and Digital Forensics There are a large number of similarities between the fields of intelligence analysis and digital forensics. Both disciplines collect data from a variety of sources, analyze data to form actionable intelligence, and report information to stakeholders in an unbiased and timely manner. In the field of digital forensics, analysis results in the formation of a report that can withstand legal scrutiny and be incorporated in legal proceedings/trials. While there are similarities in the goals and objectives of the two disciplines, there are resemblances in the current obstacles as well. Many of the challenges encountered in digital forensics are also occurring in intelligence analysis, for example, there is a need to collect and process large amounts of data (often terabytes) in a short amounts of time, there are a limited number of seasoned practitioners in the field, there is a finite pool of resources, there is a need to provide qualitative indicators of reliability and confidence to analyzed data, there is a need to provide information in an easily digestible format for stakeholders that have limited time to read them, and there is a need to overcome misinformation that appears from anti-forensics techniques. This paper/presentation will examine some of the subspecialties within digital forensics, such as media analysis, mobile device forensics, and intrusion analysis, to identify current obstacles and current solutions to these mounting problems. Comparisons and contrasts between the fields of intelligence analysis and digital forensics will be made with an emphasis on introducing solutions to overcome the current hurdles.

Patricia Rooney (National Ground Intelligence Center, US Army) and Peter Ronayne (Federal Executive Institute): Neuroscience in Intelligence Analysis: Understanding How the Brain Works and Its Impact on Analytic Decision Making: Intelligence analysis is neither straightforward nor foolproof; it is both an art and a science. Whether pursuing strategic warning, current intelligence or estimative intelligence, all analysts depend upon neuro-psychological capabilities that are in constant tension and competition with one another. Most analysts know little about this neuro-tension or the neurological processes that leave analysts vulnerable to deception in their daily analyses. This session provides participants with (1) an understanding of the neurological tensions and cognitive plasticity that analysts can draw upon for effective decision making; (2) key cognitive fitness tools to keep the decision-making apparatus in tip-top shape; and (3) an understanding of intuition (art) and rational, methodical (science) reasoning for analytic conclusions.

20

Gabriel Sebe (Bucharest University): Political Marketing Intelligence The aim of the present paper is to study the interaction in conceptual and applied terms between the intelligence realm and the political marketing field; it is organized in four sections. The first section takes into account two dimensions. The first focus on the succession series interaction between intelligence and other fields conceptually related with the political marketing realm: journalism (Park, Lippmann, etc.), public relations (Ivy Lee, Bernays, etc.), political science (Shotwell, Lasswell, etc.), international relations (Shotwell, Mitrany, etc.), etc. The shift induced by Vannevar Bush’s paradigm and Shannon’s information theory allows an integration, imposing a conceptual platform related with the informational society concept; this is the second dimension. Now an explicit relationship between Levitt’s marketing myopia, Kotler’s marketing concept broadening, Kotler and Singh marketing warfare setting, Newman and Sheth political marketing mapping, Hunt’s resource/advantage theory and some main issues from the intelligence field follows. Putting Kent’s contribution in this framework and taking into account the Kendall/Kent debate, someone infers new perspectives on the knowledge transfer problem in terms of interdisciplinarity, multidisciplinarity and transdisciplinarity. The second section introduces a segmentation compatible with Downs’ political information and Machlup’s knowledge production, adequated in terms of the intelligence cycle idea Boyd’s OODA framework. In this manner it becomes possible to take into account two process-like dimensions. First the consideration of deliberation/decision processes related with BI, CI, SI and their corresponding counterparts from the political marketing field. Secondly, the differences and similarities concerning the meaning of ‚analysis’ as usable knowledge transfer and fusion knowledge dedicated to support a competitive decision process. Beyond terminological labels, the source classification problem, the relevance criteria, the measurement of meaning paradigms, the transparency/opacity policy, the secrecy patterns, etc. and their associated processes in terms of head building through education, instruction, trening form the basis of the analogies necessary for conceptual transfer between these realms. In fact many problems treated in intelligence terms can’t be conceived without a political marketing representation; the international relations trade/conflict problem (Crucé, 1623; Polachek, 1980; etc.) is one example. Both intelligence and political marketing fields are transdisciplinary; they cannot replace each other however together in a controllable and rational knowledge fusion setting allow better solutions. The third section maps the analogies between the intelligence field components and political marketing structured processes. Putting in evidence some micro/macro principles it becomes possible to relate the specific knowledge representations of the intelligence market conceived as the environment for both an intelligence economy and a competitive political marketplace. Taking into account some preceding considerations, a certain segmentation of research (fundamental, applied, practical, commercial) is put in correspondence with distinct manners of learning generating different intellectual profiles in terms of theoretical/practical capabilities. In this manner we enlarge the meaning for both usable knowledge and knowledge transfer between theory and practice not only in conceptual, but also in human resource terms, giving a meaning for the concept of intelligence capital. The fourth section concludes on the relevance of the ‚political marketing intelligence’ paradigm from Boyd’s destruction and creation perspective.

21

Musa Tuzuner (Intelligence Studies Research Center; Turkish National Police Academy): Multi-Level Cognitive Biases in the Chain of Intelligence Flow—From Source to Consumers: Improving intelligence analysis has been attracting more attention in the last decade because of the significant intelligence failure occurring in the world. Very few scholars have examined the cognitive bias factors’ negative affects on the intelligence analysis. They believe that understanding cognitive biases affecting intelligence biases would better serve to have efficient intelligence product for the consumers. However, in order to produce better intelligence product, we need to understand the Cognitive biases, not only in the intelligence analysis, but also in the chain of Intelligence flow. From our intelligence field experience, we have found a flaw in looking for cognitive bias in the intelligence analysis (Evaluators). By the time intelligence information reaches the analysis level, a cognitive bias is already embedded when going from Informers to Collectors and so on. This is why detecting the cognitive biases in the Evaluators stage is not adequate to produce efficient intelligence for the consumers. In order to find where the biases originate, we have created a “Manual for Detecting Intelligence Failure”. In order to detect multi-level cognitive biases in the chain of intelligence flow Aydinli and I are collecting data (through interviews) from intelligence practitioners in Pakistan, Turkey, Iraq, and Lebanon. After analyzing this dataset, we will classify the cognitive origins of the defects in the intelligence flow: as individual origins of defects and institutional origins of the defects. Then we will discuss how these collection, evaluation, and sharing defects can lead to intelligence failure.

Kathleen Vogel (Cornell University): Current Intelligence Reporting and the H5N1 Bird Flu Virus: Insights from the Field of Science and Technology Studies In late 2011, virologists Ron Fouchier and Yoshihiro Kawaoka encountered a swarm of government and public controversy from their creation of novel variants of the H5N1 bird flu virus. Prior to publication of these experimental findings, the U.S. National Science Advisory Board for Biosecurity (NSABB) reviewed Fouchier’s and Kawaoka’s scientific manuscripts. The NSABB unanimously recommended that the, “conclusions of the manuscripts be published but without experimental details and mutation data that would enable replication of the experiments.” The NSABB explained its justification as being based on security concerns: “publishing these experiments in detail would provide information to some person, organization, or government that would help them to develop similar viruses for harmful purposes.” However, once the NSABB made its decision public, other flu virologists emphasized the difficulty of the experiments and how these experimental results could lead to beneficial new medical treatments. Other scientists, however, have sided with the NSABB and warned of the security dangers in publishing these bird flu results. In the middle of this context, U.S. intelligence analysts have been tasked by their superiors to provide up-to-date security assessments of these bird flu experiments as new information about the experiments continues to emerge. These intelligence analysts have struggled to make sense of the positions held by the NSABB and different scientific experts. These analysts have also encountered challenges with how to gather and evaluate information in a timely fashion to better assess the role of explicit and tacit (know-how) knowledge in these experiments. These intelligence assessments are needed to better inform policymakers about whether to censor future logical research findings and how to think about emerging biological science threats. This paper will discuss what happens when academic scholars from the social science field of science and technology studies (S&TS) are brought into unclassified dialogues with U.S. intelligence analysts as these analysts are writing their current intelligence reports on these controversial bird flu experiments. This paper will illustrate how S&TS research can provide new insights for assessing the security threat from these experiments and related biosecurity/WMD concerns, how S&TS research can more broadly inform the analytic practices that underpin current intelligence reporting, as well as challenges that exist in bringing S&TS ideas to bear on intelligence analysis.

22