By: Yvonne Marie Antonoglou, Senior Fellow
Introduction
Looking back on the past 15 years, there is little doubt that the 9/11 attacks profoundly altered the landscape of international security. Over the last decade and a half, there has been a tremendous multiplication of counter terrorism measures, laws, policies at regional, national and international levels. The events of 9/11 and -more recently- the phenomena of ISIS, “homegrown terrorism” and “foreign fighters”, set states and international bodies on an accelerated path of law making and institution-building, (Kaunert & Leonard, 2012). Domestic and international counter terrorism initiatives include specific provisions to permit, require and fund technologically innovative measures to counter terrorism; thus smart borders, (smart) surveillance, data retention/collection and sharing, passenger name record exchange etc., feature heavily in the hyper technologized operational continuum of contemporary counter terrorism, “where the production, retention, processing, and deployment of data has taken centre stage”, (De Londras, p.1., 2017). Although an array of difficult questions stems from the current risk prevention rationale regarding counter terrorism practices and the balancing act between security and democratic oversight, this study will take a step back from the specifics of technological advancements (in relation to counter terrorism measures) to focus on a more structural, ethical and deontological aspect: effectiveness. It is no secret that research conducted to evaluate the actual effectiveness of counter terrorism measures is worryingly insufficient or outright non-existent. Taking into account that the adoption of certain counter terrorism practices can potentially have implications with regard to human rights, institutional liberties, core democratic principles and the Rule of Law, it is crucial that engaged and systematic efforts in researching the effectiveness of counter terrorism measures are implemented in a rigorous way. The lack of empirical research is -by no means- a problem exclusive to counter terrorism but empirical evaluation is vital in clarifying and debunking prevailing beliefs regarding counter terrorism practices, (Jackson, 2016). It must be noted, however, that one can discern a number of challenges and technical difficulties when attempting to measure the effectiveness of counter terrorism. Van Um and Pisoiu argue that a major point of contention is the lack of a universally accepted notion of what an effective counter terrorism policy is supposed to bring about, (Van Um & Pisoiu, 2011). In other words, there is a need for a solid conceptual framework that could provide a critical overview of effectiveness -understood in terms of the extent to which objectives for the measures implemented have been achieved-.
Framing Context
It is perhaps trite to remark that terrorism –and counter terrorism for that matter- did not begin on 9/11; rather, terrorism has long been employed as a modus operandi by organizations and individuals, attempting to achieve sociopolitical and/or other goals. For certain countries, violence associated with terrorism has long been a fact of life and many of these countries have had relevant anti-terrorism legislation in place for years, (Sunga, 1997). Thus, in effect, laws, policies and regulations (occasionally with a plethora of implications for the social equilibrium) pursuant to a risk prevention/securitization rationale are nothing new at the domestic level, (de Londras, 2017).
It is also not the case that the concept of “transnational terrorism” is fundamentally or qualitatively new. Standard terrorist operating procedures -i.e. shopping center bombings, assassinations, suicide bombings, hijackings, civilian aircraft bombings, spree shootings etc. - were repeatedly observed far before 9/11, (Jarvis, 2008). Nevertheless, 9/11 did represent a significant, path-breaking moment in a broader historical trajectory, since it was collectively perceived as an unfathomable trauma, ushering in a qualitatively new time of fear and insecurity, (Jarvis, 2008). Yet, for all of that, it is safe to say that the September 11 attacks certainly marked the end of an era and the beginning of another, mainly because they sketched the contours of contemporary counter terrorism and shaped the context in which the notions of justification, effectiveness and impact (of counter terrorism measures) are analyzed.
Conceptualizing Effectiveness
The conceptualization of effectiveness (of counter terrorism measures) has been complicated by the variability and multidimensionality of the empirical phenomenon it seeks to describe. In other words, the understanding of effectiveness is largely dependent on the perspective, priorities and broader purposes of the person or entity making the assessment, (SECILE, 2016). In the abstract, effectiveness could be positivistically equated with “the achievement of general objectives”. In the context of counter terrorism, however, this simple methodological continuum is complicated by three exogenous variables: i) the fact that particular counter terrorism measures are part of a broader operational framework and may have both meta objectives (relating to overall security matters) and specific objectives (relating to the measure in particular), ii) the fact that both the meta- and specific objectives may not be lucidly elaborated iii) the possibility of certain measures devised for one purpose being applied to other issues or having unexpected effects in relation to other areas, (SECILE, 2006).
As already suggested, a universally accepted definition or framework of counter terrorism effectiveness does not exist in the literature to date, (Van Dongen, 2009). The literature is currently limited to a hypothesized conceptual self- evidence or specific indicators regarding stand-alone terms such as impact, success, effects, etc., (Van Um & Pisoiu, 2011). Subject-matter expert Martin Mazé defines effectiveness based on whether the measures “actually make a difference” and particularly focuses on the societal impact as an effectiveness indicator. He expressly notes that ex post facto evaluations rely heavily on outputs rather than impact per se; subsequently the effectiveness analysis “risks excluding key societal indicators”, (Martin-Mazé, p.5, 2013). It is worth noting at this point that the societal perception regarding the effectiveness of counter terrorism measures is also largely determined by three different arguments, central to the rationality of the Western political discourse: the exception argument, the balancing metaphor (between security and liberty) and the trade- off model, (de Londras, 2015). These conceptual continua are the ideological backbone of the societal perception of CT effectiveness. This (societal) approach of CT effectiveness is often analyzed through an output-outcome-impact trichotomy, an epistemological framework originally developed by Young (2001), (Van Um & Pisoiu, 2011, Martin-Mazé, 2013). Generally speaking, output effectiveness would refer to the adoption of a measure, policy instrument, or regulation in response to a threat; in this instance, effectiveness is assessed in relation to the behavior of the policy makers charged with the implementation, (Van Um & Pisoiu, 2011). Outcome effectiveness, contrarily, would examine the direct and measurable effects that a given policy framework has in real life; as in, it would also take into account the behavior of the targeted group (in addition to the policy-makers) in relation to the short-term objectives of the specific CT policy implemented, such as disrupting the financial flows or the recruitment process of a terrorist organization, (Van Um & Pisoiu, 2011). Finally, impact-effectiveness depends on the behavior/reaction of the targeted audience alone, with regard to the long-term objective of the CT policy, namely its success or failure in reducing or stopping terrorism, (Van Um & Pisoiu, 2011).
It is interesting to note that the effectiveness denominator is not something that most legal analyses tend to focus on, in general; they rather focus on the necessity, legality and proportionality of CT measures, (Vermeulen et. al., 2013). Vermeulen, Deering and McCarthy argue that effectiveness is predominantly evaluated in a factual sense and is assessed by reference to the intended results a CT measure is supposed to produce, (e.g. “expedition of the prosecution of suspected terrorists or the freezing of suspected terrorists’ assets”), (Vermeulen et. al., p.16, 2013). The legal perspective may not directly make out a prima facie case of effectiveness but it does consider other variables such as whether a CT measure is compliant with democratic values and principles and/or whether it is “strictly required by the exigencies of the situation"; thus, implicitly taking effectiveness into account as well, (European Commission Report on CT measures, p.6, 2010).
Measuring Operational Effectiveness (from a Law-Enforcement Perspective)
A wide spectrum of different methods has been put to use in the study of CT operational effectiveness, with qualitative methods and impact effectiveness being the most prevalent epistemological approaches, (Van Um & Pisoiu, 2011). This “clinical” analysis is somewhat understandable, since the ultimate objective of any CT measure or policy is to actually have an effect on terrorism and numbers tend to offer clearer policy-relevant indicators, (LaFree, 2006; Horgan and Braddock, 2010). However, measuring operational effectiveness practically can be a frustratingly tedious task, given the large number of quantitative studies in the field and the difficulty to clearly establish the reliability and validity of the relevant indicators, (Van Um & Pisoiu, 2011). One basic factor that influences the overall validity and reliability of indicators and complicates the structured and systematic consideration of CT measures, is the fact that most studies focus only on individual measures -i.e. the increase in law enforcement powers or legislation, criminalising acts as terrorism, racial profiling or data mining (Omotola, 2008; Harcourt, 2006; Jonas and Harper, 2006) -, or on a preferential selection thereof i.e. disrupting terrorist financing, roving patrols, security/defensive barriers, infiltration of terrorist organizations (Morag, 2005); arrests, sweeps/searches, confiscation of goods/funds, entering enemy territory, detaining or questioning, foiling attacks, sting operations, hostile actions (strikes/killings), (Pratto et. al., 2009). Along similar lines, Hewitt proposes a classification scheme albeit with largely unspecified criteria; from a law-enforcement stand point, he looks at anti-terrorist legislation, emergency powers, ceasefire and negotiations and the use of security forces, (Hewitt, 1984). Occasionally, there is no specific mention of concrete counter terrorism measures; they are rather evaluated as a whole in cost-benefit assessments, (Stewart and Mueller, 2009; Zycher, 2009).
From a law enforcement perspective, operational effectiveness is -quite often- approached in the literature through the outcome-effectiveness basis. The study of outcome-effectiveness, in this sense, can be quite problematic due to the lack of systematization, convenient selection of indicators and a certain haziness as to “whether certain elements should be considered indicators or side effects”, (Van Um & Pisoiu, p.9, 2011). Most authors tend to systematize indicators through a dichotomous categorization; the root causes of terrorism and the operational ability of the targeted organization (resources, public support, recruitment, etc.), (Della Porta, 1992). Most studies fall into the second category and take one or more operational aspects into consideration. Della Porta, for instance, argues that “the number of terrorist…arrests (can be taken) as an indicator of state success” and therefore its effectiveness, (Della Porta, p.160, 1992). Byman focuses on the “level of domestic support for counter terrorism operations, operational freedom and the disruption of the adversary’s command and control structure”, (Byman, p.413, 2003), while Perl suggests that operational effectiveness should be measured by reference to the degradation of terrorist capabilities and infrastructure, (Perl, 2007). Relatedly, Malvesti provides a comprehensive set of indicators and notes that one needs to take into account the “critical nodes” in a terrorist infrastructure, such as: the political base, financial networks, communication channels, sanctuary, intelligence network, weapons, cells and leadership, (Malvesti, p.21-25, 2002). Spencer proposes a more holistic approach, stating that the operational effectiveness of counter terrorism measures may also be purposively understood; if the implemented law enforcement measures manage to reduce the popular fear of terrorism, they can be considered effective, (Spencer, 2006).
In their report regarding the operational perspective, de Londras, Downing and Doody stress that operational effectiveness is directly proportional to “the satisfaction at operational level(s) of the original objectives that were to be achieved by the introduction of a CT policy or measure”, (de Londras et al, p.17, 2013). Nevertheless, they stress that a clear rationale for the policy and/or measure in question needs to be accurately defined along with an adequate monitoring mechanism so that effectiveness –for all intents and purposes- can actually be assessed, (de Londras et al, 2013).
It is worth noting that certain studies place a very high premium on impact effectiveness as well, with respect to the operational effectiveness of relevant (law enforcement) CT measures. Impact effectiveness is usually evaluated in relation to indicators associated with terrorist activity, i.e. a. number of attacks, recidivism rates for deradicalisation programmes of individuals (Noricks, 2009) and the degree to which group radicalization programmes are capable of preventing further terrorist activity (Ashour, 2008). In this context, Horgan and Braddock propose a more comprehensive assessment scheme, the Multi Attribute Utility Technology, consisting of two sets of indicators: a. the rate of terrorism, subordinated domestic terror rates and recidivism and b. the popular support for the government, “boosted morale and political capital”, (Horgan and Braddock, p.283, 2010). Noricks, however, notes a major difficulty within this category of effectiveness: the limited amount and reliability of data, such as the accurate reporting and tracking of re-arrests in the case of recidivism (Noricks, 2009). Numerous scholars tend to use instead of impact, outcome indicators i.e. the number of killed/arrested terrorists and the destruction of the terrorist infrastructure (Floros & Newsome, 2009, Schmid & Sing, 2008). It should be mentioned, nevertheless, that this particular indicator could lead to faulty assumptions, since operatives are easily replaceable, thus the arithmetic analogy does not necessarily reveal the potential or the size of a terrorist organization; therefore, any assessments regarding the impact, dynamics or future attacks would -at best- be probabilistic, (Probst, 2005; Spencer, 2006; Stohl, 2006).
Finally, a comparatively smaller body of literature uses output effectiveness for the evaluation of counter terrorism measures. This method is relatively unproblematic because its indicators are easily delineated. These include the production of strategies, organization reform and legislation (9/11 Commission report, 2006), the establishment of authorities or the implementation of legislation, (Alexander, 2006; Keohane, 2005).
De Londras observes that the introduction (and subsequently the effectiveness evaluation) of counter terrorism policies -particularly in the European Union- is generally structured around ex ante impact assessments, which basically “can only ascertain predicted impacts and thus estimate the proportionality, effectiveness and appropriateness of the proposed technology or measure”, (de Londras, p,12, 2017). So, in essence, ex ante impact assessments are purely speculative and they cannot determine effectiveness as a matter of fact. She proposes instead an “ex post facto” review, (currently rare in the context of counter terrorism), of how the measure or technology is actually working, how it is widely perceived, what its real impact is, how much it is costing and so on, so that effectiveness can be assessed “in the broad and vital sense”, (de Londras, p,12, 2017).
In sum, academics have used a wide variety of methods to assess the overall effectiveness of counter terrorism responses and have reached very different conclusions regarding how effective counter terrorism policies actually are.
Conclusion
While effectiveness could be regarded as one of the most essential aspects of any counter terrorism policy, its study is cumbered by both epistemological underdevelopment and a lack of a canonical scientific basis. Conceptually, there is a wide variety of indicators/measures/techniques taken into consideration, and in numerous categorizations, from comprehensive strategies to a set of measures and/or individual ones. The incremental methodological complexity is directly analogous to a series of validity and reliability issues, which explain the –partly- divergent results obtained. Another major hindrance is the data availability and quality as well as the lack of empirical data, the limited nature and/or manipulability of statistics and the “impossibility of capturing some disruptive successes in statistical form”, (SECILE, p.25, 2013). The limitations of a statistical and/or qualitative evaluation should also be born in mind, as statistics are not necessarily objective and/or exhaustive indicators of effectiveness, (SECILE, p.27, 2013). In designing mechanisms for evaluating effectiveness, qualitative empirical research should be conducted, in order to identify potential conditions that could be operationalized in assessment variables. Furthermore, policy makers must bear in mind that much of counter terrorism is -in effect- mutative in nature (shifting threats, priorities, geopolitical realities) and effectiveness itself may change over time, depending on the particular circumstances; hence, the effectiveness of CT measures requires constant re-evaluation, occasionally beyond a statistical and/or quantitative basis.
References:
- Alexander, Y., (ed.) (2006), “Counterterrorism Strategies. Successes and Failures of Six Nations”, Potomac Books, Dulles, Virginia
- Ashour, O. (2008), “Islamist De-Radicalization in Algeria: Successes and Failures”, The Middle Eastern Institute Policy Brief, 21, p. 1–10
- Byman, D. (2003), “Measuring the War on Terrorism: A First Appraisal”, Current History, 102(668), p. 411–416
- Della Porta, D. (1992), “Institutional Responses to Terrorism”, The Italian Case, Terrorism and Political Violence, 4(4), p. 151–170
- De Londras, F., Doody, J., & Downing, E., (2013), “Report on Operational Perspectives on Impact, Legitimacy and Effectiveness in Counter-Terrorism” (SECILE Consortium, 2013) Available at: http://secile.eu/report-on-operational-perspectives Accessed: 11/03/2017
- De Londras, F. (2017), “Evaluation & Effectiveness of Counter Terrorism”, Forthcoming in Wim Hardyns, Koen Ponnet, Genserik Reniers, Wim Smit, Luc Braeckmans & Barbara Segaert (eds), p.1, Socially Responsible Innovation in Security: Critical Reflections. Available at: http://ssrn.com/abstract=2801994 Accessed: 11/03/2017
- European Commission For Democracy Through Law, (2010), Report on Counter Terrorism Measures and Human Rights, Venice Commission, Available at: http://webcache.googleusercontent.com/search?q=cache:aAgFyNPOW2UJ:www.un.org/en/sc/ctc/specialmeetings/2011/docs/coe/coe-cdl-ad-2010-022-e.pdf+&cd=3&hl=el&ct=clnk&gl=gr Accessed: 13/03/2017
- Harcourt, B, (2006), “Muslim Profiles post 9/11: Is Racial Profiling An Effective Counter Terrorism Measure and Does It Violate the Right to be Free from Discrimination”, Chicago: The University of Chicago Law School
- Hewitt, C. (1984), “The Effectiveness of Anti-Terrorist Policies”, Lanham, University Press of America
- Horgan, J. & Braddock, K. (2010), “Rehabilitating the Terrorists? Challenges in Assessing the Effectiveness of De-radicalization Programs”, Terrorism and Political Violence, 22(2), p. 267–291
- Jackson, R. (2016), “Critical Terrorism Studies After 9/11”, Routledge Handbook of Critical Terrorism Studies Routledge Available at:
- Jarvis, L. (2008), “Times of Terror: Writing Temporality into the War on Terror”, Critical Studies on Terrorism 1 (2): pp.245-262
- Jonas, J & Harper, J, (2006), “Effective Counterterrorism and the Limited Role of Predictive Data Mining”, CATO Institute, Available at: https://www.cato.org/publications/policy-analysis/effective-counterterrorism-limited-role-predictive-data-mining Accessed: 13/03/2017
- Kaunert, C. & Leonard, S. (2012), “Supranational governance and European Union Security after the Lisbon Treaty- Exogenous Shocks, policy entrepreneurs and 11 September 2001” Cooperation and Conflict Vol. 47, No4
- Keohane, D. (2005), “The EU and counter-terrorism”, Centre for European Reform (CER), London
- LaFree, G. (2006), “Efficacy of Counter Terrorism Approaches: Examining Northern Ireland”, START Research Brief
- Malvesti, M. L, (2002), “Bombing bin Laden: Assessing the Effectiveness of Air Strikes as a Counter-Terrorism Strategy”, The Fletcher Forum of World Affairs, 26(1), p.17–29
- Martin- Mazé, M. (2013), “Report on Societal Understandings of Impact, Legitimacy and Effectiveness in the Counter-Terrorism Context” (SECILE Consortium, 2013), pg.5 Available at: http://secile.eu/societal-perspectives Accessed: 11/03/2017
- Morag, N. et al. (2005), “Measuring Success in Coping with Terrorism: The Israeli Case”, Studies in Conflict & Terrorism, 28(4), p. 307–320
- Mueller, J. & Stewart M.G., (2011), “Terror, Security and Money: Balancing the Risks, Benefits, and Costs of Homeland Security”, Paper prepared for presentation at the panel, "Terror and the Economy: Which Institutions Help Mitigate the Damage?” at the Annual Convention of the Midwest Political Science Association, Chicago, (April, 2011) Available at: https://global.oup.com/academic/product/terror-security-and-money-9780199795765?cc=us&lang=en& Accessed: 12/03/2017
- Newsome, B., & Floros, C. (2009), “When international counter-terrorism succeeds: Lessons from the defeat of 'revolutionary organization November 17th” Eds: CC Beyer & M Bauer, Effectively Countering Terrorism p.141-154, Sussex Academic Press
- Noricks, E. (2009), “Disengagement and Deradicalization: Processes and Programs”, In Social Science for Counterterrorism: Putting the Pieces Together, edited by P.K Davis and K. Cragin. Santa Monica, RAND, p. 11–70
- Omotola, S. (2008), “Assessing Counter-Terrorism Measures in Africa: Implications for Human Rights and National Security”, Conflict Trends, no. 2, p.41–48
- Perl, R. (2007), “Combating Terrorism: The Challenge of Measuring Effectiveness”, CRS Report for Congress, updated March 2007
- Pratto, F., Fox, B., Pitpitan, V., Tan, J., Huntington, A., Santos, Jr., E. E., Ravishanker, N. and Pearson, A. R. (2009), “The efficacy of terrorism and counter-terrorism in the second intifada: An intergroup approach”, Working Paper presented at the International Society for Political Psychology meeting, Dublin, (July 2009)
- Probst, P. (2005), “Measuring Success in Countering Terrorism: Problems and Pitfalls”, LNCS 3495, p. 315–325
- SECILE Consortium, (2013), “The Impact, Legitimacy & Effectiveness of EU Counter Terrorism”, p.12-30, Available at:
https://fdelondras.files.wordpress.com/2016/01/secile-d5-3.pdf Accessed: 10/03/2017
- Schmid, P. & Sing, R. (2009), “Measuring Success and Failure in Terrorism and Counter-Terrorism”, US Government Metrics of the Global War on Terror. In After the War on Terror: Regional and Multilateral Perspectives on Counter-Terrorism Strategy. Edited by A. Schmid and G. Hindle, London: RUSI
- Spencer, A. (2006), “The Problems of Evaluating Counter-Terrorism”, UNISCI Discussion Papers, No. 12.
- Stohl, M. (2006), “Winners and Losers in the War on Terror: The Problem of Metrics”, Paper Presented at the International Studies Association Annual Convention, San Diego, March 2006
- Sunga, L. (1997), “The Emerging System of International Criminal Law: Developments in Codification and Implementation”, Eds: Brill Publishers
- Van Dongen, T. (2009), “Break it Down: An Alternative Approach to Measuring Effectiveness in Counter Terrorism”. Economics of Security Working Paper, no 23
- Van Um, E & Pisoiu, D. (2011), “Effective Counter Terrorism: What Have We Learnt So Far?”, Economics of Security Working Paper 55, Berlin: Economics of Security. Available at:
https://webcache.googleusercontent.com/search?q=cache:d_AmpT_8n8gJ:https://www.diw.de/documents/publikationen/73/diw_01.c.386651.de/diw_econsec0055.pdf+&cd=2&hl=el&ct=clnk&gl=gr Accessed: 10/03/2017
- Vermeulen, M., Deering, D. & McCarthy, S. (with research assistance from Carolin Möller), (2013), “Report on Legal Understandings of Impact, Legitimacy and Effectiveness in Counter-Terrorism” p.14-19, (SECILE Consortium, 2013)
Available at: http://secile.eu/report- legal-understandings Accessed: 11/03/2017
- Zycher, B. (2003), “A Preliminary Benefit/Cost Framework for Counterterrorism Public Expenditures”, RAND, Santa Monica
