Awards Nomination 20+ Million Readerbase
Indexed In
  • Open J Gate
  • Genamics JournalSeek
  • SafetyLit
  • RefSeek
  • Hamdard University
  • EBSCO A-Z
  • OCLC- WorldCat
  • Publons
  • Google Scholar
Share This Page
Recommended Webinars & Conferences

41th International Conference on Dentistry & Dental Marketing

San Francisco, USA
Journal Flyer
Flyer image

Research Article - (2016) Volume 5, Issue 2

Applications and Modelling Using Multi-Attribute Decision Making to Rank Terrorist Threats

William P Fox*
Department of Defense Analysis, Graduate School of Operational and Information Sciences, USA
*Corresponding Author: William P Fox, Professor, Department of Defense Analysis, Graduate School of Operational and Information Sciences Monterey, CA 93943, USA, Tel: 831-656-3753 Email:

Abstract

In this paper we will examine a threat risk assessment process and modelling methodology that could be used by local law enforcement, homeland security, or military units to examine possible terrorist threats. We provide examples from a risk assessment process and a dark network. We apply different multi-attribute schemes to the threats. We also apply sensitivity analysis to the methods.

Introduction

Multiple-attribute decision making (MADM) refers to making decisions when there are multiple but a finite list of alternatives and multiple criteria.

Consider any of the following situations facing an organization:

• Homeland security needs a prioritization of potential threats cause many threats are made daily and they have limited assets;

• Military or government leaders need a prioritization of terrorist as targets for counterterrorism operations;

• Counterterrorist managers require a prioritization of terrorist phases targeting in order to stop acts prior to their happening;

• Organizations require finding the key nodes in a social or dark network;

• A financial institution wants a new model to build and maintain a retirement portfolio;

• An analytics company needs a procedure or methodology to compare recruiting offices, financial institutions, academic institutions, etc.

Consider a problem where homeland security in a region has a list of potential threats and limited assets so they need to know which threats to check out. Perhaps management needs to prioritize or rank order alternative choices: identify key nodes in a business Network, pick a contractor or sub-contractor, choose airports, rank recruiting efforts, ranks banking facilities, rank schools or colleges, etc. How does one proceed to accomplish this analytically?

In this chapter we will briefly present four methodologies to rank order or prioritize alternatives based upon multiple criteria. These four methodologies are:

• Data envelopment analysis (DEA)

• Simple average weighting (SAW)

• Analytical hierarchy process (AHP)

• Technique of order preference by similarity to ideal solution (TOPSIS)

For each method, we describe the method and its uses, discuss some strengths and limitations to the method, discuss tips for conducting sensitivity analysis, and present illustrative examples.

These MADM methods have been used extensively in current research and many of the research efforts are listed within each MADM technique discussion.

In this paper we will examine a threat risk assessment process that could be used by local law enforcement or homeland security to examine possible threats as in Table 1 and identifying how to find the key nodes in a terrorist or criminal network. Assume we have a social or dark network where we desire to know or find the key or influential nodes with the network. The Noordin dark network graph is provide as shown in Figure 1 and ORA output for four main metrics is provided in Table 2.

  Criteria
Reliability Casualties (millions) Psychological effect SitePop (millions) Repair cost (millions) # Tips
  Dirty Bomb 0.40 10 7 5 150 3
Alternatives Anthrax 0.45 0.80 6 2 10 12
Dc road network 0.35 0.005 4 2.5 300 8
NYC Subway 0.73 12 5 4 200 5
DC Metro 0.69 11 5 3 200 5
Bank Robbery 0.81 0.0002 2 0.05 10 16
FAA Threat 0.70 0.001 3 0.02 5 15

Table 1: Homeland security threat risk assessment priority.

Agent Total Degree centrality Betweenness Centrality Closeness Centrality Eigenvector Centrality
Agent TDC BC CC EC
a5 0.359 0.09 0.102 0.434
n2 0.333 0.182 0.103 0.35
m4 0.269 0 0.1 0.392
a6 0.256 0.033 0.1 0.325
t 0.256 0 0.099 0.376
f 0.231 0.025 0.1 0.313
j 0.231 0.034 0.099 0.32
u 0.231 0.038 0.101 0.305
s8 0.205 0 0.099 0.299
a23 0.192 0.032 0.1 0.257
b 0.192 0.028 0.099 0.279
a13 0.179 0.14 0.101 0
a22 0.179 0 0.098 0.289
d2 0.179 0.04 0.099 0.226
I7 0.179 0.163 0.099 0
m3 0.179 0 0 0.281
s5 0.179   0.098 0.264
a17 0.167 0 0.098 0.224
s6 0.167 0.039 0 0
a7 0.154 0 0 0.209

Table 2: ORA output for 4 key metrics of the top listed nodes or agents.

socialomics-Noordin-Dark

Figure 1: Noordin Dark network (79 nodes) from ORA.

Data Envelopment Analysis (DEA)

Description and uses

Data envelopment analysis (DEA) is a relatively new “data inputoutput driven” approach for evaluating the performance of entities called decision making units (DMUs) that convert multiple inputs into multiple outputs [1]. The definition of a DMU is generic and very flexible. It has been used to evaluate both the performance or efficiencies of hospitals, schools, departments, US Air Force wings, US armed forces recruiting agencies, universities, cities, courts, businesses, banking facilities, countries, regions, and the list go on. According to Cooper [1], DEA has been used to gain insights into activities that were not obtained by other quantitative or qualitative methods.

Charnes et al. [2], described DEA as a mathematical programming model applied to observational data, providing a new way of obtaining empirical estimates of relations. It is formally defined as a methodology directed to frontiers rather than central tendencies.

Methodology

The model, in simplest terms, may be formulated and solved as a linear programming problem [3,4]. Although several formulations for DEA exist, we seek the most straight forward formulation in order to maximize an efficiency of a DMU as constrained by inputs and outputs as shown in equation 1. As an option, we might normalize the metric inputs and outputs for the alternatives if poorly scaled. Otherwise, we will call this matrix, X, with entries xij. We define an efficiency unit as Ei for i=1,2,…, nodes. We let wi be the weights or coefficients for the linear combinations. Further, we restrict any efficiency from being larger than one. This gives the following linear programming formulation for single outputs but multiple inputs:

      (1)

For multiple inputs and outputs, we recommend the formulations provided by Winston [3] and Trick [5] using equation (2).

For any DMU0, let Xi be the inputs and Yi be the outputs. Let X0 and Y0 be the DMU being modeled.

image      (2)

Strengths and Limitations to DEA

DEA can be a very useful tool when used wisely. Trick [6] provides a nice list of a few of the strengths that make DEA extremely useful:

• DEA can handle multiple input and multiple output models.

• DEA doesn’t require an assumption of a functional form relating inputs to outputs.

• DMUs are directly compared against a peer or combination of peers.

Inputs and outputs can have very different units. For example, X1 could be in units of lives saved and X2 could be in units of dollars without requiring any a priori tradeoff between the two.

The same characteristics that make DEA a powerful tool can also create limitations. An analyst should keep these limitations in mind when choosing whether or not to use DEA.

Since DEA is an extreme point technique, noise in the data such as measurement error can cause significant problems.

DEA is good at estimating “relative” efficiency of a DMU but it converges very slowly to “absolute” efficiency. In other words, it can tell you how well you are doing compared to your peers but not compared to a “theoretical maximum.”

Since DEA is a nonparametric technique, statistical hypothesis tests are difficult and are the focus of ongoing research.

Since a standard formulation of DEA with multiple inputs and outputs creates a separate linear program for each DMU, large problems can be computationally intensive.

Linear programming does not ensure all weights are considered. We find that the value for weights is only for those that optimally determine an efficiency rating. If having all criteria weighted (inputs, outputs) is essential to the decision maker then we should not use DEA.

Sensitivity analysis

We like the sensitivity approach taken by Neralic [7], where he explains an increase in any output cannot worsen an already achieved efficiency rating nor can a decrease in inputs alone worsen an already achieved efficiency rating. As a result in our illustrative examples we only decrease outputs and increase inputs, as applicable.

Application to Noordin dark network

We apply the formulation to the data from ORA on the Noordin network from Table 2. We present two results. We used the formulation as presented in equation 2 and then we added a constraint that the sum of the weights must equal to one. Using the basic formulation in equation 2, we obtained the following solution using the LP software, LINDO.

The formulation from LINDO is:

MAX E1

SUBJECT TO

2)-E1+0.359 W1+0.09 W2+0.102 W3+0.434 W4=0

3) 0.333 W1+1.82 W2+1.03 W3+35 W4-E2=0

4) 0.269 W1+0.492 W3-E3=0

5) 0.256 W1+0.033 W2+0.1 W3+0.325 W4-E4=0

6) 0.256 W1+0.099 W3+0.376 W4-E5=0

7) 0.231 W1+0.025 W2+0.1 W3+0.313 W4-E6=0

8) 0.231 W1+0.034 W2+0.099 W3+0.32 W4-E7=0

9) 0.231 W1+0.038 W2+0.101 W3+0.305 W4-E8=0

10) 0.205 W1+0.099 W3+0.299 W4-E9=0

11) 0.192 W1+0.032 W2+0.1 W3+0.257 W4-E10=0

12) E1< =1

13) E2< =1

14) E3< =1

15) E4< =1

16) E5< =1

17) E6< =1

18) E7< =1

19) E8< =1

20) E9< =1

21) E10< =1

22) W1> =0.001

23) W2> =0.001

24) W3> =0.001

25) W4> =0.001

26) W1+W2+W3+W4=1

END

The solution is as follows:

LP OPTIMUM FOUND AT STEP 4.

OBJECTIVE FUNCTION VALUE

1) 0.3599123

VARIABLE VALUE REDUCED COST
E1 0.359912 0.000000
W1 0.978823 0.000000
W2 0.001000 0.000000
W3 0.001000 0.000000
W4 0.019177 0.000000
E2 1.000000 0.000000
E3 0.263795 0.000000
E4 0.256944 0.000000
E5 0.257888 0.000000
E6 0.232236 0.000000
E7 0.232378 0.000000
E8 0.232096 0.000000
E9 0.206492 0.000000
E10 0.192995 0.000000
ROW SLACK OR SURPLUS DUAL PRICES
2) 0.000000 -1.000000
3) 0.000000 0.002163
4) 0.000000 0.000000
5) 0.000000 0.000000
6) 0.000000 0.000000
7) 0.000000 0.000000
8) 0.000000 0.000000
9) 0.000000 0.000000
10) 0.000000 0.000000
11) 0.000000 0.000000
12) 0.640088 0.000000
13) 0.000000 0.002163
14) 0.736205 0.000000
15) 0.743056 0.000000
16) 0.742112 0.000000
17) 0.767764 0.000000
18) 0.767622 0.000000
19) 0.767904 0.000000
20) 0.793508 0.000000
21) 0.807005 0.000000
22) 0.977823 0.000000
23) 0.000000 -0.272217
24) 0.000000 -0.258508
25) 0.018177 0.000000
26) 0.000000 0.358280

NO. ITERATIONS=4

This indicates node #2, N2, is clearly the most efficient.

Simple Additive Weighting (SAW) Method

Description and uses

The simple additive weighting method is also called the weighted sum method attributed to Fishburn [8]. SAW is the simplest of the MADM methods, and still one of the widest used of the MADM methods. Its simplistic approach makes it easy to use. Depending on the type relational data used, we might either want the larger average or the smaller average.

Methodology

Here, each criterion (attribute) is given a weight, and the sum of all weights must be equal to 1. Each alternative is assessed with regard to every criterion (attribute). The overall or composite performance score of an alternative is given simply by Equation 3 with m criteria.

image      (3)

Previously, it was argued that SAW should be used only when the decision criteria can be expressed in identical units of measure (e.g., only dollars, only pounds, only seconds, etc.). However, if all the elements of the decision table are normalized, then this procedure can be used for any type and any number of criteria. In that case, Equation 3 will take the following form still with m criteria shown as equation 4:

image      (4)

Where (mij Normalized) represents the normalized value of mij, and Pi is the overall or composite score of the alternative Ai. The alternative with the highest value of Pi is considered the best alternative.

Strengths and limitations

The strengths are the ease of use and the normalized data allow for comparison across many differing criteria. Limitations include larger is always better or smaller is always better. There is not the flexibility in this method to state which criterion should be larger or smaller to achieve better performance. This makes gathering useful data of the same relational value scheme (larger or smaller) essential.

Sensitivity analysis

Sensitivity analysis should be applied to the weighting scheme employed to determine how sensitive the model is to the weights. Weighting can be arbitrary for a decision maker or in order to obtain weights you might choose to use a scheme to perform pairwise comparison as we show in AHP that we discuss later. Whenever subjectivity enters into the process for finding weights, then sensitivity analysis is recommended. Please see later sections for a suggested scheme for dealing with sensitivity analysis for individual criteria weights.

SAW illustrative example with the Noordin dark network

We begin with the key node analysis from ORA only for our four measures. This is shown in Table 3.

Rank Betweenness centrality Closeness centrality Eigenvector centrality Total degree centrality
1 N2 N2 A5 A5
2 I7 A5 M4 N2
3 A13 U T M4
4 A4 A13 N2 A6
5 A5 F A6 T
6 U6 M4 J F
7 A12 A6 F J
8 Z A23 U U
9 D2 T S8 S8
10 M5 I7 A22 A23
11 S6 J M3 B
12 U S8 B A13
13 J B S5 A22
14 A6 D2 A23 D2
15 A23 A17 A2 I7
16 A16 A7 I6 M3
17 B I2 D2 S5
18 P I6 A17 A17
19 F A22 I2 S6
20 A17 S5 A7 A7

Table 3: ORA’s key nodes table (abbreviated) produced by ORA developed at CASOS-Carnegie Mellon University.

We average the raw data as shown in Table 4, where the top 5 are nodes are N2, A5, M4, A4, and T.

N2 1 1 4 2 2
A5 5 2 1 1 2.25
M4   4 2 3 3
A4 4       4
T   9 3 5 5.66667
F   5 7 6 6
A13 3 4   12 6.33333
A12 7       7
A6 14 7 5 4 7.5
A6 14 7 5 4 7.5
U 12 3 8 8 7.75
Z 8       8
I7 2 10   15 9
J 13 11 6 6 9
S8   12 9 9 10
M5 10       10
S6 11       11
A22     10 13 11.5

Table 4: Ranks of key nodes.

Using weights as we describe later in the AHP section, we obtain a weighted scheme. The top 5 are N2, A5, M4, A4, and T as before shown in Table 5.

N2 1.18849 0.42807 0.16611 0.38216 0.54121
A5 0.2377 0.21403 0.66442 0.76433 0.47012
M4   0.85613 0.33221 1.14649 0.77828
A4 0.95079       0.95079
T   1.9263 0.49832 1.91082 1.44514
F   1.07017 1.16274 2.29298 1.50863
A13 3.32777 1.49823 0.83053 1.52865 1.7963
A12 3.32777       3.32777
A6 1.66389 0 0 0 0.41597
A6 2.85238 0.6421 1.32885 3.0573 1.97016
Z 1.90158       1.90158
U 3.09007 2.35436 0.99664 2.29298 2.18351
J 0.71309 0.85613 0 4.58596 1.5388
M5 0       0
S8   0 0 0 0
S6 2.61468       2.61468
I7 3.56547 1.71226   3.82163 3.03312
A23 0.4754 2.14033 0 5.73245 2.08704
B 0 0 1.66106 4.96812 1.65729
A22     1.99327 4.20379 3.09853
D2 2.13928 2.99646 2.8238 5.35028 3.32746
A17   2.78243     2.78243
S5   3.42453 0 0 1.14151

Table 5: Weighted average ranks.

Although we do not illustrate sensitivity analysis, we recommend doing it on the weights to see how a change in the weighted values affects the final ranking of the nodes.

Example 2. Homeland Security Threat Risk Assessment Recall the data and scenario provided in Table 1.

We obtain the SAW results from the normalized data and then from the weighted normalized data. We present the weighted normalized data in Table 6.

NYC Subway 0.07244 0.08958 0.01806 0.02603 0.01479 0.00388 0.03746
DC Metro 0.06847 0.08212 0.01806 0.01952 0.01479 0.00388 0.03447
Dirty Bomb 0.03969 0.07465 0.02528 0.03253 0.01109 0.00233 0.03093
Bank robbery 0.08038 1.5E-06 0.00722 0.00033 0.00074 0.01243 0.01685
Anthrax 0.04466 0.00597 0.02167 0.01301 0.00074 0.00932 0.01589
DC road network 0.03473 3.7E-05 0.01445 0.01627 0.02218 0.00621 0.01565
FAA Threat 0.06946 7.5E-06 0.01083 0.00013 0.00037 0.01165 0.01541

Table 6: Risk assessment results from SAW.

We found the top three threats were the NYC subway, DC Metro, and the dirty bomb using both methods.

Analytical hierarchy process (AHP)

Description and uses

AHP is a multi-objective decision analysis tool first proposed by Saaty [9]. It is designed when either subjective and objective measures or just subjective measures are being evaluated in terms of a set of alternatives based upon multiple criteria, organized in a hierarchical structure. At the top level, the criteria are evaluated or weighted, and at the bottom level the alternatives are measured against each criterion. The decision maker assesses their evaluation by making pairwise comparisons in which every pair is subjectively or objectively compared. The subjective method involves a 9 point scale that we present later.

We only desire to briefly discuss the elements in the framework of AHP. This can be described as a method to decompose a problem into sub-problems. In most decisions, the decision maker has a choice among many alternatives. Each alternative has a set of attributes or characteristics that can be measured, either subjectively or objectively. We will call these attributes, criteria. The attribute elements of the hierarchal process can relate to any aspect of the decision problemtangible or intangible, carefully measured or roughly estimated, well- or poorly-understood-anything at all that applies to the decision at hand.

We state simply that in order to perform AHP we need an objective and a set of alternatives, each with criteria (attributes) to compare. Once the hierarchy is built, the decision makers systematically evaluate the various elements pairwise (by comparing them to one another two at a time), with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements’ relative meaning and importance. It is the essence of the AHP that human judgments, and not just the underlying information, both can be used in performing the evaluations.

The AHP converts these evaluations to numerical values that can be processed and compared over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. This capability distinguishes the AHP from other decision making techniques.

In the final step of the process, numerical priorities are calculated for each of the decision alternatives. These numbers represent the alternatives’ relative ability to achieve the decision goal, so they allow a straightforward consideration of the various courses of action.

While it can be used by individuals working on straightforward decisions, the analytic hierarchy process (AHP) is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have long-term repercussions. It has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives.

Decision situations to which the AHP can be applied include the following where we desire ranking:

• Choice-The selection of one alternative from a given set of alternatives, usually where there are multiple decision criteria involved.

• Ranking-Putting a set of alternatives in order from most to least desirable

• Prioritization-Determining the relative merit of members of a set of alternatives, as opposed to selecting a single one or merely ranking them

• Resource allocation-Apportioning resources among a set of alternatives

• Benchmarking-Comparing the processes in one’s own organization with those of other best-of-breed organizations

• Quality management-Dealing with the multidimensional aspects of quality and quality improvement

• Conflict resolution-Settling disputes between parties with apparently incompatible goals or positions

Methodology of the Analytic Hierarchy Process

The procedure for using the AHP can be summarized as:

Step 1 Build the hierarchy for the decision

Goal Select the best alternative

Criteria c1, c2, c3… cm

Alternatives a1, a2, a3… an

Step 2 Judgments and comparison

Build a numerical representation using a 9-point scale in a pairwise comparison for the attributes criterion and the alternatives. The goal, in AHP, is to obtain a set of eigenvectors of the system that measures the importance with respect to the criterion. We can put these values into a matrix or table based on the values from Table 7.

 Intensity of Importance in Pair-wiseComparisons Definition
1 Equal Importance
3 Moderate Importance
5 Strong Importance
7 Very Strong Importance
9 Extreme Importance
2,4,6,8 For comparing between the above
Reciprocals of above In comparison of elements i and jif i is 3 compared to j, then j is 1/3 compared to i.
Rationale Force consistency; measure values available

Table 7: Saaty’s 9-point scale.

We must ensure that this pairwise matrix is consistent according to Saaty’s scheme to compute the Consistency Ratio, CR. The value of CR must be less than or equal to 0.1 to be considered consistent. Saaty’s computed the random index, RI, for random matrices for up to 10 criteria Table 8.

n 1 2 3 4 5 6 7 8 9 10
RI 0 0 0.52 0.89 1.1 1.24 1.35 1.4 1.45 1.49

Table 8: Random matrices.

Next, we approximate the largest eigenvalue, using the power method (Burden et al. 2013). We compute the consistency index, CI, using the formula:

image 

Then we compute the CR using:

image

If CR ≤ 0.1, then our pairwise comparison matrix is consistent and we may continue the AHP process. If not, we must go back to our pairwise comparison and fix the inconsistencies until the CR ≤ 0.1. In general, the consistency ensures that if A > B, B > C, that A > C for all A, B, and C all of which can be criteria or alternatives related by pairwise comparisons.

Step 3 Finding all the eigenvectors combined in order to obtain a comparative ranking.

Step 4 After the m × 1 criterion weights are found and the n x m matrix for n alternatives by m criterion, we use matrix multiplication to obtain the n × 1 final rankings.

Step 5 We order the final ranking.

Strengths and Limitations of AHP

Like all modeling methods, the AHP has strengths and limitations.

The main advantage of the AHP is its ability to rank choices in the order of their effectiveness in meeting conflicting objectives. If the judgments made about the relative importance of criteria and those about the alternatives’ ability to satisfy those objectives, have been made in good faith and effort, then the AHP calculations lead to the logical consequence of those judgments. It is quite hard, but not impossible; to manually change the pairwise judgments to get some predetermined result. A further strength of the AHP is its ability to detect inconsistent judgments in the pairwise comparisons using the CR value.

The limitations of the AHP are that it only works because the matrices are all of the same mathematical form-known as a positive reciprocal matrix. The reasons for this are explained in Saaty’s book, which is not for the mathematically daunted, so we will simply state that point. To create such a matrix requires that, if we use the number 9 to represent ‘A is absolutely more important than B’, then we have to use 1/9 to define the relative importance of B with respect to A. Some people regard that as reasonable; others do not.

Some suggest a drawback is in the possible scaling. However, understanding that the final values obtained simply say that one alternative is relatively better than another alternative. For example, if the AHP values for alternatives {A,B, C} found were (0.392, 0.406, 0.204) then they only imply that alternatives A and B are about equally good at approximately 0.4, while C is worse at 0.2. It does not mean that A and B are twice as good as C.

In less clear-cut cases, it would not be a bad thing to change the rating scale and see what difference it makes. If one option consistently scores well with different scales, it is likely to be a very robust choice.

In short, the AHP is a useful technique for discriminating between competing options in the light of a range of objectives to be met. The calculations are not complex and, while the AHP relies on what might be seen as a mathematical trick, you don’t need to understand the mathematics to use the technique. Be aware that it only shows relative values.

Although AHP has been used in many applications of the public and private sectors, Hartwich [10] noted several limitations. First and foremost, AHP was criticized for not providing sufficient guidance about structuring the problem to be solved, forming the levels of the hierarchy for criteria and alternatives, and aggregating group opinions when team members are geographically dispersed or are subject to time constraints. Team members may carry out rating items individually or as a group. As the levels of hierarchy increase, so does the difficulty and time it takes to synthesize weights. One remedy in preventing these problems is by conducting “AHP Walk-throughs” (i.e. meetings of decision-making participants who review the basics of the AHP methodology and work through examples so that concepts are thoroughly and easily understood).

Another critique of AHP is the “rank reversal” problem, i.e. changes in the importance ratings whenever criteria or alternatives are addedto or deleted-from the initial set of alternatives compared. Several modifications to AHP have been proposed to cope with this and other related issues. Many of the enhancements involved ways of computing, synthesizing pairwise comparisons, and/or normalizing the priority and weighting vectors. We mention now that TOPSIS corrects this rank reversal issue.

Sensitivity analysis

Since AHP, at least in the pairwise comparisons, is based upon subjective inputs using the 9-point scale then sensitivity analysis is extremely important. Leonelli [11] in his master’s thesis outlines procedures for sensitivity analysis to enhance decision support tools including numerical incremental analysis of a weight, probabilistic simulations, and mathematical models. How often do we change our minds about the relative importance of an object, place, or thing? Often enough that we should alter the pairwise comparison values to determine how robust our rankings are in the AHP process. We suggest doing enough sensitivity analysis to find the “break-point” values, if they exist, of the decision maker weights that change the rankings of our alternatives. Since the pairwise comparisons are subjective matrices compiled using the Saaty method, we suggest as a minimum a “trial and error” sensitivity analysis using the numerical incremental analysis of the weights.

Chen [12] grouped sensitivity analysis into three main groups: numerical incremental analysis, probabilistic simulations, and mathematical models The numerical incremental analysis, also known as One-at-a-time (OAT) or “trial and error” works by incrementally changing one parameter at a time, finding the new solution and showing graphically how the ranks change. There exist several variations of this method [13,14]. A probabilistic simulation employs Monte Carlo simulation Butler [15] that allows random changes in the weights and simultaneously explores the effect on the ranks. Modeling may be used when it is possible to express the relationship between the input data and the solution results.

We used equation (5) Alinezhad [16], for adjusting weights:

image              (5)

Where wj’ is the new weight and wp is the original weight of the criterion to be adjusted and wp’ is the value after the criterion was adjusted. We found this to be an easy method to adjust weights to reenter back into our model.

AHP illustrative example: Noordin dark network

We use the criteria weights (CR=0.00295), and the normalized data (Tables 9A-9c).

A)

TC 0.479865505 0.47986551
CC 0.262145561 0.26214556
BETW 0.155396998 0.155397
EC 0.102591936 0.10259194

B)

a5 0.08316 0.056888 0.103567 0.084387 a5
n2 0.077137 0.057446 0.209436 0.068054 n2
m4 0.062312 0.055772 0 0.07622 m4
a6 0.0593 0.055772 0.037975 0.063193 a6
t 0.0593 0.055215 0 0.073109 t
f 0.053509 0.055772 0.028769 0.060859 f
j 0.053509 0.055215 0.039125 0.06222 j
u 0.053509 0.05633 0.043728 0.059304 u
s8 0.047487 0.055215 0 0.058137 s8
a23 0.044475 0.055772 0.036824 0.049971 a23
b 0.044475 0.055215 0.032221 0.054248 b
a13 0.041464 0.05633 0.161105 0 a13
a22 0.041464 0.054657 0 0.056193 a22
d2 0.041464 0.055215 0.04603 0.043943 d2
I7 0.041464 0.055215 0.187572 0 I7
m3 0.041464 0 0 0.054637 m3
s5 0.041464 0.054657 0 0.051332 s5
a17 0.038684 0.054657 0.028769 0.043554 a17
s6 0.038684 0 0.044879 0 s6
a7 0.035673 0 0 0.040638 a7
I6 0 0.054657 0 0 I6

C)

Node AHP
n2 0.0916
a5 0.07957
I7 0.06352
a13 0.0597
a6 0.05546
u 0.05332
j 0.05261
m4 0.05234
f 0.05101
t 0.05043
a23 0.04681
b 0.04639
d2 0.04603
s8 0.04323
a17 0.04183
a22 0.03999
s5 0.03949
s6 0.02554
m3 0.0255
a7 0.02129
I6 0.01433

Table 9: Sensitivity analysis to enhance decision support tools, A) numerical incremental analysis of a weight, B) probabilistic simulations, c) mathematical models.

The top five are n2, a5, I7, a13, and a6.

We can apply sensitivity analysis on the criteria weights to determine how the results change.

Example 2. Homeland security threat risk assessment (Tables 10A-10B).

A)

Reliability of event 0.409837
estimated casualties 0.252375
psychological effects 0.115571
site of event 0.107812
cost to fix/repalce 0.064702
number of tips 0.049702

B)

  AHP
NYC Subway 0.224781
DC Metro 0.20684
Dirty Bomb 0.185582
Bank robbery 0.101095
Anthrax 0.095369
DC road network 0.093878
FAA Threat 0.092455

Table 10: Home land security threat risk assessment.

The AHP weights for our criteria (CR=0.0112) are

Using these weights and the normalized data we obtain the AHP rankings with the top three NYC subways, DC metro and dirty bomb (Table 10B).

We applied sensitivity analysis by decreasing the largest criteria weight, reliability, by 0.05 in several steps. The results are displayed in Figure 2 showing the model ranking does not change for small decreases in reliability (Figure 3).

socialomics-Reliability-criteria-weights

Figure 2: Reliability criteria weights changes and effect on rankings.

socialomics-Dark-network

Figure 3: Dark network using TOIPSIS. Shows the top nodes.

Technique of Order Preference by Similarity to the Ideal Solution (TOPSIS)

Description and uses

The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is a multi-criteria decision analysis method, which was originally developed in a dissertation from Kansas State University Hwang [17]. It has been further developed by others [18,19]. TOPSIS is based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution. It is a method of compensatory aggregation that compares a set of alternatives by identifying weights for each criterion, normalizing the scores for each criterion and calculating the geometric distance between each alternative and the ideal alternative, which is the best score in each criterion. An assumption of TOPSIS is that the criteria are monotonically increasing or decreasing. Normalization is usually required as the parameters or criteria are often of incompatible dimensions in multi-criteria problems. Compensatory methods such as TOPSIS allow trade-offs between criteria, where a poor result in one criterion can be negated by a good result in another criterion. This provides a more realistic form of modeling than non-compensatory methods, which include or exclude alternative solutions based on hard cut-offs.

We only desire to briefly discuss the elements in the framework of TOPSIS. TOPSIS can be described as a method to decompose a problem into sub-problems. In most decisions, the decision maker has a choice among many alternatives. Each alternative has a set of attributes or characteristics that can be measured, either subjectively or objectively. The attribute elements of the hierarchal process can relate to any aspect of the decision problem-tangible or intangible, carefully measured or roughly estimated, well- or poorly-understood-anything at all that applies to the decision at hand.

Methodology

The TOPSIS process is carried out as follows:

Step 1 Create an evaluation matrix consisting of m alternatives and n criteria, with the intersection of each alternative and criterion given as xij, giving us a matrix (Xij) m x n.

image 

Step 2 The matrix shown as D above then is normalized to form the matrix R=(Rij)mxn as shown using the normalization method

image

For i=1, 2…, m; j=1,2,…n.

Step 3 Calculate the weighted normalized decision matrix. First we need the weights. Weights can come from either the decision maker or by computation.

Step 3a Use either the decision maker’s weights for the attributes x1,x2,..xn or compute the weights through the use of Saaty’s (1980) AHP decision maker weights method to obtain the weights as the eigenvector to the attributes versus attribute pairwise comparison matrix.

image

The sum of the weights over all attributes must equal 1 regardless of the method used.

Step 3b Multiply the weights to each of the column entries in the matrix from Step 2 to obtain the matrix, T.

image

Step 4 Determine the worst alternative (Aw) and the best alternative (Ab): Examine each attribute’s column and select the largest and smallest values appropriately. If the values imply larger is better (profit), then the best alternatives are the largest values, and if the values imply smaller is better (such as cost), then the best alternative is the smallest value.

image

image

Where,

imageassociated with the criteria having a positive impact, and image associated with the criteria having a negative impact.

We suggest that if possible make all entry values in terms of positive impacts.

Step 5 Calculate the L2-distance between the target alternative i and the worst condition Aw

image

And then calculate the distance between the alternative i and the best condition Ab

image

Where diw and dib are L2-norm distances from the target alternative i to the worst and best conditions, respectively.

Step 6 Calculate the similarity to the worst condition:

image

siw=1 if and only if the alternative solution has the worst condition; and

siw=0 if and only if the alternative solution has the best condition.

Step 7 Rank the alternatives according to their value from siw (i=1,2,…,m).

Normalization

Two methods of normalization that have been used to deal with incongruous criteria dimensions are linear normalization and vector normalization.

Normalization can be calculated as in Step 2 of the TOPSIS process above. Vector normalization was incorporated with the original development of the TOPSIS method Yoon [18] and is calculated using the following formula:

image

For i=1, 2…, m; j= 1, 2…n.

We suggest two options for the weights in Step 3. First, the decision maker might actually have a weighting scheme that they want the analyst to use. If not, we suggest using Saaty’s 9-point pairwise method developed for the analytical hierarchy process (AHP) [9]. We refer the reader to our discussion in the AHP section for the decision weights using the Saaty’s 9-point scale and pairwise comparisons. In TOPSIS, we have the following scheme.

Objective Statement ← This is the decision desired

Alternatives: 1, 2, 3 … n

For each of the alternatives there are criteria (attributes) to compare:

Criteria (or Attributes): c1, c2 … cm

Once the hierarchy is built, the decision maker(s) systematically evaluate its various elements pairwise (by comparing them to one another two at a time), with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements’ relative meaning and importance. It is the essence of the TOPSIS that human judgments, and not just the underlying information, can be used in performing the evaluations.

TOPSIS converts these evaluations to numerical values that can be processed and compared over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. This capability distinguishes the TOPSIS from other decision making techniques.

In the final step of the process, numerical priorities or ranking are calculated for each of the decision alternatives. These numbers represent the alternatives’ relative ability to achieve the decision goal, so that they allow a straightforward consideration of the various courses of action.

While it can be used by individuals working on straightforward decisions, TOPSIS is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have long-term repercussions. It has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives.

Decision situations to which the TOPSIS might be applied are identical to what we presented earlier for AHP:

• Choice -The selection of one alternative from a given set of alternatives, usually where there are multiple decision criteria involved.

• Ranking -Putting a set of alternatives in order from most to least desirable

• Prioritization -Determining the relative merit of members of a set of alternatives, as opposed to selecting a single one or merely ranking them

• Resource allocation -Apportioning resources among a set of alternatives

• Benchmarking-Comparing the processes in one’s own organization with those of other best-of-breed organizations

• Quality management-Dealing with the multidimensional aspects of quality and quality improvement

• Conflict resolution-Settling disputes between parties with apparently incompatible goals or positions

Strengths and limitations

TOPSIS is based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution. It is a method of compensatory aggregation that compares a set of alternatives by identifying weights for each criterion, normalizing scores for each criterion and calculating the geometric distance between each alternative and the ideal alternative, which is the best score in each criterion. An assumption of TOPSIS is that the criteria are monotonically increasing or decreasing. Normalization is usually required as the parameters or criteria are often of incongruous dimensions in multi-criteria problems. Compensatory methods such as TOPSIS allow trade-offs between criteria, where a poor result in one criterion can be negated by a good result in another criterion. This provides a more realistic form of modeling than non-compensatory methods, which include or exclude alternative solutions based on hard cut-offs. TOPSIS corrects the rank reversal that was a limitation in strictly using the AHP method. TOPSIS also allows the user to state which of the criteria are maximized and which are minimized for better results. In the late 1980’s TOPSIS was a department of defense standard for performing selection of systems across all branches in tight budget years.

Sensitivity analysis

The decision weights are subject to sensitivity analysis to determine how they affect the final ranking. The same procedures discussed earlier are valid here. Sensitivity analysis is essential to good analysis. Additionally, Alinehad [16] suggests sensitivity analysis for TOPSIS for changing an attribute weight. We will again use equation (5) in our sensitivity analysis.

TOPSIS illustrative example: Noordin dark network

We revisit the dark network using TOIPSIS. Figure 4 shows the top nodes.

socialomics-substantial-changes

Figure 4: Sensitivity Analysis on the 4 criteria model top 5 with substantial changes to criterion weighting.

In our analysis, we have utilized weights as applicable to the metrics for the nodes. Weights are subjective, based upon the pairwise comparisons, even if used in AHP and TOPSIS methodologies. The literature provides no direct sensitivity analysis procedures. We recommend, as a minimum, at least a numerical trial and error approach to sensitivity analysis. Not only do we recommend altering the criterion pairwise comparison to measure the model’s robustness but also delving into break points is proven to be useful.

In our four metric models, we find that the model is quite robust and that with major changes in priority and pairwise comparison the top 5 nodes are not affected (Figure 5). We used the formula recommended Alinezhad [16] for adjusting decision maker weights:’

socialomics-Sensitivity-Analysis

Figure 5: Sensitivity Analysis.

image

Where w'j is the new weight and wp is the original weight of the criterion to be adjusted and w'p is the value after the criterion was adjusted.

In the eight metric models, we again used the formula recommended [16] for adjusting decision maker weights. We plotted the top 10 alternatives using three major adjustments in criteria weighting each time insuring a different criterion was the most heavily weighted. It is seen from the graph, Figure 6, that the top 5 never changed position.

Finding break points, if they exist

A break point is defined as the value of weight, wj’, that causes the ranking to be significantly change implying a change in the top alternative ranking. The method that we suggest is taking the largest weighted criterion and reduces it is slight increments which increases the weights of the other criteria and re-computing the rankings until another alternative is ranked number one Figure 6.

In this examination shown in Figure 6, the top ranked node, n2, never changes. We can get changes in the nodes ranked 2-4 through an increase change in the criterion weight for closeness centrality from 0.1611-0.4611, an increase of 0.3.

socialomics-break-points

Figure 6: Looking for break points.

Homeland security threat and risk assessment

The criteria and weights are the same as in the AHP process.

The AHP weights for our criteria (CR=0.0112) (Table 11A).

A)

Reliability of event 0.409837
estimated casualties 0.252375
psychological effects 0.115571
site of event 0.107812
cost to fix/replace 0.064702
number of tips 0.049702

B)

Alternatives (Threats) TOPSIS Results
DC Metro 0.731945656
NYC Subway 0.731695842
Dirty Bomb 0.594325418
DC road network 0.392397385
Anthrax 0.363152118
FAA Threat 0.146756013
Bank robbery 0.004919402

Table 11: AHP weights criteria.

Using these weights and the data from Table 1, we can use TOPSIS and obtain a rank ordering of the alternatives (Table 11B).

Using TOPSIS, the top three are still DC Metro, NYC subway, and dirty bomb although the top priority is DC metro.

We performed sensitivity analysis varying the heaviest weighted and least weight criteria. The top two criteria change position with a substantial change in the weights for tips as shown in Figures 7 and 8. In Figure 7, we see a decrease in reliability allows the NY Subway to overtake the DC metro as our primary concern. Figure 8 shows adding weight to the smallest criteria does not change the top two rankings.

socialomics-threats-risk-assessment

Figure 7: Sensitivity Analysis for threats in risk assessment.

socialomics-smallest-criteria

Figure 8: Changes in smallest criteria weights increasing by 0.05 each time.

Conclusion

We have illustrated the use of multi-attribute decision making through two examples: a dark network and a risk assessment process. Each method has its pros and cons. However, sensitivity analysis is critical to see how the ranking changes due to changes in the criteria weights.

References

  1. Cooper W, Seiford L, Tone K (2000) Data envelopment analysis. Kluwer Academic Press. London, UK.
  2. Charnes A, Cooper W, Rhodes E (1978) Measuring the efficiency of decision making units. European Journal of Operations Research 2: 429-444.
  3. Winston W (1995) Introduction to mathematical programming. Duxbury Press, Belmont, CA. 322-325.
  4. Callen J (1991) Data envelopment analysis: practical survey and managerial accounting applications, Journal of Management Accounting Research 3:35-57.
  5. Neralic L (1998) Sensitivity analysis in models of data envelopment analysis. Mathematical Communications 3: 41-59.
  6. Fishburn PC (1967) Additive utilities with incomplete product set: Applications to priorities and assignments. Operations Research Society of America (ORSA)15: 537-542.
  7. Saaty T (1980) The analytical hierarchy process. McGraw Hill, United States 161-176.
  8. Hartwich F (1999)Weighting of Agricultural Research Results: Strength and Limitations of the Analytic Hierarchy Process (AHP)UniversitatHohenheim.
  9. Leonelli R (2012) Enhancing a decision support tool with sensitivity analysis, thesis, University of Manchester, USA.
  10. Chen H, Kocaoglu D (2008) A sensitivity analysis algorithm for hierarchical decision models. European Journal of Operations Research 185: 266-288.
  11. Baker T, Zabinsky Z (2011) A multicriteriadecision making model for reverse logistics using Analytical Hierarchy Process, Omega 558-573.
  12. Hurly WJ (2001) The Analytical Hierarchy Process: A note on an approach to sensitivity which preserves rank order. Computers and Operations Research 28: 185-188.
  13. Butler J, Jia J, Dyer J (1997) Simulation techniques for the sensitivity analysis of multi-criteria decision models. European Journal of Operations Research103: 531-546.
  14. Alinezhad A, Amini A (2011) Sensitivity Analysis of TOPSIS technique: The results of change in the weight of one attribute on the final ranking of alternatives, Journal of Optimization in Industrial Engineering 7: 23-28.
  15. Hwang CL, Yoon K (1981) Multiple attribute decision making: Methods and applications. Springer-VerlagNew York, USA.
  16. Yoon K (1987) A reconciliation among discrete compromise situations. Journal of Operational Research Society 38:277-286. 
  17. Hwang CL, Lai Y, Liu TY (1993) A new approach for multiple objective decision making. Computers and Operational Research 20:889 -899.
Citation: Fox WP (2016) Applications and Modelling Using Multi-Attribute Decision Making to Rank Terrorist Threats. J Socialomics 5:162.

Copyright: © 2016 Fox WP. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.