Journal of Socialomics

Multiple-attribute decision making (MADM) refers to making decisions when there are multiple but a finite list of alternatives and multiple criteria.

Consider any of the following situations facing an organization:

• Homeland security needs a prioritization of potential threats cause many threats are made daily and they have limited assets;

• Military or government leaders need a prioritization of terrorist as targets for counterterrorism operations;

• Counterterrorist managers require a prioritization of terrorist phases targeting in order to stop acts prior to their happening;

• Organizations require finding the key nodes in a social or dark network;

• A financial institution wants a new model to build and maintain a retirement portfolio;

• An analytics company needs a procedure or methodology to compare recruiting offices, financial institutions, academic institutions, etc.

Consider a problem where homeland security in a region has a list of potential threats and limited assets so they need to know which threats to check out. Perhaps management needs to prioritize or rank order alternative choices: identify key nodes in a business Network, pick a contractor or sub-contractor, choose airports, rank recruiting efforts, ranks banking facilities, rank schools or colleges, etc. How does one proceed to accomplish this analytically?

In this chapter we will briefly present four methodologies to rank order or prioritize alternatives based upon multiple criteria. These four methodologies are:

• Data envelopment analysis (DEA)

• Simple average weighting (SAW)

• Analytical hierarchy process (AHP)

• Technique of order preference by similarity to ideal solution (TOPSIS)

For each method, we describe the method and its uses, discuss some strengths and limitations to the method, discuss tips for conducting sensitivity analysis, and present illustrative examples.

These MADM methods have been used extensively in current research and many of the research efforts are listed within each MADM technique discussion.

In this paper we will examine a threat risk assessment process that could be used by local law enforcement or homeland security to examine possible threats as in Table 1 and identifying how to find the key nodes in a terrorist or criminal network. Assume we have a social or dark network where we desire to know or find the key or influential nodes with the network. The Noordin dark network graph is provide as shown in Figure 1 and ORA output for four main metrics is provided in Table 2.

Table 1: Homeland security threat risk assessment priority.

Table 2: ORA output for 4 key metrics of the top listed nodes or agents.

Figure 1: Noordin Dark network (79 nodes) from ORA.

Description and uses

Data envelopment analysis (DEA) is a relatively new “data inputoutput driven” approach for evaluating the performance of entities called decision making units (DMUs) that convert multiple inputs into multiple outputs [1]. The definition of a DMU is generic and very flexible. It has been used to evaluate both the performance or efficiencies of hospitals, schools, departments, US Air Force wings, US armed forces recruiting agencies, universities, cities, courts, businesses, banking facilities, countries, regions, and the list go on. According to Cooper [1], DEA has been used to gain insights into activities that were not obtained by other quantitative or qualitative methods.

Charnes et al. [2], described DEA as a mathematical programming model applied to observational data, providing a new way of obtaining empirical estimates of relations. It is formally defined as a methodology directed to frontiers rather than central tendencies.

Methodology

The model, in simplest terms, may be formulated and solved as a linear programming problem [3,4]. Although several formulations for DEA exist, we seek the most straight forward formulation in order to maximize an efficiency of a DMU as constrained by inputs and outputs as shown in equation 1. As an option, we might normalize the metric inputs and outputs for the alternatives if poorly scaled. Otherwise, we will call this matrix, X, with entries xij. We define an efficiency unit as Ei for i=1,2,…, nodes. We let wi be the weights or coefficients for the linear combinations. Further, we restrict any efficiency from being larger than one. This gives the following linear programming formulation for single outputs but multiple inputs:

      (1)

For multiple inputs and outputs, we recommend the formulations provided by Winston [3] and Trick [5] using equation (2).

For any DMU0, let Xi be the inputs and Yi be the outputs. Let X0 and Y0 be the DMU being modeled.

      (2)

Strengths and Limitations to DEA

DEA can be a very useful tool when used wisely. Trick [6] provides a nice list of a few of the strengths that make DEA extremely useful:

• DEA can handle multiple input and multiple output models.

• DEA doesn’t require an assumption of a functional form relating inputs to outputs.

• DMUs are directly compared against a peer or combination of peers.

Inputs and outputs can have very different units. For example, X1 could be in units of lives saved and X2 could be in units of dollars without requiring any a priori tradeoff between the two.

The same characteristics that make DEA a powerful tool can also create limitations. An analyst should keep these limitations in mind when choosing whether or not to use DEA.

Since DEA is an extreme point technique, noise in the data such as measurement error can cause significant problems.

DEA is good at estimating “relative” efficiency of a DMU but it converges very slowly to “absolute” efficiency. In other words, it can tell you how well you are doing compared to your peers but not compared to a “theoretical maximum.”

Since DEA is a nonparametric technique, statistical hypothesis tests are difficult and are the focus of ongoing research.

Since a standard formulation of DEA with multiple inputs and outputs creates a separate linear program for each DMU, large problems can be computationally intensive.

Linear programming does not ensure all weights are considered. We find that the value for weights is only for those that optimally determine an efficiency rating. If having all criteria weighted (inputs, outputs) is essential to the decision maker then we should not use DEA.

Sensitivity analysis

We like the sensitivity approach taken by Neralic [7], where he explains an increase in any output cannot worsen an already achieved efficiency rating nor can a decrease in inputs alone worsen an already achieved efficiency rating. As a result in our illustrative examples we only decrease outputs and increase inputs, as applicable.

Application to Noordin dark network

We apply the formulation to the data from ORA on the Noordin network from Table 2. We present two results. We used the formulation as presented in equation 2 and then we added a constraint that the sum of the weights must equal to one. Using the basic formulation in equation 2, we obtained the following solution using the LP software, LINDO.

The formulation from LINDO is:

MAX E1

SUBJECT TO

2)-E1+0.359 W1+0.09 W2+0.102 W3+0.434 W4=0

3) 0.333 W1+1.82 W2+1.03 W3+35 W4-E2=0

4) 0.269 W1+0.492 W3-E3=0

5) 0.256 W1+0.033 W2+0.1 W3+0.325 W4-E4=0

6) 0.256 W1+0.099 W3+0.376 W4-E5=0

7) 0.231 W1+0.025 W2+0.1 W3+0.313 W4-E6=0

8) 0.231 W1+0.034 W2+0.099 W3+0.32 W4-E7=0

9) 0.231 W1+0.038 W2+0.101 W3+0.305 W4-E8=0

10) 0.205 W1+0.099 W3+0.299 W4-E9=0

11) 0.192 W1+0.032 W2+0.1 W3+0.257 W4-E10=0

12) E1< =1

13) E2< =1

14) E3< =1

15) E4< =1

16) E5< =1

17) E6< =1

18) E7< =1

19) E8< =1

20) E9< =1

21) E10< =1

22) W1> =0.001

23) W2> =0.001

24) W3> =0.001

25) W4> =0.001

26) W1+W2+W3+W4=1

END

The solution is as follows:

LP OPTIMUM FOUND AT STEP 4.

OBJECTIVE FUNCTION VALUE

1) 0.3599123

NO. ITERATIONS=4

This indicates node #2, N2, is clearly the most efficient.

Description and uses

The simple additive weighting method is also called the weighted sum method attributed to Fishburn [8]. SAW is the simplest of the MADM methods, and still one of the widest used of the MADM methods. Its simplistic approach makes it easy to use. Depending on the type relational data used, we might either want the larger average or the smaller average.

Here, each criterion (attribute) is given a weight, and the sum of all weights must be equal to 1. Each alternative is assessed with regard to every criterion (attribute). The overall or composite performance score of an alternative is given simply by Equation 3 with m criteria.

      (3)

Previously, it was argued that SAW should be used only when the decision criteria can be expressed in identical units of measure (e.g., only dollars, only pounds, only seconds, etc.). However, if all the elements of the decision table are normalized, then this procedure can be used for any type and any number of criteria. In that case, Equation 3 will take the following form still with m criteria shown as equation 4:

      (4)

Where (mij Normalized) represents the normalized value of mij, and Pi is the overall or composite score of the alternative Ai. The alternative with the highest value of Pi is considered the best alternative.

The strengths are the ease of use and the normalized data allow for comparison across many differing criteria. Limitations include larger is always better or smaller is always better. There is not the flexibility in this method to state which criterion should be larger or smaller to achieve better performance. This makes gathering useful data of the same relational value scheme (larger or smaller) essential.

Sensitivity analysis

Sensitivity analysis should be applied to the weighting scheme employed to determine how sensitive the model is to the weights. Weighting can be arbitrary for a decision maker or in order to obtain weights you might choose to use a scheme to perform pairwise comparison as we show in AHP that we discuss later. Whenever subjectivity enters into the process for finding weights, then sensitivity analysis is recommended. Please see later sections for a suggested scheme for dealing with sensitivity analysis for individual criteria weights.

SAW illustrative example with the Noordin dark network

We begin with the key node analysis from ORA only for our four measures. This is shown in Table 3.

Table 3: ORA’s key nodes table (abbreviated) produced by ORA developed at CASOS-Carnegie Mellon University.

We average the raw data as shown in Table 4, where the top 5 are nodes are N2, A5, M4, A4, and T.

Table 4: Ranks of key nodes.

Using weights as we describe later in the AHP section, we obtain a weighted scheme. The top 5 are N2, A5, M4, A4, and T as before shown in Table 5.

Table 5: Weighted average ranks.

Although we do not illustrate sensitivity analysis, we recommend doing it on the weights to see how a change in the weighted values affects the final ranking of the nodes.

Example 2. Homeland Security Threat Risk Assessment Recall the data and scenario provided in Table 1.

We obtain the SAW results from the normalized data and then from the weighted normalized data. We present the weighted normalized data in Table 6.

Table 6: Risk assessment results from SAW.

We found the top three threats were the NYC subway, DC Metro, and the dirty bomb using both methods.

Description and uses

AHP is a multi-objective decision analysis tool first proposed by Saaty [9]. It is designed when either subjective and objective measures or just subjective measures are being evaluated in terms of a set of alternatives based upon multiple criteria, organized in a hierarchical structure. At the top level, the criteria are evaluated or weighted, and at the bottom level the alternatives are measured against each criterion. The decision maker assesses their evaluation by making pairwise comparisons in which every pair is subjectively or objectively compared. The subjective method involves a 9 point scale that we present later.

We only desire to briefly discuss the elements in the framework of AHP. This can be described as a method to decompose a problem into sub-problems. In most decisions, the decision maker has a choice among many alternatives. Each alternative has a set of attributes or characteristics that can be measured, either subjectively or objectively. We will call these attributes, criteria. The attribute elements of the hierarchal process can relate to any aspect of the decision problemtangible or intangible, carefully measured or roughly estimated, well- or poorly-understood-anything at all that applies to the decision at hand.

We state simply that in order to perform AHP we need an objective and a set of alternatives, each with criteria (attributes) to compare. Once the hierarchy is built, the decision makers systematically evaluate the various elements pairwise (by comparing them to one another two at a time), with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements’ relative meaning and importance. It is the essence of the AHP that human judgments, and not just the underlying information, both can be used in performing the evaluations.

The AHP converts these evaluations to numerical values that can be processed and compared over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. This capability distinguishes the AHP from other decision making techniques.

In the final step of the process, numerical priorities are calculated for each of the decision alternatives. These numbers represent the alternatives’ relative ability to achieve the decision goal, so they allow a straightforward consideration of the various courses of action.

While it can be used by individuals working on straightforward decisions, the analytic hierarchy process (AHP) is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have long-term repercussions. It has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives.

Decision situations to which the AHP can be applied include the following where we desire ranking:

• Choice-The selection of one alternative from a given set of alternatives, usually where there are multiple decision criteria involved.

• Ranking-Putting a set of alternatives in order from most to least desirable

• Prioritization-Determining the relative merit of members of a set of alternatives, as opposed to selecting a single one or merely ranking them

• Resource allocation-Apportioning resources among a set of alternatives

• Benchmarking-Comparing the processes in one’s own organization with those of other best-of-breed organizations

• Quality management-Dealing with the multidimensional aspects of quality and quality improvement

• Conflict resolution-Settling disputes between parties with apparently incompatible goals or positions

The procedure for using the AHP can be summarized as:

Step 1 Build the hierarchy for the decision

Goal Select the best alternative

Criteria c1, c2, c3… cm

Alternatives a1, a2, a3… an

Step 2 Judgments and comparison

Build a numerical representation using a 9-point scale in a pairwise comparison for the attributes criterion and the alternatives. The goal, in AHP, is to obtain a set of eigenvectors of the system that measures the importance with respect to the criterion. We can put these values into a matrix or table based on the values from Table 7.

Table 7: Saaty’s 9-point scale.

We must ensure that this pairwise matrix is consistent according to Saaty’s scheme to compute the Consistency Ratio, CR. The value of CR must be less than or equal to 0.1 to be considered consistent. Saaty’s computed the random index, RI, for random matrices for up to 10 criteria Table 8.

Table 8: Random matrices.

Next, we approximate the largest eigenvalue, using the power method (Burden et al. 2013). We compute the consistency index, CI, using the formula:

Then we compute the CR using:

If CR ≤ 0.1, then our pairwise comparison matrix is consistent and we may continue the AHP process. If not, we must go back to our pairwise comparison and fix the inconsistencies until the CR ≤ 0.1. In general, the consistency ensures that if A > B, B > C, that A > C for all A, B, and C all of which can be criteria or alternatives related by pairwise comparisons.

Step 3 Finding all the eigenvectors combined in order to obtain a comparative ranking.

Step 4 After the m × 1 criterion weights are found and the n x m matrix for n alternatives by m criterion, we use matrix multiplication to obtain the n × 1 final rankings.

Step 5 We order the final ranking.

Strengths and Limitations of AHP

Like all modeling methods, the AHP has strengths and limitations.

The main advantage of the AHP is its ability to rank choices in the order of their effectiveness in meeting conflicting objectives. If the judgments made about the relative importance of criteria and those about the alternatives’ ability to satisfy those objectives, have been made in good faith and effort, then the AHP calculations lead to the logical consequence of those judgments. It is quite hard, but not impossible; to manually change the pairwise judgments to get some predetermined result. A further strength of the AHP is its ability to detect inconsistent judgments in the pairwise comparisons using the CR value.

The limitations of the AHP are that it only works because the matrices are all of the same mathematical form-known as a positive reciprocal matrix. The reasons for this are explained in Saaty’s book, which is not for the mathematically daunted, so we will simply state that point. To create such a matrix requires that, if we use the number 9 to represent ‘A is absolutely more important than B’, then we have to use 1/9 to define the relative importance of B with respect to A. Some people regard that as reasonable; others do not.

Some suggest a drawback is in the possible scaling. However, understanding that the final values obtained simply say that one alternative is relatively better than another alternative. For example, if the AHP values for alternatives {A,B, C} found were (0.392, 0.406, 0.204) then they only imply that alternatives A and B are about equally good at approximately 0.4, while C is worse at 0.2. It does not mean that A and B are twice as good as C.

In less clear-cut cases, it would not be a bad thing to change the rating scale and see what difference it makes. If one option consistently scores well with different scales, it is likely to be a very robust choice.

In short, the AHP is a useful technique for discriminating between competing options in the light of a range of objectives to be met. The calculations are not complex and, while the AHP relies on what might be seen as a mathematical trick, you don’t need to understand the mathematics to use the technique. Be aware that it only shows relative values.

Although AHP has been used in many applications of the public and private sectors, Hartwich [10] noted several limitations. First and foremost, AHP was criticized for not providing sufficient guidance about structuring the problem to be solved, forming the levels of the hierarchy for criteria and alternatives, and aggregating group opinions when team members are geographically dispersed or are subject to time constraints. Team members may carry out rating items individually or as a group. As the levels of hierarchy increase, so does the difficulty and time it takes to synthesize weights. One remedy in preventing these problems is by conducting “AHP Walk-throughs” (i.e. meetings of decision-making participants who review the basics of the AHP methodology and work through examples so that concepts are thoroughly and easily understood).

Another critique of AHP is the “rank reversal” problem, i.e. changes in the importance ratings whenever criteria or alternatives are addedto or deleted-from the initial set of alternatives compared. Several modifications to AHP have been proposed to cope with this and other related issues. Many of the enhancements involved ways of computing, synthesizing pairwise comparisons, and/or normalizing the priority and weighting vectors. We mention now that TOPSIS corrects this rank reversal issue.

Sensitivity analysis

Since AHP, at least in the pairwise comparisons, is based upon subjective inputs using the 9-point scale then sensitivity analysis is extremely important. Leonelli [11] in his master’s thesis outlines procedures for sensitivity analysis to enhance decision support tools including numerical incremental analysis of a weight, probabilistic simulations, and mathematical models. How often do we change our minds about the relative importance of an object, place, or thing? Often enough that we should alter the pairwise comparison values to determine how robust our rankings are in the AHP process. We suggest doing enough sensitivity analysis to find the “break-point” values, if they exist, of the decision maker weights that change the rankings of our alternatives. Since the pairwise comparisons are subjective matrices compiled using the Saaty method, we suggest as a minimum a “trial and error” sensitivity analysis using the numerical incremental analysis of the weights.

Chen [12] grouped sensitivity analysis into three main groups: numerical incremental analysis, probabilistic simulations, and mathematical models The numerical incremental analysis, also known as One-at-a-time (OAT) or “trial and error” works by incrementally changing one parameter at a time, finding the new solution and showing graphically how the ranks change. There exist several variations of this method [13,14]. A probabilistic simulation employs Monte Carlo simulation Butler [15] that allows random changes in the weights and simultaneously explores the effect on the ranks. Modeling may be used when it is possible to express the relationship between the input data and the solution results.

We used equation (5) Alinezhad [16], for adjusting weights:

              (5)

Where wj’ is the new weight and wp is the original weight of the criterion to be adjusted and wp’ is the value after the criterion was adjusted. We found this to be an easy method to adjust weights to reenter back into our model.

AHP illustrative example: Noordin dark network

We use the criteria weights (CR=0.00295), and the normalized data (Tables 9A-9c).

A)

B)

C)

Table 9: Sensitivity analysis to enhance decision support tools, A) numerical incremental analysis of a weight, B) probabilistic simulations, c) mathematical models.

The top five are n2, a5, I7, a13, and a6.

We can apply sensitivity analysis on the criteria weights to determine how the results change.

Example 2. Homeland security threat risk assessment (Tables 10A-10B).

A)

B)

Table 10: Home land security threat risk assessment.

The AHP weights for our criteria (CR=0.0112) are

Using these weights and the normalized data we obtain the AHP rankings with the top three NYC subways, DC metro and dirty bomb (Table 10B).

We applied sensitivity analysis by decreasing the largest criteria weight, reliability, by 0.05 in several steps. The results are displayed in Figure 2 showing the model ranking does not change for small decreases in reliability (Figure 3).

Figure 2: Reliability criteria weights changes and effect on rankings.

Figure 3: Dark network using TOIPSIS. Shows the top nodes.

Description and uses

The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is a multi-criteria decision analysis method, which was originally developed in a dissertation from Kansas State University Hwang [17]. It has been further developed by others [18,19]. TOPSIS is based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution. It is a method of compensatory aggregation that compares a set of alternatives by identifying weights for each criterion, normalizing the scores for each criterion and calculating the geometric distance between each alternative and the ideal alternative, which is the best score in each criterion. An assumption of TOPSIS is that the criteria are monotonically increasing or decreasing. Normalization is usually required as the parameters or criteria are often of incompatible dimensions in multi-criteria problems. Compensatory methods such as TOPSIS allow trade-offs between criteria, where a poor result in one criterion can be negated by a good result in another criterion. This provides a more realistic form of modeling than non-compensatory methods, which include or exclude alternative solutions based on hard cut-offs.

We only desire to briefly discuss the elements in the framework of TOPSIS. TOPSIS can be described as a method to decompose a problem into sub-problems. In most decisions, the decision maker has a choice among many alternatives. Each alternative has a set of attributes or characteristics that can be measured, either subjectively or objectively. The attribute elements of the hierarchal process can relate to any aspect of the decision problem-tangible or intangible, carefully measured or roughly estimated, well- or poorly-understood-anything at all that applies to the decision at hand.

Methodology

The TOPSIS process is carried out as follows:

Step 1 Create an evaluation matrix consisting of m alternatives and n criteria, with the intersection of each alternative and criterion given as x_ij, giving us a matrix (X_ij) m x n.

Step 2 The matrix shown as D above then is normalized to form the matrix R=(R_ij)_mxn as shown using the normalization method

For i=1, 2…, m; j=1,2,…n.

Step 3 Calculate the weighted normalized decision matrix. First we need the weights. Weights can come from either the decision maker or by computation.

Step 3a Use either the decision maker’s weights for the attributes x1,x2,..xn or compute the weights through the use of Saaty’s (1980) AHP decision maker weights method to obtain the weights as the eigenvector to the attributes versus attribute pairwise comparison matrix.

The sum of the weights over all attributes must equal 1 regardless of the method used.

Step 3b Multiply the weights to each of the column entries in the matrix from Step 2 to obtain the matrix, T.

Step 4 Determine the worst alternative (Aw) and the best alternative (Ab): Examine each attribute’s column and select the largest and smallest values appropriately. If the values imply larger is better (profit), then the best alternatives are the largest values, and if the values imply smaller is better (such as cost), then the best alternative is the smallest value.

Where,

associated with the criteria having a positive impact, and associated with the criteria having a negative impact.

We suggest that if possible make all entry values in terms of positive impacts.

Step 5 Calculate the L2-distance between the target alternative i and the worst condition A_w

And then calculate the distance between the alternative i and the best condition Ab

Where d_iw and d_ib are L2-norm distances from the target alternative i to the worst and best conditions, respectively.

Step 6 Calculate the similarity to the worst condition:

s_iw=1 if and only if the alternative solution has the worst condition; and

s_iw=0 if and only if the alternative solution has the best condition.

Step 7 Rank the alternatives according to their value from s_iw (i=1,2,…,m).

Normalization

Two methods of normalization that have been used to deal with incongruous criteria dimensions are linear normalization and vector normalization.

Normalization can be calculated as in Step 2 of the TOPSIS process above. Vector normalization was incorporated with the original development of the TOPSIS method Yoon [18] and is calculated using the following formula:

For i=1, 2…, m; j= 1, 2…n.

We suggest two options for the weights in Step 3. First, the decision maker might actually have a weighting scheme that they want the analyst to use. If not, we suggest using Saaty’s 9-point pairwise method developed for the analytical hierarchy process (AHP) [9]. We refer the reader to our discussion in the AHP section for the decision weights using the Saaty’s 9-point scale and pairwise comparisons. In TOPSIS, we have the following scheme.

Objective Statement ← This is the decision desired

Alternatives: 1, 2, 3 … n

For each of the alternatives there are criteria (attributes) to compare:

Criteria (or Attributes): c1, c2 … cm

Once the hierarchy is built, the decision maker(s) systematically evaluate its various elements pairwise (by comparing them to one another two at a time), with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements’ relative meaning and importance. It is the essence of the TOPSIS that human judgments, and not just the underlying information, can be used in performing the evaluations.

TOPSIS converts these evaluations to numerical values that can be processed and compared over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. This capability distinguishes the TOPSIS from other decision making techniques.

In the final step of the process, numerical priorities or ranking are calculated for each of the decision alternatives. These numbers represent the alternatives’ relative ability to achieve the decision goal, so that they allow a straightforward consideration of the various courses of action.

While it can be used by individuals working on straightforward decisions, TOPSIS is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have long-term repercussions. It has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives.

Decision situations to which the TOPSIS might be applied are identical to what we presented earlier for AHP:

• Choice -The selection of one alternative from a given set of alternatives, usually where there are multiple decision criteria involved.

• Ranking -Putting a set of alternatives in order from most to least desirable

• Prioritization -Determining the relative merit of members of a set of alternatives, as opposed to selecting a single one or merely ranking them

• Resource allocation -Apportioning resources among a set of alternatives

• Benchmarking-Comparing the processes in one’s own organization with those of other best-of-breed organizations

• Quality management-Dealing with the multidimensional aspects of quality and quality improvement

• Conflict resolution-Settling disputes between parties with apparently incompatible goals or positions

Strengths and limitations

TOPSIS is based on the concept that the chosen alternative should have the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution. It is a method of compensatory aggregation that compares a set of alternatives by identifying weights for each criterion, normalizing scores for each criterion and calculating the geometric distance between each alternative and the ideal alternative, which is the best score in each criterion. An assumption of TOPSIS is that the criteria are monotonically increasing or decreasing. Normalization is usually required as the parameters or criteria are often of incongruous dimensions in multi-criteria problems. Compensatory methods such as TOPSIS allow trade-offs between criteria, where a poor result in one criterion can be negated by a good result in another criterion. This provides a more realistic form of modeling than non-compensatory methods, which include or exclude alternative solutions based on hard cut-offs. TOPSIS corrects the rank reversal that was a limitation in strictly using the AHP method. TOPSIS also allows the user to state which of the criteria are maximized and which are minimized for better results. In the late 1980’s TOPSIS was a department of defense standard for performing selection of systems across all branches in tight budget years.

Sensitivity analysis

The decision weights are subject to sensitivity analysis to determine how they affect the final ranking. The same procedures discussed earlier are valid here. Sensitivity analysis is essential to good analysis. Additionally, Alinehad [16] suggests sensitivity analysis for TOPSIS for changing an attribute weight. We will again use equation (5) in our sensitivity analysis.

TOPSIS illustrative example: Noordin dark network

We revisit the dark network using TOIPSIS. Figure 4 shows the top nodes.

Figure 4: Sensitivity Analysis on the 4 criteria model top 5 with substantial changes to criterion weighting.

In our analysis, we have utilized weights as applicable to the metrics for the nodes. Weights are subjective, based upon the pairwise comparisons, even if used in AHP and TOPSIS methodologies. The literature provides no direct sensitivity analysis procedures. We recommend, as a minimum, at least a numerical trial and error approach to sensitivity analysis. Not only do we recommend altering the criterion pairwise comparison to measure the model’s robustness but also delving into break points is proven to be useful.

In our four metric models, we find that the model is quite robust and that with major changes in priority and pairwise comparison the top 5 nodes are not affected (Figure 5). We used the formula recommended Alinezhad [16] for adjusting decision maker weights:’

Figure 5: Sensitivity Analysis.

Where w'_j is the new weight and wp is the original weight of the criterion to be adjusted and w'_p is the value after the criterion was adjusted.

In the eight metric models, we again used the formula recommended [16] for adjusting decision maker weights. We plotted the top 10 alternatives using three major adjustments in criteria weighting each time insuring a different criterion was the most heavily weighted. It is seen from the graph, Figure 6, that the top 5 never changed position.

Finding break points, if they exist

A break point is defined as the value of weight, wj’, that causes the ranking to be significantly change implying a change in the top alternative ranking. The method that we suggest is taking the largest weighted criterion and reduces it is slight increments which increases the weights of the other criteria and re-computing the rankings until another alternative is ranked number one Figure 6.

In this examination shown in Figure 6, the top ranked node, n2, never changes. We can get changes in the nodes ranked 2-4 through an increase change in the criterion weight for closeness centrality from 0.1611-0.4611, an increase of 0.3.

Figure 6: Looking for break points.

Homeland security threat and risk assessment

The criteria and weights are the same as in the AHP process.

The AHP weights for our criteria (CR=0.0112) (Table 11A).

A)

B)

Table 11: AHP weights criteria.

Using these weights and the data from Table 1, we can use TOPSIS and obtain a rank ordering of the alternatives (Table 11B).

Using TOPSIS, the top three are still DC Metro, NYC subway, and dirty bomb although the top priority is DC metro.

We performed sensitivity analysis varying the heaviest weighted and least weight criteria. The top two criteria change position with a substantial change in the weights for tips as shown in Figures 7 and 8. In Figure 7, we see a decrease in reliability allows the NY Subway to overtake the DC metro as our primary concern. Figure 8 shows adding weight to the smallest criteria does not change the top two rankings.

Figure 7: Sensitivity Analysis for threats in risk assessment.

Figure 8: Changes in smallest criteria weights increasing by 0.05 each time.

We have illustrated the use of multi-attribute decision making through two examples: a dark network and a risk assessment process. Each method has its pros and cons. However, sensitivity analysis is critical to see how the ranking changes due to changes in the criteria weights.