Reassessing the Mathematical Consistency and Ranking Reliability of Additive Ratio Assessment and Combinative Distance-Based Assessment in Multi-Criteria Decision-Making

The proliferation of multi-criteria decision-making (MCDM) models has posed a significant challenge in applied research. Which method should be selected for a given problem, and how should its suitability be established? What if different methods yield different solutions? These questions, together with proposed approaches to addressing them, have been discussed in previous studies [1], [2], [3].

A further concern in recent publications is the insufficient justification provided for selecting particular MCDM methods to address applied problems. In some cases, new approaches or modifications of existing methods are proposed without adequate evidence of their effectiveness or feasibility. Consequently, their reliability and practical applicability remain insufficiently established, and their use should be approached with caution until appropriate validation has been conducted.

Addressing this problem remains difficult because methodological shortcomings may not be identified until after a method has been published and applied. For example, Xu [4] showed that the objective weights obtained by the method proposed by Ma et al. [5] differ significantly from those obtained using Shannon entropy method [6]. The availability of a large number of methods (including hybrid models) to researchers has determined a modern trend in MCDM: the synthesis of decisions based on multiple models, termed as the Multi Methods Model (3M) in Mukhametzyanov and Pamucar [2] (see also Section 2.2). We use “3M” in the context of synthesizing various methods. It is assumed that combining solutions obtained using a large number of methods increases the reliability of decisions. This is consistent with the majority principle as the primary method for making collegial decisions. However, historical experience shows that this principle sometimes leads to unsatisfactory results.

According to Mukhametzyanov and Pamucar [2], the application of the 3M approach raises five fundamental questions:

(i) Which MCDM methods should be used to solve a specific problem, and how many methods should be included;

(ii) How to compare the results obtained using different methods;

(iii) How to evaluate the significance (weight) of methods;

(iv) Whether methods should be grouped and how to form groups; and

(v) How to synthesize a solution based on multiple models.

In terms of grouping MCDM methods, the current result is a ranking obtained by aggregating the results of three ranking methods: the ratio system, the reference point approach, and the full multiplicative form within the Multi-Objective Optimization by Ratio Analysis plus the Full Multiplicative Form (MULTIMOORA) concept [7], [8].

For solution synthesis, several approaches can be employed, including dominance theory, arithmetic/geometric means, the Borda approach [9], and the Copeland method [10].

A notable project is PYREPO-MCDA, a Python 3 library for multi-criteria decision analysis (MCDA). This library includes:

(1) MCDA methods (which serve as aggregation functions in the 3M model);

(2) distance metrics (for goal or reference level models);

(3) methods for normalizing the decision matrix;

(4) weighting methods for determining criterion weights (objective and subjective methods); and

(5) methods for determining compromise rankings based on several rankings obtained with different MCDA methods (synthesis of solutions from multiple models).

All components of this project are designed around the integration of MCDM methods to construct an MCDM model and then synthesize solutions based on multiple models.

One commonly adopted approach to selecting an MCDM method is to draw on previous experience and select methods that have been successfully applied to problems with similar characteristics or within the same application domain.

An alternative is the typical MCDM methodology: select a method or rank methods using multicriteria selection. In this approach, the MCDM methods themselves serve as the alternatives to be evaluated. Based on this view of MCDM alternatives, it is necessary to define a set of criteria for evaluating MCDM methods. After this, a standard procedure for selecting the best method should be carried out. In our opinion, the most significant contribution in this area is the study by Saaty and Ergu [1], who proposed 16 criteria based primarily on subjective assessments using three categories: high, medium, and low. However, the proposed set of largely subjective quality criteria for methods requires an expert approach to evaluation within the selected scale. Despite the presence of a significant subjective component in this method, one can only conclude that it is the best solution to date. The problem has been formulated, and its solution requires the efforts of the entire MCDM community in expert evaluation of a wide variety of MCDM methods.

Nevertheless, further efforts are needed to strengthen the objective component in MCDM methods. Mukhametzyanov and Pamucar [2], [3] proposed a simple and effective analysis method based on the relative performance indicator (RPI) (see Section 2.2). Six MCDM methods were examined: 1) the Weighted Sum Model (WSM), 2) the Ratio System (RS) approach, 3) the Multi-Attributive Border Approximation area Comparison (MABAC), 4) the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) using the $L_{1}$ metric, 5) the Multi-Attributive Ideal-Real Comparative Analysis (MAIRCA), and 6) the Ranking of Alternatives with Weights of Criterion (RAWEC), provided that each aggregation method incorporates the same linear attribute normalization technique. The authors subsequently extended this analysis to the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method and established its equivalence to WSM under Max normalization and reverse sorting (ReS) inversion.

This result allows us to eliminate equivalent methods from the MCDM toolbox. When solving MCDM problems, the simplest and most easily interpreted of these methods, WSM, is recommended. The authors recommend this methodology as mandatory for analyzing new or hybrid MCDM methods to eliminate duplication of existing methods.

In the context of MCDM model selection, Mukhametzyanov and Zulkarnay [11] conducted a comprehensive analysis of the complex proportional assessment (COPRAS) method. The analysis demonstrated that COPRAS does not substantiate its claimed ability to distinguish the separate effects of benefit and cost criteria on the overall evaluation results. The assertion that COPRAS eliminates the need for cost-criterion inversion was also found to be unsupported. It was shown that, in the case of a single cost criterion, COPRAS is equivalent to the WSM model using Sum/iSum normalization/inversion. In some of the cases examined in that study, the differences were small, and COPRAS produced results comparable to those obtained with WSM. The study concluded that COPRAS provided no additional decision-making information or clearer interpretation of the contribution of cost criteria, despite its more complex calculation formula.

An analysis of another widely used MCDM method, VIse Kriterijumska Optimizacija kompromisno Resenje (VIKOR), revealed its shortcomings in calculating performance in various metrics and in assessing the distinguishability of alternative rankings. The current analysis and method-selection procedure are summarized in Figure 1.

The existing literature in this area has largely focused on developing hybrid MCDM approaches and validating their performance, with comparatively less attention devoted to critically examining the mathematical properties and limitations of individual methods. This study is a continuation of a series of works on MCDM model selection. In this article, we analyze the Additive Ratio Assessment (ARAS) method [12] and the Combinative Distance-Based Assessment (CODAS) method [13] in terms of the correct processing of the initial information describing the properties of the objects under study. Both methods employ a non-linear inversion of cost criteria, resulting in a distortion of proportions within the initial data. A linear inversion method—the ReS algorithm—is recommended; however, this does not eliminate the methods’ fundamental drawbacks.

Regarding ARAS, the consequences of introducing an ideal alternative into the decision matrix remain an open question. A primary concern is whether the well-known Rank Reversal Phenomenon (RRP) occurs upon the addition of a new alternative [14]. Another issue is the extent to which the resulting solution differs from that of the WSM model, given the structural similarity between ARAS and WSM [12]. In this paper, we demonstrate that incorporating an optimal alternative into the decision matrix leads to disproportionate changes in alternative ratings, potentially resulting in a reordering of the rankings. The ARAS method does not utilize any additional information contained within the decision matrix. It is a typical constructive analogue of the WSM that, among other things, distorts the data ratios.

One of the primary motivations for analyzing the CODAS method is that Keshavarz Ghorabaee et al. [13] did not explain why they combined two distance measures to calculate the score for each alternative. We question the rationale and justification behind the CODAS method. The core premise of CODAS is the combination of two distance measures to compute the score for each alternative. Our analysis shows that the taxicab term is activated conditionally across pairwise comparisons, so its contribution can vary across alternatives and may affect their relative scores and rankings.

These findings suggest that ARAS and CODAS should be applied with caution, taking into account the effects of normalization and aggregation choices under the conditions examined. The paper is structured as follows. Section 2 provides a brief description of the theoretical background necessary for conducting a quantitative and qualitative analysis regarding the MCDM ranking model, the method for comparing the rankings of two or more MCDM methods based on RPI, and a step-by-step algorithm for the basic ARAS and CODAS methods.

Section 3 analyses the ARAS method and demonstrates its equivalence to the WSM under Max and Max–Min normalization. It is shown that introducing the optimal alternative into the decision matrix produces a rank reversal in approximately 13% of cases compared to the WSM method (analogous to RRP). Based on this, the ARAS method should be used with caution under the examined conditions.

Section 4 examines how the conditional combination of Euclidean and taxicab distances influences CODAS scores and rankings. The taxicab term is not activated in every pairwise comparison, and its contribution can vary across alternatives. Numerical simulations involving 10,000 generated problems illustrate this variability.

These findings suggest that the threshold rule and its effects on the resulting rankings should be considered when applying CODAS. Each of the properties and assertions in this article is accompanied by a corresponding numerical example. The conclusion summarizes the findings, including the recommendation of WSM as an alternative to ARAS and CODAS because of its reliable basis for data transformation and straightforward interpretation of ratings.

This section gives a brief description of the theoretical information required for quantitative and qualitative analysis of MCDM ranking methods, it also describes the RPI, which is used to compare rankings generated by two or more MCDM methods, and presents step-by-step algorithms for the ARAS and CODAS methods.

Rank-based MCDM approaches can be broadly classified into three categories [8]:

(1) Value measurement methods, such as WSM [15] and the weighted product model (WPM) [16];

(2) Goal or reference-level models, such as TOPSIS [15] and VIKOR [17];

(3) Outranking techniques, such as the Preference Ranking Organization Method for Enrichment of Evaluations (PROMETHEE) [18] and Organisation, Rangement Et Synthèse de données relationnelles (ORESTE) [19].

Mukhametzyanov and Pamucar [2] introduced the 3M model, which combines results obtained using different methods for ranking alternatives. Within this framework, a value measurement model is specified by a triplet comprising an aggregation method ($F$), a normalization method (Norm), and a criterion-weighting method ($W$), as expressed in Eq. (1):

where, the rating $Q_{i}$ of alternative $A_{i}$ (or the performance indicator, or quantitative assessment) is determined by the choice of the decision matrix normalization method ($x_{ij}=\operatorname{Norm}(a_{ij})$), the choice of the criterion significance assessment method $C_{i}$ ($w_j=W(C_1,\ldots,C_n)$) and the choice of the aggregation method $F$ for the normalized values of alternative attributes ($F$).

To identify a model of group 2, it is additionally necessary to define a distance metric in the attribute space to determine the distance from the ideal solution.

To identify a model of group 3, it is additionally necessary to define a preference function for each criterion (attribute), which is analogous to the data normalization procedure (“Norm”) but replaces formal normalization with the construction of decision maker preferences.

It should also be noted that if the deviation from the ideal solution in a Group 2 model is considered as the performance of alternatives, then the Group 1 and 2 models can be classified as belonging to the same category. Study by Mukhametzyanov and Pamucar [2] demonstrates the equivalence of the TOPSIS(Norm, $W$, $L_{1}$) and WSM(Norm, $W$) models when linear normalization methods are used.

According to Eq. (1), the $Q_{i}$ rating is the assessment score of an alternative, formed from partial normalized feature values using the aggregation method and attribute significance. Based on these rating values, the alternatives are ranked, with Rank 1 assigned to the best alternative.

In a simple version, the rank is determined based on the rule “the higher (or lower) the rating, the better”, as expressed in Eq. (2):

where, the “$\succ$” sign indicates that alternative $A_{p}$ is preferred to alternative $A_{q}$.

A more complex ranking method considers whether the ratings of alternatives differ significantly. Alternatives with indistinguishable ratings may be assigned the same rank.

A ranking model requires specification. For example, one of the simplest and most widely used MCDM models is the weighted sum of the normalized feature values of each alternative (WSM), as given in Eq. (3):

Even in this simple case, the ranking directly depends on the choice of weight estimation and normalization method. Both factors constitute the part of the model specification. Therefore, without specifying these methods, the model remains undefined. Examples of specifications for the 3M model include:

WSM(“Max”, “Entropy”)—uses the Max normalization method ($x_{ij}=a_{ij}/a_j^{\max}$) and the entropy weight estimation method.

WSM(“Sum”, “AHP/EV”)—uses the Sum normalization method and analytic hierarchy process (AHP) prioritization and weight estimation with the eigenvector (EV) method. TOPSIS(“Max”, “AHP/LLSM”, $L_{2}$-metric) uses the TOPSIS aggregation method [15], the Max normalization method, AHP prioritization and weight estimation with the logarithmic least squares method (LLSM), and the $L_{2}$ metric for determining the distance from the ideal/anti-ideal.

Aytekin [20] provided an illustrative example showing that, in some problems, changing individual components of the model specification alters both alternative ratings and their ranks. In this example, for the problem of selecting 8 alternatives based on 5 criteria, applying 6 different linear normalization methods results in a change in the top-ranked alternative. It should also be noted that within the normalization framework, a specification is also required for the cost criteria inversion method [21].

Let us compare solutions to the same MCDM problem obtained using two different ranking models. The models produce two ordered lists of ratings, $Q_{i}^{(1)}$ and $Q_{i}^{(2)}$. Ranking (according to the adopted rule) yields two ranking lists, $R_{i}^{(1)}$ and $R_{i}^{(2)}$, $i=1,\ldots,m$. The value of $R_{i}$ determines the number of the alternative of the $i$-th rank. Rank 1 corresponds to the best alternative. In some situations (indistinguishable ratings of alternatives), the ranks of alternatives may coincide.

Unification is achieved using the RPI for an ordered list of alternatives, as defined by Mukhametzyanov and Pamucar [2] [3], and expressed in Eq. (4):

subject to the condition $Q_{1} \ne Q_{m}$, which means that at least two alternatives have different rankings.

According to Mukhametzyanov and Pamucar [2] [3], two MCDM models are equivalent if they produce identical rank lists and the same RPI values, as expressed in Eq. (5) and Eq. (6):

where, $Q_{i}$ is the value of the efficiency indicator corresponding to the alternative of the $i$-th rank (ordered list).

The conditions in Eq. (6) are equivalent to the perfect correlation (according to Pearson) of two rating lists, as expressed in Eq. (7):

or, equivalently, the linear transformation in Eq. (8):

On one hand, Eq. (7) is easier to apply than Eq. (6). On the other hand, we note that the RPI of alternatives $\Delta Q_{i}$ for an ordered list has an important application for analyzing the distinguishability of alternative rankings [2].

Using RPI, Mukhametzyanov and Pamucar [2], [3] established the equivalence of WSM, RS, MABAC, TOPSIS with the $L_1$ metric, MAIRCA, and RAWEC when the same linear attribute normalization technique is applied. The authors subsequently extended this analysis to MARCOS and established its equivalence to WSM under Max normalization and ReS inversion.

This avoids duplicating equivalent methods in the 3M approach. When solving MCDM problems, the WSM is recommended as the simplest and most readily interpretable of these methods.

The ARAS method [12] is positioned in the literature as an effective method for multicriteria analysis.

According to Section 2.1, the basic ARAS method can be specified as WSM(“Sum/iSum”, $W$) for the extended decision matrix $D^{*}[m+1,n]$. The matrix $D^{*}$ is formed from the decision matrix $D[m,n]$ by adding the best values (optimal values) of the features for each criterion as the first row.

In the basic version, the sum normalization method is used for the benefit criteria, as given in Eq. (9):

The inversion of the cost criteria is performed using the nonlinear function iSum, as given in Eq. (10):

The rating of each alternative (including the integrated ideal alternative) is formed as a weighted sum of the normalized features $x_{ij}$.

The final rating of the initial alternatives is normalized by dividing by the rating of the ideal alternative, as expressed in Eq. (11):

The alternative with the higher rating is considered the best.

In the CODAS method, the overall performance of alternatives is determined based on a matrix of feature deviations from the negative ideal solution [13]. The weighted distance from the negative ideal solution in the Euclidean and taxicab metrics is used as the alternatives’ performance. These two intermediate ratings are then combined to obtain the final distance (rating). An alternative with a greater distance from the negative ideal is considered more desirable.

The step-by-step algorithm of the CODAS method is as follows [13]:

1. Construct a decision matrix: a matrix is constructed with alternatives as rows and criteria as columns, with each cell representing the performance of an alternative on a specific criterion:

$D=(a_{ij})$

2. Normalize the data: the data are normalized to bring all criteria into a comparable, dimensionless form, as given in Eq. (12):

3. Calculate the weighted normalized decision matrix using Eq. (13):

4. Determine the negative-ideal solution: this solution is a point representing the worst-case scenario for each criterion, as defined in Eq. (14):

5. Calculate distances: the Euclidean and taxicab distances are calculated for each alternative from the negative-ideal solution.

Euclidean distance: the primary measure, calculated using the standard Euclidean formula in Eq. (15):

Taxicab distance: the secondary measure, calculated as the sum of the absolute differences between the weighted normalized values and the negative-ideal solution, as given in Eq. (16):

6. Combine the distances: the two distances are combined to form a single assessment score. If alternatives are incomparable by Euclidean distance, the taxicab distance is used to make the final decision. Construct the relative assessment matrix, using Eq. (17):

where, $\psi$ denotes a threshold function to recognize the equality of the Euclidean distances of two alternatives, and is defined in Eq. (18):

In this function, $\tau$ is the threshold parameter that can be set by the decision maker. It is suggested to set this parameter at a value between 0.01 and 0.05. If the difference between Euclidean distances of two alternatives is less than $\tau$, these two alternatives are also compared by the taxicab distance. The recommended value is $\tau=0.02$.

It should be noted The threshold function reported by Keshavarz Ghorabaee et al. [13] and reproduced here in Eq. (18) is inconsistent with their verbal description of the threshold rule. This point is discussed in greater detail below in Section 4.3. In our study, for every calculation and simulation, we use the verbal description of the criterion and the corrected Eq. (19) for the valve function $\psi$ in the form:

7. Rank alternatives: the alternative with the greater combined distance score is considered more desirable. Calculate the assessment score (rating) of each alternative, using Eq. (20):

The alternative with the highest rating is the best among all the options. The alternatives are ranked in descending order of their ratings.

The first criticism of the ARAS method concerns data inversion. When combined with the sum normalization method, the basic algorithm utilizes nonlinear data inversion (Eq. (10)). Nonlinear inversion introduces distortions into the original data, or more precisely, into the proportions between feature values. In all cases of linear and nonlinear normalization, the ReS algorithm of inversion is correct [21].

The second criticism concerns the calculation of the final rating of the original alternatives by dividing by the rating of the ideal alternative (Eq. (11)). According to Section 2.2, this transformation does not change the RPI or the ranking, making this operation unnecessary for ranking. The authors may be hoping to obtain a normalized rating for subsequent comparison, but this normalization is inappropriate. The correct approach for comparing ratings obtained by different methods is Max-Min normalization, as discussed by Mukhametzyanov and Pamucar [2].

The third and most important remark concerns the choice of the Sum normalization method (Eq. (9)) and the integration of the optimal alternative. There are no formal criteria for selecting a normalization method, and the choice is made based on principles. Mukhametzyanov [21] provided an adequate classification of normalization methods and a reasonable approach to their selection.

First, we note that if Max/iMax normalization (Eq. (12)) or Max-Min normalization is used, then following Section 2.2, it is easy to see that ARAS is equivalent ($\simeq$) to WSM:

$\operatorname{WSM}(\mathrm{Max/iMax},W)\simeq\operatorname{ARAS}(\mathrm{Max/iMax},W)$, $\operatorname{WSM}(\mathrm{Max\text{-}Min/ReS},W)\simeq\operatorname{ARAS}(\mathrm{Max\text{-}Min/ReS},W)$.

Since the maximum and minimum values do not change when an additional optimal alternative is introduced into the problem, the attribute normalization coefficients do not change as well. They are the same as in the WSM method.

When using sum normalization, the normalization coefficient increases due to the integration of a new alternative, and, importantly, this increase is not uniform for each attribute. This leads to disproportionate changes in attribute values. As a result, the rankings of alternatives change unevenly. Furthermore, the phenomenon of RRP—a change in rank when adding or removing a new alternative—is also possible. Analysis suggests that, in the ARAS method, RRP occurs both when using the linear inversion ReS and the nonlinear inversion iSum. The deviation of rankings in the ARAS method from the WSM method is more pronounced when using nonlinear inversion.

It should be noted that RRP occurs when adding a new alternative because the normalization coefficients of some attributes change disproportionately to the others. This only needs to happen for one attribute. Clearly, the rankings will change. However, it is important to note that this procedure also changes the proportions of the rankings of alternatives. Then, using a linear aggregation function (WSM) or a homogeneous aggregation function (TOPSIS), RRP is possible. It is not surprising that the result depends on the specific situation (the relationship between attribute values), i.e., the task. Therefore, this phenomenon is referred to as a phenomenon—an unusual, special occurrence, a rare fact.

Therefore, the combined process of integrating a new alternative into the decision matrix and choosing a normalization method, in which the normalization coefficients vary differently for different criteria, such as Sum, Vec, Z-score, etc., alters the structure of the original data. Consequently, the evaluations may depend on changes in the normalization coefficients introduced by the additional alternative.

Assuming that the WSM model adequately determines rankings, we conducted a numerical test to determine how the WSM ranks differ from those obtained using the ARAS method. As noted in the previous section, the result depends on the problem (data). Therefore, we investigated the change in ranks using numerical simulations of MCDM problems of the same dimension. The data for the initial problem were taken from the study by Chakraborty and Zavadskas [16] and are given in Eq. (21):

$[m,n]=\operatorname{size}(D_0)=[ 7,5]$;

$w=[ 0.036\quad0.192\quad0.326\quad0.326\quad0.120]$;

Max/Min (benefit/cost): $\mathrm{MM}=[ 1\quad-1\quad1\quad1\quad1]$;

For any finite-dimensional problem, it is easy to calculate ratings and ranks using the WSM and ARAS methods. We will generate a large number of problems of the same dimension based on the matrix $D_0=(a_{ij}^{0})$ such that the values of each attribute $a_{ij}$ belong to the interval $[a_j^{0\min},a_j^{0\max}]$.

The technique for generating such a decision matrix $D$ is based on the generation of uniformly distributed random values. For each attribute, it is necessary to generate $m$ random values ($m$ alternatives) from a given interval of values.

The algorithm is described by Eq. (22):

where, $\operatorname{rnd}(1)$ is a random number from $[ 0,1]$.

Such decision matrices could hypothetically define the same decision-making problem as defined by the matrix $D_0$, but with a different set of alternatives.

For each such decision matrix, a ranking is performed in the WSM(Sum/iSum, $w$) and ARAS(Sum/iSum, $w$) models. The ranking may be the same or different for different methods.

For $N=10{,}000$ simulations, rank 1 alternatives did not match (regardless of other ranks) in 1.8% of problems, i.e., 180 problems; rank 2 alternatives did not match in 4.3% of problems; ranks 1 and 2 alternatives did not match simultaneously in 1.8% of problems; and the ranks of at least one alternative did not match in 13.2% of problems.

It should be noted that the elements of the new matrix are selected independently, as independent random variables are generated. The determining factor is not the random number generator itself, but the value distribution law—which is uniform. Therefore, the generator’s initial value is irrelevant.

In accordance with Eq. (22), the elements of the new decision matrix are uniformly distributed over the interval $[a_j^{0\min},a_j^{0\max}]$. This results in the formation of a different (new) problem, as the priorities for each attribute change. The normalization/inversion method applied at each stage is indicated as Sum/iSum. The weighting coefficients are determined by the initial problem $D_{0}$, since, formally, each problem involves the same criteria as the original one.

Reproducibility of the results is ensured by their statistical robustness regarding the generation procedure based on the initial matrix $D_0$, as defined above. Regardless of the random number generator used, the results are reproducible with an error of no more than 0.5%. This implies that 10,000 generation runs are entirely sufficient for the conclusions of the numerical simulation (numerical experiment). Throughout all calculations, the authors used MATLAB software, version R2017b.

Thus, the numerical experiment showed that introducing an ideal alternative into the decision matrix produces a rank reversal in an average of 13% of cases compared to the WSM method (analogous to RRP). Critical to decision-making is the discrepancy between alternatives of ranks 1 and 2, which accounted for about 1.8% of the total number of generated problems.

These findings suggest that ARAS should be applied with caution, particularly when sum normalization is used and the inclusion of an ideal alternative affects the normalization coefficients. The ARAS method does not utilize any additional information contained in the decision matrix. It is a typical constructive analogue of the WSM, which, among other things, distorts the data relationships. The WSM is therefore recommended as an alternative to ARAS because of its transparent data-transformation procedure and straightforward interpretation of the resulting rankings.

One of the main motivations for analyzing the CODAS method is that Keshavarz Ghorabaee et al. [13] did not explain why they combined two distance measures to calculate the evaluation score for each alternative.

We divide $H$ in Eq. (17) into two parts according to the distance metrics $L_{2}$ and $L_{1}$:

$H = H^{(2)} + H^{(1)}$ with the elements:

\[h_{ik}^{(2)} = E_{i} - E_{k}\]

\[h_{ik}^{(1)} = T_{i} - T_{k}\]

Then

\[ h_{ik}=h_{ik}^{(2)}+\psi_{ik}\cdot h_{ik}^{(1)} \]

where, the valve function $\psi_{ik}$ is either 0 or 1 in accordance with Eq. (19).

It should be noted that if $h_{ik} = h_{ik}^{(2)}$, then, in accordance with Section 2.2, the evaluation score of each option $Q_i=\sum_k h_{ik}^{(2)}$ is equivalent to the evaluation score $Q_{i}^{(2)} = E_{i}$, since $\Delta Q_i=\Delta Q_i^{(2)}$.

This equivalence follows from the property of positive affine transformations described in Section 2.2. Consequently, Eq. (17) does not introduce additional information into the evaluation score of the alternatives. Since $H^{(2)}$ is a matrix and $E_{i}$ is a vector, this additional procedure only complicates the method’s algorithm.

Similarly, if $h_{ik} = h_{ik}^{(1)}$, then, in accordance with Section 2.2, the evaluation score of each option $Q_i=\sum_k h_{ik}^{(1)}$ is equivalent to the evaluation score $Q_i^{(1)}=T_i$, since $\Delta Q_{i} = \Delta Q_{i}^{(1)}$. Thus, Eq. (17) does not introduce additional information into the evaluation score of the alternatives.

Let us illustrate this fact with an example. The initial data for this example are the same as in Example 1 of Section 3.2. The threshold value $\tau$ for the CODAS method is 0.02 in all calculations. The reference ratings calculated using the CODAS method match those reported by Keshavarz Ghorabaee et al. [13]:

$Q=[ 0.5122\quad1.4633\quad1.0715\quad-0.2125\quad-1.8515\quad-1.1717\quad0.1887]$.

The results of rating calculations using the deviations ($E_{i} - E_{k}$) and ($T_{i} - T_{k}$) and without using the deviations, using $E_{i}$ and $T_{i}$, are presented in Table 1. The presented results demonstrate that, although the alternative ratings differ across the calculation variants, the RPIs (Eq. (4)) are consistent.

Therefore, the alternative rankings will always match, and there is no need to complicate the ranking calculation using the ($E_{i} - E_{k}$) and ($T_{i} - T_{k}$) deviation matrices.

If we assume that $\psi_{ik}=1,\ \forall i,k\ (i\ne k)$ and truncate Eq. (17), leaving only the second term, then the estimated score for each option is $Q_i=\sum_k h_{ik}^{(1)}$.

Then we can conclude that the second term is equivalent to the result of applying the WSM model. This equivalence follows from the property of positive affine transformations described in Section 2.2. In the CODAS method, these are two successive linear transformations: Eq. (16) and Eq. (20). We denote this result as $\operatorname{WSM}(\mathrm{iMax})\simeq\operatorname{CODAS}(\mathrm{iMax},L_1)$—the truncated CODAS method with the taxicab distance metric is equivalent to the WSM model. In both cases, Max/iMax normalization is used. Note that from here on, this and other properties hold for any linear normalization and linear inversion ReS of cost attributes. Let us illustrate this conclusion with an example. The initial data for this example are the same as in Example 1 of Section 3.2. The threshold value $\tau$ for the CODAS method is 0.02 in all calculations. The results of ranking calculations using the WSM method, separately in the $L_{2}$ metric: CODAS($L_{2}$), separately in the $L_{1}$ metric: CODAS($L_{1}$), and synthesis using the CODAS method are presented in Table 2.

The result demonstrates the equivalence of the methods: $\operatorname{WSM}(\mathrm{iMax})\simeq\operatorname{CODAS}(\mathrm{iMax},L_1)$. Note that the use of nonlinear normalization for cost attributes distorts the initial data [21].

Eq. (17) of the CODAS method involves summing distances calculated using two metrics. However, when ranking alternatives, the addition to the “Euclidean component” is not applied to all alternatives. The maximum number of summations is $m-1$. The last column of Table 2 shows the number of instances $k(\tau)$—out of six possible instances—in which summation was performed; that is, the number of cases where the condition in Eq. (19) was met. A more detailed analysis of this point is presented below in Section 4.3.

Keshavarz Ghorabaee et al. [13] explain that if the difference between the Euclidean distances of two alternatives is less than $\tau$, the two alternatives are also compared using the taxicab distance. The recommended value for $\tau$ is between 0.01 and 0.05. However, this contradicts Eq. (18), which is interpreted as the opposite: if the difference between the Euclidean distances of two alternatives is greater than $\tau$, these two alternatives are also compared using the taxicab distance. One might assume that this is a technical error, but the control calculations reported by Keshavarz Ghorabaee et al. [13] in Examples 1 and 2 show that they used Eq. (10) of their paper..

Chen and Gou [22] previously noted that when the difference x between the Euclidean distances of two alternatives is greater than or equal to a specified threshold value, the Euclidean distance is considered sufficient to distinguish between the two alternatives, rendering the Hamming distance unnecessary. Based on this, they made fundamental changes to the threshold function reproduced here in Eq. (18), although they described these changes as improvements to the function’s definition rather than as corrections of an error. However, to prevent incorrect results when applying the CODAS method to practical and theoretical MCDM problems, we assert that the valve function defined by Keshavarz Ghorabaee et al. [13] and reproduced here in Eq. (18) is illogical and erroneous.

Another reason for using the taxicab distance may be that if the difference between the Euclidean distances of two alternatives is less than $\tau$, this indicates weak distinguishability of the two alternatives in the Euclidean metric. However, in this case, it is unclear why the value of $\tau$ is between 0.01 and 0.05. Distinguishability can be easily established using the relative (scaled) deviation of the indicator values [3].

Chen and Gou [22] also proposed an improved method employing weighted Hamming distance to compensate for the shortcomings of classical methods.

The problem with synthesis is that the ranks of alternatives in the CODAS($L_{2}$) and CODAS($L_{1}$) metrics do not always coincide, as is directly evident from the calculation results for the example in Table 2. Let us explain why this occurs. When determining the ranking of alternatives, the taxicab term is not added in every pairwise comparison. Eq. (17) of the CODAS method defines the summation of distances in two metrics. The maximum number of summations is $m-1$. The last column of Table 2 shows the number of cases (out of 6) for which the summation was performed, i.e., the condition in Eq. (19) was satisfied. The difference in values indicates that for some alternatives, the assessment is strengthened/weakened to a greater extent than for others. In the example under consideration, the final rating for four of the seven alternatives is calculated as the sum of the Euclidean and Manhattan components, whereas for three alternatives, it is based solely on the Euclidean distance from the anti-ideal. Thus, for some alternatives, the rating is strengthened/weakened to a greater extent than for others.

To test the latter thesis, we analyzed this phenomenon using 10,000 generated problems (see Section 3.2). Figure 2 presents the results of this numerical analysis. The threshold value $\tau$ for the CODAS method is 0.02 in all calculations.

In the CODAS algorithm, the summation of the “Euclidean” and “Taxicab” distance ratings does not occur for all elements of matrix $H$. Figure 2 shows the percentage of the total number of problems (10,000) on the x-axis. The y-axis shows the average percentage of switch activations (relative to the maximum possible)—that is, the percentage of the total number ($m^{2} - m$) of elements $h_{ik}$, for which summation takes place. For example, according to Figure 2, with $m=6$, the summation of the “$E+T$” rating occurred for an average of 25% of the elements $h_{ik}$ in 18–22% of the problems; this corresponds to 7.5 instances, or an average of one instance per alternative.

Figure 2 demonstrates that the number of switch activations for calculating the rating based on the two metrics varies across both alternatives and problems.

Another issue is that the additive component in the rating calculation is applied unevenly across different alternatives. In other words, the rating for some alternatives is calculated differently than for others. This process is embedded within the CODAS algorithm and remains hidden from the decision-maker, precluding potential oversight.

In the numerical experiment, we record the number of switch activations for each alternative in each experiment. This value varies within the interval $[ 0,m-1]$ and is defined in Eq. (23):

where, $j$ is the experiment number, and $N$ is the number of generated problems.

To characterize the unevenness of the “Taxicab” component’s contribution to the rating, we use the range of the number of switch activations across all alternatives in each experiment, as defined in Eq. (24):

Based on the statistic in Eq. (24), we group the results $K(j)$ into $m$ value groups $G=\{0,1,2,\ldots,m-1\}$, where 0 indicates that the number of switch activations across all alternatives was identical, while the limiting value of $m-1$ indicates the maximum difference in the number of activations of the valve function across alternatives within a single experiment. The results for the numerical experiment shown in Figure 2 ($N=10{,}000$) are presented in Table 3.

Thus, the Euclidean and taxicab components are combined differently across comparisons, which may affect the resulting scores and rankings.

Keshavarz Ghorabaee et al. [13] may have believed that calculating rankings based on pairwise comparisons increases the method’s informativeness compared with methods such as WSM. However, in Section 4.1, we demonstrated the invariance of the final result.

The key premise of the CODAS method is to combine two distances to calculate the evaluation score of each option (alternative). However, our analysis showed that when determining the ranking of alternatives, the taxicab term is not added in every pairwise comparison. The numerical analysis in Section 4.3 illustrates how the contribution of the taxicab term can vary across alternatives, with possible effects on their relative scores and rankings.

These findings suggest that CODAS should be applied with caution, with particular attention to the threshold rule and its influence on alternative scores and rankings. An alternative to the CODAS method is to use the original WSM model, which has a reliable basis for data transformation and a simple interpretation of the ranking.

The trend towards synthesizing solutions obtained from multiple MCDM models is based on the premise that different methods capture different relationships between alternatives, thereby increasing the overall information content and potentially improving the reliability of the results. This premise motivates the identification of a set of suitable methods for solving a specific problem. These methods should be independent and reflect different relationships between alternatives. One approach to the quantitative analysis of MCDM methods is based on the RPI.

This study examined the ARAS and CODAS methods in the context of MCDM method selection. Because these methods may produce different rankings as a result of implicit and potentially ambiguous effects arising from their algorithmic formulations, their use in MCDM problems is not recommended without careful evaluation of their limitations. The WSM is proposed as an alternative to ARAS and CODAS because of its transparent data-transformation procedures and straightforward interpretation of alternative ratings.

The authors specifically highlight the error in using nonlinear inversion of cost criteria in MCDM methods. A universal (applicable to all normalization methods) ReS algorithm for linear inversion has been described in detail in the scientific literature since 2019.

Further research should evaluate the independence, informativeness, and correctness of MCDM methods used to synthesize solutions based on multiple models.