Theory and Methodology
Eliciting and mapping qualitative preferences to numeric rankings in group decision making

https://doi.org/10.1016/S0377-2217(98)00081-2Get rights and content

Abstract

Group work is becoming the norm in organizations. From strategy planning committees to quality management teams, organizational members are collaborating on problem solving. One area of team support that is often desired is the scoring and ranking of decision alternatives on qualitative/subjective domains, and the aggregation of individual preferences into group preferences. In this paper we present a new conceptual approach to qualitative preference elicitation and aggregation. This approach is based on well established decision analysis techniques. It significantly advances the state of the art of group decision making by addressing four common limitations: (1) the inability to deal with vagueness of human decision makers in articulating preferences; (2) difficulties in mapping qualitative evaluation to numeric estimates; (3) problems in aggregating individual preferences into meaningful group preference; and (4) the lack of simple user friendly techniques for dealing with a large number of decision alternatives. Our approach is easy to implement in stand alone personal computers and groupware. We illustrate this with a real-world problem.

Introduction

From strategy planning committees to quality management teams, organizational members are collaborating on problem solving. Flexible organizations, in the form of teams consisting of individuals with diverse skills from different disciplines, seem to be the preferred approach for dealing the complexity and diversity of everyday business problems (Finholt and Sproull, 1990). In response to these changes in the organization of work, researchers and developers are building and studying a new class of computer systems, called groupware. Early groupware focused exclusively on group decision making (Huber, 1984; Kull, 1982); but more recently other types have emerged that target a range of coordination and collaboration activities. Recent studies have also shown that some groupware features: (1) the evaluation of decision alternatives, (2) voting (Kull, 1982; Nunamaker et al., 1989); and (3) group memory retention (Grohowski et al., 1990; Finholt and Sproull, 1990) contribute to group process gains in groupwork. And although, much research has been conducted on the use of groupware, little attention is given to improving voting and evaluation techniques currently implemented in groupware.

A common activity of group work is evaluating and deciding upon various decision alternatives. Take for example, the situation in which a team must evaluate a set of proposals and agree upon one or a subset of the alternatives for implementation. Each member of the team is required to evaluate all proposals based on some set of criteria (some of which maybe qualitative) and to provide a composite score for each proposal. The scores are then analyzed and merged into a “group rank” for each proposal, then final decision is made. The expression of individual preferences among a set of decision alternatives may appear to be simple and straight forward on the surface, but there are several issues that must be addressed. In this paper, we present an approach to preference evaluation that involves a multi-stage qualitative discriminant process. Our approach extends evaluation techniques currently used in groupware, and overcomes their main limitations that have been identified in the literature. It offers simple techniques for: (1) eliciting preferences from users of diverse backgrounds; (2) mapping qualitative evaluations to numeric estimates; (3) analyzing data relevant to evaluating consensus formation; (4) easy implementation in manual and computer supported group activities.

Section snippets

Preference elicitation and evaluation techniques

Current preference elicitation and evaluation numerical techniques fall into one of four general categories: (1) point estimates on interval scales; (2) point estimates on ratio scales; (3) interval estimates on ratio scales; and (4) interval estimates on interval scales. More recently, point estimate techniques have been criticized for the following limitations: (1) They do not address the fuzziness which is characteristic of many human decision making problems (Bryson et al., 1995; Weber, 1987

The qualitative discriminant process

The approach presented here provides a process and a structured comparison technique for making qualitative distinctions among decision alternatives, which overcome the four main limitations of current groupware techniques. It also facilitates the mapping of the qualitative distinctions to numeric estimates from vague real numbers (VRN, cf. Parik, 1983). Our approach falls within and area of measurement theory concerned with preference elicitation and representation (Krantz, 1968; Roberts, 1979

The general procedure

A tree structure (see Fig. 1), of qualitative categories is used to discriminate among the objects. Let B be the bucket that contains the set of all objects before ranking, and Bi, Bij, and Bijk be the buckets for objects assigned to Qi, Qij, and Qijk respectively. Then, B is the root of the tree and applies to the set of objects, while Bi, Bij, and Bijk, represent successive levels. Let B* be the bucket that contains the ordered set of all objects after ranking. Although we have defined only

Case illustration

The information management steering committee of Midwest American Manufacturing Corp. (MAMC), which comprises (1) the Chief Executive Officer, (2) the Chief Information Officer, and (3) the Chief Operating Officer, must prioritize for development and implementation a set of ten information technology improvement projects, which have been proposed by area managers. The committee is concerned that the projects are prioritized from highest to lowest potential contribution to the firm's strategic

Conclusions

We have presented a qualitative discriminant process for scoring and ranking in groupware. The QDP addresses four major limitations of current systems. These are: (1) the inability to deal with vagueness in human decision making; (2) difficulties in mapping qualitative evaluation to numeric estimates; (3) problems in aggregating individual preferences into meaningful group preference; and (4) the inability to deal with a large number of decision alternatives. The QDP was developed to support

References (27)

  • J.S. Dyer et al.

    Group preference aggregation rules based on strength of preference

    Management Science

    (1979)
  • T. Finholt et al.

    Electronic groups at work

    Organization Science

    (1990)
  • S.T. Goddard

    Ranking in tournaments and group decision making

    Management Science

    (1983)
  • Cited by (42)

    • K-means clustering for the aggregation of HFLTS possibility distributions: N-two-stage algorithmic paradigm

      2021, Knowledge-Based Systems
      Citation Excerpt :

      The generation of complex and flexible qualitative evaluation data sets occurs frequently in group-decision-making contexts, especially when a significant number of experts are involved [1–3].

    • Generalized asymmetric linguistic term set and its application to qualitative decision making involving risk appetites

      2016, European Journal of Operational Research
      Citation Excerpt :

      Meanwhile, our daily life is repeated with qualitative information, which could be considered a database of information as it closely resembles human subjective cognition. Therefore, many related QDM theories and methods have been proposed and applied (Brugha, 1998; Sabbadin, Fargier, & Lang, 1998; Ngwenyama & Bryson, 1999; Türkşen, 2002; Paul et al., 2002; Lee & Hurst, 2007; Willow, 2007; Montibeller & Belton, 2009; Wu, Zhang, Wu, & Olson, 2010; Ramey, 2013; Liao, Xu, Zeng, & Merigó, 2015; Cabrerizo, 2015). The complexity of objectives, the fuzziness of human thought, and the uncertainty associated with the qualitative information have led to the proposal of a specific technique allowing computing with words (CWW) by Zadeh (1975, 1996) and its subsequent application to the area of QDM (Rubin, 1998; Herrera & Martinez, 2000; Ying, 2002; Herrera, Alonso, Chiclana, & Herrera-Viedma, 2009; Zadeh, 2012; Yager, 2014; Massanet et al., 2014).

    • Graph-based multi-agent decision making

      2012, International Journal of Approximate Reasoning
    • A hybrid Delphi-SWOT paradigm for oil and gas pipeline strategic planning in Caspian Sea basin

      2012, Energy Policy
      Citation Excerpt :

      In spite of the potential for the Caspian states to meet the increasing global demand for energy resources, only a few Caspian oil and natural gas export projects have become operational in the region over the last decade (Shaffer, 2010). Bilgin (2007, 2010), Guliyev and Akhrarkhodjaeva (2009), Kakachia (2011), Newnham (2011), Pasquarè et al. (2011), Shaffer (2010) and Umbach (2010) have introduced a large number of factors that has played a significant role in shaping the Caspian energy developments. These factors take into consideration political (Russian influence in the region), economical, social and geological issues.

    View all citing articles on Scopus
    View full text