Elsevier

Applied Soft Computing

Volume 32, July 2015, Pages 468-480
Applied Soft Computing

A new fuzzy peer assessment methodology for cooperative learning of students

https://doi.org/10.1016/j.asoc.2015.03.056Get rights and content

Highlights

  • A new fuzzy peer assessment methodology is proposed.

  • A synthesis of perceptual computing and a fuzzy ranking algorithm.

  • Providing both crisp scores and recommendations.

  • A case study in Universiti Malaysia Sarawak is reported.

  • An illustrative example is adopted to detect possible free riders.

Abstract

In this paper, a new fuzzy peer assessment methodology that considers vagueness and imprecision of words used throughout the evaluation process in a cooperative learning environment is proposed. Instead of numerals, words are used in the evaluation process, in order to provide greater flexibility. The proposed methodology is a synthesis of perceptual computing (Per-C) and a fuzzy ranking algorithm. Per-C is adopted because it allows uncertainties of words to be considered in the evaluation process. Meanwhile, the fuzzy ranking algorithm is deployed to obtain appropriate performance indices that reflect a student's contribution in a group, and subsequently rank the student accordingly. A case study to demonstrate the effectiveness of the proposed methodology is described. Implications of the results are analyzed and discussed. The outcomes clearly demonstrate that the proposed fuzzy peer assessment methodology can be deployed as an effective evaluation tool for cooperative learning of students.

Graphical abstract

The proposed Fuzzy peer assessment methodology.

  1. Download : Download full-size image

Introduction

Cooperative learning is an educational approach or strategy in which students work in small groups to help each other to learn academic content [1]. The importance of cooperative learning in engineering disciplines has been explained and highlighted in [2], [3], [4]. Cooperative learning plays an important role to improve students’ soft skills, e.g., communication and teamwork, which is the essence in engineering studies [2]. While cooperative learning offers remarkable benefits (e.g., improving collaborative and critical thinking skills) to students at the tertiary level (i.e., the third stage of learning after graduating from the secondary school) [1], it is yet to be widely adopted owing to a number of practical challenges [5]. A search in the literature reveals that the assessment of an individual student in a group is not an easy task, since a group mark is often not a clear and fair reflection of each individual's effort [5], [6], [7]. Besides that, it is difficult for an instructor to closely monitor each student's efforts in a group; therefore it is not suitable for the instructor to assess each student's contribution [8], [9]. To tackle these challenges, peer assessment has been introduced to evaluate each student's contribution in a group work [5], [6], [7], [8], [9], [10], [11], [12], [13]. Several successful case studies in peer assessment have been reported, e.g., in civil engineering [5], biological sciences [10], primary mathematics education [11], and computer studies [13]. In addition, substantial evidence to show that peer assessment can lead to improvements in quality and effectiveness learning is available [14].

Generally, there are two types of peer assessment [10]: (i) involving students in a class to assess other students’ work; (ii) involving students to assess the contribution/performance of other students within the same group. These two types of peer assessment can be further classified into two; i.e., formative or summative assessment [15], [16]. The goals of formative assessment are to monitor students’ learning capabilities, gather their ongoing feedbacks, and improve their learning experience [15], [16]. On the other hand, summative assessment evaluates students’ learning capabilities at the end of an instructional unit [15], [16]. Typically, an instructor needs to decide whether to use the formative or summative form of peer assessment [14]. This paper focuses on summative assessment, which focuses on the outcome of a learning process [9]. The procedure for summative assessment is further detailed in Section 2.4.

Traditionally, the Likert scale (a numerical grading scale) is used in a way equivalent to psychological measurement [6], [7], [10], [12]. As an example, a numerical grading scale (e.g., 1 to 5) can be used for assessment of group members [6], [7], [12], whereby “1” indicates “didn’t contribute”, “2” indicates “willing but not successful”, “3” indicates “average”, “4” indicates “above average”, and “5” indicates “outstanding” [6], [7], [12]. Even though the use of numerals in peer assessment is popular, it suffers from problems associated with psychological measurement in terms of the meaning pertaining to the numerals used (see [17] for a study on the theoretical relationship between measurement and marking). It would be more natural to define assessment grades using subjective and vague linguistic terms [17]. Furthermore, the conventional method aggregates individual scores to produce a total score. In some situations (as illustrated in Section 4.3), it is difficult to distinguish the ranking order of students using the same numeral score.

Fuzzy set theory has been used in education assessment [9], [18], [19], [20], [21], [22], [23]. It is useful to deal with linguistic grades such as “didn’t contribute”, “willing but not successful”, and “average” in a grading system, which involve a substantial amount of fuzziness and vagueness [18]. It is worth mentioning that fuzzy set theory is an efficient and effective method to represent uncertainties [18], [19]. Comparing with methods based on numerical grading scores [6], [7], [12], fuzzy set theory offers an alternative to linguistic evaluation in which “fuzzy” words, instead of numerals, are used during the assessment procedure [9], [18], [19], [20], [21], [22], [23]. Besides that, “computing with words”, as coined by Zadeh, is also a methodology related to fuzzy set theory, whereby the objects of computation are words and propositions drawn from a natural language [24], [25].

Motivated by the success of fuzzy set theory in education assessment [9], [18], [19], [20], [21], [22], [23], this paper aims to propose a fuzzy peer assessment methodology that evaluates each student's contribution in a group work. The proposed methodology is a synthesis of perceptual computing (Per-C) [25], [26], [27], [28], [29] and a fuzzy ranking algorithm that uses fuzzy preference relations [30]. The rationale for the proposed methodology hinges on a number of imperatives. Firstly, the available information is too imprecise to be justified with numerals, which is more suitable to be represented using words [25]. In this paper, Per-C is adopted owing to its effectiveness in handling inherent uncertainties in words [25]. Specifically, Per-C is able to handle subjectivity, vagueness, imprecision, and uncertainty while achieving tractability and robustness in modelling human decision-making behaviours [25], [26], [27], [28], [29]. Comparing with type-1 fuzzy models [9], [18], [19], [20], [21], [22], [23], Per-C adopts interval type-2 fuzzy sets (IT2FSs) in tackling a decision making problem [25], [26], [27], [28], [29]. IT2FS has more flexibility in preserving and processing uncertainties than type-1 fuzzy set [25]. Indeed, Per-C has been successfully implemented to undertake a number of fuzzy multiple criteria hierarchical decision making problems [25], [26], [27], [28], [29]. In [25], [26], [27], [28], [29], Per-C focuses on ranking the sequence of outcomes, mapping the outcomes into words and/or classifying the outcomes into different categories. Nevertheless, the use of Per-C in peer assessment is still new. In this paper, the relative importance of the outcomes, i.e., the contribution of each student with respect to those from other students, is examined in detail, which is yet to be investigated in the literature, e.g. [25], [26], [27], [28], [29]. In this aspect, our preliminary work [30], as discussed in Section 2.3, is further extended to serve this purpose. Then, the effectiveness and practicality of the proposed methodology are evaluated with a case study in an engineering course (i.e., Multiprocessors Architecture) at Universiti Malaysia Sarawak. The results from the conventional and proposed methodologies are analyzed and discussed. In essence, this paper contributes to a new fuzzy peer assessment methodology in which human linguistic words are adopted in the entire assessment process. Besides that, the proposed methodology provides an insight pertaining to each individual's contribution; therefore providing personalized assessment in a cooperative learning environment.

The rest of this paper is organized as follows. In Section 2, the background of fuzzy sets, perceptual computing, fuzzy ranking algorithms and peer assessment in problem-based learning is presented. In Section 3, a new technique for fuzzy peer assessment is explained in detail. In Section 4, a case study is conducted to demonstrate the usefulness of the proposed fuzzy peer assessment methodology. Concluding remarks and suggestions for future research are presented in Section 5.

Section snippets

Preliminaries

A number of notations and definitions related to type-1 fuzzy sets (T1FSs) and interval type-2 fuzzy sets (IT2FSs) are presented in Section 2.1. A review on perceptual computing is presented in Section 2.2. Our preliminary work related to a fuzzy ranking algorithm is reviewed in Section 2.3. Finally, an overview on peer assessment in problem-based learning is presented in Section 2.4.

A new methodology for fuzzy peer assessment

In this section, the proposed new methodology for fuzzy peer assessment is described. Fig. 6 shows the overall fuzzy peer assessment framework. In this paper, the focus is on enhancing the 5th and 6th stages of the conventional methodology (as illustrated in Fig. 4) by using the Per-C paradigm [25], [26], [27], [28], [29]. The details of the proposed methodology are summarized in stages 5.1–5.4 and stage 6.0, as follows.

A case study

In this section, a real case study is conducted. A variety of aspects from the initial survey to the end results are discussed and analyzed.

Conclusions

In this paper, a new fuzzy peer assessment methodology is proposed. The proposed methodology is the synthesis of Per-C and a fuzzy ranking algorithm that uses fuzzy preference relations. Per-C is able to provide assessment pertaining to students’ contributions in term of recommendations in words. Meanwhile, the fuzzy ranking algorithm is able to provide indices that reflect students’ contributions in a group, and subsequently rank them accordingly. These indices are employed to derive students’

Acknowledgements

The financial support of the FRGS grant (i.e., FRGS/1/2013/ICT02/UNIMAS/02/1) and RACE grant (i.e., RACE/F2/TK/UNIMAS/5) is gratefully acknowledged.

References (38)

  • Y.M. Wang et al.

    Risk evaluation in failure mode effects analysis using fuzzy weighted geometric mean

    Expert Syst. Appl.

    (2009)
  • J.J. Wang et al.

    Review on multi-criteria decision analysis aid in sustainable energy decision making

    Renew. Sustain. Energy Rev.

    (2009)
  • R.E. Slavin

    Cooperative Learning and Cognition in Education

    (2011)
  • J.E. Mills et al.

    Engineering education – is problem-based or project-based learning the answer?

    Australas. J. Eng. Educ.

    (2003)
  • L. Springer et al.

    Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis

    Rev. Educ. Res.

    (1999)
  • P.C. Wankat et al.

    The scholarship of teaching and learning in engineering

    Disciplinary Styles in the Scholarship of Teaching and Learning: Exploring Common Ground

    (2002)
  • K.P. Nepal

    An approach to assign individual marks from a team mark: the case of Australian grading system at universities

    Assess. Eval. High. Educ.

    (2012)
  • R. Conway et al.

    Peer assessment of an individual's contribution to a group project

    Assess. Eval. High. Educ

    (1993)
  • L.K.Y. Lim

    Some refinements on peer assessment of group projects

    Assess. Eval. High. Educ

    (2001)
  • Cited by (0)

    View full text