Agenda

Title: Refining the performance metrics and related mathematical operators for ordinal classification problems

Abstract: In multilabel classification (MLC) problems, the standard confusion matrix is generally replaced by a set of confusion matrices, with one matrix per label. Although ordinal classification problems are slightly different from MLC, it shares the same set of performance metrics used for MLC. We contend that ordinal classification has nuances which makes it different from MLC. As in binary classification, the notion of false positive and false negative are still valid in MLC problems in a sense that any wrongly predicted label can be termed as false positive (or false negative). However, in ordinal classification, these notions are replaced by overestimates and underestimates. We propose the use of different mathematical operators to quantify the strength of overestimates and underestimates. The proposed metrics tailored for ordinal classifiers not only introduces a novel framework for evaluation but also signifies a crucial step forward in enhancing the performance evaluation of ordinal classification algorithms. We demonstrate its practical usefulness with the help of a number of case studies.

Bio: Dr Siraj is working as Associate Professor in Business Analytics at Leeds University Business School. He has worked as a design engineer and software developer in the technology sector, primarily focusing on machine learning and decision support. He did his PhD in Computer Science from The University of Manchester focusing on the field of Decision Support Systems, and has since then worked on various projects with industry. Some of his recent projects include the identification of musical talent using machine learning techniques; enhancing customer experience of a transit bus system in the UK; improving and optimising the customer returns processes for one of the top retail companies in the UK. He has also developed a number of software solutions for decision making. One of his open-source software tools is called Priority Estimation Tool (or PriEsT), which been downloaded more than 50,000 times around the globe, and has helped numerous organisations in their decision making exercises.