Gil-Gonzalez, J., Giraldo, J.-J., Alvarez-Meza, A.M. et al. (2 more authors) (2021) Correlated chained Gaussian processes for datasets with multiple annotators. IEEE Transactions on Neural Networks and Learning Systems, 34 (8). pp. 4514-4528. ISSN 2162-237X
Abstract
The labeling process within a supervised learning task is usually carried out by an expert, which provides the ground truth (gold standard) for each sample. However, in many real-world applications, we typically have access to annotations provided by crowds holding different and unknown expertise levels. Learning from crowds (LFC) intends to configure machine learning paradigms in the presence of multilabelers, residing on two key assumptions: the labeler's performance does not depend on the input space, and independence among the annotators is imposed. Here, we propose the correlated chained Gaussian processes from the multiple annotators (CCGPMA) approach, which models each annotator's performance as a function of the input space and exploits the correlations among experts. Experimental results associated with classification and regression tasks show that our CCGPMA performs better modeling of the labelers' behavior, indicating that it consistently outperforms other state-of-the-art LFC approaches.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Multiple annotators; Correlated Chained Gaussian Processes; Variational inference; Semi-parametric latent factor model |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Sciences Research Council EP/R034303/1; EP/T00343X/1 Rosetrees Trust n/a |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 22 Oct 2021 06:14 |
Last Modified: | 24 Jun 2024 15:27 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Refereed: | Yes |
Identification Number: | 10.1109/tnnls.2021.3116943 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:179506 |