Scriminaci, M., Lommatzsch, A., Kille, B. et al. (5 more authors) (2016) Idomaar : a framework for multi-dimensional benchmarking of recommender algorithms. In: Guy, I. and Sharma, A., (eds.) Proceedings of the Poster Track of the 10th ACM Conference on Recommender Systems (RecSys 2016). 10th ACM Conference on Recommender Systems (RecSys 2016), 17 Sep 2016, Boston, USA. CEUR Workshop Proceedings
Abstract
In real-world scenarios, recommenders face non-functional requirements of technical nature and must handle dynamic data in the form of sequential streams. Evaluation of recommender systems must take these issues into account in order to be maximally informative. In this paper, we present Idomaar—a framework that enables the efficient multi-dimensional benchmarking of recommender algorithms. Idomaar goes beyond current academic research practices by creating a realistic evaluation environment and computing both effectiveness and technical metrics for stream-based as well as set-based evaluation. A scenario focussing on “research to prototyping to productization” cycle at a company illustrates Idomaar’s potential. We show that Idomaar simplifies testing with varying configurations and supports flexible integration of different data.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2016 The Authors. This is an author-produced version of a paper subsequently published in Proceedings of the Poster Track of the 10th ACM Conference on Recommender Systems (RecSys 2016). Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 14 Jun 2021 11:18 |
Last Modified: | 14 Jun 2021 11:18 |
Published Version: | http://ceur-ws.org/Vol-1688/ |
Status: | Published |
Publisher: | CEUR Workshop Proceedings |
Refereed: | Yes |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:175097 |