Hopfgartner, F. orcid.org/0000-0003-0380-6088, Gurrin, C. and Joho, H. (2019) Rethinking the test collection methodology for personal self-tracking data. In: Cheng, W.-H., Kim, J., Chu, W.-T., Cui, P., Choi, J.-W., Hu, M.-C. and De Neve, W., (eds.) Proceedings of the 26th International Conference on Multimedia Modeling, MMM2020. 26th International Conference on Multimedia Modeling 2020, 05-08 Jan 2020, Daejeon, South Korea. Lecture Notes in Computer Science, 11962 . Springer , pp. 463-474. ISBN 9783030377335
Abstract
While vast volumes of personal data are being gathered daily by individuals, the MMM community has not really been tackling the challenge of developing novel retrieval algorithms for this data, due to the challenges of getting access to the data in the first place. While initial efforts have taken place on a small scale, it is our conjecture that a new evaluation paradigm is required in order to make progress in analysing, modeling and retrieving from personal data archives. In this position paper, we propose a new model of Evaluation-as-a-Service that re-imagines the test collection methodology for personal multimedia data in order to address the many challenges of releasing test collections of personal multimedia data.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2020 Springer Nature Switzerland AG. This is an author-produced version of a paper subsequently published in Cheng WH. et al. (eds) MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | lifelogging; evaluation; self-tracking |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 29 Oct 2019 14:16 |
Last Modified: | 24 Dec 2020 01:38 |
Status: | Published online |
Publisher: | Springer |
Series Name: | Lecture Notes in Computer Science |
Refereed: | Yes |
Identification Number: | 10.1007/978-3-030-37734-2_38 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:152523 |