Demartini, G. (2015) Hybrid human-machine information systems: Challenges and opportunities. Computer Networks, 90. pp. 5-13. ISSN 1389-1286
Abstract
Micro-task Crowdsourcing has been used for different purposes: creating training data for machine learning algorithms, relevance judgments for evaluation of information systems, sentiment analysis, language translation, etc. In this paper we focus on the use of crowdsourcing as core component of data-driven systems. The creation of hybrid human–machine systems is a highly promising direction as it allows leveraging both the scalability of machines over large amounts of data as well as keeping the quality of human intelligence in the loop to finally obtain both efficiency and effectiveness in data processing applications.
Such a hybrid approach is a great opportunity to develop systems that are more powerful than purely machine-based ones. For example, it is possible to build systems that can understand sarcasm in text at scale. However, when designing such systems it is critical to take into account a number of dimensions related to human behavior as humans become a component of the overall process.
In this paper, we overview existing hybrid human–machine systems presenting commonalities in the approaches taken by different research communities. We summarize the key challenges that one has to face in developing such systems as well the opportunities and the open research directions to make such approaches the best way to process data in the future.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2015 Elsevier. This is an author produced version of a paper subsequently published in Computer Networks. Uploaded in accordance with the publisher's self-archiving policy. Article available under the terms of the CC-BY-NC-ND licence (https://creativecommons.org/licenses/by-nc-nd/4.0/) |
Keywords: | Human computation; Crowdsourcing; Database; Semantic web; Information retrieval |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 09 Jun 2015 14:51 |
Last Modified: | 09 Jul 2016 16:18 |
Published Version: | http://dx.doi.org/10.1016/j.comnet.2015.05.018 |
Status: | Published |
Publisher: | Elsevier |
Refereed: | Yes |
Identification Number: | 10.1016/j.comnet.2015.05.018 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:86863 |