Sanchez, Beatriz A., Barmpis, Konstantinos, Neubauer, Patrick orcid.org/0000-0002-9811-4772 et al. (2 more authors) (2018) Restmule:Enabling resilient clients for remote APIs. In: Proceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018. 15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018, 28-29 May 2018 IEEE Computer Society , SWE , pp. 537-541.
Abstract
Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2018 Copyright held by the owner/author(s). Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Keywords: | HTTP API clients,openAPI specification,resilience |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 30 Aug 2018 08:50 |
Last Modified: | 11 Jan 2025 00:11 |
Published Version: | https://doi.org/10.1145/3196398.3196405 |
Status: | Published |
Publisher: | IEEE Computer Society |
Identification Number: | 10.1145/3196398.3196405 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:135104 |