Testing software services in cloud ecosystems

Testing in the Cloud is far more challenging than testing individual software services. A multitude of factors affect testing, including variations across platforms and infrastructure. Architectural issues include differences between private, public Clouds, multi-Clouds and Cloud-bursting. Platform issues include cross-vendor incompatibility, and diverse locales of service deployment and consumption. Software issues include integration with third-party services, the desire to validate competing service offerings to similar standards and need to re-validate services at different stages of service lifecycle. A complete approach to testing whole Cloud ecosystems should involve all relevant stakeholders, such as service provider, consumer and broker. When testing Clouds, the methodologies used should not hinder the advantages Cloud usage brings to the users or programmers and more importantly be simple and cost effective. However, these testing methodologies differ according to the various kinds of Cloud ecosystems and the different user perspectives of the actors involved such as the end-user, the infrastructures, or the different software (i.e. web services). This paper also studies the state-of-the-art in Cloud testing where most research focuses predominantly on web services, functional testing and quality-of-service, usually being considered separately. The authors suggest a framework, Quality-as-a-Service (QaaS) which integrates quality issues such as functional behaviour and performance monitoring with lifecycle governance and security of the service. This paper maps out the themes in the contemporary research literature and links them with the service lifecycle process for validating future Cloud services. Along the way, the authors identify important research questions that the future Cloud service testing agenda should seek to address.