To mix or not to mix? A meta-method approach to rethinking evaluation practices for improved effectiveness and efficiency of psychological therapies illustrated with the application of perceptual control theory

Progress in the development of more effective and efficient psychological therapies could be accelerated with innovative and nuanced approaches to research methodology. Therapy development has been dominated by a mono-methodology attitude with randomized controlled trials (RCTs) regarded as a “gold standard” despite the concept of a single methodology being ascribed gold standard status having been called into question. Rather than one particular methodology being considered superior to all others, the gold standard approach should be matching appropriate methodologies to important research questions.

The way in which that matching should occur, however, is far from clear. Moving from a mono-methodological approach to mixed-method designs has not been straightforward. The ways in which methods should be mixed, to arrive at robust and persuasive answers to genuine research questions, is not entirely clear. In this paper, we argue that attention

to the meta-methods underpinning all research designs will improve research precision and provide greater clarity about the contribution of any particular program of research to scientific progress in that field. From a meta-method perspective, the matter of what changed can be delineated from why or how these changes occurred. Different methods

and different types of mixing can be justified for each meta question. A meta-method approach should make explicit the assumptions that guide the development of research designs and also promote the articulation of putative mechanisms that might be relevant. By paying greater attention to assumptions such as how causality occurs, and important

mechanisms of change, the mixing of methodologies that are still not mainstream in this area such as routine outcome monitoring and evaluation and functional model building, can occur. By adopting methodologies that focus on learning about a program’s strengths

and weaknesses rather than presiding over judgments of whether or not the program is deemed to be effective, we will move much closer to a position of being able to understand what programs under which conditions people find most helpful for their purposes.