A systematic review of economic evaluations alongside studies within a trial (SWATs) for improving recruitment and retention in randomised controlled trials

Aim To review the cost-effectiveness of strategies to improve participant recruitment and retention in randomised controlled trials. Methods All included studies from the latest Cochrane recruitment and retention reviews were considered. To identify articles published since the Cochrane reviews, electronic databases were searched until March 2021. Hand searching of conference databases and journals was also undertaken. The inclusion criteria included Studies within a Trial (SWATs). The main outcome was the incremental cost-effectiveness ratio (ICER). Quality assessment of papers used the Cochrane risk of bias 1 tool. The CRD guidance was used to assess the quality of economic evaluation. Random-effect meta-analyses were undertaken. The GRADE certainty of evidence was applied for each strategy, and Trial Forge Guidance 2 was used for strategies included in meta-analyses to evaluate the uncertainty of the findings. Cost-effectiveness ranks summarise the cost-effectiveness of all strategies. Results We identified 6569 records and included 29 SWATs (earliest conducted in 1999 and latest in 2021) including more than 35,800 participants. There is no strategy we would recommend trial teams and researchers adopt with complete statistical certainty. Recruitment strategies which could be cost-effective include financial incentives, trial-branded pens, telephone reminders and pre-notification leaflets. Retention strategies which could be cost-effective include vouchers and trial-branded pens. Conclusion Future SWATs should replicate existing recruitment and retention strategies, rather than evaluate novel ones. We recommend that economic evaluations be carried out alongside all future SWATs, costs and benefits be recorded transparently, and the cost-effectiveness of existing recruitment or retention strategies be evaluated.


Introduction
Recruitment of participants into randomised controlled trials (RCTs) is usually poor. 1 Under-recruited, and hence under-powered, trials result in research waste. 2 Another main challenge with RCTs is attrition, which occurs when recruited participants fail to complete follow-up assessments. A systematic review of 151 trials associated with the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme has found the median retention rates to be 89%. 3 Poor retention not only diminishes the power of the trial but also can introduce attrition bias, thus threatening the statistical analysis of RCTs. 4 Studies Within A Trial (SWATs) are a study design for identifying strategies to improve recruitment and retention in RCTs. 5,6 SWATs' primary objective is to improve trial methodology and efficiency. 6 Two systematic reviews have appraised the evidence on the effectiveness of strategies for improving recruitment 2 and retention 7 in RCTs. Both reviews have implied poor progress on identifying effective recruitment/retention strategies.
Moreover, no appraisal of the evidence on the costeffectiveness of recruitment and retention strategies has been undertaken so far. Given the anticipated high direct and indirect costs of poor recruitment and retention rates, 8,9 economic evaluations of recruitment and retention strategies alongside SWATs are useful. More broadly, an "economic evaluation offers an organised consideration of the range of possible alternative courses of action and the evidence of the possible effects of each. This is more likely to lead to better decisions that improve overall social value". 10 There is an urgent need to strengthen the evidence arising from SWATs towards identifying strategies that could improve recruitment and retention in RCTs. There is a further need to develop a framework for the economic analysis of SWATs to enable research organisations to make more informed decisions.
This review accumulates and critically appraises the existing evidence on economic evaluations alongside SWATs for improving recruitment and retention in SWATs. The primary aim is to improve trial efficiency by increasing the evidence available for making trial process decisions. The secondary aim is to make recommendations for improvement of future economic evaluations alongside recruitment and retention SWATs.

Methods
A protocol is registered on PROSPERO (CRD record code: 42021236824), in line with the 2020 PRISMA (Preferred Reporting Items for Systematic Reviews and Metaanalyses) guidelines. 11 The studies eligible for inclusion were quasi-or fully randomised SWATs. The corresponding host trials of SWATs had to be quasi-or fully randomised, and within the context of healthcare or any field applicable to healthcare settings. Hypothetical studies (i.e. studies that ask potential patients whether they would participate in a trial that will not take place in reality) were excluded as these were assessed to have a high Cochrane risk of bias, 2 and hence were also excluded from the retention review. 7 Any strategies designed to improve recruitment and/or retention of participants in RCTs were eligible for inclusion in the study. The target population was any potentially eligible trial participants. For SWATs associated with improving recruitment, the strategies were aimed at potential trial participants who could be recruited to a host trial. For SWATs associated with improving retention, the strategies were aimed at already randomised trial participants who were asked to provide follow-up data. In contrast to the Cochrane recruitment and retention reviews, strategies aimed at collaborators or research ethics committees were not considered. 2,7 There were no restrictions regarding comparators. There are several potential types of economic evaluation alongside SWATs, including cost-effectiveness analysis, cost-benefit analysis, cost-consequence analysis, and cost-utility analysis. Therefore, the primary outcome was reported in terms of the incremental cost-effectiveness ratio (i.e. the incremental cost per additional patient recruited or per additional participant retained), the (monetary) net benefit of a given strategy or the willingness to pay (WTP) for a given strategy. The secondary outcomes were any costs and health utilities (benefits) of recruitment/ retention strategies. The measures of effect could be reported as incremental/unit/total costs, or incremental utilities/effects/benefits expressed in recruitment or retention rates. If the primary outcome in a study was unavailable but its secondary outcomes were reported appropriately, such a study would not be excluded on these grounds.
As this review focused on SWATs of recruitment or retention strategies that included economic evaluations in their analyses, the SWATs from the most recent Cochrane recruitment 2 and retention 7 reviews were considered in the study selection process. Further potential SWATs were identified after the final dates of the study searches in these Cochrane reviews, that is, on and after 12 February 2015 until 3 March 2021 for recruitment strategies 2 and on and after 1 March 2020 until 3 March 2021 for retention strategies. 7 Thus, we developed a search strategy for the identification of more recent SWATs on recruitment and/or retention strategies that involved economic evaluation. The search strategy is available in Supplemental Material 2.
We searched the following electronic databases: · MEDLINE (OVID) · Embase (OVID) · CINAHL · Cochrane Methodology Review Group Specialised Register (CMR) in the Cochrane Library · Science Citation Index and Social Citation Index Hand searching of conference abstracts associated with SWATs was also undertaken. Journals were also hand searched, including ClinicalTrials.gov, OpenTrials, EU Clinical Trials Register, Cochrane Central Register of Controlled Trials (CENTRAL) and the Online Resource for Recruitment research in Clinical triAls (ORRCA). The search dates were the same as those for the electronic databases.
The titles, abstracts, and full texts of identified records were independently screened by two authors (AG and AE).
We independently extracted the data through a standardised data extraction form on Microsoft Word, which included information on both the host trial and the SWAT. The data extraction form provided information about the host trial name, design, location, clinical setting, population, intervention(s), and comparator(s) (Supplemental Material 1). With respect to SWATs, the data extraction form provided information about the design, the strategy (-ies), the comparator(s), study objective, time horizon, frequency and timing of strategy, measure(s) of benefit and costs, type of economic evaluation, numbers, and proportions of participants in the intervention and control groups, results of health economic outcomes in the intervention and control groups, and perspective adopted in the economic evaluation (Supplemental Material 1). No automation tools were applied in the data collection process. We did not need to contact any study investigators to obtain further data or to ask for clarification of published data.
In line with the Cochrane recruitment and retention reviews' methodology, the Version 1 Cochrane risk of bias tool was used to appraise the quality of the included studies. 12 For included studies that were obtained from the recruitment and retention reviews, 2,7 the risk of bias presented in these reviews was assumed to be valid, and hence their quality appraisal was adopted from the corresponding reviews 2,7 to avoid duplication of effort. In addition, a descriptive quality assessment was independently undertaken to assess the quality of the included SWATs (including those from the Cochrane reviews) with respect to their economic evaluation. Such assessment followed explicitly the University of York's Centre for Reviews and Dissemination (CRD) guidance on systematic reviews of economic evaluations by considering the following: methods of deriving the effectiveness data, cost analysis, valuation and measurement of health benefit, methods of synthesising the costs and effects, and, if applicable, analysis of uncertainty. 13 To allow for broader inclusion of studies associated with economic evaluations alongside SWATs, studies with a high risk of bias, or studies with a low quality of economic evaluation, or studies that were not peer-reviewed, were still included if they met the inclusion criteria.
Random-effect meta-analyses, through the Cochran-Menthel-Haenszel weighting method, were undertaken for the primary outcome. Since all included SWATs were associated with cost-effectiveness analysis, this was carried out by initially obtaining the odds ratios (ORs) of the recruitment or retention rates for each strategy. Then, the ORs were converted to effect sizes, that is incremental recruitment or retention rates, by dividing the natural logarithm of the OR with 1.81. 14 This conversion is assumed for continuous outcomes, 14 such as the incremental recruitment or retention rate, and hence we applied such a conversion in our study.
(i.e. effect size = incremental recruitment rate/retention rate = ln ðORÞ 1:81 ). The Cochran-Menthel-Haenszel method was applied for weighting the incremental costs of each strategy from each included study, to obtain the aggregate figure for the incremental cost of each strategy. The final step was to calculate the incremental cost per patient recruited or participant retained for each strategy by dividing the incremental cost with the incremental recruitment or retention rate. 95% confidence intervals are presented for the primary outcome, the OR, and the incremental recruitment or retention rate. RevMan was the software used for metaanalysis. 15 The figures for the primary outcome were adjusted to 2019 USD Purchasing Power Parity (PPP) rates. The use of PPP, defined by the International Monetary Fund (IMF) as "the rate at which the currency of one country would have to be converted into that of another country to buy the same amount of goods and services in each country", 16 can reflect more accurately any cost variations among countries. We anticipated the included studies to be potentially subject to between-group (study) heterogeneity; hence the I 2 statistic, ranging from 0% to 100%, was computed for all strategies, whose primary outcome was obtained through the inclusion of multiple SWATs. The greater a reported value of I 2 , the greater the extent of between-study heterogeneity. More details about the metaanalyses of each recruitment and retention strategy can be found in Supplemental Material 3 and Supplemental Material 4 respectively.
Following meta-analysis, the GRADE approach was applied to the effect measure (i.e. the OR) and consequently the primary outcome (i.e. the ICER), to assess the certainty of the evidence for each recruitment and retention strategy; this tool explores the extents of risk of bias, imprecision, inconsistency, indirectness, and publication bias in the included studies. Such an assessment was undertaken by the two reviewers (AG and AE), and details about the GRADE assessment of each recruitment and retention strategy can be found in Supplemental Material 5. Furthermore, the Trial Forge Guidance 2 was explicitly used to qualitatively assess whether more SWATs should be conducted for recruitment and/or retention strategies included in the study's metaanalyses. 17 The assessment using Trial Forge Guidance 2 comprises of five criteria: risk of bias, imprecision, inconsistency, balance of benefit and disadvantage to participants, and balance of benefit and disadvantage to the host trial. 17 Such an assessment was undertaken by the two reviewers (AG and AE), and details about the application of Trial Forge Guidance 2 to each recruitment and retention strategy comprising of at least two SWATs can be found in Supplemental Material 6.
Cost-effectiveness ranks of strategies for improving recruitment and retention in RCTs are presented. The rank was based on the mean ICER of each recruitment or retention strategy, whether the lower 95% odds ratio confidence intervals indicate the associated strategy is significantly effective, and the GRADE certainty of evidence.

Searching of records
The full texts of 68 studies from the recruitment review 2 and 71 studies from the retention review 7 were assessed for inclusion to our study. Following the searches, 8113 records were retrieved from the electronic databases overall. 28 additional studies were identified from manually searched registers. Nine studies were identified from hand searching reference lists of two studies that were retrieved from the electronic databases. After deduplication, 6569 records were screened, and 267 full texts were assessed for eligibility. 22 studies were included in this review. A PRISMA flow diagram is shown in Figure 1.

Characteristics of the included studies
Three studies had more than one SWATs, [18][19][20] and hence 22 studies (29 SWATs) were included in this review. Nine studies (15 SWATs) assessed recruitment strategies, whereas 15 studies (16 SWATs) evaluated retention strategies. Two studies included SWATs with strategies targeting both recruitment and retention. 21,22 The characteristics of the included studies, that is, study (author, date, country), host trial design, participants, SWAT intervention(s) (and comparator(s)), and SWAT outcome(s), are presented in Table 1.
All SWATs of retention strategies were already included in the retention review. 7 However, two of these SWATs had data that were publicly inaccessible at the time of the publication of the retention review. 23,24 These SWATs eventually became publicly accessible in journals. As there was an uncertainty regarding which data the reviewers from the retention review had accessed, the papers' risk of bias was re-assessed.  For recruitment strategies, four included studies (four SWATs) were retrieved from the electronic databases, 22,[25][26][27] one study (one SWAT) was retrieved from manual searching, 28 four studies (10 SWATs) were already included in the recruitment review, 2 and one study (one SWAT) was already included in the retention review. 20 Most SWATs had individually randomised designs; however, two studies (two SWATs) were quasi-randomised, 29,30 and two studies (two SWATs) were cluster randomised. 22,31 Primary outcomes were available in seven out of 22 studies (11 out of 29 SWATs) and reported in terms of the incremental cost per additional patient recruited/incremental cost per additional participant retained, respectively. Accordingly, the incremental cost-effectiveness ratio (ICER), for recruitment strategies, was defined as:

Cost of recruitment strategy
Àcost of baseline strategy Recruitment rate of recruitment strategy ÀRecruitment rate of baseline strategy For retention strategies, the ICER was defined as:

Cost of retention strategy
Àcost of baseline strategy Retention rate of retention strategy ÀRetention rate of baseline strategy Therefore, cost-effectiveness analysis was the sole method of economic evaluation alongside the included SWATs. The primary outcome was manually computed by the reviewers in the remaining 15 studies (18 SWATs), using the incremental costs and the incremental recruitment and/or retention rates of a given strategy to obtain the ICER. The perspective adopted by all economic evaluations related to the trial teams, that is, the reported effects and costs of recruitment or retention strategies were direct and associated with the trial teams' budget. In total, 35,864 participants from 29 SWATs were involved.
The Cochrane risk of bias was assessed to be high for four included studies, unclear for nine studies, and low for nine studies (Table 2). In terms of the quality of the economic evaluation, this was assessed to be low for seven studies, moderate for six studies, unclear for one study, and high for eight studies (Table 3). One included study affected the quality appraisal detrimentally, as a full text was unavailable. 26 Recruitment strategies Financial incentives. The ICER of a financial incentive, against no financial incentive, was estimated from two studies (six SWATs). 18,19 With an odds ratio of 1.65 (95% CI: 0.86, 3.18) and an incremental cost of US$133.44, it costs US$476.57 (95% CI: from US$208.50 to N/A 1 ) to recruit an additional patient (see Table S1 and Figure S1 in Supplemental Material 3 for more details). All SWATs have a low Cochrane risk of bias 18,19 , but moderate quality of economic evaluation 18,19 . The I 2 statistic is 49%, signalling evidence of substantial between-study heterogeneity. There are three potential sources of between-group heterogeneity;  The GRADE certainty of evidence is moderate for this recruitment strategy, due to inconsistency. In line with Trial Forge Guidance 2, 17 the GRADE criterion is met, the cumulative meta-analysis criterion is met, the PICOT criterion 2 is partially met, the balance of benefit and disadvantage to participants criterion is not met, and the balance of benefit and disadvantage to host trial criterion is not met. We suggest further studies including different monetary incentives be conducted in the future so that a figure of additional patients recruited by a US$1 increase in monetary incentive be obtained.
Nudge interventions. The ICER of nudge interventions against usual recruitment procedures was estimated from three studies (three SWATs). 19,22,27 Nudge interventions related to recruitment included: quotes from existing  participants over text messages, a multimedia information resource that was developed through patient and public involvement (PPI) contributors and researchers, and a DVD presentation containing an audio-visual presentation explaining the host trial. With an odds ratio of 1.13 (95% CI: 0.72, 1.77) and an incremental cost of US$22.00, it costs US$314.29 (95% CI: from US$68.75 to N/A) to recruit an additional patient (see Table S2 and Figure S2 in Supplemental Material 3 for more details). The risk of bias is high for one study 27 , unclear for one study 22 , and low for one study. 19 In addition, the quality of economic evaluation is moderate for two studies 19,27 and low for one study. 22 The I 2 statistic is 74%, signalling evidence of substantial betweenstudy heterogeneity. There are four potential sources of such heterogeneity; (1) variations in healthcare settings across the host trials of the SWATs; (2) variations in the "nudge interventions"; (3) variations in the populations across SWATs and (4) variations in the designs of the included SWATs. The GRADE certainty of evidence is very low for this recruitment strategy, due to risk of bias, inconsistency, and indirectness of the included studies. In line with Trial Forge Guidance 2, 17 the GRADE criterion is met, the cumulative meta-analysis criterion is met, the PICOT criterion is partially met, the balance of benefit and disadvantage to participants criterion is met, and the balance of benefit and disadvantage to host trial criterion is partially met. Therefore, further replications of SWATs associated with nudge interventions are encouraged.

Screening of a trial by a senior investigator
The cost-effectiveness of this strategy was estimated according to a single SWAT. 29 Screening for the host trial undertaken by a senior investigator, versus screening for the host trial undertaken by a research assistant, is not costeffective, since the odds ratio is 0.19 (95% CI: 0.11, 0.32), with an incremental cost of US$37.05 (see Table S3 in Supplemental Material 3 for more details). Given the low sample size of the included study, 29 and its high Cochrane risk of bias, the GRADE certainty of evidence is low due to imprecision and risk of bias. The included study has a moderate quality of economic evaluation.

Text messages versus telephone calls
The cost-effectiveness of this strategy was estimated according to a single SWAT. 19 Table S3 in Supplemental Material 3 for more details). Given the sample size of the included study, 19 and its low Cochrane risk of bias, the GRADE certainty of evidence is moderate due to imprecision. The included study has a moderate quality of economic evaluation.

Pre-notification leaflet
The cost-effectiveness of this strategy was estimated according to a single SWAT. 26 With an odds ratio of 1.17 (95% CI: 0.87, 1.57) and an incremental cost of US$2.25, it costs US$25.97 (95% CI: from US$9.00 to N/A) to recruit an additional patient (see Table S3 in Supplemental Material 3 for more details). Given the sample size of the included study 26 , and its low Cochrane risk of bias, the GRADE certainty of evidence is moderate due to imprecision. The included study has a high quality of economic evaluation.

Telephone reminders versus text reminders
The cost-effectiveness of this strategy was estimated according to a single SWAT. 25 With an odds ratio of 1.37 (95% CI: 0.95, 1.98) and an incremental cost of US$3.98, it costs US$23.37 (95% CI: from US$10.47 to N/A) to recruit an additional patient (see Table S3 in Supplemental Material 3 for more details). Given the sample size of the included study 25 , and its low Cochrane risk of bias, the GRADE certainty of evidence is moderate due to imprecision. The included study has a high quality of economic evaluation.

Invitation packs by GP
The cost-effectiveness of this strategy was estimated according to a single SWAT. 28 With an odds ratio of 7.75 (95% CI: 1.04, 57.97) and an incremental cost of US$1.13, it costs US$1.00 (95% CI: from US$0.50 to US$57.47) to recruit an additional patient (see Table S3 in Supplemental Material 3 for more details). However, since these figures were obtained from an abstract 28 , the GRADE certainty of evidence is very low due to risk of bias, imprecision, indirectness and publication bias. The included study has an unclear quality of economic evaluation.

Trial-branded pen
The cost-effectiveness of this strategy was estimated according to a single SWAT. 21 With an odds ratio of 1.04 (95% CI: 0.65, 1.66) and an incremental cost of US$0.47, it costs US$21.41 (95% CI: from US$1.68 to N/A) to recruit an additional patient (see Table S3 in Supplemental Material 3 for more details). Given the sample size of the included study, 21 and its low Cochrane risk of bias, the GRADE certainty of evidence is moderate due to imprecision. The included study has a high quality of economic evaluation.
In line with Trial Forge Guidance 2, 17 we encourage all the recruitment strategies to be replicated in future SWATs, but the comparison of pre-notification leaflet against no leaflet.

Ranking recruitment strategies
The cost-effectiveness rank of the eight recruitment strategies is available in Table 4. Providing financial incentives might be an effective recruitment strategy, but its ICER is relatively high, at US$476.57, thus questioning its costeffectiveness; more SWATs of financial incentives with moderate amounts (i.e. significantly less than £100) are needed to estimate the ICER. The following may be considered cost-effective strategies: providing a telephone reminder versus a SMS reminder, or a branded pen versus no pen, or a pre-notification leaflet versus no leaflet. However, whereas their corresponding ICERs are relatively low, their OR lower bounds signal they may not actually be effective recruitment strategies. Providing primary text message, versus primary call and no text message, might be a very cost-effective strategy; however, more SWATs of this strategy are needed since its GRADE certainty of evidence is moderate. It remains uncertain whether the remaining recruitment strategies are cost-effective since their GRADE certainty of evidence is either low or very low.
Overall, there is no complete certainty up to date on which recruitment strategies would be cost-effective for trial teams to use for recruiting eligible patients to their trials. Nevertheless, strategies such as financial incentives, trialbranded pens, telephone reminders and pre-notification leaflets could possibly provide recruitment benefits to future trials in a cost-effective manner. More evidence is needed to determine the cost-effectiveness of such strategies.

Retention strategies
Trial-branded pen. The ICER of providing a trial-branded pen versus no pen was estimated from three studies (three SWATs). 21,32,33 With an odds ratio of 1.14 (95% CI: 1.00, 1.30) and an incremental cost of US$0.52, it costs US$6.98 (95% CI: from US$3.63 to N/A) for an additional participant to be retained in a host trial (see Table S4 and Figure  S3 in Supplemental Material 4 for more details). One included study has a low Cochrane risk of bias, 21 whereas the remaining studies have an unclear risk of bias. 32,33 All studies have a high quality of economic evaluation. The I 2 statistic is negligible at 0%, signalling no evidence of substantial between-study heterogeneity. However, there are four potential sources of such heterogeneity; (1) variations in healthcare settings across the included SWATs; (2) variations in retention periods among SWATs; (3) variations in the populations across the SWATs and (4) variations in the SWATs' designs. The GRADE certainty of evidence for this retention strategy is moderate, due to inconsistency. In line with Trial Forge Guidance 2, 17 the GRADE criterion is met, the cumulated evidence criterion is not met, the PICOT criterion is partially met, the balance of benefit and disadvantage to participants criterion is met and the balance of benefit and disadvantage to host trial criterion is not met.  Table S5 and Figure S4 in Supplemental Material 4 for more details). Once the quasi-randomised SWAT included in the three SWATs of financial incentives is removed, the result is not significantly affected: the OR slightly falls to 1.32 (95%: 1.01, 1.73), the incremental cost increases to US$11.08 and the ICER slightly increases to US$22.05 (95% CI: from US$11.17 to US$615.21). The Cochrane risk of bias is low in one study, 34 unclear in one study, 20 and high in one study. 30 Furthermore, two studies 20,34 have a low quality of economic evaluation, whereas one study 30 has a high quality of economic evaluation. The I 2 statistic is 37%, signalling low evidence of substantial between-study heterogeneity. There are two sources of such heterogeneity; (1) variations in retention periods among SWATs and (2) variations in the monetary incentives among SWATs. The GRADE certainty of evidence for this retention strategy is moderate due to risk of bias. In line with Trial Forge Guidance 2, 17 , the GRADE criterion is met, the cumulative meta-analysis criterion is not met, the PICOT criterion is partially met, the balance of benefit and disadvantage to participants criterion is partially met, and the balance of benefit and disadvantage to host trial criterion is not met. We encourage replications of further SWATs associated with financial incentives as a strategy for improving participant retention in RCTs. Three further studies (or three SWATs) were not included in this meta-analysis. 20,31,35 The results of these studies, and the reasons for which they were not included are available in Supplemental Material 7.

Nudge interventions
The ICER of a nudge intervention versus usual retention was estimated from three studies (three SWATs). 22,36,37 Nudge interventions related to retention included: a personalised text message instead of a generalised one, a multimedia information resource that was developed through patient and public involvement (PPI) contributors and researchers, and a social incentive cover letter instead of a standard one. With an odds ratio of 1.14 (95% CI: 0.94, 1.39) and an incremental cost of US$0.84, it costs US$11.55 (95% CI: from US$4.61 to N/A) for an additional participant to be retained in a host trial (see Table S6 and Figure  S5 in Supplemental Material 4 for more details). The Cochrane risk of bias is low in two studies, 36,37 and unclear in one study. 22 The quality of economic evaluation is high in one study, 36 moderate in one study, 37 and low in one study. 22 . The I 2 statistic is negligible at 0%, signalling low evidence of substantial between-study heterogeneity. However, there are still three sources of such heterogeneity; (1) variations in "nudge" interventions among SWATs; (2) variations in retention periods among SWATs and (3) variations in the SWATs' designs. The GRADE certainty of evidence for this retention strategy is moderate due to inconsistency. In line with Trial Forge Guidance 2, 17 the GRADE criterion is met, the cumulative meta-analysis criterion is met, the PICOT criterion is partially met, the balance of benefit and disadvantage to participants criterion is not met, and the balance of benefit and disadvantage to host trial criterion is partially met. We encourage replications of further SWATs associated with nudge interventions for improving participant retention in RCTs.
Unconditional monetary incentive versus conditional monetary incentive. The ICER of an unconditional monetary incentive, versus a conditional one, was estimated from two studies (two SWATs). 23,24 With an odds ratio of 0.90 (95% CI: 0.31, 2.64) and an incremental cost of US$18.61, such a strategy is not cost-effective, since its estimated odds ratio is less than 1 (see Table S7 and Figure S6 in Supplemental Material 4 for more details). The Cochrane risk of bias is unclear for both studies, whereas the quality of economic evaluation is high in one study 23 and moderate in the other study. 24 The I 2 statistic is 93%, demonstrating high evidence of substantial between-study heterogeneity. There are five potential sources of such heterogeneity; (1) variations in healthcare settings between the host trials of the included SWATs; (2) variations in the populations between SWATs; (3) differences in the interventions between SWATs; (4) variations in retention periods between SWATs and (5) variations in the SWATs' designs. The GRADE certainty of evidence is low, due to risk of bias and inconsistency. In line with Trial Forge Guidance 2, 17 the GRADE criterion is met, the cumulative meta-analysis criterion is met, the PICOT criterion is partially met, the balance of benefit and disadvantage to participants criterion is met, and the balance of benefit and disadvantage to host trial criterion is met. We highly encourage replications of further SWATs comparing unconditional with conditional monetary incentives for improving participant retention in RCTs.

Pre-notification card
The cost-effectiveness of this strategy was estimated according to a single SWAT. 38 With an odds ratio of 1.26 (95% CI: 0.99, 2.19) and an incremental cost of US$1.02, it costs US$4.86 (95% CI: from US$2.76 to US$N/A) to retain an additional participant in a host trial (see Table S8 in Supplemental Material 4 for more details). Given the low sample size of the included study 38 and its low Cochrane risk of bias, the strategy's GRADE certainty of evidence is moderate, due to imprecision. The included study has a moderate quality of economic evaluation.

Electronic prompts
The cost-effectiveness of this strategy was estimated according to a single SWAT. 39 With an odds ratio of 1.48 (95% CI: 0.81, 1.96) and an incremental cost of US$0.12, it costs US$0.55 (95% CI: from US$0.28 to US$N/A) to retain an additional participant in a host trial (see Table S8 in Supplemental Material 4 for more details). Given the low sample size of the included study 39 and its unclear Cochrane risk of bias, the strategy's GRADE certainty of evidence is low, due to risk of bias and imprecision. The included study has a low quality of economic evaluation.

Trial-branded pen (before recruitment)
The cost-effectiveness of this strategy was estimated according to a single SWAT. 21 With an odds ratio of 8.27 (95% CI: 1.04, 66.00) and an incremental cost of US$0.47, it costs US$0.40 (95% CI: from US$0.20 to US$23.50) to retain an additional participant in a host trial (see Table S8 in Supplemental Material 4 for more details). Given the sample size of the included study 21 and its low Cochrane risk of bias, the strategy's GRADE certainty of evidence is moderate, due to imprecision. The included study has a high quality of economic evaluation.
In line with Trial Forge Guidance 2, 17 we highly encourage the aforementioned retention strategies to be replicated in future SWATs.
Ranking retention strategies. A summary of the costeffectiveness rank of the seven retention strategies is provided on Table 5. Providing pens before patient recruitment to an RCT is potentially a very cost-effective strategy, with an ICER of US$0.40. However, as this finding is derived from a single study with a low sample size, more evidence is needed to confirm the figure. A retention strategy, which also seems to be cost-effective with moderate GRADE certainty of evidence, is the provision of £5 up to £10 vouchers; the ICER is relatively low at US$15.89. Providing a trial-branded pen is potentially another cost-effective retention strategy, with its ICER being very low, at US$6.98. However, its lower bound OR = 1, meaning there is still a chance such a strategy is not (cost-) effective. Due to either low GRADE certainty of evidence or wide confidence intervals of the remaining retention strategies, it is inconclusive whether these strategies are cost-effective or not.
We encourage trial researchers to consider financial incentives of up to £10 or/and trial-branded pens as retention strategies, while we recommend more SWATs of these strategies be undertaken. Despite the reported lower bound OR, we still encourage pens as a retention strategy due to its low reported ICER and low incremental costs, especially for trials involving postal questionnaires.

Summary of findings
Whereas Cochrane reviews have explored the effectiveness of strategies for improving recruitment and retention in RCTs, 2,7 this review additionally appraises the costeffectiveness of recruitment and retention strategies. The findings demonstrate an uncertainty regarding which strategies are cost-effective for improving participant recruitment and/or retention in RCTs. For both recruitment and retention strategies, the uncertainty of the evidence primarily originates from the evaluation of several potential strategies from single studies, but without any replications. The corresponding Cochrane reviews on recruitment and retention in RCTs suggested that replications of SWATs with strategies having a moderate GRADE certainty of evidence be undertaken, a recommendation we also make for bolstering the evidence on the cost-effectiveness of strategies for improving recruitment or retention in SWATs.
Overall, there is no retention strategy which we would recommend trial teams and researchers adopt with complete statistical certainty. Providing vouchers of up to £10 during follow-up could be a cost-effective retention strategy with an estimated ICER of US$15.89; it costs only US$15.89 for an additional participant to be retained in a host trial. Providing a trial-branded pen may also be a cost-effective strategy, with an ICER of US$6.98, yet not statistically significant since its lower bound OR = 1 (hence its lower bound effectiveness is zero). Also, providing a trial-branded pen before recruitment, may be a cost-effective strategy, with an ICER of US$0.40 which is also statistically significant. However, the GRADE certainty of evidence for both strategies is moderate, meaning that additional SWATs of these strategies could be beneficial for making more certain inferences about their costeffectiveness. ICERs were derived for further retention strategies; however, it remains inconclusive whether these are costeffective due to their low or very low GRADE certainty of evidence. Whereas the retention review found the inclusion of self-kits or a diary to be effective strategies, 7 no data about their cost-effectiveness were available. Therefore, we highly encourage the conduct of future SWATs of these strategies and the inclusion of economic evaluations alongside such SWATs. Similarly to the retention review, 7 we also encourage the conduct of further SWATs associated with the costeffectiveness of patient and public involvement (PPI) interventions, since PPI is a key unanswered question about trial retention. 40 Overall, due to the low ICER and incremental costs we recommend trial teams use trial-branded pens as a retention strategy, especially in trials involving postal questionnaires for follow-up. Providing vouchers of up to £10 could be another beneficial retention strategy for trial-teams.
Also, there is no recruitment strategy which we would recommend trial teams and researchers adopt with complete statistical certainty. Including a branded pen with a trial invitation pack, or a telephone reminder versus an SMS reminder, could be cost-effective strategies, with their ICERs being low, at US$21.41 and US$23.37 respectively. However, as their lower bound ORs are less than 1, their costeffectiveness is not statistically significant. In addition, their GRADE certainty of evidence is moderate, implying these strategies would benefit from further SWATs to determine their cost-effectiveness with less uncertainty. Another costeffective strategy could be the provision of primary text message versus primary call and no message, with its ICER estimated at $4.41 (95% CI; $2.45, $23.38); however, its GRADE certainty of evidence is also moderate. Providing financial incentives may be an effective yet a costly strategy, with US$476.57 required to recruit an additional patient. However, there is substantial heterogeneity among the associated SWATs, since very different monetary incentives were present (i.e. from £5 up to £100). Therefore, we encourage the cost-effectiveness of moderate financial incentives (i.e. less than £100 per participant recruited 17 ) to be evaluated in future SWATs. Moreover, we could not estimate the cost-effectiveness of an open design, compared to a placebo-controlled design, as the associated SWATs did not undertake any relevant economic evaluations or provide costs related to such strategies. Since this strategy appears to be effective at improving recruitment, 2 economic evaluations of such a strategy alongside future SWATs are welcome. We also encourage the estimation of the cost-effectiveness of incorporating user-testing for improving the participant information leaflet (PIL) in future SWATs.

Recommendations for future economic evaluations alongside SWATs
To minimise the uncertainty regarding the findings from SWATs on the cost-effectiveness of recruitment and/or retention strategies, we highly recommend the application of Value of Information (VoI) analyses. Such an analysis can inform decision makers on whether more trials are needed to minimise the uncertainty of the cost-effectiveness of a strategy. A VoI analysis could be used in line with the Trial Forge Guidance 2 17 to confirm whether a further SWAT associated with a recruitment/retention strategy should be undertaken. For instance, since we concluded trial-branded pens to be a potentially cost-effective retention strategy with moderate GRADE certainty of evidence, and the Cochrane review concluded pens to be a potentially effective strategy with low GRADE certainty of evidence, the GRADE criterion is met in Trial Forge Guidance 2 and hence further SWATs on pens are recommended. However, it seems that such a strategy could be a very cost-effective one for participant retention, and hence it may not be necessary to undertake another SWATs, which would require the financing of an additional SWAT from a constrained allocated budget . To determine whether more SWATs are needed for determining its effectiveness, a VoI analysis for trial-branded pens could be undertaken. A framework of VoI analysis related to SWATs which trial researchers could follow is available in the literature, and applicable after a standard meta-analysis of a recruitment or retention strategy. 41 A concern was that although 139 studies were originally included in the recruitment and retention reviews, 2,7 only 17 of these studies were included in our review. Therefore, economic evaluations were not undertaken alongside the majority of SWATs. Whereas capturing the effectiveness of different recruitment or retention strategies is useful, cost considerations are equally important due to limited availability of financial resources. We highly encourage trialists and researchers to undertake economic evaluations alongside all SWATs in the future. In addition, the costs of obtaining outcome data were not provided in six studies (10 SWATs), meaning that the cost-effectiveness of some strategies may have been overestimated. Therefore, the reporting of costs should be transparent, expressed in unit terms, and stratified into different types of direct and indirect costs, including the costs of obtaining outcome data. Finally, in all cost-effectiveness analyses there is a defined cost-effectiveness threshold to determine whether a given intervention, which is both more effective compared to existing interventions and costlier, is cost-effective. As long as the ICER is less (more) than the threshold, then a strategy is (not) cost-effective. We recommend trial researchers to define such a threshold for determining which recruitment or retention strategies should be considered as costeffective. In our review, we did not set out a costeffectiveness threshold, as there has not been any research in this area; instead we presented cost-effectiveness ranks of recruitment and retention strategies for comparisons with respect to their cost-effectiveness to be made.
The perspective all SWATs followed was related to the trial teams. However, poor recruitment into RCTs may also lead to indirect costs through the generation of foregone health benefits to an affected population not experiencing the clinical benefits of a potentially effective intervention. For instance, a study modelled the impact on human lives lost due to poor recruitment in the COVID-19 RECOVERY trial, which showed that over 2800 lives could have been saved in the UK. 42 Similarly, the financial costs of poor attrition can be significant, with the time costs of researchers dealing with follow-up being dominant. 43 When the follow up to a funded RCT is poor, this may generate huge costs for RCT funders, as they could have instead provided funding to trials with better follow-up rates and hence with more statistical accuracy in their results. Therefore, in future SWATs it is recommended that researchers adopt a broader perspective where possible when conducting economic evaluations alongside SWATs, such as the perspective of a national healthcare system or the societal perspective (i.e. through cost-benefit analysis instead of cost-effectiveness analysis).

Strengths and limitations of the review
The major limitations in our study were the differential definitions and computations of cost-effectiveness outcomes among the included studies. These were partially captured through manual conversions of ICERs, or any other secondary economic outcome, into unit incremental costs, by stringently following the definition of ICER (equations (1) or (2)) and the reported recruitment or retention rates. This approach enabled us to obtain costeffectiveness figures from 20 out of 22 studies, or from 26 out of 29 included SWATs in a homogeneous manner. Another limitation could be our flexible approach towards including studies with high Cochrane risk of bias or low quality of economic evaluation, or studies that have not been peer reviewed or published yet. However, as an appraisal of the cost-effectiveness of strategies for improving participant recruitment and retention in RCTs has not been explored before a, we encouraged this flexible approach during the screening of records and inclusion of studies. Finally, there were differences in the definitions of "recruitment rate" or "retention rate", especially in terms of the recruitment and retention periods, across the included studies. However, we encouraged flexibility in the definitions of such terms by the same means.
Overall, the review benefits from such flexibility so that the evidence on the cost-effectiveness or recruitment and/or retention strategies is fully captured. In addition, all studies were subject to extensive quality appraisals, including the Cochrane risk of bias and quality of economic evaluation. Moreover, the certainty of the evidence for each recruitment and/or retention strategy was extensively assessed through the GRADE approach and Trial Forge Guidance 2. We believe the use of multiple tools strengthens the reliability of our findings. Finally, our review could motivate the research community to undertake economic evaluations alongside all future SWATs; we have also made recommendations on how such economic evaluations could be undertaken.

Conclusion
There is no recruitment or retention strategy which we would recommend trial teams and researchers adopt with full certainty. Improving recruitment and retention in RCTs is a priority for trial teams, reflected through the emergence of SWATs as a study design to improve trial efficiency. It is of paramount importance for future SWATs to replicate existing recruitment and/or retention strategies, rather than focus on novel strategies. We also recommend that economic evaluations be carried out alongside all future SWATs, costs and benefits be reported clearly and transparently, the costeffectiveness of existing recruitment or retention strategies be repeatedly evaluated, and broader perspectives be adopted in future SWATs if applicable. Finally, we encourage researchers to undertake VoI analyses for each recruitment and retention strategy, in combination with Trial Forge Guidance 2, to minimise the uncertainty of the evidence.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.