A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment in a randomised controlled trial of yoga for older adults with multimorbidity

Background: Monetary and other incentives may increase recruitment to randomised controlled trials. Methods: 2x2 factorial ‘study within a trial’ of including a pen and/or £5 (GBP) in cash with a postal recruitment pack to increase the number of participants randomised into the host trial (‘Gentle Years Yoga’) for older adults with multimorbidity. Secondary outcomes: return, and time to return, of screening form, and the cost per additional participant randomised. Binary data were analysed using logistic regression and time to return using Cox proportional hazards regression. Results: 818 potential host trial participants were included. Between those sent a pen (n=409) and not sent a pen (n=409), there was no evidence of a difference in the proportion of participants randomised (15 (3.7%) versus 11 (2.7%); OR 1.38, 95% CI 0.63–3.04), in returning a screening form (66 (16.1%) versus 61 (14.9%); OR 1.10, 95% CI 0.75–1.61) nor in time to return the screening form (HR 1.09, 95% CI 0.77–1.55). Between those sent £5 (n=409) and not sent £5 (n=409), there was no evidence of increased randomisation (14 (3.4%) versus 12 (2.9%); OR 1.18, 95% CI 0.54–2.57), but more screening forms were returned (77 (18.8%) versus 50 (12.2%); OR 1.67, 95% CI 1.13–2.45) and there was decreased time to return screening form (HR 1.56, 95% CI 1.09–2.22). No significant interaction between the interventions was observed. The cost per additional participant randomised was £32 and £1000 for the pen and £5, respectively. Conclusion: A small, monetary incentive did not result in more participants being randomised into the host trial but did encourage increased and faster response to the recruitment invitation. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in this population. Pens were cheaper but did not provide evidence of benefit.


Introduction
Efficient recruitment to randomised controlled trials (RCTs) is important to achieve the target sample size and statistical power within the planned budget and time frame. Incentives, monetary or otherwise, are sometimes used to increase trial recruitment. 1 Financial incentives have been found to increase recruitment by 4% (95% CI -1-8%) in a metaanalysis. 1 However, most of the included studies used an incentive of £100, which is larger than publicly funded trials can usually afford. There remains, therefore, uncertainty as to whether financial incentives should be used and, if so, what amount.
Offering a potential participant a gift such as a pen may make them more likely to take up the invitation to enrol in a trial. It is also possible that the convenience of having a pen to hand upon receipt of the invitation may help facilitate a swifter response. However, a previous SWAT conducted by the York Trials Unit evaluated the use of pens as an incentive for recruitment into the OTIS trial of older adults and showed no difference in proportion of participants randomised (pen 4.5%; no pen 4.3%, odds ratio (OR) 1.04, 95% CI 0.65-1.67, p = 0.86), or screened (pen 14.2%, no pen 11.7%, OR 1.25, 95% CI 0.94-1.67, p = 0.12), or in time to return screening form (hazard ratio (HR) 1.23, 95% CI 0.94-1.60, p = 0.13). 2 To our knowledge, this is the only previous RCT to evaluate pens to increase trial recruitment, so more evidence was needed. We conducted a methodological 'study within a trial' (SWAT) to evaluate the effects of including a small, unconditional financial incentive and/or a pen in the postal recruitment pack on the proportion of participants randomised into the host trial. We hypothesised that receipt of a pen or financial incentive would improve the response to the trial invite, encourage a faster response and ultimately result in more participants being randomised into the host trial. We expected an additive effect of receiving both incentives, and that they would not interact.

Design
This 2x2 randomised factorial SWAT was embedded in the Gentle Years Yoga (GYY) trial, which is a multi-centre RCT of the clinical and cost-effectiveness of a yoga programme for older adults with multimorbidity conducted in England (recruitment complete and trial in follow-up at the time of writing; ISRCTN13567538, registered 18/03/2019 https://www.isrctn.com/ISRCTN13567538). The SWAT was registered with the Northern Ireland Network for Trials Methodology Research SWAT repository on 01/04/2018 (SWAT94; https://www.qub.ac.uk/sites/TheNorthernIreland-NetworkforTrialsMethodologyResearch/SWATSWARInformation/Repositories/SWATStore/). The GYY trial, and its embedded sub-studies, received approval from the North East-York Research Ethics Committee on 24/04/2019 (19/NE/0072), and the Health Research Authority.

Participants
In order to identify potential participants for the GYY host trial, we asked General Practitioners (GPs) to screen their practice lists for patients who may be eligible and to mail them a recruitment pack. The first 850 patients (see Sample size and randomisation section) mailed a recruitment pack, as identified by four participating GP practices, were included in this SWAT. The standard GYY recruitment pack contained an invitation letter, participant information sheet (PIS), consent and contact details form, screening form, and prepaid envelopes to return documentation to the York Trials Unit, University of York. A random sample of packs additionally included a financial incentive and/or a pen as part of this SWAT. For these packs, the PIS included the following text:

REVISED Amendments from Version 1
In response to the reviewers' very helpful comments, we have made the following amendments to the manuscript. The word 'rates' has been removed from the title. The Abstract has been amended so that the primary outcome is reported before the secondary outcomes in both the Results and Conclusion sections. General edits have also been made to keep the Abstract within the word limit of 300.
Within the main body of the manuscript, text from the original Intervention section has been moved to the Introduction, and additional details about the incentives added to the Intervention section (including a new Figure 1 depicting the pen). Text from the Blinding and consent section (now just called 'Blinding') has been moved to the Participants section. Detail on how the cost per additional participant recruited was calculated has been added. Some text in the Results section, that was presented in tabular form in Table 1, has been removed as one of the reviewers thought this was confusing and unnecessary. We have revised Table 2 so that the primary outcome is reported before the secondary outcomes. The description of the £5 incentive has been changed in some of the text, tables and figures to 'cash' for international audiences (to avoid sole use of a label that designates the exact value in British currency). The findings from the meta-analysis have been moved from the Results to the Discussion section, and the figures renumbered. A recommendation for future SWATs to consider additional qualitative work has been added to the Discussion. Other general edits have been made throughout the paper to improve clarity and readability.
Any further responses from the reviewers can be found at the end of the article Please find enclosed a complimentary £5 AND/OR pen given as a thank you for considering taking part. If you choose not to take part you can still keep this.
The packs were sent out in August 2019.
Potential participants for the host trial were not informed in advance that they were to be sent a recruitment pack for the GYY trial. They were hence also not informed in advance about the SWAT being embedded in the host trial, i.e. that they may receive a pen or some money in the recruitment pack, and that the incentive they received (if allocated to receive a pack containing one) had been chosen through a process of randomisation. (However, as explained above, the PIS, when received, did reference that the pack included £5 and/or a pen as a thank you for considering taking part in GYY.) This means that specific consent for the SWAT was not obtained; this was approved by the Research Ethics Committee as it was considered low risk. As described above, written informed consent for the GYY main trial was obtained from all participants who took part.

Financial incentive
A £5 (GBP cash) note was enclosed within the recruitment pack for the host trial. The amount of £5 was chosen as we wanted to include a note, rather than coins, as they are lighter (and so did not add any additional postage costs). £5 is the smallest denomination of GBP note; this was deemed sufficient as a thank you to participants for considering taking part in GYY, and including anything larger, e.g. £10, was less likely to be cost-effective.

Pen
A retractable ballpoint, black ink pen, branded with the GYY trial logo, was enclosed within the recruitment pack for the host trial ( Figure 1).

Financial incentive + pen
Both a pen and the cash note were enclosed within the recruitment pack for the host trial.

Control
The host trial recruitment pack was sent without the inclusion of a pen or cash.

Outcome measures
The primary outcome was the proportion of participants randomised into the GYY host trial. Secondary outcomes were return, and time to return, of a screening form to the York Trials Unit, and the cost per additional participant recruited. This was calculated by working out the additional cost of each incentive (i.e. what enclosing a pen or the £5 in cash cost in addition to mailing the standard recruitment pack), and multiplying this by the 'number needed to treat' (NNT). The formula for NNT is 1/the absolute risk reduction (ARR). For example, in this case, if the proportion of participants randomised among those who received a pen is X%, and the proportion randomised among those who did not receive a pen is Y%, then the ARR is (X-Y)*100, and the number of people we would need to send a pen to achieve one extra randomised participant is 1/(X-Y)*100.

Sample size and randomisation
Due to financial restrictions, we could afford to involve a sample of 850 recruitment packs in this SWAT. This would give 80% power (two-sided α=0.05) to detect a difference in the proportion of participants randomised of 4% (from 3% to 7%) for either of the interventions, relative to not receiving that intervention.
Block randomisation of size 4 was used to allocate recruitment packs 1:1:1:1 to: no pen or £5; £5 only; pen only; or pen and £5. A trial statistician, not involved in the production of recruitment packs or recruitment of participants, generated the sequence using Stata v15 (RRID: SCR_012763). Stata is a proprietary software but an open-access alternative in which the sequence could have been generated is Microsoft Excel (RRID: SCR_016137).

Blinding
Neither the participants nor statisticians analysing the data were blinded to allocation.

Statistical analysis
The primary comparisons in this trial are the main effects of being sent a pen, and of being sent £5. Returning a screening form and being randomised into the GYY trial were both analysed using multivariable logistic regression, including the two interventions (pen and £5). Time to return the screening form (in days from the date the recruitment pack was sent out to the date it was returned) was analysed using Cox proportional hazards regression. Screening forms that were not returned were censored at eight weeks after they were sent out. These analyses provide an estimate of the average effect of each intervention, assuming there is no interaction between them. In secondary analyses, the interaction between the two interventions was tested by extending the original models to include the interaction term. Analyses were conducted in Stata v16 (RRID: SCR_012763 ). An open-access alternative that can perform an equivalent function to Stata for analysis is R, a free software environment for statistical computing and graphics (RRID: SCR_001905).
The additional cost of including a pen in the postal mailout was £0.32; the inclusion of £5 additionally cost only the value of the note itself. Given the 1% increase in participants randomised when sent a pen, 100 (1/0.01) pens would need to be  sent to recruit one additional participant at a cost of £32 (100Â£0.32). Given the 0.5% increase in participants randomised when sent £5, we would need to send 200 participants £5, at a cost of £1000, to recruit one extra participant ((1/0.005)*5).

Discussion
There was no evidence of a difference in the proportion of participants randomised into the host trial between those sent a pen or £5 and those who did not receive the respective incentive. The proportions of participants randomised in the 'no intervention' arms (2.7% and 2.9%) were similar to the 3% assumed in the sample size calculation but the observed group differences were smaller than the 4% difference we were powered for; therefore, this SWAT was underpowered to detect the differences observed.
There was little or no evidence that sending a pen increased the proportion of participants returning a screening form or decreased time to return the form.
A small, monetary incentive did not result in more participants being randomised into the host trial but was effective at prompting return of the screening form, and of a swifter return. Since sending a financial incentive (even a small one) is, by its very nature, relatively costly, we do not recommend this intervention for use to increase recruitment in older adults with multimorbidity. The pen was cheaper but provided little evidence of benefit. If the observed effect of a 1% difference was true then we would need sufficient SWATs to provide an overall sample size of around 11,000 participants to confirm this. In a meta-analysis with the OTIS SWAT, the pooled OR associated with receipt of a pen was 1.12 (95% CI 0.75-1.67, p = 0.58) (Figure 4). Because the extra cost of recruiting an additional participant is relatively small, more SWATs are required to assess whether this difference is a true effect, since sending pens could be a cost-effective intervention for recruitment.
Although no statistically significant interactions between the pen and £5 were observed, as was expected, this cannot be ruled out as the sample size of this trial was likely insufficient to be powered to detect an interaction.
For both the pen and cash, the same wording on the participant information sheet was used, which stated that the pen and/or £5 were complimentary and sent as a thank you for people considering taking part in the host trial. Some anecdotal evidence from the GYY trial's process evaluation suggested that participants felt it unnecessary to receive £5 with their No cash As the response rate was less than 50%, the median time to return could not be calculated, so the 10th percentile survival times were estimated in days: pen 12, 95% CI 11 to 18; no pen 13, 95% CI 11 to 21; Cash (£5 GBP note) 11, 95% CI 11 to 13; no cash (£5 GBP note) 18, 95% CI 11 [upper limit not calculable] recruitment pack as they would willingly have joined the trial without this purely to help themselves, others and the research. In addition, this may have caused potential confusion if participants discussed receiving £5 during their yoga sessions as to why some received it and some did not. Such sentiments were not expressed in relation to being sent a pen, potentially suggesting that people view non-monetary incentives differently (more like a gift) than monetary incentives.
We were unable to obtain views on receiving the incentives from people sent a recruitment pack but whom either did not return it, were ineligible or declined participation in the GYY host trial. Future SWATs may want to consider ways to obtain qualitative accounts from such participants, as this would add further useful context in which to interpret the findings.
In conclusion, we did not find evidence that the inclusion of a pen and/or £5 was particularly effective or represented good value for money for improving recruitment into a trial of Gentle Years Yoga for older adults with multimorbidity.

Data availability
Underlying data OSF: Underlying data for 'A 2x2 randomised factorial SWAT of the use of a pen and small, financial incentive to improve recruitment rates in a randomised controlled trial of yoga for older adults with multimorbidity'

Open Peer Review
This was a 2x2 factorial 'study within a trial' of including a pen and/or £5 with a postal recruitment pack to improve randomisation rate (primary outcome) into the host Gentle Years Yoga trial in older adults with multimorbidity. The authors conclude that including a small, monetary incentive encouraged an increased and faster response to the recruitment invitation but did not result in more participants being randomised into the host trial.

Major
The major issue is the phrasing of the primary outcome as the rate of randomisation. You didn't analyse the time to randomisation in your analysis, you analysed the proportion randomised. I believe a better description is the proportion randomised to the host trial or just simply number of participants randomised to the host trial.
Given my comments, I would remove the term 'rates' from the title also.
The presentation of secondary outcomes prior to presenting the primary outcome could be described as selective outcome reporting. Please adjust this throughout and present your primary outcome first, including in your tables.

General Comments
Check your tenses throughout. Sometime you say 'was' and sometimes you say 'has'. 'Consist of' should be 'consisted of'. Check the English language also as sometime some sentences are phrased a little unusually or are missing the indefinite article or definite article.
In your results you say there was no evidence of a difference in the likelihood of being randomised. This is nitpicking a little but I think it is clear to stick to the number or proportion randomised.
Conclusion of abstract: Again modify your response here and put your primary outcome first.

Methods Section
Design I think you should include information in this section on the consent. I can see it further down. Perhaps reconsider all the smaller headings to bring the information together. This is a very small issue.

Participants
Remove the word 'appeared' from the first sentence as it introduces unnecessary uncertainty around your eligibility criteria.

Intervention
The information you have currently given in the intervention section belongs in the introduction section. In this section, I would expect to see what the intervention was. I would also expect to see what the control group was in this SWAT. I assume it is no incentive, but it should be clearly stated.

Outcomes
Again, consider the primary outcome here. Also, when you state the secondary outcomes, list them all together. The cost per additional participant recruited looks like it was not part of the secondary outcomes as it is in a separate sentence.

Results
Change 'potential participants' to 'participants' as the SWAT is completed. Delete the following text as it is all evident in Table 1 and is actually easier to follow in the table.
"In these analyses, the potential participants who were sent a pen (n = 409) consist of the group who received a pen and a £5 note (n = 203), and the group who received a pen only (n = 206). These are compared with those who were not sent a pen (n = 409), consisting of the group who were only sent a £5 note (n = 206), and the group who were sent neither a pen nor £5 (n = 203). Similarly, those who were sent £5 (n = 409) consist of the group sent both a pen and £5 (n = 203), and the group sent £5 only (n = 206). These are compared with the potential participants who were not sent £5 (n = 409), consisting of the group who were sent a pen only (n = 206), and the group who were sent neither a pen nor £5 (n = 203)."

Conclusion
Your primary outcome should be reported first.

If applicable, is the statistical analysis and its interpretation appropriate? Partly
Are all the source data underlying the results available to ensure full reproducibility? Yes

Are the conclusions drawn adequately supported by the results? Partly
Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Trial Methodology; Epidemiology
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

University of New Brunswick, Fredericton, NB, Canada
Thank you for the opportunity to review this very interesting 2x2 experimental SWAT study which I enjoyed reading. The paper is well-written and interesting. Below are some comments, the authors might want to consider for a potential revision of their manuscript.

Abstract:
Minor suggestion is to re-phrase "into the host Gentle Years Yoga trial in older adults with multimorbidity" to "into the host trial Gentle Years Yoga in older adults with multimorbidity". Separating 'host' and 'trial' was confusing on the first reading. There are a couple more instances in the paper which might benefit from keeping host trial as one phrase. Maybe 'in' should be 'for' in this sentence?
Conclusions: "Including a small, monetary incentive encouraged increased and faster response to the recruitment invitation but did not result in more participants being randomised into the host trial. Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in this population.". I think this is accurate, but wonder if you want to consider putting the main message first for your primary outcome (numbers randomised), rather than the secondary outcomes of response rate and speed?
Conclusions: "Pens are cheaper but did not provide evidence of benefit.". I suppose that depends on the pen that is chosen. Either phrase in the past ("were cheaper") or consider omitting. Not sure it adds much to the conclusions.
Conclusions: the last sentence seems to include very little study-specific information that wouldn't be true for almost every research study ("Further studies may be required.") and could be omitted without losing any critical content.

Introduction:
Overall I do like a focused and succinct introduction but wondered whether a little more information was required to contextualise the study some more. I found a lot of relevant contextual information in the methods section under 'Interventions' which I would have expected in the introduction. I wonder if that can be moved up within the manuscript. Below I suggest additional information for the methods section which outlines the intervention in somewhat more detail.
Did the authors have any study aims or a priori hypotheses they were testing within the study? If yes then it might be useful to explicitly state these at the end of the introduction.

Methods
Very minor point: Did participants receive multiple 'prepaid envelopes' or just one (all intervention material is described in singular except the envelopes)? The reason I am asking is because the intervention was included as part of the whole package. The more content in the package, and the more complex the response to the invite (e.g. through having to navigate multiple envelopes), the less likely people are to respond and miss the intervention.
Under intervention, it would be useful to move the current text to the introduction as it provides a rationale for the study, rather than describing the intervention. It would be useful to describe intervention details, including: How was the value of 5 pounds decided?
○ Was the money provided in cash?
○ Why would the pen and the money components interact? Are there any theoretical reason why such an interaction might occur?
○ What did patients get told about the money, if anything? This is important as having seemingly randomly a 5er in a research recruitment package might be confusing to people, unless there is some kind of explanation why it was sent. Incentives and rewards typically have an element of contingency attached to them.

○
More details on the pen would be useful, as there is a difference between a cheap throwaway pen and a more substantial and durable one. A picture of the pen would be useful. In fact, a picture of the entire online package with all its content would be even better.

○
Could the authors explain how they calculated the following "The cost per additional participant recruited was calculated for each intervention."?
All the Tables and figures have one group designated as pen/no pen, and the other as £5/ No £5. Would it be better to talk about 'money' or 'cash' for international audiences (i.e. intervention groups = 'pen/no pen' and 'money/ no money), rather than use the label which designates the exact value in British currency?

Results
The meta-analysis in the results section is somewhat surprising given that it has not been mentioned in the methods section and does not appear to be related to a research question. It might be more useful to compare the results of this study with other similar studies in the discussion section.

Discussion
In line with the conclusions in the abstract, and for consistency of message, I wonder if the authors might want to consider to state no difference in randomisation between the groups, rather than using more ambiguous phrasing of 'marginally more significant.
The discussion of the 'anecdotal evidence' is interesting, adding qualitative accounts of participants would have added context and additional information useful to interpret the findings. It would have been particularly informative to ask those who returned but declined, as well as those who never returned. Perhaps add a sentence for future SWATs to consider?
The sentence "Since it is relatively costly, we do not recommend this intervention for use to increase recruitment in older adults with multimorbidity." could be increased in clarity. The previous paragraph discusses both pen and cash, so it is not clear if the authors mean either or only cash with 'this intervention'. I assume the money, given the paragraph structure, but a quick read of the paper might lead to some confusion. Replacing 'this intervention' with 'a 5 pounds incentive' or something might be useful.
I think that the interpretation of the findings in relation to the incentive depends on how the money was presented in the context of the recruitment package -see methods comments on additional information which would add value to the paper. The conclusion might benefit from taking into consideration that the form of delivery of the incentive could have affected how people saw it, which subsequently impacted their decision making around participation.
I wonder if a concluding sentence might be beneficial to add to finish the discussion.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Partly

Are the conclusions drawn adequately supported by the results? Yes
Thank you for your thorough and constructive review. We have changed the Abstract according to all your suggestions, plus a few others to keep it within the word limit of 300. We have moved text from the Intervention section to the Introduction, where we agree it is better placed, and have added in our hypotheses to the end of the Introduction. In relation to the content of the recruitment packs, all potential participants of the host trial, GYY, received the same standard recruitment pack in the post. This included an invitation letter, participant information sheet (PIS), consent and contact details form, screening form and two prepaid envelopes. The reason that two envelopes were included is that, if the person wished to take part in the trial, we asked them to return their consent form and contact details in one envelope, and their completed screening form in the other (all documents were linked by a unique, prepopulated trial ID number). This is so that, should the post be intercepted, an individual's contact details are separated from the confidential information they provide in the screening form. In the SWAT, for those randomised to receive a pen and/or £5, the allocated intervention was enclosed within the standard recruitment pack for the host trial. The only other difference was some wording on the PIS, to include: "Please find enclosed a complimentary £5 AND/OR pen given as a thank you for considering taking part. If you choose not to take part you can still keep this." Therefore, participants in all four groups of the SWAT received the exact same contents in the recruitment pack APART FROM the inclusion of a pen and/or £5 if allocated, and the additional wording in the PIS.
In the Intervention section, we have elaborated on the incentives, providing detail about why the value of £5 was chosen and a description of the pen. More detail has also been included on how the 'cost per additional participant recruited' was calculated. We have heeded your suggestion to change £5 to 'cash' in some of the text and tables for international audiences, and have moved the pen meta-analysis to the Discussion.
We have made changes to the Discussion as per all your suggestions, and added in the following concluding sentence: "In conclusion, we did not find evidence that the inclusion of a pen and/or £5 was particularly effective or represented good value for money for improving recruitment into a trial of Gentle Years Yoga for older adults with multimorbidity." We greatly appreciate your comments and believe the changes we have made in response to them have really improved the paper. If you have any other suggestions for improvement, we would be happy to receive them.