Exploring engagement with authors of randomised controlled trials to develop recommendations to improve allocation concealment implementation and reporting [version 1; peer review: awaiting peer review]

Background: Reviews have consistently shown that allocation concealment is frequently implemented and reported suboptimally in randomised controlled trials (RCTs). This research aims to pilot engaging with authors of RCTs to explore their knowledge and understanding of allocation concealment implementation and reporting to ascertain areas and mechanisms for their improvement. Methods: Authors that published RCTs in core clinical journals in one month in 2019 were identified. Authors were invited to complete questionnaires to elicit their views and experiences on the implementation and reporting quality focussing on allocation concealment. Methodological quality of allocation concealment was evaluated in this sample by assessing adherence to the Consolidated Standards of Reporting Trials (CONSORT). Results: Reporting was suboptimal, with only 57% of allocation concealment methods reported to be implemented which were judged as adequate, with 18% using sealed envelopes and more than 40% not adequately reporting allocation methods. When exploring allocation concealment, implementation and reporting questionnaires were found to elicit a low response rate amongst authors of RCTs. Discussion: Following analysis of the themes that emerged from the questionnaires, the main recommendations to improve reporting quality are: journals need to endorse, adhere and promote reporting guidelines, a methodologist could review methodological details of publications simultaneously to peer review, envelopes as a form of allocation concealment are poorly implemented and reported, so careful review of these is required, funders need to insist on more robust allocation concealment methods are employed if the RCT


Introduction
Allocation concealment is a key criterion of quality for a rigorously conducted randomised controlled trial (RCT). It is the most basic methodological piece of information that should be reported 1,2 . Allocation concealment has been defined by the Consolidated Standards of Reporting Trials (CONSORT) group as: 'A technique used to prevent selection bias by concealing the allocation sequence from those assigning participants to intervention groups, until the moment of assignment. Allocation concealment prevents researchers from (unconsciously or otherwise) influencing which participants are assigned to a given intervention group.' Allocation Concealment, CONSORT.
A common misconception is that allocation concealment is the same as blinding 3,4 . Unlike allocation concealment, blinding is not always possible or even desirable to be carried out at all in RCTs 5 . This confusion may inadvertently lead to the use of poor allocation concealment.
Historically, sequentially numbered, opaque, sealed, envelopes (SNOSE) were used to conceal allocation sequences and are still widely used. Despite descriptions of 'relatively' tamper proof methods using envelopes 6 they can still be easily subverted, such as by opening them in advance of patient recruitment, therefore findings from RCTs using envelopes ought to be open to greater levels of scrutiny 3,7 . If envelopes are to be used then methods minimising tampering need to be put into place and reported. A more secure approach for allocation concealment is to use central randomisation-this is a method which is more difficult to subvert as it separates the randomisation sequence generation from the allocation of participants 3,7 . Research has shown that trials with inadequate or unclear allocation concealment are associated with increased effect sizes of greater than 40% 1 .
Unfortunately, RCT publications frequently do not report essential methodological factors in full, with many methodological elements, including allocation concealment, missing entirely 4,8 In 1996 the CONSORT statement was published, after a growing body of evidence identified a lack of consistent and high-quality reporting of RCTs 9 , with further revisions for different research designs 8,10,11 . Many, but not all, journals are now 'CONSORT endorsing' journals as they have adopted the CONSORT checklist for RCTs as a basic methodological checklist that authors should adhere to 12 .
Research recognises that insecure allocation concealment methods are still being implemented and reporting is still suboptimal despite CONSORT [13][14][15][16] . An area that needs exploring is understanding why personnel involved in the conduct of RCTs are implementing insecure methods and why these methods are still not being properly reported. Engaging with this group will enable targeted recommendations to be made to improve both the implementation and reporting of allocation concealment.

Aims of this research
-Pilot the use of a questionnaire to engage with authors of published RCTs to investigate their knowledge and understanding of allocation concealment implementation and reporting.
-Use these findings to ascertain areas and mechanisms which are acceptable to these authors to improve implementation and reporting of allocation concealment.
-Assess whether a questionnaire is an effective way to engage with authors of RCTs regarding methodological information to inform future research.
-Determine the reporting quality of allocation concealment in this sample by assessing the adherence to CONSORT.

Methods
We identified all RCTs published in one month in 2019 in the 'Abridged Index Medicus'. We do not have ethical approval to specify the month that the articles are published as this would identify the authors invited to participate in the research and not keep their identities secure. We undertook double data extraction of the randomisation method and allocation concealment method where the data was initially extracted and then second checked for any typological errors. We used this data to assess whether papers adhered to CONSORT. Table 1 details the criteria used. Robust method used to conceal the randomisation sequence from all personnel involved in the study until after allocation has occurred.
If envelopes were used as an allocation concealment method they are only judged as adequate if they were sequentially numbered sealed opaque envelopes (SNOSE), the personnel who created and opened them were both stated, and these were not the same person.
Central randomisation was judged as adequate Double-blind placebo-controlled trials were considered adequate as they are considered a robust trial design and difficult to subvert.
We contacted corresponding authors via the email address cited in the publication and asked them to anonymously fill in a short questionnaire 17 assessing their views on trial reporting. The questionnaire consisted of two questions with closed responses and one open ended question. The purpose of this questionnaire was to establish if authors could define, and identify what were adequate methods of, allocation concealment. Participants were asked to leave their email address if they consented to fill in an in-depth 18 item questionnaire. Both questionnaires were created and distributed using QUALTRICS 18 . A reminder email was sent two weeks after the initial, requesting participation in the survey.
The purpose of the second questionnaire was to elicit comprehensive knowledge and views on implementation, reporting quality and mechanisms to improve trial reporting quality in relation to allocation concealment.
Authors were sent the questionnaire and asked to respond within two weeks, after one week a reminder email was sent. The QUALTRICS survey was closed two months after the end date to ensure if there were late responses could still be submitted. For both questionnaires, where there were open/ free text boxes we performed a thematic analysis of the content of the questionnaire responses. The analysis was performed by LC and NM. Text was systematically examined, and themes identified by coding and categorising text according to key themes that emerged Data was initially organised in order to determine what participants knew about allocation concealment methodology, what method they had experience of using, if they knew what essential methodological information to report and what mechanisms could improve the reporting quality of RCTs and the opinions of suggestions to improve reporting quality of allocation concealment methodoloy These categories were further coded and mapped, as detailed participant responses were analysed to identify beliefs, experiences and barriers and facilitators to improving reporting quality, these lead to identifying and developing emergent themes.
An internal pilot was performed on each questionnaire with research staff in the York Trials Unit who had experience of being authors of RCTs and trial methodology, following piloting we reordered some of the questions but did not change the content Ethical approval was granted by the University of York Health Sciences Research Governance Committee in March 2019. Information regarding study information and the process of consent was provided at the start of the questionnaire, submission of the completed questionnaire by the participant was considered as implicit consent. We provided contact details to enable recipients of the questionnaire to contact the research team with any queries, advised them on the length of time the survey was expected to take, that all responses were anonymous and findings from the survey would be published.

Results
In total, 95 RCTs were identified 19 , and 87% (n=83) were published in CONSORT endorsing journals. Of these, 94 had a corresponding author email address. The corresponding authors were a combination of clinicians and academics.

Assessment of allocation concealment reporting
Of the identified RCTs, 75.8% (n = 72) reported the allocation sequence generation method and 71.6% (n =68) reported the allocation concealment method (see Table 2 for details), with CONSORT endorsing journals reporting these methodological details more frequently. Table 3 shows that almost half of RCTs in CONSORT endorsing journals did not report with sufficient clarity that a judgement could be made as to the robustness of allocation concealment. It also indicates that 21.1% (n=20) were double blind placebo blind trials which were judged as adequate, of these only 60% (n=12) reported their allocation sequence generation method.

Allocation concealment methods: envelopes and central randomisation
Envelopes were used to conceal the allocation sequence in17.9% (n=17) of the included RCTs, with 52.9% (n = 9) describing the security of the envelope, 29.4% (n=5) stating who prepared the envelopes, and 35.3% (n=6) stating who opened the envelope. The reporting quality of necessary methodological information was poor for studies using envelopes; only 17.6% (n=3) were published adequately with all necessary information to assess the methodological rigour reported. Central randomisation was performed in 33 (34.7%) RCTs.
Questionnaire analysis. In total, 12 (12.7%) anonymous authors returned the shorter questionnaire. The participants had a varied range of experience of how many RCTs they had conducted. Most participants (n=8) had conducted >3 RCTs with one participant stating they had conducted >300 RCTs. Six authors (6.4%) consented to being sent the second larger questionnaire and five authors completed this. From examining the email addresses, one participant was a clinician and five were academics. Figure 1 shows the summary of response rates to the questionnaires.

Allocation concealment implementation
All participants would use both central randomisation services and envelopes in future research. Of the participants, 80% (n = 4) believed that central randomisation should be the gold standard if appropriate to the trial design. Most participants supported the use of envelopes but did acknowledge that they were more at risk of subversion than central randomisation.

Envelope use
There was a large amount of support for the implementation of envelopes as an allocation concealment method where appro-  One participant selected that they did not think envelopes were an adequate allocation concealment method but they had used them in the past due to budget constraints.

Central randomisation
Central randomisation was accepted and described as being the 'best' with no elaboration as to why. One author considered the problematic reporting of central randomisation.
'Just saying its 'done centrally' [when reporting allocation concealment methods implemented] I also find problematic. If you can call someone up to randomise and they have an open allocation list, there is still a risk that there is some collusion between the two parties to ensure certain patients are randomised one way or the other' (ID 2.5)

Essential methodological information knowledge
We asked authors to list the top three methodological items to consider when conducting and reporting an RCT. Each response included the randomisation method employed with only one response stating that allocation concealment as important.
Of the responses gained, 75% included blinding as an important methodological item to report, which could suggest that blinding and allocation concealment are being confused as past research has highlighted 3,4 .
From the short questionnaire 60% of participants stated they could not define allocation concealment. The most general way of defining allocation concealment was to discuss how the randomisation sequence should be hidden, but with no consideration as to why as defined by CONSORT only one participant included selection bias in their response. This suggests that many of those involved within the conduct of RCTs do not appear to see allocation concealment as an essential element to report.
There was a misconception that allocation concealment was the same as blinding the patients and outcome assessors by one participant, when asked to state what is important to report in RCTs.

'Method of concealing the allocated treatments (blinding of patients and outcome assessors) [is important to report in RCTs]' (ID 2.4)
Envelopes: methodology knowledge We determined whether participants were aware of how to adequately implement and report envelopes in RCTs by asking them to state what essential methodological information is necessary. No author knew all essential quality factors to include within a publication. Only one participant stated that the individual who opened the envelope should be reported, and no authors stated that it was essential to report who prepared the envelopes. All participants knew that envelopes should be sequentially numbered, and that this should also be reported. Participants were mostly aware of additional security measures regarding the envelope that could be taken and reported to decrease the risk of subversion. 'There are probably extra measures which may help to prevent subversion such as instructing staff to write the patient name etc. on the envelope prior to opening it, and ensuring that the seal is tamper proof -not e.g. a staple which can be carefully unpicked and re-stapled if someone was so motivated. A wax seal comes to mind although that sounds rather impractical too!' (ID 2.5) Investigator integrity emerged as a theme here, with participants claiming that investigators had to implement envelopes correctly but not acknowledging what needed to be reported. This also supports the theme that there is a lot of focus on trial conduct rather than trial reporting, this is discussed further on in this publication

'If I had to pick something [essential methodological information when reporting envelope use within an RCT] it would be processes to ensure adherence to correct use of the envelopes/prevent violation of the study protocol.' (ID 2.1)
Improving reporting quality Participants recognised that reporting quality needed improving, and different mechanisms were explored: Role of journal editor. Of those surveyed, 80% (n=4) believed that the journal editor should ensure that key methodological information is reported within an RCT publication. It was also acknowledged that the editor does not necessarily have the expertise to do this.

'Journal editors should do their best to review methodological information but if they don't have sufficient methodology expertise across all experimental designs that may be included in submis-
sions to their journal, then they are not the most appropriate person for this task' (ID: 2.1)

Word limits imposed by journals.
We asked participants if they thought word limits imposed by journals impacted the amount of methodological information reported and thus the quality of the publication. There was agreement that although it could impact the quality, increasing the word count would most likely not help as the additional word limit may not be used to report the methodology.
'Increasing journal word counts is merely one mechanism to facilitate (but not ensure) more comprehensive reporting of methods/findings. At authors discretion they could simply use the additional word count to expand their discussion' (ID 2.1) Pre-submission checks. We found there was minimal support from our surveyed population for journals to perform presubmission checks on a manuscript prior to full submission.
We proposed that key methodological information of an RCT including allocation concealment and randomisation procedures were submitted to a journal for review. Only those manuscripts that include transparent and detailed methodological information would then be invited to submit their full manuscript. This could lead to improved reporting quality of allocation concealment and randomisation methodology.
Linked to this was the criticism of journals and the time it takes for the peer review process to be completed. This influenced participant opinions when considering improvements to reporting quality. Although all participants agreed there was a need to increase reporting quality, they did not want to add any additional barriers to slow down publications.
'It could add value but the length of the peer review process already substantially delays the release of scientific advances, and adding another rate-limiting step would exacerbate the delay' (Paper ID 2.1)

'All these points are important [to improve reporting quality] but I don't want more "process" [when trying to submit a manuscript to a journal]' (ID 2.3)
Methodologist. The largest support was given to a methodologist always assessing the methods section of any publication submitted to a journal, similar to the statistical review some journals subject manuscripts to. Support was given to this if it was performed in a streamlined way and done simultaneously whilst the manuscript was going through the peer review process; participants were not supportive of any method of improve=ing trial reporting that they perceived to hold the process up further.
'There is no reason why specialised methodology review could not be included as part of the standard external peer review process.' (ID 2.1)

Role of Funders.
Support for funders allocating specific funding for a third-party randomisation service if the RCT setting and design suited this emerged.
'These [allocated funding for a third-party randomisation service] can help ensure robust methods before trial conduct, and thus the quality of the trial and report' (ID 2.2) There was also a suggestion for funding organisations to mandatorily require and fund external independent assessors

'It would strengthen the robustness of any trial results if funding organizations would demand and financially support external independent assessors' (ID: 2.4)
Both of these points however fail to recognise that robust trial design and methodology does not necessarily mean the RCT publication will be written to reflect this; these suggestions will not prevent poor reporting quality they only impact trial conduct. This theme is considered later in this manuscript.
Awareness of and adherence to guidelines All participants stated that they used CONSORT guidelines when writing their RCT. Participants emphasised that some authors may not be aware of CONSORT, and that even though some journals do endorse the CONSORT statement there is a difference in how they are adhered to between journals which may impact reporting quality. There was universal support for CONSORT guidelines and strong agreement that there were adequate guidelines and support to aid the robust and transparent publication of research; rather it was adherence to the existing guidelines that is needed, with a focus on researchers ensuring this occurred.
'Guidelines for acceptable processes are readily available and reporting of that information is commonly required in key research reporting guidelines. If all researchers adhere to those recommendations we should have very little risk of selection bias in published research…..Spend the effort/time/money advocating for better adherence to existing reporting guidelines.' (ID 2.1) Linked to this was that there were already mechanisms in place to enable trials to be reported adequately and that these need to be adhered to.
'Many journals require submission of a supplementary document demonstrating adherence to an appropriate reporting guideline, which commonly directs readers to the appropriate section/page of a manuscript to find relevant information.' (ID 2.1) The translation of trial design and conduct to the publication A recurring theme was the idea that if the trial was conducted robustly it would lead to a robust publication automatically. A lot of emphasis was placed on those researchers conducting themselves correctly, with integrity and following the protocol with the implication this would lead to a robust publication.
'These options [suggestions to aid reporting quality] can help ensure robust methods before trial conduct, and thus the quality of the trial and report' (ID 2.2) '[to improve reporting quality] ensure adherence to correct use of the envelopes/prevent violation of the study protocol' (ID: 2.1)

Discussion
Our research highlighted again that there is suboptimal implementation and reporting of allocation concealment. Only 57% of RCTs published in CONSORT endorsing journals contained adequate allocation concealment methods.
Analysis of the questionnaire data indicated that some authors could not accurately define allocation concealment, and it was at times confused with blinding. Only one participant considered it to be within the top three methodological elements of an RCT to report, indicating that it may not be reported adequately as it is not seen as a priority.
To improve the reporting of allocation concealment there was support for a methodologist to review submitted articles to ensure essential methodological information was reported.
There are a number of barriers to operationalise this particular mechanism. One solution could be to nominate a trial team member to be the guarantor for required methodological information on submission of a manuscript, if journals do not have the capacity to support this specialist process.
The views of this sample of participants suggest that envelopes are going to be employed as a method of allocation concealment in future research for a variety of reasons such as budget, but authors had poor knowledge on which methodological factors should be reported.
Central randomisation was accepted as the gold standard but cost was a barrier for its implementation. Authors did not consider that choosing a less expensive allocation concealment method could result in inadequate methods being employed, which, could lead to research that is judged as a high risk of bias 20,21 . We agree with the discussion point raised by a participant stating that randomisation that was 'performed centrally' does not give the reader enough information on what was carried out, this is suboptimal reporting.

RCT author engagement
This research demonstrated that a survey elicits a low response rate when engaging with authors regarding allocation concealment knowledge, and different mechanisms to engage are needed. The corresponding author of a publication in healthcare is frequently a clinician and not a methodologist, which could explain the low response rate to a questionnaire about methodological information. We intentionally did not limit word counts within the free text boxes to enable full exploration and elicitation of views, this enabled us to capture rich data. Despite the response rate the data was consistent amongst the participants, those that did respond showed high engagement with the topic and this has enabled a targeted list of recommendations to be made on improving the implementation and reporting of allocation concealment for this sample.

Limitations
There are many methodological quality factors that could have been addressed when assessing reporting quality. Nevertheless, allocation concealment was our focus due to it being the central part of an RCT. There was a low response rate to the questionnaire with the majority of responses from academics rather than clinicians, which limits the generalisability of the findings amongst the RCT author population. The low survey response rate is a finding that has been observed in other research with a trend of decreasing participant engagement over time [22][23][24] , it is postulated that frequent survey requests and work schedules may influence participation, particularly with health professionals 25 . We did not repeat the survey to increase the sample size as an aim of this research was to assess the effectiveness of this method on author engagement, the response rate demonstrated that this method did not yield a high participation rate for this research area.

Future research
Further work into how best to engage with RCT authors is necessary, interviews may be a better method to engage and unpick what is underpinning suboptimal implementation and reporting of allocation concealment. Identifying who is the methodologist within the author list and engaging with this individual may enable us to determine how the methods section was written and reported and why poor reporting was occurring by examining the internal editing processes that occurs within research teams, including any comments received from reviewers and methodological information that could have been lost during the publication process. Clinicians were under represented in this research, and exploring their views may elicit different mechanisms to improve allocation concealment implementation and reporting.
Work into exploring different mechanisms and reporting platforms where there is an instant publication and a transparent reviewing process is recommended (such as with F1000 Research) which could lead to recommendations in speeding up reviewing time and processes which was highlighted as an issue with this surveyed sample.

Recommendations and conclusion
This research highlighted that there needs to be greater knowledge of allocation concealment amongst authors of RCTs, as well as the understanding that securely implementing it does not automatically generate a well written RCT publication.
From the findings of this research to improve allocation concealment implementation and reporting we recommend: -CONSORT is endorsed and adhered to by each journal as it was accepted by all participants in this research -Trials using envelopes need to be discouraged and if they are to be used then they should be robustly implemented and reported with careful checking that key methodological information is included in publications, as knowledge around this was suboptimal -Funders assess proposed methodology and insist on the use of robust allocation concealment methods if the RCT setting allows (such as central randomisation) -If central randomisation is implemented full methodological details are reported within the publication, including the names of external centres involved and the method of randomisation (simple, restricted) and details such as stratification variables and block size.
-Authors need to take responsibility for ensuring key methodological factors are included in their RCT publications and should report all methods transparently even if not methodologically robust -A methodologist reviews the RCT publication to ensure key methodological information is included simultaneously to the peer review process. Although this is a recommendation based from the findings of this research the difficulties in the practicalities is discussed. We can only currently share the questionnaires sent to participants and the non-identifiable data from the data extraction of the allocation concealment judgement where we identified included studies. We are only able to share the data extracted related to the journal CONSORT endorsement, quality judgements we made on the allocation concealment methods and the breakdown of different methods used (such as central randomisation and envelopes use), as to share the full extracted methods section and title from each included paper would enable the paper and therefore participants to be identified.

Data availability
If readers or reviewers would like to apply for access to the data they would have to contact Laura Clark at laura.clark@ york.ac.uk. We are required to liaise with the University of York Health Sciences Research Governance Committee (https:// www.york.ac.uk/healthsciences/research-information/rsg/) to request permission to share the requested data and adhere to the conditions under which they will grant access.

The benefits of publishing with F1000Research:
Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com