Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that enables research into pragmatic trials. It is a platform that collects and shares clean trial data and ratings using PRECIS-2 which allows for multiple and varied meta-epidemiological studies that compare treatment effects estimates across trials with different levels of pragmatism, as well as other design features.
Background
Pragmatic trials are becoming more widely acknowledged as providing evidence from the real world to support clinical decision-making. The term "pragmatic", however, is not used in a consistent manner and its definition and measurement require clarification. Pragmatic trials should be designed to guide clinical practice and policy decisions, rather than to prove the validity of a clinical or physiological hypothesis. A pragmatic trial should try to be as close as is possible to actual clinical practices which include the recruitment of participants, setting, designing, delivery and execution of interventions, determination and analysis results, as well as primary analysis. This is a significant distinction from explanatory trials (as described by Schwartz and Lellouch1) which are intended to provide a more complete confirmation of an idea.
Truely pragmatic trials should not conceal participants or the clinicians. This can result in an overestimation of the effect of treatment. Pragmatic trials will also recruit patients from different healthcare settings to ensure that the results can be applied to the real world.
Finally, pragmatic trials must concentrate on outcomes that are important to patients, like the quality of life and functional recovery. This is particularly important when it comes to trials that involve the use of invasive procedures or potentially dangerous adverse events. The CRASH trial29, for example focused on the functional outcome to evaluate a two-page case report with an electronic system for the monitoring of patients in hospitals suffering from chronic heart failure, and the catheter trial28 focused on urinary tract infections caused by catheters as the primary outcome.
In addition to these features pragmatic trials should reduce the trial's procedures and data collection requirements in order to reduce costs. Additionally pragmatic trials should strive to make their results as applicable to real-world clinical practice as they can by making sure that their primary analysis follows the intention-to treat approach (as described in CONSORT extensions for pragmatic trials).
Many RCTs that do not meet the criteria for pragmatism but contain features contrary to pragmatism have been published in journals of varying kinds and incorrectly labeled pragmatic. This can lead to false claims of pragmatism and the term's use should be made more uniform. The creation of a PRECIS-2 tool that provides an objective and standardized assessment of pragmatic features is a first step.
Methods
In a pragmatic study the aim is to inform policy or clinical decisions by demonstrating how an intervention can be integrated into routine treatment in real-world contexts. This is different from explanatory trials that test hypotheses regarding the cause-effect connection in idealized conditions. In this way, pragmatic trials could have a lower internal validity than explanation studies and be more susceptible to biases in their design, analysis, and conduct. Despite their limitations, pragmatic studies can be a valuable source of information to make decisions in the healthcare context.
The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatic). In this study the areas of recruitment, organisation, flexibility in delivery, flexible adherence, and follow-up received high scores. However, the principal outcome and the method for missing data was scored below the pragmatic limit. This suggests that it is possible to design a trial using excellent pragmatic features without harming the quality of the results.
It is difficult to determine the amount of pragmatism in a particular trial because pragmatism does not have a binary attribute. Certain aspects of a study can be more pragmatic than others. The pragmatism of a trial can be affected by modifications to the protocol or the logistics during the trial. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. The majority of them were single-center. They are not in line with the usual practice, and can only be considered pragmatic if their sponsors agree that such trials aren't blinded.
A common aspect of pragmatic research is that researchers try to make their findings more relevant by studying subgroups within the trial. This can result in imbalanced analyses and lower statistical power. This increases the risk of omitting or misinterpreting differences in the primary outcomes. In the case of the pragmatic trials included in this meta-analysis, this was a significant problem since the secondary outcomes weren't adjusted for differences in baseline covariates.
Additionally practical trials can have challenges with respect to the collection and interpretation of safety data. It is because adverse events tend to be self-reported and are susceptible to errors, delays or coding errors. It is important to improve the accuracy and quality of the outcomes in these trials.
Results
While the definition of pragmatism does not require that all trials are 100 100% pragmatic, there are advantages of including pragmatic elements in clinical trials. These include:
By including routine patients, the results of trials can be translated more quickly into clinical practice. However, pragmatic trials can also have drawbacks. The right type of heterogeneity, for example, can help a study expand its findings to different patients or settings. However the wrong type of heterogeneity could reduce the assay sensitivity and thus decrease the ability of a study to detect small treatment effects.
A variety of studies have attempted to classify pragmatic trials using a variety of definitions and scoring methods. Schwartz and Lellouch1 created a framework to differentiate between explanation studies that support the physiological hypothesis or clinical hypothesis, and pragmatic studies that help inform the selection of appropriate therapies in real world clinical practice. Their framework comprised nine domains, each scored on a scale of 1-5, with 1 being more informative and 5 indicating more practical. The domains included recruitment, setting, intervention delivery, flexible adherence, follow-up and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. Koppenaal et. al10 devised an adaptation of the assessment, called the Pragmascope which was more user-friendly to use for systematic reviews. They discovered that pragmatic systematic reviews had higher average score in most domains, but lower scores in the primary analysis domain.
This difference in primary analysis domain can be explained by the way most pragmatic trials analyse data. Some explanatory trials, however don't. The overall score was lower for pragmatic systematic reviews when the domains of the organization, flexibility of delivery and follow-up were combined.
It is important to remember that a pragmatic trial doesn't necessarily mean a low-quality trial, and in fact there is an increasing number of clinical trials (as defined by MEDLINE search, however it is neither specific or sensitive) which use the word "pragmatic" in their title or abstract. These terms may indicate that there is a greater awareness of pragmatism within titles and abstracts, but it isn't clear whether this is evident in the content.
Conclusions
As the value of real-world evidence grows widespread the pragmatic trial has gained momentum in research. They are clinical trials that are randomized that evaluate real-world alternatives to care instead of experimental treatments under development. They include patient populations which are more closely resembling the patients who receive routine care, they employ comparisons that are commonplace in practice (e.g. existing drugs), and they depend on participants' self-reports of outcomes. This method has the potential to overcome limitations of observational studies that are prone to limitations of relying on volunteers, and the limited availability and coding variability in national registries.
Other benefits of pragmatic trials include the possibility of using existing data sources, as well as a higher likelihood of detecting meaningful changes than traditional trials. However, they may be prone to limitations that compromise their credibility and generalizability. For example the participation rates in certain trials could be lower than anticipated due to the healthy-volunteer effect as well as incentives to pay or compete for participants from other research studies (e.g. industry trials). The necessity to recruit people in a timely manner also restricts the sample size and the impact of many practical trials. Additionally certain pragmatic trials lack controls to ensure that the observed differences aren't due to biases in the conduct of trials.
프라그마틱 슈가러쉬 of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatic. The PRECIS-2 tool was employed to evaluate the degree of pragmatism. It includes domains such as eligibility criteria, recruitment flexibility as well as adherence to interventions and follow-up. They found that 14 of these trials scored as highly or pragmatic sensible (i.e. scores of 5 or higher) in one or more of these domains, and that the majority of them were single-center.
Trials with a high pragmatism rating tend to have higher eligibility criteria than traditional RCTs, which include very specific criteria that are unlikely to be found in the clinical environment, and they contain patients from a broad range of hospitals. These characteristics, according to the authors, can make pragmatic trials more useful and useful in everyday clinical. However, they don't guarantee that a trial is free of bias. In addition, the pragmatism that is present in trials is not a predetermined characteristic; a pragmatic trial that does not possess all the characteristics of an explanatory trial can produce valuable and reliable results.