Abstract
This paper focuses on basic meta-analyses using the updated RevMan Web version, based on the Cochrane Handbook of Systematic Reviews of Interventions for clinical trials. Theoretical statistical knowledge, such as the REML method for estimating heterogeneity variance in random-effects meta-analyses, the HKSJ method for reflecting the uncertainty of pooled estimates, and the prediction interval in a random-effects model for exploring true treatment effects in a future trial, is briefly described. Examples with synthetic data are presented to help with the understanding of meta-analysts.
-
Keywords: RevMan web; Random effects meta-analysis; HKSJ method; Prediction interval
Introduction
Meta-analysis is a quantitative method that synthesizes results from two or more separate studies under systematic review or the development of clinical practice guidelines. Meta-analysis of randomized trials generates an overall pooled estimate with its confidence interval that summarizes the effectiveness of an experimental intervention compared with a comparison [
1]. There are many statistical software packages such as SAS, R studio, STATA, SPSS and Cochrane's Review Manager (RevMan) with free access or a subscription [
2].
However, clinicians, nurses, or researchers need up-to-date theoretical knowledge and properly implement the software program to conduct a correct meta-analysis. Especially, researchers should consider the following three methods whether they apply the up to date approach in their random effects meta-analysis: the first, the Cochrane Handbook for Systematic Reviews of Interventions updated that the Restricted Maximum Likelihood (REML) method offers a more reliable estimation of heterogeneity variance of random-effects meta-analyses rather than the DerSimonian and Laird moment-based method, because the fact that most systematic reviews and meta-analyses do not have enough studies to allow for reliable investigation of heterogeneity’s causes [
3]. In the RevMan web version, Restricted Maximum Likelihood (REML) is set as the default method for estimating between-study variance, although the DerSimonian and Laird moment-based approach is retained as an additional option [
4].
Second, meta-analysts often face the problem of a small number of available studies due to the lack of large trials. Moreover, small studies have been found to show more heterogeneity than large trials [
5]. When the number of studies is small with different sample sizes and there is moderate or substantial heterogeneity, the Hartung–Knapp–Sidik–Jonkman (HKSJ) method yields more accurate confidence intervals for the summary effect compared with the commonly used DerSimonian and Laird random effects method [
6]. By constructing the confidence interval based on the t-distribution with k-1 degrees of freedom, the HKSJ method improved coverage probability compared to the DerSimonian and Laird method using a Wald-type statistic based on the standard normal distribution [
7]. This approach generally inflates the variance of the summary effect and widens the confidence interval to reflect the uncertainty in estimating between-study heterogeneity. However, when the number of studies included in the meta-analysis is few or the dataset contains rare events, HKSJ confidence intervals may become overly wide [
5]. Hence, confidence intervals must be calculated using Wald-type CI methods when meta-analyses include two or fewer studies [
4]. In contrast, the use of the HKSJ method is recommended when the between-study variance estimate exceeds zero and the number of available study results is greater than two, although the default confidence interval method for the summary effect in the RevMan web version is the Wald-type method [
1].
Third, neither of the measures of between-study heterogeneity such as τ², nor the inconsistency measure I², can provide insight into the clinical implications of the observed heterogeneity. The prediction interval in a random-effects model contains highly probable values for the true treatment effects in future settings, if those settings are similar to the settings in the meta-analysis[
8]. The confidence interval only addresses the accuracy of the combined effect of the existing studies in a meta-analysis, whereas the prediction interval encompasses both the uncertainties in the combined effect and the potential heterogeneity between a future study and the existing evidence [
9]. The values in the prediction interval can be compared with clinically relevant thresholds to see whether they correspond to benefit, null effects or harm [
8]. Depending on the extent of heterogeneity, the prediction interval is generally wider than the corresponding confidence interval; in the absence of observed heterogeneity, the prediction interval is identical to the confidence interval [
9]. However, in a meta-analysis with few studies (eg, fewer than 5 studies), the prediction interval may be particularly imprecise, resulting from an imprecise estimation of the summary effect size and heterogeneity parameter [
10]. Prediction intervals can optionally be calculated in RevMan web.
Step-By-Step Guide To Meta-Analysis Using Revman Web Version
Access to the RevMan web
The RevMan web version is available through
https://revman.cochrane.org/info. Recently, Cochrane RevMan was changed from free software to an annual subscription. Cochrane reviewers are available free in RevMan, whereas the others require a yearly subscription. In updating profiles, the discount rate is automatically applied for students and academics.
Using the Cochrane account, users can log in to RevMan. Individuals or Cochrane users create and manage an unlimited number of reviews to conduct meta-analysis and present the results in forest plots with funnel plots [
11]. In this article, synthetic data were used to compare high-intensity laser therapy (HILT) with controls for pain reduction in musculoskeletal conditions, including low back pain, frozen shoulder, and neck pain, as examples. The paper focused on basic meta-analysis, although RevMan web allows the import of Risk of Bias 1 or 2 assessments for both Cochrane and non-Cochrane intervention reviews, and it supports GRADEpro Guideline Development Tool integration exclusively for Cochrane reviews.
After creating a Cochrane account, users could log in and add a practice review to try out this software for 30 days without a subscription. Practice reviews will be available to only one person and cannot be exported. If users are subscribers of RevMan web version, researchers can create a new review, including the title, and select review types such as intervention review, rapid review, prognosis review, and others, by clicking 'New review' as a permanent review in the Portfolio (
Fig. 1).
After generating my reviews, the users need to click on the review, the Dashboard will be displayed on the webpage. In the left column, the first item under the Data section is Review Criteria, which defines the eligibility framework for the review. These include Interventions, Intervention Groupings, Outcomes, Covariates, Characteristics, and Risk of Bias.
Interventions specify the most detailed list of eligible interventions and controls. Intervention Groupings enable the restructuring of multi-arm trials (more than two interventions) into two-arm groups. Outcomes define the endpoints of interest, allowing users to specify outcomes with descriptions, data types, units of measurement, and directions (e.g., higher values indicate improvement, while lower values indicate deterioration). Among the Review criteria, researchers should specify Interventions, Intervention groupings, Outcomes, and Risk of bias to conduct either automatic or manual analysis. However, if researchers plan to conduct subgroup analyses using study-centric data, covariates need to be determined at this stage. Covariates indicate study characteristics that might influence the size of an intervention effect (
Fig. 2).
In RevMan, there are two approaches: manual input analyses or study-centric data analyses (
Table 1). Manual input analysis is aligned with the previous RevMan 5 software, in which researchers manually input each study's data, prepared using a wide-form dataset. The characteristic of wide format is that all information pertaining to a single entity is contained within one row. If the outcome data type is binary, the event numbers and the total numbers in the result data in each arm should be included. For continuous outcome variables, the mean and standard deviation of each study arm are required to conduct manual input analysis in the RevMan Web version.
Study-centric data analyses are new data analytic ways introduced in the RevMan web version. According to Cochrane RevMan Web Knowledge base [
12], study-centric data management constitutes an efficient approach to data handling in meta-analysis. By defining synthesis criteria and planned analyses in RevMan at the protocol stage, reviewers can extract data through standardized templates, using either arm-level data or contrast-level data with a long-form data format (
Fig. 3). This allows for the subsequent import of study data with minimal procedural effort and the automatic population of the analyses. This process reduces dependence on manual data entry and minimizes the risk of error.
To conduct correct study-centric data analyses, users should consider the following points. First, users should prepare data extraction templates that are aligned exactly with the “Review criteria”. For example, suppose researchers input “Outcome” as an “adverse events” in the “Review criteria”. In that case, the Outcome variable of the standardized templates needs to be entered as “adverse events” rather than “adverse event”. Second, two data types might be used in the standardized templates of RevMan web version. The contrast level in the data type refers to the relative treatment effects (e.g., natural log risk ratio, ln (RR), or mean difference, along with standard error and 95% confidence intervals) across the trials. Whereas the arm-level data type indicates data for all study arms that are available (e.g., number of cases/events and total numbers for each group, or mean, standard deviation, and total numbers for each arm). In automatic analyses under “Data source”, users could choose to include a) only arm-level data, b) only contrast-level data, c) contrast- and arm-level data (preferring arm-level data where both exist), d) contrast- and arm-level data (preferring contrast-level data where both exist), depending on their prepared data template from studies included in the meta-analysis [
13].
Study-centric data (automatic) analysis composed of a long-form dataset
Researchers should plan how to perform meta-analyses during the protocol phase to enable automated analysis with study-centric data input. After creating the “Review criteria” aligned with standardized templates, users can import their results data for automatic analyses in Dashboard, and the analyses will be populated in one go. Finally, reviewers should review the analyses that RevMan has automatically created. When importing study data into the Dashboard, users select the file type to upload, which can be either a CSV file or a JSON file. Because downloaded standardized data extraction templates are CSV files, users carefully manage the file to import CSV files, not Excel files. Both the study arms and the study results data CSV file, previously prepared in a long-form dataset, are required to conduct a meta-analysis. When files contain inconsistent data with the Review criteria, validation issues occur. After users need to resolve potential validation errors, and then click “import study data” to complete the automatic analyses. Researchers go back to the analyses and find the forest plot in the Analyses (
Fig. 4).
Manual input analysis using a wide-form dataset
To perform manual input analysis, reviewers need to enter arm-level data in “Study arms” and “Result data” as well as new study information such as General after clicking “add study” in “included” of “Studies” in “Data” section (
Fig. 5). Users can select “Add data row” to add relevant studies from the list of included studies. In manual analysis, like the previous RevMan 5 software, users can enter data by typing it manually into the table cells or pasting data in tabular form from a wide-form dataset in an Excel file.
To create funnel plot of forest plot, users click “Figures” in left column of Default view, and click “ Add Figure” then scroll down in “Figure type” as funnel plot, and select the analysis (
Fig. 6).
Conclusion
The recently updated RevMan web version provides user-friendly software that enables researchers to conduct basic meta-analyses using either automatic or manual input analysis. Additionally, RevMan web is available for the implementation of up-to-date theoretical knowledge of meta-analyses, including the REML method to estimate heterogeneity variance of random-effects meta-analyses, the HKSJ method to reflect the uncertainty of the pooled estimates, and the prediction interval in a random effects model to explore the true treatment effects in a future study. However, because it does not support more advanced analytical methods, such as meta-regression or network meta-analysis, and requires an annual software subscription, alternative software programs, such as R Studio with free access, might be considered when conducting complex analyses.
Notes
Conflict of Interest
No potential conflict of interest relevant to this article was reported.
Data Availability Statement
All data generated or analyzed during this study are included in this published article.
Ethics Approval and Consent to Participate
Not applicable.
Authors Contributions
Study conception, Design acquisition, Drafting and critical revision of the manuscript - HJS.
Acknowledgments
During the preparation of this work, the authors utilized ChatGPT (OpenAI) to enhance the clarity of the language. The author reviewed and edited the content as needed and takes full responsibility for the content of the published article. A preliminary version of this work was presented as an educational webinar at the Korean Society of Evidence-Based Medicine in July 2025.
Fig. 1.Creation of my reviews, either practice review or permanent review, according to subscription.
Fig. 2.Set up the review criteria, including interventions, intervention groupings, and outcomes.
Fig. 3.Exemplar of study arms file and study results data file, prepared as arm-level data type in a long-form dataset.
Fig. 4.Main process of study-centric data analysis in RevMan Web.
Fig. 5.Manual input analyses prepared as arm arm-level data type.
Fig. 6.Creating a Funnel plot of a forest plot in Figures
Table 1.Types of the Two Analytic Approaches in the RevMan web Version
|
Manual input analyses |
Study-centric data(automatic) analyses |
|
Analyses without subgroups |
☑ |
☑ |
|
Subgrouping by a characteristic of the included studies |
☑ |
☑ |
|
define covariates and relevant categories in the Review criteria. |
|
Combining arms |
☑ |
☑ |
|
by using the calculator. |
|
Splitting control arms |
☑ |
☑ |
|
by splitting the control arm |
only relevant when subgrouping by intervention is conducted. |
|
Create analyses with contrast data |
☑ |
☑ |
|
Create analyses with different types of specifications of interventions |
☑ |
☑ |
|
Subgroup by the most granular interventions |
☑ |
☑ |
|
Subgroup by variants of the outcome |
☑ |
X |
|
Subgroups within studies |
☑ |
X |
References
- 1. Deeks JJ, Higgins JPT, Altman DG, McKenzie JE, Veroniki AA. Chapter 10: Analysing data and undertaking meta-analyses [last updated November 2024]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5: Cochrane; 2024. Available from: https://www.cochrane.org/handbook
- 2. Tantry TP, Karanth H, Shetty PK, Kadam D. Self-learning software tools for data analysis in meta-analysis. Korean J Anesthesiol 2021; 74: 459-61.
- 3. Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, et al. A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses. Res Synth Methods 2019; 10: 83-98.
- 4. Veroniki AA, McKenzie J. Introduction to New Random-Effects Methods in RevMan: Cochrane; [2024 Oct 23]. Available from: https://training.cochrane.org/sites/training.cochrane.org/files/public/uploads/Introduction%20to%20new%20random-effects%20methods%20in%20RevMan.pdf
- 5. Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol 2015; 15: 99.
- 6. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol 2014; 14: 25.
- 7. Sidik K, Jonkman JN. A simple confidence interval for meta‐analysis. Stat Med 2002; 21: 3153-9.
- 8. IntHout J, Ioannidis JP, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open 2016; 6: e010247.
- 9. Al Amer Fahad M, Lin Lifeng. Empirical assessment of prediction intervals in Cochrane meta-analyses. Eur J Clin Invest 2021; 51(7):e13524.
- 10. Spineli M, Pandis N. Prediction interval in random-effects meta-analysis. Am J Orthod Dentofacial Orthop 2020; 157: 586-8.
- 11. RevMan Feaures: Cochrane; 2025 [2025 Sep 1]. Available from: https://subscribe.cochrane.org/info/features#features
- 12. Study centric data management; 2025 [2025 Sep 1]. Available from: https://documentation.cochrane.org/revman-kb/study-centric-data-management-117379417.html
- 13. Arm vs Contrast level data; 2025 [2025 Sep 1]. Available from: https://documentation.cochrane.org/revman-kb/arm-vs-contrast-level-data-312606799.html
Figure & Data
Citations
Citations to this article as recorded by
