|
Sign In to gain access to subscriptions and/or personal tools.
|
The Interpretation of Random-Effects Meta-Analysis in Decision Models
A. E. Ades, PhD
Medical Research Council, Health Services Research Collaboration, Canynge Hall, Bristol, UK, I.Ades{at}bristol.ac.uk
G. Lu
Medical Research Council, Health Services Research Collaboration, Canynge Hall, Bristol, UK
J. P. T. Higgins, PhD
Medical Research Council Biostatistics Unit, Institute of Public Health, Cambridge, UK
This article shows that the interpretation of the random-effects models used in meta-analysis to summarize heterogeneous treatment effects can have a marked effect on the results from decision models. Sources of variation in meta-analysis include the following: random variation in outcome definition (amounting to a form of measurement error), variation between the patient groups in different trials, variation between protocols, and variation in the way a given protocol is implemented. Each of these alternatives leads to a different model for how the heterogeneity in the effect sizes previously observed might relate to the effect size(s) in a future implementation. Furthermore, these alternative models require different computations and, when the net benefits are nonlinear in the efficacy parameters, result in different expected net benefits. The authors’ analysis suggests that the mean treatment effect from a random-effects meta-analysis will only seldom be an appropriate representation of the efficacy expected in a future implementation. Instead, modelers should consider either the predictive distribution of a future treatment effect, or they should assume that the future implementation will result in a distribution of treatment effects. A worked example, in a probabilistic, Bayesian posterior framework, is used to illustrate the alternative computations and to show how parameter uncertainty can be combined with variation between individuals and heterogeneity in meta-analysis.
Key Words: variation • uncertainty • random effects • meta-analysis • probabilistic decision analysis • Bayesian
References
- National Research Council. Science and Judgment in Risk Assessment. Washington, DC: National Academy Press; 1994.
- Hattis D, Burmaster D. Assessment of variability and uncertainty distributions for practical risk analysis. Risk Anal. 1994;14:713-730.[CrossRef]
- Thompson KM, Graham JD. Going beyond the single number: using probabilistic risk assessment to improve risk management. Human and Ecological Risk Assessment. 1996;2:1008-1034.[Web of Science]
- Nayak TK, Kundu S. Calculating and describing uncertainty in risk assessment. Human and Ecological Risk Assessment. 2001;7: 307-328.[CrossRef]
- Thompson KM. Variability and uncertainty meet risk management and risk communication. Risk Anal. 2002;22:647-654.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Critchfield GC, Willard KE. Probabilistic analysis of decision trees using Monte Carlo simulation. Med Decis Making. 1986;6:85-92.[Abstract/Free Full Text]
- Doubilet P, Begg CB, Weinstein MC, Braun P, McNeill BJ. Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach. Med Decis Making. 1985;5:157-177.[Free Full Text]
- Thompson KM, Burmaster DE, Crouch EAC. Monte Carlo techniques for quantitative uncertainty analysis in public health risk assessments. Risk Anal. 1992;12:53-63.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Eddy DM, Hasselblad V, Shachter R. Meta-analysis by the Confidence Profile Method. London, UK: Academic Press; 1992.
- Dilks DW, Canale RP, Meier PG. Development of Bayesian Monte Carlo techniques for water quality model uncertainty. Ecol Modell. 1992;62:149-162.[CrossRef]
- Ades AE, Cliffe S. Markov Chain Monte Carlo estimation of a multi-parameter decision model: consistency of evidence and the accurate assessment of uncertainty. Med Decis Making. 2002;22:359-371.[Abstract/Free Full Text]
- Hunink M, Glasziou P, Siegel J, et al. Decision Making in Health and Medicine: Integrating Evidence and Values. Cambridge, UK: Cambridge University Press; 2001.
- Hoffman FO, Hammonds JS. The propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Anal. 1994;14:707-712.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Bogen KT. Methods to approximate joint uncertainty and variability in risk. Risk Anal. 1995;15:411-419.[CrossRef]
- Frey CH, Rhodes DS. Characterizing, simulating, and analyzing variation and uncertainty: an illustration of methods using an air toxic emissions example. Human and Ecological Risk Assessment. 1996;2:762-797.
- Rai SN, Krewski D. Uncertainty and variability analysis in multiplicative risk models. Risk Anal. 1998;18:37-45.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Krewski D, Rai SN, Zielinski JM, Hopke PK. Characterization of uncertainty and variability in residential radon cancer risks. Ann NY Acad Sci. 1999;895:245-272.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Burmaster DE, Wilson AM. An introduction to second-order random variables in human health risk assessment. Human and Ecological Risk Assessment. 1996;2:892-919.
- DerSimonian R, Laird N. Meta-analysis of clinical trials. Control Clin Trials. 1986;7:177-188.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Smith TC, Spiegelhalter DJ, Thomas SL. Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med. 1995;14:2685-2699.[Web of Science][Medline]
[Order article via Infotrieve]
- Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18:2693-2708.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Higgins J, Thompson S, Deeks J, Altman DG. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002;7:51-61.[Abstract/Free Full Text]
- Petitti DB. Meta-analysis, Decision Analysis, and Cost-Effectiveness Analysis: Methods for Quantitative Synthesis in Medicine. New York, NY: Oxford University Press; 2000.
- Mandelblatt JS, Fryback DG, Weinstein MC, Russell LB, Gold MR, Hadorn DC. Assessing the effectiveness of health interventions. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-Effectiveness in Health and Medicine. New York, NY: Oxford University Press; 1996. p 135-175.
- Spiegelhalter DJ, Thomas A, Best N, Lunn D. WinBUGS User Manual: Version 1.4. Cambridge, UK: MRC Biostatistics Unit; 2001.
- Ades A, Lu G, Claxton K. Expected value of sample information calculations in medical decision modeling. Med Decis Making. 2004;24:207-227.[Abstract/Free Full Text]
- Thompson SG, Smith TC, Sharp SJ. Investigating underlying risk as a source of heterogeneity in meta-analysis. Stat Med. 1997;16:2741-2758.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21:1559-1574.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Glasziou PP, Sanders SL. Investigating causes of heterogeneity in systematic reviews. Stat Med. 2002;21:1503-1511.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Thompson SG. Why sources of heterogeneity in meta-analyses should be investigated. BMJ. 1994;309:1351-1355.[Free Full Text]
- Davey Smith G, Egger M, Phillips AN. Meta-analysis: beyond the grand mean? BMJ. 1997;315:1610-1614.[Free Full Text]
- Lau J, Ioannidis JPA, Schmid CH. Summing up evidence: one answer is not always enough. Lancet. 1998;351:123-127.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- McIntosh MW. The population risk as an explanatory variable in research synthesis of clinical trials. Stat Med. 1996;15:1713-1728.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
Medical Decision Making, Vol. 25, No. 6,
646-654 (2005)
DOI: 10.1177/0272989X05282643
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
G. Lu and A. Ades
Modeling between-trial variance structure in mixed treatment comparisons
Biostat.,
October 1, 2009;
10(4):
792 - 805.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
G. Livesey and R. Taylor
Fructose consumption and consequences for glycation, plasma triacylglycerol, and body weight: meta-analyses and meta-regression models of intervention studies
Am. J. Clinical Nutrition,
November 1, 2008;
88(5):
1419 - 1437.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A E Ades, N. J Welton, D. Caldwell, M. Price, A. Goubar, and G. Lu
Multiparameter evidence synthesis in epidemiology and medical decision-making
J Health Serv Res Policy,
October 1, 2008;
13(suppl_3):
12 - 22.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. J. Sutton, N. J. Cooper, S. Goodacre, and M. Stevenson
Integration of Meta-analysis and Economic Decision Modeling for Evaluating Diagnostic Tests
Med Decis Making,
September 1, 2008;
28(5):
650 - 667.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. P A Ioannidis, N. A Patsopoulos, and H. R Rothstein
Reasons or excuses for avoiding meta-analysis in forest plots
BMJ,
June 21, 2008;
336(7658):
1413 - 1415.
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. Helfand and L. Saxton
Medical Decision Making and Electronic Publishing
Med Decis Making,
March 1, 2007;
27(2):
98 - 100.
[PDF]
|
|
|
|
|
|
|
N.J. Welton, I.R. White, G. Lu, J.P.T. Higgins, J. Hilden, and A.E. Ades
Correction: Interpretation of Random Effects Meta-analysis in Decision Models
Med Decis Making,
March 1, 2007;
27(2):
212 - 214.
[PDF]
|
|
|
|
|
