| 课程目标与工具（Goals and Tools） |
At the end of this module, you will be able to:
Later in this module, we will have you look at a simple meta-analysis, and critically appraise it. Please have the following article available:
It is also available online, where you can view it or download a printable PDF file. |
| Meta分析概述 |
The second, third, and fourth modules have helped you learn how to read articles about diagnosis, therapy, and prognosis respectively. Now, it's time to learn about how to read an article about meta-analysis. While on the surface more complex and involved than most studies, by applying the principles learned in the first few modules you'll soon be comfortable reading and interpreting these studies. One area of confusion is the distinction between review articles, systematic reviews, and meta-analyses. The diagram below illustrates their relationship using a Venn diagram:
Review articles (also called overviews) are the broadest category. Most review articles are unsystematic, because the author does not look at all of the evidence. A systematic review has a formal approach to gathering, evaluating, and presenting the evidence. Some systematic reviews are meta-analyses; a meta-analysis goes the final step by using formal statistical methods to calculate a summary result or results. There are two major reasons to do a meta-analysis:
Meta-analyses of the first kind can help resolve medical controversies caused by conflicting studies, are an inexpensive alternative to very large randomized trials, and can in this way shape health policy. The second kind of meta-analysis is particularly useful for designing future studies, by systematically identifying key patient and study characteristics from previous work. Meta-analyses were initially used in the social sciences in the mid-1970's, and were adapted to medical data sets in the early to mid-1980's. The number of meta-analyses is growing rapidly, based on the results of this Medline search for the keyword "meta-analysis":
Meta-analyses often form the initial step of a cost-effectiveness analysis, decision analysis, or grant application. Once you've done all of that background work, why not publish it?!?! While the results usually correspond to later randomized trials, they do not always (LeLorier, 1997):
In the next section, you will learn the steps in a good meta-analysis. Understanding these steps is important, even if you never plan to do one yourself, because it helps you understand how to critically appraise this type of study. | ||||||||||||||||||||||||
| Meta分析的步骤 |
There are four basic steps to any good meta-analysis: We will discuss each of these steps below. 1. Identification The first step in a meta-analysis is to find all of the pertinent articles on your topic. Important sources of information for a meta-analysis include:
While MEDLINE, the database of the National Library of Medicine, is a good starting point, it is not the only source of information. MEDLINE indexes approximately 4100 journals, dating from 1966 to the present. It also has an excellent feature called clinical queries. There are also CD-ROM based search engines from BRS Colleague, WinSPIRS, and others which offer different search options, but use essentially the same underlying database. The European version of MEDLINE is called EMBASE, and is a Dutch/English collaboration. Depending on the topic, it may be appropriate to search the more specialized National Library of Medicine databases, such as CancerLit, AIDSLine, and ToxLine. The Cochrane Collaboration Controlled Trials Register, established in 1993, is an important source of studies for a meta-analysis. The Register includes abstracts of thousands of trials. It includes all controlled trials in the MEDLINE and EMBASE, as well as the results of hand searches by Cochrane Collaboration volunteers of thousands of journals not indexed by MEDLINE or EMBASE. MSU students, faculty and staff can best access the Cochrane through the MSU Library. Remember Index Medicus? I'm old enough to recall the intimidating row of thick (and I mean THICK) books that were my only way to find medical research articles as a medical student in the 1980's. They can still be useful when it is important to search for articles published before 1966, when MEDLINE and the other electronic databases were established. Finally, there are other sources of "fugitive literature" that may be important for the author of a meta-analysis (some of which may be found in the Cochrane Controlled Trials Register):
It's important to know that different search strategies can result in different results (Table 4.5, Petitti):
Note: CTR = Controlled Trials Register If you are thinking about doing a meta-analysis of your own, it is important to enlist the aid of an expert Medline searcher such as a medical librarian. The above table also highlights the importance of using the Cochrane Controlled Trials Register. 2. Selection Once the author of a meta-analysis has assembled a large number of studies, it is important to select the right ones! There are a variety of possible inclusion (also called eligibility) criteria:
3. Abstraction Once an appropriate group of studies has been identified, the author(s) have to abstract the relevant data from each study. There are many sources of potential error in data abstraction:
A good meta-analysis will take some or all of the following steps to minimize errors:
Bias can also creep into a meta-analysis. For example, the authors may be biased in favor of (or against!) well known researchers. Also, prominent journals may be given greater weight or authority (rightly or wrongly). It is therefore best (although not often done) to have identifiers eliminated from articles. Finally, part of the data abstraction phase is an assessment of study quality. Chalmers has proposed a fairly complex set of criteria which apply well to randomized controlled trials. Simpler criteria may be sufficient. For example, in a diagnostic meta-analysis, simply assuring a high quality gold standard, independent assessment of reference and study tests, and blinding may be adequate. Too often, the quality assessment is done, then ignored! Ideally, the results of the quality assessment should inform the analysis and interpretation of results. 4. Analysis There are many issues and controversies in the analysis of meta-analytic data. First, let's define some important terms:
Fixed and random effects models can give very different answers, and you can create examples where either model gives counterintuitive results (see Petitti, page 92). Usually, though, the answers provided by these different modeling assumptions are similar. Differences only arise when studies are nothomogenous. In a comparison of 22 meta-analyses, fixed and random effects models gave the same answer in 19 out of 22. In 3 cases, fixed effects models were significant while random effects models were not (Berlin, 1989 in Petitti textbook, pg 94). When there is significant heterogeneity, the between-study variance becomes much larger than the within, and studies of different sample size receive relatively similar weight. When there is homogeneity, sample size dominates, and both models give similar results. Random effects models are therefore more "conservative" and generate a wider confidence interval. Put another way, a random effects model is less likely to show a significant treatment effect than a fixed effects model. In general, if the studies are homogenous, the researchers should use a fixed effects model. If the studies are heterogeneous, the researchers (and you, the reader) should first ask why! While it may be appropriate to do a random effects analysis on all of the studies, it may be better to identify an important subgroup difference (i.e. studies using one dose showed significant effect, while lower dose did not) and then do a fixed effects analysis of each and report all of the results. A term you will encounter in many meta-analyses is "sensitivity analysis". A sensitivity analysis is a way of looking at only certain studies, certain groups of patients, or certain interventions. For example, a meta-analysis of aspirin in prevention of acute MI might first analyze all studies, but then also look separately at only studies of men and studies of women. The article by Hasselblad is an excellent starting point for budding meta-analysts, with lots of examples and formulae. Meta-analysis of diagnostic tests is another area of growing interest - how do you combine sensitivities, specificities, and so on. However, details of calculations for homogeneity, fixed effects models, and random effects models are beyond the scope of this course. Advanced students and those with a special interest in this topic may wish to review the following sections: |
| Meta分析证据的评价 |
Now that we have reviewed the steps in a meta-analysis, we can develop a formal approach to their critical appraisal. As with other kinds of studies, there are three basic steps:
A study which meets all three of these criteria is a POEM (Patient-Oriented Evidence that Matters). The first two criteria have to do with relevance, while the third is concerned with validity. 1. Would the results change my practice if valid? This question must be answered by the individual clinician. A meta-analysis which confirms existing practice need not be subjected to rigorous evaluation, since they results don't change what you are already doing. On the other hand, if the results would change what you do for your patients, it is incumbent on you to take the next step. 2. Are the outcomes important to my patients? Most meta-analyses use patient-oriented outcomes such as morbidity, mortality, and symptoms. However, if a study uses surrogate or intermediate outcomes such as FEV1 or hemoglobinA1C, the results should be interpreted cautiously. Proponents of the use of POEMs would argue that if a meta-analysis does not use patient-oriented outcomes, you don't have to proceed to the next step. 3. Are the results valid? If a study would change practice, and uses patient-oriented outcomes, you are obligated to evaluate its validity. Key criteria include:
In the next section, we will go into more detail on the evaluation of validity. For more on this approach to evaluating relevance, see the section on POEMs and DOEs. |
| Meta-Analysis真实性评价 |
In the previous section, we listed key criteria for the evaluation of a meta-analysis include. Let's talk a little more about each one, then work through a sample study. 1. Did the authors ask a focused clinical question? Better meta-analyses don't try to do it all. It's a lot like developing a question for your own research; consider the following questions as examples:
The second question is more focused regarding treatment (antibiotics), patient population (otherwise healthy adults), and diagnosis (acute bronchitis) than the first. 2. Were the criteria used to select articles for inclusion appropriate? Inclusion criteria can include criteria involving the treatment, patient population, study design, and diagnosis. Possible problems include:
3. Is it unlikely that important, relevant studies were missed? Remember, just using MEDLINE may not be adequate, and a meta-analysis may be more convincing if it includes unpublished trials and studies from the Cochrane Controlled Trials Register which don't appear in MEDLINE. 4. Was the validity of the included studies appraised (study quality)? Assessment of study quality means that the authors carefully read each study, and rated it on a number of quality measures. Important aspects of quality for an RCT include:
In addition, do the authors of the meta-analysis use this information? For example, do they drop studies which don't meet a certain minimum level of quality, or do they calculate results for high-quality studies separately from those for all studies or low-quality studies? 5. Were assessments of studies reproducible (data abstraction)? There should be a description of how data were abstracted. Ideally, at least two people should abstract each study, then compare results and have a formal mechanism for resolving conflicts. 6. Were the results similar from study to study (homogeneity)? When study results are homogenous, using a Q statistic or chi-square test, it is much more likely that the meta-analysis reflects "the truth". When studies are heterogeneous (i.e. some find benefit, some do not) the authors should be very cautious about combining the results, and if they do should use a random effects model. Now, let's look at the meta-analysis we asked you to pull earlier with these questions in mind. This article was selected because it is a topic of interest to all generalist physicians, and is fairly straightforward. |
| Meta-Analysis结果使用 |
Most meta-analyses will initially give you a table of included studies. In our study, that table is published on the Web only. We see that most studies used doxycycline, sulfa/trimethoprim, or erythromycin. This is a possible limitation - maybe the pharmaceutical reps are right, and Biaxin or Zithromax are really better, although I doubt it! Let's look at the results now. The three key figures are reproduced below. In Figure 2, the first column has the study name; the second column the proportion receiving antibiotic with productive cough at follow-up; the third column the proportion receiving placebo with productive cough at follow-up. Let's stop and think here. If antibiotics work, patients receiving them should be less likely to have cough at follow-up. That would make the relative risk of cough less than 1.0. In the light blue "relative risk diagram" below, the vertical line indicates a relative risk of 1.0. The horizontal dots and bars represent the relative risk and 95% confidence interval for each study. The weight is proportional to the inverse of the variance of the study - high variance, associated with a small study, means less weight given to that study, and vice versa. The relative risk is shown numberically in the final column of each table. Finally, the summary estimate of relative risk with its confidence interval is shown in the last row, along with an estimate of homogeneity (the chi-square). Figure 2. Productive cough at 7 to 11 days follow-up.
Looking at this table, you see that the summary estimate of relative risk just barely includes one, suggesting that the association is NOT statistically significant. Look at the next figure below: Figure 3. Failure to clinically improve at 7 to 11 days follow-up.
In this case, the confidence interval of the summary relative risk also includes 1.0 (it is 0.36 - 1.09). Figure 4. Adverse drug effects.
Finally, while there appear to be more adverse drug effects with antibiotic use, this association is also not statistically significant. So, have we learned nothing? No. Even meta-analyses can suffer from Type II error, i.e. too small a sample size. The total number of patients studied was only about 600 in the 6 studies, and it is interesting to note that while the results were not significant, they were consistent. That is, 5 out of 6 studies fell either to the left or right of the vertical bar for each outcome. |
