Pierpont, peer review, and expert witnesses

If I may jump in here as an editor of medical journals, I would just like to note that single case reports and case series are a major part of their content. That is how information is shared. When the authors are noticing something new (or more often, noticing something in a new way), they are clear about the limitations of a case series for making conclusive statements. For example, this is from an article I read last week about a new surgical technique:

It is important to note that there are limitations to our study, primarily because it is a case series. Therefore, we have no comparison group and there may be selection bias, though we attempted to minimize this by reviewing our entire endoscopic database and selecting all patients who underwent this procedure, not just a subset. Although this is the largest series presented to date, it still comprises only 12 patients, all of whom were seen at a single GI referral center with expertise in esophageal dilation and treatment of esophageal strictures. Therefore, the results may not be generalizable to other settings.

Those "limitations" did not prevent it from being published in a peer-reviewed journal. They are normal.

Nina Pierpont notes the limitations of her work (pp. 124-125): interview-only and limited medical records, comparison by memory, inaccurate reporting by subjects, English-only, small sample, limited follow-up. (She also describes how she endeavored to minimize each these effects.) On page 123, she notes the research that needs to be done to more definitively describe the syndrome, risk factors, and mechanisms. (Another limitation, mentioned in the text (p. 39-40), is that "participation by Americans was limited by non-medical factors such as turbine leases or neighbor contracts prohibiting criticism, court decisions restricting criticism of turbine projects, and community relationships".)

It is important to note that authors typically suggest the reviewers for their papers. It is not adversarial, as detractors seem to imply. Successful peer review just means you've been able to hold up your end of the conversation, not that you've slain all skeptics. It is more a process of refining the paper, as I believe Pierpont did with her panel of reviewers (and why in part it took so long to get into print), whose reports are on pp. 287-292. In turn, most journals also include a lively letters section in which the real peer review is shown to begin after publication.

Choosing the most effective expert witness must of course be done according to each case's focus and strategy, but I dare say, opposing counsel would have a lot harder time undercutting Pierpont's work than they would that of most "peer-reviewed experts".

In other words, "peer reviewed" is not a defense. Conversely, lack of one form of peer review (that for journal publication) is not much of a charge. A court room is perhaps the ultimate peer review.

These comments are in response to some of the reactions to Wind Turbine Syndrome that have been written. There is no question of WTS, whether you call it that or not. There remain questions of mechanism, but not of cause. And determining its extent and risk requires large epidemiologic studies. These are not criticisms. Pierpont says them herself. That is the process of science.

If only more people in power were so skeptical about wind industry claims!