De Cochrane Collaboration en 'evidence-based medicine'

Brisbane (Australia), Oxford (England), november 1995,

There is now substantial evidence that clinical investigators are responsible for biased underreporting of research.1 Compared with studies yielding unremarkable point estimates of effects, studies which have yielded relatively dramatic point estimates of effects are more likely to be selected for presentation at scientific meetings;2 more likely to be reported in print;1 more likely to be published as full reports;3 more likely to be published in journals that are widely read;45 and more likely to be cited in reports of subsequent, related studies.6 These reporting patterns introduce both bias and imprecision into assessments of the effects of health care, and this phenomenon must inevitably mean that decisions taken in health care and health research are less well informed than they could be.

Opinions differ about the consequences – in practice – of selective reporting of well-designed clinical research. In a letter to The Lancet published in 1993, de Melker, Rosendaal and Vandenbroucke suggested that the importance of publication bias was being overstated and that it was unlikely to pose any substantive threat to the wellbeing of patients.7 Although it is not difficult to think of examples which challenge this view,89 that is not the reason that we feel prompted to write to Nederlands Tijdschrift voor Geneeskunde.

In a recent commentary published in this journal, Vandenbroucke challenges the view that selective reporting of well-designed clinical research is not only unscientific but unethical (1995;1476-7). As authors of two articles on the ethics of underreporting of clinical research (which were written independently of each other),1011 we wish to challenge Vandenbroucke's views.

In our view, investigators embarking on clinical research projects enter into implied if not formal contracts, both with the funding bodies through which the resources (often public) required to support their research have been provided, and with the individual patients who are recruited to participate in the research. We suggest that research funding bodies and the vast majority (if not all) of the patients participating in clinical research assume that their involvement is contributing to a growth in knowledge. This implied contract is violated by investigators who, having conducted well-designed research, fail to make the results of their investigations publicly available.

On both scientific and ethical grounds, we find it difficult to understand Vandenbroucke's position on selective reporting of well-designed research; but it is his apparent rejection of the ethical concerns raised by this practice we find most disturbing. It would be interesting to know the extent to which his views are shared by other researchers, research funding bodies, research ethics committees, clinicians and the public in general in the Netherlands.

Leiden, december 1995,

The basic difference in our opinions is rather simple. It can be summarized in one sentence by Cornfield, in a theoretical paper on randomization and randomized controlled trials in 1976.1 Cornfield wrote that there are statistical issues in randomized trials for which unambiguous answers appear unattainable, and that this must be so theoretically. That the results of clinical trials can never be unambiguous is due to several mechanisms, mostly relating to prior and therefore subjective beliefs. The overenthusiastic and selective reporting of findings that raise hope and the quiet burial of findings that are disappointing, form only one aspect. How clinical trial results can ride upon a ‘random high’ or a ‘random low’ has been explored by Pocock and Hughes in the case of early stopping of a trial.2 These mechanisms also operate in trials that end at their preplanned endpoint, and in non-randomized studies. For example: side effects are discovered because of a cluster – most likely the cluster is a ‘random high’, and the real incidence will be lower, but it still helps us to discover the side effect. In the case of a clinical trial, a result that is spuriously high when there is no real difference (neither beneficial nor adverse) will give rise to an overenthusiastic report and to a temporary fashion in therapy or diagnosis that will neither do good, nor harm. One day or another, it will be quietly replaced.

Subjectivity is simply inescapable in all phases of clinical research: planning, execution, analyzing and reporting. The ultimate subjectivity is in the interpretation of published results. For example, I do not believe the results of the randomized trial that showed that smoking cessation does not prolong life3 – as a matter of fact, few epidemiologists ever discuss that trial or use it in any meta-analysis or overview about smoking and disease. It has been quietly buried. Similarly, I do completely ignore all trials showing that homeopathy does work, since such trials are randomizations between two placebos, and therefore in essence meaningless.

Understanding the world necessitates integration of new data into existing knowledge. That knowledge is imperfect, but so are the new data. This view is not new. Let me, in true Anglo-Saxon medical tradition, take refuge and solace in the writings of Sir William (Osler), for example in his 1906 Harveian Oration upon ‘The growth of Truth’.4 Mark the sentences outlining his ideas: ‘In the first place, like a living organism, Truth grows, and its gradual evolution may be traced from the tiny germ to the mature product (...) Truth may suffer all the hazards incident to generation and gestation’, and, ‘Secondly, all scientific truth is conditioned by the state of knowledge at the time of its announcement (...) as Harvey says: “opinion is the source of opinion” (...)’. I counsel the remainder of the text as salutary reading to those with greater passion than I have.