IJE Advance Access originally published online on September 15, 2006
International Journal of Epidemiology 2006 35(5):1368-1369; doi:10.1093/ije/dyl182
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Book Review |
Statistical Evidence in Medical Trials: What do the Data Really Tell Us? Stephen D. Simon. OUP, 2006. £65. ISBN 0 19 856760 X
E-mail: becky.rooney{at}bristol.ac.uk
Aimed at health professionals who need to understand published research, this book takes a light-hearted approach to what is often (understandably) seen as a dry subject. The emphasis is on helping the reader critically evaluate research papers and extract useful information. The book is informally written and seeks to provide an entertaining yet informative account of how to avoid common mistakes when reading the literature. Many of the chapters begin with a cartoon sketch chosen to illustrate the focus of the chapter. Analogies are used (such as ‘apples and oranges’ for comparison groups and a courtroom for collecting evidence) and are returned to as a useful tool for tying the chapters together. There are many examples and case studies, the majority of which are from medical research plus a handful from other areas used to explain difficult concepts or to add a humorous twist. Sometimes there are a few too simple examples and not enough more complex ones but on the whole they work very well. Medical terminology is often used, although the intended medical audience will not find this a problem. In the majority of cases, the examples still make sense even if the medical language does not! All examples are referenced and footnotes indicate where the reader can get more information. A basic knowledge of statistics and statistical terminology is assumed from the start (for example mean, confidence intervals, odds ratios) with this material not being explained in detail until Chapter 6. There are also occasions where a case study discusses a concept before it is explained (e.g. randomization). This should not prove troublesome but may be a bit confusing.
Each chapter begins with a series of questions that someone reading research papers might consider. Each of these questions is then discussed in more depth in the next sections. Examples are given of both good and bad practice. Many chapters end with a ‘counterpoint’ section, which gives an alternative viewpoint on some of the issues discussed. At the end of each chapter, there is an ‘on your own’ section. These are usually extracts from real papers about which questions are posed to help the reader consider some aspects of the chapter. As group discussion exercises they might work but individual readers may find them daunting, especially as there are no sample answers.
The Overview contains an example of a review into the quality of clinical trials for treating schizophrenia and describes the reviewers' findings—that many of the studies looked at the wrong patients, in not enough numbers, over too short time periods and then did not measure the outcomes consistently. This demonstrates the importance of thinking through how the research was carried out as a means to reading between the lines of the published report and achieving a thorough critical appraisal. Chapter 1 introduces the running analogy of apples and oranges, i.e. who is the research comparing and with whom? The chapter covers randomization, observational studies, matching, and statistical adjustment. Chapter 2 is about ‘who was left out’, so refusals, exclusions, dropouts, and non-conformers. It starts with an example of a study into smoking cessation in people aged less than 18. The study under-represented eligible individuals from ethnic minorities and required parental permission, thus excluding people who did not want their parents to know that they smoked. There was also a high dropout rate. Only 15% of the initial volunteers actually completed the study and the results were hard to generalize. This nicely illustrates why studies need to be well designed and begins to introduce the concept of bias. Examples of previous research where samples have been biased are given. It then moves onto ‘intention to treat’ and a consideration of how the analysis can be conducted to minimize bias. Chapter 3 starts with the example of vaccines—how much more prevalent would the disease be if people did not use the vaccine? Does the risk of side effects outweigh the benefit of vaccination? There is a discussion of whether the researchers measured the right thing and did it well, as well as whether the results are clinically important. In other words, whether the results are a ‘mountain or a molehill’. Chapter 4 introduces a courtroom analogy. With different research projects as witnesses it makes a clear point about quality and credibility of ‘evidence’. There are then a series of questions posed with which to cross-examine the ‘witnesses’, e.g. ‘is there a strong association?’, ‘is it plausible?’, ‘is there a conflict of interest?’. It is made clear that often ‘evidence’ is not black and white but is only informative in conjunction with other factors. Chapter 5 talks about systematic reviews and meta-analysis following the apple/orange analogy—were apples and oranges combined, were apples left on the tree, were the apples rotten, etc. This chapter is very detailed and possibly a bit more than necessary for understanding what others have written. However, the reader will be well equipped to determine whether what they are reading has avoided bias. Chapter 6 provides an overview of basic statistical ideas and terminology covering sampling error, confidence intervals, P-values, odds ratios, relative risk, correlation, survival curves, prevalence, and incidence. These are all explained clearly and well with plenty of examples. Chapter 7 introduces how to begin the search for evidence. There is quite an in-depth guide to using PubMed including Mesh headings, tags, filters, etc. This is a useful guideline, whilst directing the reader to a librarian for more advanced help.
This book is a clear, concise, and interesting read and should prove to be a useful guide. The examples and case studies make it easy to understand difficult concepts and the jokes and stories make it fun. There are some salient points and hopefully the reader will be enthused about looking at the published research and be more confident about distinguishing between the good and the bad.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||