1 00:00:03,070 --> 00:00:08,139 So you've been learning about evidence based medicine this week, and most of you are interested in the area. 2 00:00:08,140 --> 00:00:12,510 And I thought I'd talk a little bit about the past way. Where did the term come from? 3 00:00:12,520 --> 00:00:18,669 What's its historical roots? And then I'm going to move on to a snapshot of the present and then particularly 4 00:00:18,670 --> 00:00:22,780 talking about things that I see that need to improve in evidence based medicine, 5 00:00:22,780 --> 00:00:30,740 directions that will go on in the future. So I have to begin with a definition, first of all, of what evidence based medicine is. 6 00:00:30,760 --> 00:00:33,490 This is the standard one that's out of the textbook of IBM, 7 00:00:33,820 --> 00:00:38,680 which says it's the integration of best research evidence with clinical expertise and patient's values. 8 00:00:39,200 --> 00:00:40,830 Then you need the clinical expertise. 9 00:00:40,840 --> 00:00:46,780 You can't practice medicine without it, and you need to communicate with the patient, find out what they're concerned about, 10 00:00:46,780 --> 00:00:52,870 what they would like to know about the treatment options, etc. And that's part of modern medical practice. 11 00:00:52,870 --> 00:00:57,880 The tricky bit is this best research evidence of trying to get hold of that as well. 12 00:00:58,630 --> 00:01:01,900 And I actually see this as being involving two components. 13 00:01:02,320 --> 00:01:06,670 One is having a sceptical attitude because most things in medicine don't work. 14 00:01:07,240 --> 00:01:11,890 We have lots of theories about why things should work, but when we test them in practice, 15 00:01:12,250 --> 00:01:16,210 most things, most treatments don't work, most tests aren't accurate, 16 00:01:16,540 --> 00:01:23,500 and we need the evidence to filter out those things that are actually going to be very useful in making diagnoses and in benefiting patients. 17 00:01:24,610 --> 00:01:27,399 And the other part of that attitude is not only being sceptical, 18 00:01:27,400 --> 00:01:32,170 but also favouring knowledge that comes from experiments rather than theories about things. 19 00:01:32,590 --> 00:01:38,559 So there's this so-called hierarchy of evidence which basically says good experiments go on the top, bad experiments, 20 00:01:38,560 --> 00:01:44,350 etc. further down, and then our theories about things are right on the bottom, which is somewhat of a turnaround. 21 00:01:44,350 --> 00:01:48,460 And it goes contrary to certainly what I was taught at medical school and is 22 00:01:48,470 --> 00:01:51,760 still a controversial thing about how things should fit into the hierarchy. 23 00:01:52,570 --> 00:01:55,690 These ideas, though, are not that new. 24 00:01:56,800 --> 00:01:59,190 You could probably trace them back a very long way, 25 00:01:59,200 --> 00:02:08,770 but one person who comes to mind is Al Razi in 900 A.D. So he was a Persian physician who worked mostly, though, in Baghdad, 26 00:02:09,790 --> 00:02:15,550 and he tried to compile a list of all of the therapies that were being used across the world 27 00:02:16,360 --> 00:02:20,800 and put together this couple of volumes sort of encyclopaedia that was like a pharmacopoeia. 28 00:02:21,460 --> 00:02:25,720 But he was also interested in the evidence. Beyond that, he wanted to know that these things work. 29 00:02:25,750 --> 00:02:31,450 He didn't write down just anything. So, for example, this is a quote from one of the volumes where he says, for once, 30 00:02:31,450 --> 00:02:38,530 I saved one group by the therapy that he was using, while intentionally neglected another group, a controlled trial. 31 00:02:39,010 --> 00:02:46,389 And he says by that I wish to reach a conclusion. So he was interested in he was sceptical, first of all, about whether things worked or not, 32 00:02:46,390 --> 00:02:51,520 one of the full list, but wanted to forget in his volumes what things actually worked for patients. 33 00:02:52,180 --> 00:02:56,979 He didn't have the ideas of randomised trials and blinding and all of those sorts of things, 34 00:02:56,980 --> 00:03:02,170 but he had the clear idea of testing things in practice and at least using a comparative group. 35 00:03:04,480 --> 00:03:12,760 Okay. Are we the first recognised controlled experiment though a very deliberate experiment was James Lynn Swann on board 36 00:03:12,760 --> 00:03:21,700 the the Salisbury where he was treating scurvy and he tried out different things for scurvy and actually allocated 37 00:03:21,970 --> 00:03:29,380 pairs of sailors to six different things and discovered the two that were on the the citrus juices recovered 38 00:03:29,590 --> 00:03:36,729 relatively rapidly and started treating the others he's held responsible for for doing the first controlled trial, 39 00:03:36,730 --> 00:03:38,889 though it took a long time to implement in practice. 40 00:03:38,890 --> 00:03:44,049 The British Navy didn't adopt this for at least 50 years, and other navies around the world took even longer. 41 00:03:44,050 --> 00:03:52,810 So this even back then, there was the evidence practice gap which lasted a long time, and other person around the same period was Pierre Louis, 42 00:03:52,840 --> 00:04:02,229 who, like Al-razi, was interested in things that came from empirical observations, and he was made a major change, particularly with bloodletting. 43 00:04:02,230 --> 00:04:08,890 So he he demonstrated that if you bleed patients later rather than earlier, they actually did better rather than worse. 44 00:04:09,730 --> 00:04:16,240 Published his results. It was rather controversial, but the number of leeches being sold in Paris eventually plummeted over a sort of five or 45 00:04:16,240 --> 00:04:21,160 ten year period after he persuaded people that they needed to use this numerical method, 46 00:04:21,160 --> 00:04:24,700 is what he called it, that is making direct empirical observations. 47 00:04:25,720 --> 00:04:34,360 A real turning point was Bradford Hill who laid down a lot of the principles about medical statistics and how to do experiments correctly, 48 00:04:34,660 --> 00:04:37,389 and in particular wrote the principles of medical statistics. 49 00:04:37,390 --> 00:04:45,010 And was the statistician involved in the MRC trial of streptomycin, which most people take as the first randomised trial, 50 00:04:45,010 --> 00:04:49,870 very explicitly randomised trial though there are if you go to the James Lynn Library, 51 00:04:49,870 --> 00:04:57,040 you'll find a lot of alternate allocations that were very close to Randomisation long before Bradford Hill came along. 52 00:04:57,340 --> 00:05:07,090 But that was a real turning point. Was that particular trial in terms of the, the, the further application of epidemiology to clinical medicine? 53 00:05:07,540 --> 00:05:14,439 A key person was Alvin Fine's team, who published his book on clinical judgement in 1967 and coined probably coined the 54 00:05:14,440 --> 00:05:18,759 term clinical epidemiology and developed a lot of the principles that we now use. 55 00:05:18,760 --> 00:05:23,620 So he was also a seminal figure and was the sort of grandfather of many of the clinical 56 00:05:23,620 --> 00:05:28,330 epidemiologists who have been active in the recent past or who are still active today. 57 00:05:28,750 --> 00:05:38,079 So he was also a very influential figure and helped develop groups in all around the world, including the group at McMaster, 58 00:05:38,080 --> 00:05:42,700 who are probably seen as the fathers of the sort of clinical evidence based medicine movement. 59 00:05:43,090 --> 00:05:50,890 So too interesting things were happening about the same time the McMaster Group at this new university were developing clinical epidemiology, 60 00:05:52,090 --> 00:05:55,690 and this was Sackett and his colleagues. And at the same time, 61 00:05:55,690 --> 00:06:03,160 Archie Cochrane on this side of the Atlantic was writing his book on effectiveness and efficiency from a public health perspective as well. 62 00:06:03,940 --> 00:06:10,540 And I think that had reasonably similar ideas, though this group was closer to a sort of bedside practice of ABM, 63 00:06:10,540 --> 00:06:14,229 but they both had the general principles that everybody else in the past had been 64 00:06:14,230 --> 00:06:17,920 developing and and were trying to push them more to the forefront of medicine. 65 00:06:19,210 --> 00:06:22,450 At this stage. It was still called, as you see, clinical epidemiology. 66 00:06:22,450 --> 00:06:27,820 The term ABM had not been developed as a long history, but prior to the term being developed, 67 00:06:28,570 --> 00:06:35,210 the birth of the term itself time when there was a Use a Guide series that was being developed for JAMA. 68 00:06:35,450 --> 00:06:41,830 The McMaster Group had written a series for the Canadian Medical Association Journal on how to read a paper, 69 00:06:42,790 --> 00:06:46,449 and they decided things had changed and moved on. And I wanted to write a new series. 70 00:06:46,450 --> 00:06:53,709 I persuaded JAMA to use this, to pick this up, and they wanted a name for the series and they said, We can't call it clinical epidemiology. 71 00:06:53,710 --> 00:06:57,250 That's really boring. Nobody gets interested in that. What are we going to do? 72 00:06:57,250 --> 00:07:01,690 And so some suggestions were, Let's scratch out that. What about scientific medicine? 73 00:07:01,720 --> 00:07:04,570 And I said, Well, everyone would claim that that got scientific medicine. 74 00:07:04,870 --> 00:07:08,770 And Gordon Guyot pictured here said, Well, what about evidence based medicine? 75 00:07:09,190 --> 00:07:13,780 That should be pretty irritating, shouldn't it? So I said, Yeah, that's not a bad term actually. 76 00:07:13,930 --> 00:07:17,499 And so that's how this that's how the group so this, first of all, 77 00:07:17,500 --> 00:07:23,860 this evidence based medicine group got formed and how the series got developed and the term was born. 78 00:07:25,030 --> 00:07:31,600 So even though you could mark the beginning of it from this use guide in 1991, actually it had a long history before that. 79 00:07:32,500 --> 00:07:38,500 This was a seminal turning point that because this really attracted attention and people started to take notice both of the series, 80 00:07:38,500 --> 00:07:47,560 but of even more generally, shortly after the series started in 1993, Dave Sackett, one of the four sort of leaders at McMaster, 81 00:07:47,830 --> 00:07:51,399 moved to Oxford and founded the Centre for Evidence based Medicine. 82 00:07:51,400 --> 00:07:55,690 Muir Grey was responsible for getting him over and he was a general physician. 83 00:07:55,690 --> 00:08:00,070 He'd actually retrained who'd been an epidemiologist and retrained in general internal medicine. 84 00:08:00,580 --> 00:08:04,000 And at the John Radcliffe up the road here, he was doing one. 85 00:08:04,200 --> 00:08:08,160 Rounds using this strange vehicle here called the evidence card. 86 00:08:08,580 --> 00:08:14,969 The evidence card was a data projector, a laptop computer with a whole lot of evidence resources on it, including Medline, 87 00:08:14,970 --> 00:08:21,540 the Cochrane Library, their past approaches, topics, past issues of the ICP Journal Club, etc. and had a printer on it as well. 88 00:08:22,170 --> 00:08:27,090 And they would look up two or three questions per patient on the ward round. 89 00:08:27,780 --> 00:08:31,889 It would take them 15 to 90 seconds to find that information. 90 00:08:31,890 --> 00:08:37,100 And it changed importantly one third of decisions that they made in managing this. 91 00:08:37,110 --> 00:08:40,380 So almost every patient, it would actually change a clinical decision. 92 00:08:40,710 --> 00:08:45,990 As you might imagine, mortar rounds took longer. Carl Hennigan was actually a medical student at the time. 93 00:08:45,990 --> 00:08:53,280 Dave Sackett was doing this and says that the ward rounds could take four or 5 hours, which seems ridiculous. 94 00:08:53,280 --> 00:08:59,579 But in a sense what Sackett was doing was moving medical education out of the lecture theatre conferences, 95 00:08:59,580 --> 00:09:03,209 etc., which he didn't go to, he never went to any of those sorts of things. 96 00:09:03,210 --> 00:09:06,810 He said the best place to do medical education is at the bedside. 97 00:09:07,350 --> 00:09:10,920 Why don't we do it where it really matters when we're making patient decisions? 98 00:09:12,180 --> 00:09:15,389 Okay. You're all thinking, this is crazy. 99 00:09:15,390 --> 00:09:19,470 Nobody would do this right or you're never going to do it. 100 00:09:20,520 --> 00:09:27,060 So I put these rings down here to remind me, to tell you these guys were the Olympic champions of evidence based medicine, the gold medallists. 101 00:09:27,150 --> 00:09:36,960 Right? This is thousands of hours of practice, searching, appraising, etc., to be able to get to this point where you could actually do that. 102 00:09:37,290 --> 00:09:41,429 I wouldn't have dreamed of doing anything like this when I first started trying to do IBM. 103 00:09:41,430 --> 00:09:44,610 Answering one question a week was quite hard enough. Thank you very much. 104 00:09:45,060 --> 00:09:46,710 But you gradually get faster at it. 105 00:09:47,100 --> 00:09:52,350 And after your point, I actually felt comfortable in being able to search in front of patients, but that took me a couple of years. 106 00:09:52,620 --> 00:09:56,130 So it's actually a long curve to be able to get to do this sort of thing. 107 00:09:58,140 --> 00:10:04,799 Okay, so things have evolved a lot. I've got some of the the products that have occurred since then here. 108 00:10:04,800 --> 00:10:09,450 So the Cochrane Collaboration in the Cochrane Library started in the early 1990s, 109 00:10:09,450 --> 00:10:16,620 the IBM journal Evidence-Based Nursing Bmj's clinical evidence I think was 1999 or 2000. 110 00:10:17,370 --> 00:10:22,440 There have been these improved filters that I'll show you a little bit later in Medline called The Clinical Queries, 111 00:10:22,890 --> 00:10:26,580 there's a database of physio physiotherapy called Pedro. 112 00:10:27,150 --> 00:10:29,100 I'm developed by Rob Herbert in Sydney. 113 00:10:29,760 --> 00:10:35,520 There's an occupational therapy database called Otsuka Evidence Updates, which are part of the engine behind this, 114 00:10:35,820 --> 00:10:40,410 says all sorts of evidence products that keep developing and they'll keep developing, which is fantastic. 115 00:10:40,800 --> 00:10:45,240 We need better evidence resources to make it easier for people to look up that information. 116 00:10:45,690 --> 00:10:50,790 What David Sackett was doing was actually brilliant in using very primitive tools. 117 00:10:51,690 --> 00:10:55,260 We've got much better tools now and I'm sure they'll continue to get better. 118 00:10:56,040 --> 00:11:00,899 The literature is also improving. We know more about the methods of doing IBM, the Use of Gods series, 119 00:11:00,900 --> 00:11:09,240 etc. Things have evolved to make it easier for us to read the structured abstract, the consort statement to present controlled trials. 120 00:11:09,840 --> 00:11:13,919 The Grey Group has been working on trying to classify evidence more clearly to 121 00:11:13,920 --> 00:11:18,659 get beyond just the simple levels of evidence that we use of randomised trial, 122 00:11:18,660 --> 00:11:22,140 cohort, study, etc. to look at it in a more fine grained way. 123 00:11:22,590 --> 00:11:24,149 There are areas that have been neglected. 124 00:11:24,150 --> 00:11:30,660 We've recently been working on evidence based monitoring, so a lot of tests get done for diagnostic purposes, 125 00:11:30,660 --> 00:11:38,160 but actually probably about one third of all testing is for monitoring purposes and that monitor evidence based monitoring has been a neglected field, 126 00:11:38,160 --> 00:11:43,680 despite being one of the fastest growing areas of testing. And there's a big interest in knowledge translation. 127 00:11:43,680 --> 00:11:47,100 And this is Sharon Strauss's recent book on the area as well. 128 00:11:47,850 --> 00:11:53,909 So things are evolving and it's been specified now as part as a as an appropriate curriculum 129 00:11:53,910 --> 00:11:58,500 to be part of medicine that we should train health care students generally in this. 130 00:11:58,500 --> 00:12:01,350 So there was the Sicily statement. There's a regular meeting in Sicily, 131 00:12:01,350 --> 00:12:08,040 and the curriculum was basically formulated around the four steps of the Abbey and formulate an answerable question, 132 00:12:08,060 --> 00:12:15,570 track down the best evidence, critically appraise that and integrate the decision with your expertise and the patient's values. 133 00:12:17,040 --> 00:12:20,099 Unfortunately, it's not necessarily happening everywhere. 134 00:12:20,100 --> 00:12:23,580 It's very patchy in and in the degree of training around the world. 135 00:12:23,970 --> 00:12:29,310 We did a survey of UK medical schools and got 20 replies from 32 medical schools. 136 00:12:29,310 --> 00:12:37,350 A lot of them didn't have somebody who was who could tell us about what the ABM curriculum was, so we couldn't find a person who was in charge of it. 137 00:12:37,980 --> 00:12:42,960 So this is this is an an optimistic view in a way, because I my guess is that the others are worse. 138 00:12:43,530 --> 00:12:50,160 These are the things like being able to search databases, appraise therapy articles, understand confidence intervals and P values. 139 00:12:50,580 --> 00:12:56,820 As you can see, most of the topics, at least the main ones get covered, but actually there's very little practice in them. 140 00:12:57,750 --> 00:13:03,720 Okay. Most of the medicals, only about half the medical schools do some practice in these, and it's much more rarely assessed. 141 00:13:05,910 --> 00:13:12,059 So it seems like this is sort of superficial coverage, but not actually training in the practical skills of ABM at the moment. 142 00:13:12,060 --> 00:13:15,750 And it's patchy. Some medical schools, I must say, were absolutely superb, 143 00:13:15,990 --> 00:13:22,049 and there are others where the term was a bit foreign to them and it'll be a generation or two. 144 00:13:22,050 --> 00:13:29,370 But I think before we've really moved to it being considered just part of, you know, it's, it's just like having a stethoscope now. 145 00:13:29,370 --> 00:13:33,870 You just wouldn't think of practising medicine without a stethoscope in, in 20 years time. 146 00:13:33,870 --> 00:13:37,380 You won't think of doing it without understanding what evidence is all about. 147 00:13:38,310 --> 00:13:42,930 And McKibben, who works at McMaster, did this interesting study as part of her Ph.D., 148 00:13:43,320 --> 00:13:47,400 where she looked at people in general practice or in internal medicine. 149 00:13:47,640 --> 00:13:51,660 She gave them a set of questions to answer to What do you think the answer to these are? 150 00:13:52,770 --> 00:14:00,329 And she said, okay, now do a search, see if you can find an answer, some information about that, and then re ask them the questions. 151 00:14:00,330 --> 00:14:04,770 Okay, now what do you think about that to see whether it would improve their answers. 152 00:14:05,370 --> 00:14:11,819 So in 28% of cases, they went from right to right, 13% of cases they went from wrong to right. 153 00:14:11,820 --> 00:14:15,090 They corrected their answer. That's the good news. 154 00:14:16,410 --> 00:14:20,520 The bad news is that right to wrong was 11%. 155 00:14:21,780 --> 00:14:28,200 In other words, people could look up stuff, get misled by information and get the wrong answer because of their you know, 156 00:14:28,200 --> 00:14:32,220 they looked up Google and found some trash website and were misled by it. 157 00:14:32,580 --> 00:14:36,299 And the other disappointing part is the wrong to wrong. That's not quite as disappointing. 158 00:14:36,300 --> 00:14:42,120 I think it's the balance between these two that overall there was a 2% improvement by doing a search. 159 00:14:42,510 --> 00:14:44,460 A couple of similar studies have been done. 160 00:14:44,820 --> 00:14:51,510 One is a group in New South Wales, in Australia and RICO careers group developed a system called Quick Clinical, 161 00:14:51,990 --> 00:14:58,020 which was an interface to try and guide you to the best information to look at and go through guidelines, 162 00:14:58,740 --> 00:15:07,050 systematic reviews to PubMed, using the clinical queries, filters that I'll show you, etc. And this was just with GPS and they did a bit better. 163 00:15:07,350 --> 00:15:12,360 32% went from wrong to right and only 7% went from right to wrong. 164 00:15:12,960 --> 00:15:19,410 Okay, and 40% still went from wrong to wrong. So at least here there was a substantial improvement in the number. 165 00:15:19,410 --> 00:15:24,720 Correct, which is great. So I think quick clinical is actually a very good system. 166 00:15:25,260 --> 00:15:33,180 But are we there yet? No, I still took minutes to do this and 40% of them are still going from wrong to wrong. 167 00:15:33,180 --> 00:15:39,180 So there's still a lot of room for improvement in terms of either training people or improving the interfaces. 168 00:15:39,180 --> 00:15:43,169 And I'm not sure it would be nice if we could just do it through the interfaces. 169 00:15:43,170 --> 00:15:46,080 Somehow we found search engines that could make this easy. 170 00:15:46,590 --> 00:15:52,860 A couple of other studies by Hirsch and Hirsch these were both by Hirsch on medical students and nurses, and again, 171 00:15:53,520 --> 00:15:58,670 a little better than with the with the McKibbin study, but they probably had more room for improvement here. 172 00:15:58,680 --> 00:16:06,970 They were basically getting more wrong in the first place. One of the problems in all of this is just so much research is actually poor. 173 00:16:07,540 --> 00:16:11,769 So for the album Journal, we scanned 140 journals, primary journals, 174 00:16:11,770 --> 00:16:16,360 and get about 60,000 articles per year and ask some simple questions about whether 175 00:16:16,570 --> 00:16:20,170 there's a first screen about whether these articles are valid in these journals. 176 00:16:20,710 --> 00:16:25,810 For an intervention we need the study needs to be randomised and have at least 80% follow up. 177 00:16:26,230 --> 00:16:31,990 That's it. Okay for a prognosis study has to be an inception cohort. 178 00:16:32,590 --> 00:16:37,090 The rules are actually a relatively low bar. This isn't the full scale critical appraisal. 179 00:16:37,090 --> 00:16:41,680 This is a low bar. 5% pass that stage. 180 00:16:41,680 --> 00:16:46,419 And then we do a test on relevance. We asked people about would this be important to change your practice, 181 00:16:46,420 --> 00:16:51,350 etc. And we actually filtered down to a very small number of things that we actually pick up for the BMJ Journal. 182 00:16:52,120 --> 00:16:57,910 But the consequence is that the number needed to read to find one valid article is about 20, 183 00:16:58,750 --> 00:17:02,980 which means most issues of most journals won't contain a single valid article, 184 00:17:04,300 --> 00:17:08,380 and the number needed to redefine a valid and relevant article is about 200. 185 00:17:10,030 --> 00:17:16,900 So if you do a lot of reading, so actually, I mean, one of the purposes of journals like IBM is to help you with that scanning process. 186 00:17:18,190 --> 00:17:21,190 Finding stuff is good. We need to be able to peer review it as well. 187 00:17:21,190 --> 00:17:24,130 And unfortunately, our peer review system isn't particularly good. 188 00:17:24,400 --> 00:17:31,090 So we should you imagine that the good stuff goes into these journals and the rest goes into the bias and confounding trash can? 189 00:17:31,600 --> 00:17:39,100 Unfortunately, that's not true. One of the most cynical studies I've ever seen was Sarah Schroeder's where how this passed the Ethics Committee. 190 00:17:39,100 --> 00:17:47,240 I don't know. They got 607 reviews of the BMJ and they inserted errors into the papers and simply 191 00:17:47,650 --> 00:17:52,270 without telling the reviewers just to see whether they detected the errors or not. 192 00:17:52,660 --> 00:17:57,820 These 14 errors, nine they classified as major and three and five minor errors. 193 00:17:58,210 --> 00:18:02,400 And on average, less than three of the nine major errors were detected. 194 00:18:03,010 --> 00:18:08,260 And unfortunately, our peer reviewers at the moment are not trained properly in critical appraisal. 195 00:18:08,260 --> 00:18:15,249 They don't know how to detect the bad things from the good things. The last area I wanted to mention was the application. 196 00:18:15,250 --> 00:18:20,200 I think this is probably the biggest one that we have to work on over the next couple of decades. 197 00:18:20,590 --> 00:18:24,960 And that's we've got great systematic reviews now that's all happening in Cochrane, 198 00:18:25,810 --> 00:18:31,300 but there's a real problem in the application of the results to individual patients, and there's a couple of areas. 199 00:18:31,600 --> 00:18:35,290 One is taking the average result and finding out what it means for the individual. 200 00:18:35,530 --> 00:18:39,320 But the other is the how to do it, the ways to individualise. 201 00:18:39,340 --> 00:18:42,999 I think you can think of it in several different areas in chronic disease as you 202 00:18:43,000 --> 00:18:47,110 can do a single patient trial of monitoring an adjustment in acute disease. 203 00:18:47,110 --> 00:18:52,030 You've got to try and predict it and adjust for the predictive tools. 204 00:18:52,390 --> 00:18:58,600 And finally, in prevention, you've got to predict the future risk for people at high risk, basically have more to gain than the people at low risk. 205 00:18:59,080 --> 00:19:03,250 I'm not going to go through that one in detail because that's our sort of whole talk by itself. 206 00:19:03,640 --> 00:19:08,140 But I'd recommend this very good book that Peter Rothwell has put together on treating individuals, 207 00:19:08,410 --> 00:19:12,460 which goes through looking at very different slices of this whole problem. 208 00:19:13,030 --> 00:19:19,179 Peter is a neurologist here in Oxford and it's a very good compilation of essays that were originally in The Lancet, 209 00:19:19,180 --> 00:19:21,700 and it's an extended version of them that's in the book. 210 00:19:23,500 --> 00:19:28,680 The other one I said that I wanted to talk a bit more about was the idea of some of what the treatment is. 211 00:19:28,690 --> 00:19:36,760 I'm just going to give you an example of this. This is a paper that featured recently in the BMJ on long term benefits of reduced salt intake. 212 00:19:37,630 --> 00:19:43,380 We've known for a long time that reducing your salt. Leads to lower blood pressure. 213 00:19:43,390 --> 00:19:47,580 But we'd never been out. No one had been able to prove that it actually improved cardiovascular outcomes. 214 00:19:47,580 --> 00:19:51,870 And this trial, the torch trial did well, sorry, the study did. 215 00:19:52,680 --> 00:19:56,560 Here's the description of the patient of the sodium reduction that was in this paper. 216 00:19:56,580 --> 00:20:00,479 Individual and weekly group counselling sessions were offered initially with 217 00:20:00,480 --> 00:20:04,350 less intensive counselling and support thereafter specific to sodium reduction. 218 00:20:05,220 --> 00:20:13,740 So I want you to try and imagine what you would be telling a patient next Monday if you wanted to advise them about salt reduction, 219 00:20:13,740 --> 00:20:17,370 picture, the sorts of things you're going to do with them based on this information. 220 00:20:18,040 --> 00:20:21,759 We'll try to track this down so we track through the references in the papers. 221 00:20:21,760 --> 00:20:30,540 It was one of one of the ways that we tracked this down. And here's the fuller description that's in a journal you have to pay for bottom line. 222 00:20:30,570 --> 00:20:36,030 So this is not free information that BMJ research article is free because they make all their research articles free. 223 00:20:36,690 --> 00:20:41,969 But here it said, this is an individual session followed by ten weekly group 90 minute sessions with a nutritionist, 224 00:20:41,970 --> 00:20:44,640 followed by a transitional stage of some additional sessions. 225 00:20:45,420 --> 00:20:49,680 Topics in the weekly sessions included getting started certain basics, the morning meal, mid-day sources of sodium, 226 00:20:49,680 --> 00:20:53,610 the mind meal, planning ahead, creative cooking, eating out food cues and social support. 227 00:20:54,360 --> 00:20:59,130 The sessions included sampling of foods, discussion of articles on sodium, reduction in problem solving, 228 00:20:59,340 --> 00:21:03,390 and patients kept diaries at least six days a week and urine sodium were measured. 229 00:21:03,630 --> 00:21:08,459 Today, no one imagine all of those like anyone. 230 00:21:08,460 --> 00:21:11,910 Imagine even half of them you probably wouldn't have. 231 00:21:12,060 --> 00:21:17,879 So you couldn't pick this up from that description. And even this I first of all, it sounds impractical. 232 00:21:17,880 --> 00:21:21,030 This is not something I'm going to be doing in general practice next Monday. 233 00:21:21,330 --> 00:21:23,729 But even if I was absolutely dedicated to this, 234 00:21:23,730 --> 00:21:29,800 I couldn't replicate this without getting hold of the manuals and seeing a lot more about how this whole process actually runs. 235 00:21:30,180 --> 00:21:37,080 So this is not a replicable intervention. It's a nice proof of concept, if you like, but it's not actually something you should pick up on. 236 00:21:37,590 --> 00:21:41,549 The editor of the BMJ had actually written an editorial saying All clinicians should be doing this. 237 00:21:41,550 --> 00:21:42,900 And I thought, well, doing what? 238 00:21:45,240 --> 00:21:53,490 So this this sort of thinking has led us to ask about the problem of getting the descriptions from trials or systematic reviews in practice. 239 00:21:54,450 --> 00:22:02,549 So Carl Henning and I did a study a couple of years ago where we looked at the adequacy of the descriptions that were in the journal 240 00:22:02,550 --> 00:22:08,610 things this is so these were studies that we thought were valid and then we thought were important that should change practice. 241 00:22:09,090 --> 00:22:13,440 And we looked at 80 of them, which was a year worth of things from the Journal. 242 00:22:13,740 --> 00:22:17,970 And the question we asked is, could we replicate this tomorrow if we saw a patient with it? 243 00:22:18,420 --> 00:22:22,230 And the overall answer is just just under 50% of them were replicable. 244 00:22:23,880 --> 00:22:29,490 Trials were better than systematic reviews. Drugs were better than non-drug therapies, as you might expect. 245 00:22:30,000 --> 00:22:33,390 So the bad news is a lot of things are not replicable. 246 00:22:34,110 --> 00:22:36,059 The good news is that we tried to fix it. 247 00:22:36,060 --> 00:22:43,410 We wrote to authors, tracked down references, did all sorts of things to try and get the additional information that made it replicable. 248 00:22:43,410 --> 00:22:49,870 And we could fix about half of this. Which is fantastic because it was about a day's work to do this. 249 00:22:50,230 --> 00:22:55,120 I think it's the most cost effective thing I can think of to do in medicine that we could potentially do. 250 00:22:55,120 --> 00:23:02,250 If you think of the costs of a trial and you spend one extra day to get the information that would fix one in four of those, 251 00:23:02,260 --> 00:23:07,000 that is just incredibly cost effective. That's more cost effective than the original trial. 252 00:23:08,590 --> 00:23:14,919 As a follow up to this. Ian and Ian Chalmers and I wrote a paper recently about the avoidable waste in the production of research. 253 00:23:14,920 --> 00:23:19,930 And I've just been talking about the last part here, a usable report. 254 00:23:20,320 --> 00:23:24,520 There are all sorts of problems in the reports that we actually have available. 255 00:23:24,820 --> 00:23:31,420 One of them is the description of the interventions, but others are poor reporting of what was actually the primary outcome. 256 00:23:31,430 --> 00:23:34,719 People swap around things when it actually comes to publication, 257 00:23:34,720 --> 00:23:39,700 so there are bits missing that don't allow you to appraise the paper, etc. So that's poor. 258 00:23:39,970 --> 00:23:45,070 But we decided there are actually four stages that we could break research production down into. 259 00:23:45,730 --> 00:23:51,160 Are the questions relevant to patients and clinicians asking the right questions in the first place? 260 00:23:51,160 --> 00:23:56,230 Did they use appropriate design methods? Did they ever publish it? 261 00:23:56,620 --> 00:24:01,600 And was the report usable? The three of the stages we could actually quantify. 262 00:24:01,630 --> 00:24:06,880 There are various things that says less than 50% of the articles actually have an appropriate method in design. 263 00:24:07,270 --> 00:24:14,010 There are flaws in the way that the trials are set up, and there are flaws in not looking at previous studies. 264 00:24:14,020 --> 00:24:21,880 Most of them didn't access a systematic review even when it was already available at the time of writing the protocol, which is just amazing. 265 00:24:22,870 --> 00:24:28,749 The other appalling thing is that less than 50% of studies never get published in full of abstracts. 266 00:24:28,750 --> 00:24:36,820 For example, presented at oncology, trial meetings, conferences, less than 50% of them, 267 00:24:37,000 --> 00:24:41,560 sorry, about 50% of them have been published after a period of eight years. 268 00:24:42,700 --> 00:24:48,760 And there's been a systematic review of Sally Hopewell that suggested that that's probably about right for the overall 50%. 269 00:24:49,210 --> 00:24:57,880 So if you multiply those things together you can get roughly something like about an 85% loss that occurs in the usability of publications, 270 00:24:58,900 --> 00:25:02,110 not counting this first one, addressing the right questions in the first place. 271 00:25:02,770 --> 00:25:09,460 The world expenditure on research at the moment is about it's over 100 billion USD per year. 272 00:25:10,000 --> 00:25:14,860 So you could say in a rough estimate of the waste is about $85 billion per year. 273 00:25:15,940 --> 00:25:21,969 This is just an amazing thing that we're allowing to happen at the moment when some of it is very difficult to fix. 274 00:25:21,970 --> 00:25:25,060 But other elements of this are actually incredibly easy to fix. 275 00:25:25,060 --> 00:25:31,120 And a small investment could save billions of dollars worth of wasted research that we're not using at the moment. 276 00:25:32,920 --> 00:25:34,960 Okay. I want to end on a brighter note than that. 277 00:25:35,620 --> 00:25:42,220 So I'm going to talk the last couple of slides will be about team based, even digesting the evidence. 278 00:25:42,940 --> 00:25:47,469 When I first started doing evidence based medicine, I thought of it as a sort of solo thing that you did, 279 00:25:47,470 --> 00:25:53,800 basically as a form of continuing medical education. But more and more, I'm becoming convinced that you actually have to do this as a team, 280 00:25:54,190 --> 00:25:58,620 because often the whole team in a practice has to make the appropriate changes. 281 00:25:58,620 --> 00:25:59,979 It's not individual practice, 282 00:25:59,980 --> 00:26:07,480 but that the coordination between different members is important and getting the appropriate training and infrastructure set up in order to do things. 283 00:26:08,200 --> 00:26:14,830 So we run in my practice, a fortnightly journal club and I spoke to several of you last night about this next actions thing. 284 00:26:15,190 --> 00:26:21,190 Not only is do we read the evidence, but we need to agree on what the basic conclusions the clinical bottom line is, 285 00:26:21,490 --> 00:26:24,580 but also to organise what the next actions would be. 286 00:26:25,760 --> 00:26:30,470 And I would like to see that I know what happens in happens very little in primary care. 287 00:26:31,790 --> 00:26:35,390 There are several practices in Oxfordshire now that this is happening in. 288 00:26:35,510 --> 00:26:40,490 Which is great. The other thing that I'd like to see happening is collaboration between these practices then, 289 00:26:40,940 --> 00:26:46,730 because often there's a lot of effort that goes into working out how to implement something in the individual practice. 290 00:26:47,330 --> 00:26:52,490 For example, there are things that have taken me months to get implemented in my practice to sort out the bugs. 291 00:26:53,240 --> 00:26:57,110 And it would be nice that once you've done that, you could share that with another practice. 292 00:26:57,680 --> 00:27:03,950 So we're trying to establish collaborations both here within Oxford, but we've got some funding to do this within Milton Keynes, 293 00:27:04,340 --> 00:27:07,610 where we're getting the practices not only to run the journal clubs, 294 00:27:07,610 --> 00:27:15,380 but also to share ideas across the practice and also work with the primary care trust in order to implement those in practice. 295 00:27:15,650 --> 00:27:22,820 So where there's something where the trust would need to put in more resources to have something happening, they'll actually help out in doing that. 296 00:27:23,210 --> 00:27:27,710 So these are called impact groups. Yeah, that was the best acronym we could come up with so far. 297 00:27:27,710 --> 00:27:32,390 If anyone ever comes up with a really sexy acronym or name for journal clubs, I'd love to know. 298 00:27:33,770 --> 00:27:38,240 And I just wanted to give some examples of some of the projects that they're doing carpal tunnel syndrome, 299 00:27:38,480 --> 00:27:47,330 where you inject patients before you refer them, which is happens in a few places, but now there's specific trained people to be able to do that, 300 00:27:47,330 --> 00:27:51,110 and that's decreasing the number of people that need referral for carpal tunnel syndrome. 301 00:27:51,620 --> 00:27:57,110 One group implemented delayed antibiotic prescriptions and could record it through setting up an E Ms. code. 302 00:27:57,590 --> 00:28:00,210 Smoking cessation has probably been the most successful. 303 00:28:00,290 --> 00:28:06,709 That's been a coordination between both the practices of pharmacists and the C leading to huge reductions 304 00:28:06,710 --> 00:28:12,410 in the number of people taking up at least nicotine replacement therapy for smoking cessation. 305 00:28:12,650 --> 00:28:17,150 And there's a whole series of others that have happened as well. Okay. 306 00:28:17,630 --> 00:28:25,730 So just as a summary slide, I think the gloss for IBM for where we are present and where we need to go in the future is half empty and half full. 307 00:28:26,070 --> 00:28:31,220 There's been this amazing growth in research and trials since that first trial in 1948. 308 00:28:31,850 --> 00:28:36,820 So we're now up to probably close to a million trials now, one and a half thousand per year, 309 00:28:37,970 --> 00:28:42,500 but much of it is poor and synthesised or unusable, as we saw. 310 00:28:43,250 --> 00:28:49,729 Search engines are improving all the time. So can you imagine trying to do ABM in the days when you used to have the Piper Index? 311 00:28:49,730 --> 00:28:54,770 Medicus. It would just be impossible and we've moved on a lot by having it electronic, 312 00:28:54,770 --> 00:29:00,050 but now we're getting improved filters and improved systematic reviews and ways of trying to organise that information. 313 00:29:00,530 --> 00:29:04,849 But it's still very disorganised and little effort goes into that compared with 314 00:29:04,850 --> 00:29:08,690 the the effort to do the primary research in the first place to make it usable. 315 00:29:09,050 --> 00:29:15,020 So half, half empty and half full. And finally, the skills in ABM, I think they're increasing. 316 00:29:15,020 --> 00:29:18,259 So if I'd done that survey that I talked about 20 years ago, 317 00:29:18,260 --> 00:29:21,740 they would have said what's evidence based medicine quite rightly, because the term hadn't been invented. 318 00:29:22,460 --> 00:29:25,700 But if I'd said What was clinical epidemiology, it probably wasn't being done. 319 00:29:25,700 --> 00:29:27,829 It's now happening in many medical schools. 320 00:29:27,830 --> 00:29:34,129 And I see interestingly, when I go around the world, there's a huge interest in this and it's happening in a lot of places around the world, 321 00:29:34,130 --> 00:29:41,060 in the Middle East and Asia, as well as the sort of developed places of the of Canada, the US, etc. 322 00:29:41,390 --> 00:29:44,720 But it's still very patchy and it's still ignored in many medical schools. 323 00:29:45,080 --> 00:29:50,540 So I think the future is bright. But for those interested in doing work in ABM, there's a lot more still to do. 324 00:29:50,780 --> 00:29:51,590 Thank you very much.