1 00:00:00,420 --> 00:00:03,720 Well, thank you very much for coming in tonight. We have a treat. 2 00:00:04,650 --> 00:00:07,650 We have one of the treasures of Britain, the treasure. 3 00:00:09,250 --> 00:00:14,430 Something that's hard to got me, who is editor in chief of the BMJ. 4 00:00:14,820 --> 00:00:18,690 I think it's actually pretty correct. 5 00:00:19,770 --> 00:00:25,230 I first met Fiona 25 years ago for years. 6 00:00:25,300 --> 00:00:31,470 But one of the characters who was there was the headline, The Fierce Criticism of the Richest Man, 7 00:00:31,470 --> 00:00:36,180 one of the most charismatic, interesting people that you could ever meet. 8 00:00:36,390 --> 00:00:43,500 And we were in a in a consensus of people looking at peer review, and it was called Loch Ness. 9 00:00:44,400 --> 00:00:49,080 And that's where I met people. They got interested in their view. Within a few things together. 10 00:00:50,140 --> 00:00:54,540 And ever since then, she has just grown and grown and grown. 11 00:00:54,870 --> 00:01:01,409 And I think that the most important thing that she has done to fund my personal opinion as editor 12 00:01:01,410 --> 00:01:09,809 in chief of the BMJ open and and follows in the footsteps of one of the great predecessors, 13 00:01:09,810 --> 00:01:24,390 Hugh Clegg, who grocery Hugh Clegg once said if something is in the public interest, it should be in the public domain and it should be kept there. 14 00:01:24,780 --> 00:01:29,460 And that's exactly what Fiona has done. So Lindsey Graham on the ongoing. 15 00:01:34,370 --> 00:01:39,740 Thank you very much. Good luck saying that Tom has been an enormous influence on my life. 16 00:01:40,160 --> 00:01:47,840 This is quirky. You had a small room. And so I can as have other people in this room and all of us in our careers. 17 00:01:48,380 --> 00:01:52,310 You don't necessarily know where you're heading. And certainly I didn't. 18 00:01:52,880 --> 00:01:58,850 And a lot of the directions you take are down to the people you meet and people 19 00:01:58,850 --> 00:02:02,989 you learn to admire and the people who who welcome you into their world. 20 00:02:02,990 --> 00:02:12,559 And Ian is one of the many people around the world, and Tom will certainly be one of those and who has taken us, 21 00:02:12,560 --> 00:02:20,150 I think, to the edge of our own sanity and our own ability to respond to the demands of the work involved. 22 00:02:20,150 --> 00:02:29,209 And Tom has been a real a real exemplar of of devotion to this important cause. 23 00:02:29,210 --> 00:02:36,080 So I'm very honoured to be with you today. I thought it might be useful to let you know the BMJ very well. 24 00:02:36,320 --> 00:02:42,020 Just to give you a quick discussion, overview about the BMJ has been around for a long time. 25 00:02:42,020 --> 00:02:46,970 It's changed a lot in the last nearly 200 years, 118 90 years. 26 00:02:47,240 --> 00:02:52,819 We have a mission as anyone would expect. We want to be the world's most influential medical journal. 27 00:02:52,820 --> 00:02:56,149 We may not be the biggest or the most the highest impact factor, 28 00:02:56,150 --> 00:03:01,220 but we do want to change things and we want to be the change we want to see in the world. 29 00:03:02,030 --> 00:03:09,829 And we believe in integrity, independence, partnership with patients evidence base, answering questions and questioning answers. 30 00:03:09,830 --> 00:03:13,640 So this is the kind of sceptical core within the BMJ team. 31 00:03:13,910 --> 00:03:21,020 We want to challenge the status quo and we have begun increasingly to campaign and our aim is to do that in the public interest. 32 00:03:21,020 --> 00:03:27,080 So although we are owned by a medical doctors organisation, we do not speak for doctors, we speak in the public interest. 33 00:03:28,130 --> 00:03:29,990 We have a number of tools at our disposal, 34 00:03:30,950 --> 00:03:38,209 so we have at our disposal a number of tools to try to do what we aim to do in a lot of top your research education using these campaigns. 35 00:03:38,210 --> 00:03:44,810 And just on the research front, our research is open access, open peer reviewed, 36 00:03:45,560 --> 00:03:51,920 and we try and increase it in comment opinion and also increasingly individual summaries. 37 00:03:51,920 --> 00:03:54,979 And this is one of the ones particularly proud of. We've got metastatic capital. 38 00:03:54,980 --> 00:04:01,400 Will Star Timmins who does these infographics. It's it's a complex issue called a graphical user evidence reviews. 39 00:04:01,400 --> 00:04:11,150 And the idea is that one would eventually be able to slot in data information from individual trials and participant characteristics, 40 00:04:11,150 --> 00:04:13,010 study quality, study findings. 41 00:04:13,610 --> 00:04:20,480 So it's a kind of way of displaying complex data which you might like to look at that could go for graphical and evidence reviews. 42 00:04:21,170 --> 00:04:26,330 So on the research front, we still think there's a way to go to improve the presentation. 43 00:04:26,570 --> 00:04:33,229 But on the campaign front, we have become, as I said, a campaigning journal and the sort of permission, 44 00:04:33,230 --> 00:04:38,720 if you like, historically for us to do this stems back to the editor in the Victorian age, Ernest Hart, 45 00:04:39,200 --> 00:04:50,840 who in the in those days babies were being it is different babies were being farmed out to so-called foster parents who were being 46 00:04:51,140 --> 00:04:57,350 paid for taking on board these babies that in fact were just leaving them out in the cold and killing them or letting them die. 47 00:04:57,710 --> 00:05:05,120 And it was a big issue in Victorian England and that is half of the editor of the BMJ thought that it was something the BMJ should have a view on, 48 00:05:05,120 --> 00:05:11,629 and he advertised in some newspapers as a pretend father of an illegitimate child in 49 00:05:11,630 --> 00:05:16,780 order to attract these foster parents and to identify what was going on with it. 50 00:05:17,690 --> 00:05:26,509 And as Tom says, that permission to campaign continues through into the forties and fifties and sixties, through through Hugh Clegg, 51 00:05:26,510 --> 00:05:33,620 who is the grandfather of Nick Clegg, who said this a subject that needs reform to be kept before the public until it demands reform. 52 00:05:33,620 --> 00:05:40,590 So this is our in our DNA, we feel and it gives us permission to do things that other journals may not want, 53 00:05:40,640 --> 00:05:44,330 people may not feel they have permission to do. I mentioned the campaigns. 54 00:05:44,330 --> 00:05:51,050 You'll see at the top that a glance, better evidence is the one I'm going to be talking about in relation to the what you are all doing. 55 00:05:51,860 --> 00:05:54,950 So why do we need better evidence? 56 00:05:54,950 --> 00:06:01,970 Well, Doug Altman, in a really iconic editorial back in 1994, talked about the scandal of poor medical research. 57 00:06:02,590 --> 00:06:07,129 And in it he said, what should we think about a doctor who uses the wrong treatment, 58 00:06:07,130 --> 00:06:10,880 either wilfully or through ignorance, or he uses the right treatment wrongly? 59 00:06:11,210 --> 00:06:16,580 Most people would agree that such behaviour is unprofessional, arguably unethical and certainly unacceptable. 60 00:06:16,880 --> 00:06:22,910 What then should we think about researchers who use the wrong techniques, either wilfully or they don't use the right techniques wrongly, 61 00:06:22,910 --> 00:06:29,120 misinterpret the results, report their results selectively, cite the literature selectively, and draw conclusions, unjustified conclusions. 62 00:06:29,450 --> 00:06:32,450 We should be appalled. And it was. 63 00:06:32,910 --> 00:06:40,620 We need better evidence that the evidence and evidence for the right reason to research or best research and research for the right reasons. 64 00:06:40,620 --> 00:06:46,110 And these were some of the problems with the evidence that Doug identified and some others have identified, 65 00:06:46,130 --> 00:06:53,340 adding to those asking the wrong questions, not involving patients from techniques, wrong analysis, misinterpretation of results, 66 00:06:53,460 --> 00:06:56,970 manipulation to get the desired answer, overstating the conclusions. 67 00:06:57,270 --> 00:07:00,690 Selective reporting of benefits and harms. Poor Quality Peer Review. 68 00:07:00,690 --> 00:07:03,240 These are additional to some of the things that dog identified. 69 00:07:03,600 --> 00:07:08,370 Lack of engagement with Post-publication peer review is an issue that we as editors fret about. 70 00:07:08,490 --> 00:07:12,810 So once you published your work, do you engage with the reader if they criticise you? 71 00:07:13,050 --> 00:07:19,260 Hidden data. I'll talk more about that. Financial and academic vested interests, positive publication, bias, fabrication, 72 00:07:19,260 --> 00:07:25,440 falsification and plagiarism, which is the standard definition of fraud and regulatory failure. 73 00:07:26,220 --> 00:07:30,630 And the reason these are worry is because they cause research, waste and clinical harm. 74 00:07:30,680 --> 00:07:39,540 And in and Glasgow Publishing in The Lancet in 2009 and subsequently have looked at the for and possibly more before 75 00:07:39,540 --> 00:07:46,259 in this article stages of research and the result of these different types of waste leads in their estimates. 76 00:07:46,260 --> 00:07:56,370 85% of research effort wasted over a billion hundred billion dollars of research funding per year or research effort per year. 77 00:07:57,510 --> 00:08:04,980 So this is a major problem with this research being done that is not achieving what we would all wish it could achieve. 78 00:08:05,310 --> 00:08:08,370 And just to talk about hidden data, this is a again, 79 00:08:08,880 --> 00:08:12,300 what I would consider an iconic piece of work published in the BMJ in 2010 80 00:08:12,600 --> 00:08:18,149 about the drug war boxer team and these authors who basically quit in Germany, 81 00:08:18,150 --> 00:08:24,660 which is the the German equivalent of Nice, were asked whether they should fund or provide insurance. 82 00:08:24,660 --> 00:08:32,490 In Germany, the drug were boxed on the public purse and they felt sure that there was data that they were not being shown. 83 00:08:32,910 --> 00:08:41,160 So they asked the manufacturer for the data. And Pfizer, who produced this drug, which is an antidepressant, it was being used as such, 84 00:08:42,480 --> 00:08:46,379 said that they had provided all the data and they said, well, you clearly haven't. 85 00:08:46,380 --> 00:08:48,060 We've got we're aware that you haven't. 86 00:08:48,330 --> 00:08:56,640 And eventually they persuaded Pfizer by saying that if you don't give us your data, we won't allow your drug to be made available on health insurance. 87 00:08:57,060 --> 00:09:02,100 So Pfizer presented them with the data and it turned out that three quarters of it had never been published before. 88 00:09:02,940 --> 00:09:10,540 And when these authors did the review of the effect of combining the public and the unpublished data, you can see the results. 89 00:09:10,540 --> 00:09:14,640 So in each case we've got remission published unpublished total, published unpublished total. 90 00:09:14,910 --> 00:09:18,239 And then here the reverse is the withdrawal of adverse effects. 91 00:09:18,240 --> 00:09:20,400 So published a nice result, 92 00:09:20,670 --> 00:09:29,940 but boxing better unpublished no result total no effect response same again and then withdrew the reverse to the adverse effects. 93 00:09:29,940 --> 00:09:37,620 So no difference in the published data but with the unpublished data and because the combined drug is only ineffective but also harmful. 94 00:09:38,130 --> 00:09:47,790 So this is, I think, an example beautifully illustrated of of a problem that we know to be pretty endemic throughout clinical trials, 95 00:09:48,150 --> 00:09:51,810 but also specifically in industry funded trials. 96 00:09:52,440 --> 00:10:01,740 So the idea of simply doing a meta analysis based on the published results is obviously long gone as something we should accept. 97 00:10:02,100 --> 00:10:05,579 But I think often there's much more hidden data. 98 00:10:05,580 --> 00:10:09,840 Even if you start trying to look, we don't really take this out a great deal. 99 00:10:10,560 --> 00:10:14,219 I was very fascinated about this case because it turned out when it was published that this 100 00:10:14,220 --> 00:10:19,500 drug had been used in animal studies because it was thought to be so effective in humans. 101 00:10:19,830 --> 00:10:22,950 It was used as animal studies for the animal model for antidepressants. 102 00:10:23,760 --> 00:10:31,890 So the way they test it is whether the rat struggles or whether it swims a swim test in a struggle test if you hold the rat by its tail. 103 00:10:32,760 --> 00:10:41,330 And when these data come out, the animal models have had to change the drug that they use because in reverse, 104 00:10:41,380 --> 00:10:51,810 you mean it's not really transferable to humans? So now for the story that most changed my mind or mind, 105 00:10:51,810 --> 00:11:00,930 the way my brain worked or majorly increased my scepticism about the value of published evidence is the story of Tamiflu. 106 00:11:01,350 --> 00:11:11,190 And this goes back to 2008, 2009, when there was suddenly a huge fear that we were going to be overwhelmed by swine flu, 107 00:11:11,550 --> 00:11:14,850 and governments around the world were being encouraged to stockpile as well. 108 00:11:14,850 --> 00:11:16,140 Time of Tamiflu. 109 00:11:16,800 --> 00:11:28,230 And the UK Government asked the Cochrane Collaboration to update its review on Tamiflu to see if it was still there was any new evidence to include. 110 00:11:29,040 --> 00:11:32,300 I'm sitting here with Tom Hager, with my parents, the. 111 00:11:32,390 --> 00:11:36,680 My version of events may be slightly different. You have the time, but as I understand it, 112 00:11:36,680 --> 00:11:44,329 the Japanese paediatricians contacted the Cochrane Group to say understanding they were about to review and renew their update, 113 00:11:44,330 --> 00:11:48,500 their review to ask them if they had been really appreciative of the fact that 114 00:11:48,500 --> 00:11:53,330 of the ten trials that had been used in that earlier review of the data, 115 00:11:54,170 --> 00:12:02,900 all of them had been funded or performed by Roche employees and only two of them had ever been published in a journal. 116 00:12:03,320 --> 00:12:10,760 The other eight had been in abstracts only. So this prompted Tom and his colleagues to think, goodness, we ought to look a bit more at this. 117 00:12:10,760 --> 00:12:17,629 So the first step was to speak to the systematic review who had reviewed these trials to ask him if he had the data. 118 00:12:17,630 --> 00:12:21,410 He said he didn't. So they then went to the individual trials. Did they have the data? 119 00:12:21,980 --> 00:12:27,090 They didn't. They said that Tom and his colleagues would have to go to Rush to get the data. 120 00:12:27,090 --> 00:12:35,090 The drug company had the data. So Tom started the long journey of trying to get the data from the manufacturer. 121 00:12:35,090 --> 00:12:41,419 And at some point in this proceedings early on, I think he asked if we might help and it seemed an important thing to do. 122 00:12:41,420 --> 00:12:52,430 So the BMJ joined forces with Cochrane to try to get hold of the data and so ensued a lengthy, very, very exciting, interesting. 123 00:12:53,150 --> 00:13:01,340 But certainly for Tom, a huge amount of work for us, a very exciting journey to try and find the truth about Tamiflu. 124 00:13:02,660 --> 00:13:05,959 And on the way we described what we were doing. 125 00:13:05,960 --> 00:13:09,560 So this is an article by Deborah Cohen, who is then our investigations editor, 126 00:13:09,800 --> 00:13:14,630 about the Cochrane's attempt to reproduce an analysis underpinning the use of virtual cannabis. 127 00:13:14,900 --> 00:13:21,080 So the Lancet review published prior to this had concluded that Tamiflu did reduce your 128 00:13:21,080 --> 00:13:25,790 risk of complications and ending up in hospital if you had an influenza like illness. 129 00:13:26,750 --> 00:13:31,190 And the Cochrane really were looking to confirm that. 130 00:13:31,520 --> 00:13:38,510 But at some point in the in the process they set themselves a deadline and said if we don't have the data by that date, 131 00:13:38,810 --> 00:13:43,550 we will publish an updated review which basically says we do not know, we cannot tell. 132 00:13:43,790 --> 00:13:48,830 Having said that, we knew we now know that we don't know and we're not going to make any conclusions. 133 00:13:49,580 --> 00:14:00,980 So this was the story of that approach and an excellent article by Tom's collaborator, Peter Doshi, who's now an associate editor on the BMJ, 134 00:14:01,370 --> 00:14:09,529 looking at the story behind this this Cochrane Review that had concluded that we could not say and what's rather marvellous to have got to see it. 135 00:14:09,530 --> 00:14:14,689 But inside that article obviously is this box with these two columns about whether 136 00:14:14,690 --> 00:14:18,260 or not the drug works for complications to prevent complications of influenza. 137 00:14:18,590 --> 00:14:22,489 And these are the people who say it does work and these are the people who say it doesn't work. 138 00:14:22,490 --> 00:14:28,640 So in this column you've got on the good work side, you've got Kaiser, you've got who's the systematic review, 139 00:14:28,940 --> 00:14:32,420 you've got the European medicines, you've got CDC in America, you've got Australia. 140 00:14:32,780 --> 00:14:34,129 And on the side that it doesn't work. 141 00:14:34,130 --> 00:14:41,240 You've got the US FDA disagreeing with the CDC, you've got Japan and Canada, but at the very top you've got good. 142 00:14:41,240 --> 00:14:44,000 I thought you'll see that both of these top Roche. 143 00:14:44,720 --> 00:14:50,480 So Roche on the one hand says it does recommendations and on the other hand it says it doesn't work complications. 144 00:14:50,750 --> 00:14:54,950 And if you go to this website, it says this website is intended only for Americans. 145 00:14:55,970 --> 00:14:59,030 So you've got an extraordinary situation. I mean, this is awful. 146 00:14:59,630 --> 00:15:02,810 But this this is this is where things stood. 147 00:15:03,080 --> 00:15:06,830 And I guess this is where things stand for quite a few drugs. If only we were to look. 148 00:15:09,070 --> 00:15:17,260 So at the time we wrote an editorial in the Journal saying it is a legitimate scientific concern that data used to support important 149 00:15:17,260 --> 00:15:23,470 public health policy decisions held only by commercial companies and have not been subject to independent external scrutiny. 150 00:15:24,310 --> 00:15:32,710 And we started what became our third campaign with Tamiflu as our poster child and started doing things like writing open letters to Roche. 151 00:15:33,040 --> 00:15:38,199 This one in particular is to John Bell, based in Oxford, which I had a great deal of help writing. 152 00:15:38,200 --> 00:15:40,660 I can't promise I went to work, but I did sign it. 153 00:15:43,540 --> 00:15:52,540 And it's really saying you, John Bell, as a member of Roche's commercial board, not the scientific advisory board, but he's on that commercial board. 154 00:15:52,540 --> 00:15:53,470 I don't know if he still is. 155 00:15:53,830 --> 00:16:00,490 He's he's one of the top doctors in the country, the regents professor of medicine at Oxford, and he's on Russia's commercial board. 156 00:16:00,600 --> 00:16:06,730 So, I mean, in itself is pretty shocking. But the fact that he's on Russia's border and Russia's behaving like this seems doubly shocking. 157 00:16:07,090 --> 00:16:12,910 And we wrote this letter presenting him with the fact that if he were whether or not he replied, we would publish it. 158 00:16:13,210 --> 00:16:21,250 We would prefer to publish it with his reply. He wrote me a very brief email in response, saying, Is this really what we've got to do? 159 00:16:22,030 --> 00:16:29,260 Which I. I didn't know whether I said that. John Bell has responded, saying that he has referred the letter to Roche. 160 00:16:30,940 --> 00:16:33,040 So we did get the reply, but nothing publicly. 161 00:16:34,180 --> 00:16:45,549 This was one of many open letters and also on our website a whole trail of correspondence that Tom and Peter Doshi managed between them and the CDC, 162 00:16:45,550 --> 00:16:56,020 W.H.O., the FDA, a whole host of regulatory and other bodies asking them and of course, Roche asking them to release the data. 163 00:16:56,350 --> 00:17:04,659 They did release the data and this is just a sign that we did get quite enough news coverage and there should be a picture here of Tom. 164 00:17:04,660 --> 00:17:07,380 But I don't know if you can kind of. Kim Kardashian. 165 00:17:07,400 --> 00:17:16,820 But this has been documented and we got very good wide coverage of the final result, which was that when the data arrived, 166 00:17:16,820 --> 00:17:24,350 eventually just on some disks, having been told it was an enormous confidentiality and all sorts of reasons why they couldn't share the data. 167 00:17:24,830 --> 00:17:33,470 And Tom and his team analysed the data, found that there was very little evidence of benefit that it had never been compared with placebo, 168 00:17:34,070 --> 00:17:42,590 saw it with paracetamol, the alternative influenza perhaps, and also that there were harms that had not been previously reported. 169 00:17:44,240 --> 00:17:48,380 So what did the Tamiflu saga do? 170 00:17:49,640 --> 00:17:55,130 We found that a vast amount of money had been spent. Poland was only countries that didn't stockpile the drug. 171 00:17:56,120 --> 00:17:59,120 I think they took a look at the evidence and thought it wasn't very sound. 172 00:17:59,540 --> 00:18:09,919 But in the UK, I'm going to get this figure right, 500 million if I got that right pound, which is half of a percent of the NHS. 173 00:18:09,920 --> 00:18:17,180 Total annual budget was spent on Tamiflu and they then renewed that with 50 million subsequent to that. 174 00:18:17,180 --> 00:18:24,020 So and stockpiling of the drugs sitting in, in warehouses properly used at various points, 175 00:18:25,490 --> 00:18:31,880 we, I mean it was a number of different pieces to this investigation. 176 00:18:32,570 --> 00:18:35,270 And I suppose the question we have to ask ourselves and Tom would be the best 177 00:18:35,300 --> 00:18:40,460 is to tell us this is have we really achieved is it really achieve anything, 178 00:18:41,600 --> 00:18:46,250 concrete things? Tamiflu came off the essential drug, the subject of central drug list. 179 00:18:47,240 --> 00:18:50,330 I think I think governments around the world did get a certain message. 180 00:18:50,330 --> 00:18:56,540 It was a great backlash from Roche. There was a whole saga about observational studies being better than trials. 181 00:18:56,540 --> 00:19:03,919 In this context, Carl and I appeared in front of the Public Accounts Committee where they had a very good discussion. 182 00:19:03,920 --> 00:19:12,140 And Sally DAVIES, the chief medical officer, obfuscated in a way that did not make me proud of our civil service. 183 00:19:12,680 --> 00:19:18,050 And that was that was the sort of the end of that saga. 184 00:19:18,890 --> 00:19:22,820 But Tamiflu is still out there, and I'm sure it has yet to. 185 00:19:23,180 --> 00:19:29,120 I mean, it will rear itself again. It will be found to be, you know, necessary or needed in some way. 186 00:19:29,150 --> 00:19:32,660 And so I think the most important thing was not about Tamiflu. 187 00:19:32,770 --> 00:19:35,479 That that is that is one area. 188 00:19:35,480 --> 00:19:42,710 The most important thing that it did was to show us what is needed if you really want to get the truth about a single drug. 189 00:19:42,980 --> 00:19:52,070 And I think I'm right in saying, Tom, it was the first, but it was the first real knowledge that was gained about CSL clinical study 190 00:19:52,070 --> 00:19:56,479 reports that prior to that we hadn't really understood the what they were, 191 00:19:56,480 --> 00:19:59,480 how they were kept, who had them, what they would show us. 192 00:19:59,900 --> 00:20:05,840 And so I feel it's been the sort of progenitor of a whole, whole new discipline, which is this much more in-depth, 193 00:20:06,320 --> 00:20:12,590 systematic reviewing that we are going to need to do if unless we move into a whole new era of transparency, 194 00:20:12,590 --> 00:20:22,219 where these things on the table and we began to sort of look more generally at the whole question of hidden data. 195 00:20:22,220 --> 00:20:25,790 And this is an editorial from Elizabeth Loder and Richard Lehmann. 196 00:20:26,030 --> 00:20:31,099 There is an Alice in Wonderland feel to these investigators searching over Hillandale and among the paperwork of regulatory 197 00:20:31,100 --> 00:20:35,899 bodies and drug companies to put together pieces of data that should have been freely available in the first place. 198 00:20:35,900 --> 00:20:39,139 So the question is, why are these data not available in the first place? 199 00:20:39,140 --> 00:20:45,680 What how have we got to a situation where it takes all this absurd effort and and and that's just on one drug. 200 00:20:46,910 --> 00:20:52,940 And we did a number of collections of articles where we asked people externally to send us studies, 201 00:20:52,940 --> 00:21:02,629 research and commentary and narrative and review on the issue of hidden data and what we could do to bring these data to the fore. 202 00:21:02,630 --> 00:21:08,090 And also discussions about what we mean by data and how we manage confidentiality and anonymization, 203 00:21:08,420 --> 00:21:17,220 which are all issues that are still, still ongoing. And we published an editorial looking at what we would do. 204 00:21:17,230 --> 00:21:22,470 The Journal Because I'm always very keen, I think I showed you the mission statement beginning that we want to walk the walk. 205 00:21:23,050 --> 00:21:29,200 It's very easy for journal editors to sort of just go around, say everyone should be better and and do better with you guys. 206 00:21:29,650 --> 00:21:35,800 But we always would like, where possible, to do something ourselves to show that we have skin in the game. 207 00:21:35,810 --> 00:21:43,180 And for us, this was that we would start requiring, if people wanted to publish a clinical trial with us, 208 00:21:43,510 --> 00:21:46,630 that they had to commit to sharing their data on reasonable request. 209 00:21:47,200 --> 00:21:54,180 Now, arguably, it's not much of a problem for us because we don't get nearly the same number of clinical trials sent to us as Lancet in JAMA, 210 00:21:54,190 --> 00:21:59,980 a New England Journal, the other big journals. And so we're less reliant on this kind of type of article. 211 00:22:00,610 --> 00:22:05,260 So if people suddenly decided to stop sending us stuff because of this requirement. 212 00:22:06,340 --> 00:22:11,500 We would not it wouldn't, you know, we wouldn't have great holes in our reprint revenue or, you know, 213 00:22:11,910 --> 00:22:16,850 the money that comes into journals to set of clinical trial reprints wouldn't have been so. 214 00:22:16,910 --> 00:22:21,320 So, I mean, I say this because that's been the pushback kind of said I wrote easy enough for the BMJ. 215 00:22:22,540 --> 00:22:29,260 But actually what's happened is we haven't seen a drop off in our trials and that maybe because we don't publish many industry trials, 216 00:22:29,710 --> 00:22:35,020 but but we have found that people seem to be willing to do this. And I think there is beginning to be a shift in the culture. 217 00:22:37,820 --> 00:22:45,649 And the other thing that came out of this, I think, which was brewing in the background and because of Peter Doshi connection with the Journal and 218 00:22:45,650 --> 00:22:51,020 because of the work that he and others had done and done with them on Tamiflu and calm as well. 219 00:22:51,020 --> 00:22:58,460 And the whole sort of bubbling up under the surface was this new initiative called Restoring Invisible and Abandoned Trials or Riot. 220 00:22:58,940 --> 00:23:04,489 And this front cover, which is where we launched the initiative in the Journal, is really just showing that, 221 00:23:04,490 --> 00:23:10,400 you know, you've got you've got the public trial like a proud flag on top of the iceberg. 222 00:23:11,060 --> 00:23:18,950 And everyone thinks that the totality of the evidence base, but underneath the water is this massive other hidden iceberg. 223 00:23:19,340 --> 00:23:22,729 And here we all are trying to look for evidence, 224 00:23:22,730 --> 00:23:31,670 trying to work out what's going on in his Tom Jefferson and Peter Doshi and others trying to look at the underbelly of the evidence base. 225 00:23:32,150 --> 00:23:35,110 And this, I think, is a really exciting initiative. 226 00:23:35,120 --> 00:23:41,540 And the idea is that where trials have been done and either badly reported, misreported or not reported, 227 00:23:42,230 --> 00:23:49,010 that these that this initiative can step into into play and that the original authors are given an 228 00:23:49,010 --> 00:23:55,160 opportunity to re draft their own work or to draft it in the first place if they haven't done that. 229 00:23:55,460 --> 00:23:59,090 But if they if they either say they haven't got the time or energy or didn't want to do it, 230 00:23:59,090 --> 00:24:04,190 or if they don't reply after a certain period, the idea is that others can then step in and do this. 231 00:24:04,460 --> 00:24:12,680 And we as a journal committed to publishing the results of those reanalysis in the same way we would as an original analysis of the original research. 232 00:24:13,280 --> 00:24:22,700 And the famous case that we knew was in the background was the case across the GlaxoSmithKline trial into antidepressants in teenagers, 233 00:24:22,700 --> 00:24:27,860 which had already been a subject of a lot of concern and public display about 234 00:24:28,430 --> 00:24:33,200 suicidal ideation and self-harm and in children who were taking paroxetine. 235 00:24:33,740 --> 00:24:40,580 So study 3 to 9 was already in the in the aether is something that had been proven to have been 236 00:24:40,580 --> 00:24:46,880 ghostwritten by an industry funded author and was already the subject of a vast legal case in America. 237 00:24:47,210 --> 00:24:53,860 And GSK was fined a large amount of money for wrongfully marketing this drug in this case. 238 00:24:53,870 --> 00:24:57,019 So there was a lot of activity around this drug. 239 00:24:57,020 --> 00:25:03,700 And and because the files because of the legal case that the data were available to it 240 00:25:03,710 --> 00:25:09,440 to be reanalysed and the authors who wanted to do that took it took on the analysis. 241 00:25:09,440 --> 00:25:11,280 It was a vast piece of work for them, 242 00:25:11,280 --> 00:25:19,099 and it was also a vast piece of work for my colleagues at the BMJ to make sure that we were already right about this, 243 00:25:19,100 --> 00:25:24,350 that one of the problems was that the authors themselves had been expert witnesses in this legal case, 244 00:25:24,860 --> 00:25:31,430 and so were considered to some extent to have them potentially not malicious, but at a certain angle on these data. 245 00:25:31,790 --> 00:25:38,269 And so we insisted that the authors introduce an independent review, separate to them, which caused another full saga of delay. 246 00:25:38,270 --> 00:25:46,009 But the data, certainly in terms of the adverse effects of this drug in teenagers, seemed to be quite compelling. 247 00:25:46,010 --> 00:25:53,750 So here is the original paper published still not retracted of the ghostwritten industry funded report. 248 00:25:54,080 --> 00:25:58,280 And these are adverse effects with Broxton, imipramine and placebo. 249 00:25:59,060 --> 00:26:09,830 And so a few, a few suicidal and self-injurious events in the peroxide group, but not vastly different to imipramine. 250 00:26:10,130 --> 00:26:19,760 And then in the SmithKline Beecham, which is the GSK analysis, then you can see rather more suicidal ideation and events in the APA group. 251 00:26:20,150 --> 00:26:28,760 And then this is the right response to a trial which found additional, definitely possible suicide in the jurors events. 252 00:26:29,750 --> 00:26:38,960 So this this seemed to be an important contribution to the role of hidden data in distorting things. 253 00:26:39,350 --> 00:26:47,270 So we published the restored trial and I think again, that has been quite an iconic contribution to the whole debate. 254 00:26:49,100 --> 00:26:54,470 So we now move to our own where we want to take it from here. 255 00:26:55,460 --> 00:27:05,240 And working with Karl and his unit in Oxford, we've produced an evidence manifesto which is looking to improve the development dissemination, 256 00:27:05,240 --> 00:27:10,340 implementation of research, evidence for better health. And so I'll just say what that says. 257 00:27:10,880 --> 00:27:15,140 We believe that the design conduct, quoting healthcare research should be better serve the needs of patients and the public. 258 00:27:15,140 --> 00:27:23,180 Better evidence leads to better healthcare. And we have here four things that we are aiming to do which have come out of the evidence manifesto, 259 00:27:23,660 --> 00:27:27,590 how we want to expand the role of patients and other users in research, in healthcare. 260 00:27:28,040 --> 00:27:31,550 We want to increase the systematic use of existing evidence for better decision making. 261 00:27:31,970 --> 00:27:36,180 We want to make research evidence relevant, replicable, accessible to healthcare professionals, patients, and. 262 00:27:37,010 --> 00:27:40,610 And to take a stand on financial interests by reducing questionable research practices. 263 00:27:41,720 --> 00:27:44,299 And that last thing is something I haven't talked a lot about, 264 00:27:44,300 --> 00:27:54,320 but we've got a big push at the Journal to move towards more independent research and to try to remove the clear, 265 00:27:55,190 --> 00:28:01,639 irreducible conflict of interest that commercial players have in in designing 266 00:28:01,640 --> 00:28:07,070 and performing and reporting their own evaluations of their own products. 267 00:28:08,600 --> 00:28:12,080 So this is a better evidence campaign, 268 00:28:12,890 --> 00:28:17,690 and I've talked a bit about some of this is on this is where you'll find is under that 269 00:28:17,690 --> 00:28:23,420 tab on the website I've mentioned in and Paul's work on 85% of research going to waste. 270 00:28:23,870 --> 00:28:27,589 Half of the results from half of all trials never publish positive results. 271 00:28:27,590 --> 00:28:33,490 Twice as likely to be published. Over 4/5 of a sample of Cochrane reviews did not include data on the main harm outcome. 272 00:28:34,580 --> 00:28:38,990 A systematic review of 39 studies found no robust studies evaluating shared decision making strategies. 273 00:28:39,170 --> 00:28:44,900 Is a real absence of the patient voice. The drug industry has been defined for criminal behaviour, 274 00:28:45,710 --> 00:28:50,600 but little happens to prevent such problems occurring again because these fines, although they seem enormous to us, 275 00:28:50,600 --> 00:28:57,140 are just like water off a duck's back for these large corporations, conflicts of interest, 276 00:28:57,890 --> 00:29:04,460 questionable research practices and manipulation of data happening as routine amongst some of these. 277 00:29:06,020 --> 00:29:13,820 So what is the BMJ currently doing? We working with Oxford Medicine on Evidence Live, which is a I think marvellous thing of course, 278 00:29:14,510 --> 00:29:20,420 which is an annual meeting where we get together to discuss some of these issues and try to work out better ways of doing things. 279 00:29:21,590 --> 00:29:25,790 We've launched the Evidence Manifesto, which I think importantly is intended. 280 00:29:25,790 --> 00:29:29,120 Whether we're succeeding or not, we need to keep questioning. 281 00:29:29,360 --> 00:29:32,530 But what was intended as a way of a bottom up? 282 00:29:32,540 --> 00:29:34,890 It's not intended as a kind of we've got the answer. 283 00:29:34,910 --> 00:29:41,720 It's intended to try to get as many people involved in finding solutions to these endemic problems within the evidence base. 284 00:29:43,160 --> 00:29:50,960 And and then we're looking at this data sharing. So this is the sharing of clinical trial data, the storing invisible about the trials. 285 00:29:52,010 --> 00:29:53,479 We've got a number of other initiatives. 286 00:29:53,480 --> 00:29:59,010 We're working hard on patient partnership because we are convinced that actually the very best type of research, 287 00:29:59,600 --> 00:30:06,649 research where patients have been involved from the outset and also potentially even where patients are leading the research. 288 00:30:06,650 --> 00:30:16,270 And there's an initiative hopefully coming soon in Oxford where that patient led research will be the aim in honour of Rosalind Snow, 289 00:30:16,280 --> 00:30:22,010 who was our patient advocate but sadly died a year ago, but maybe two years ago. 290 00:30:22,680 --> 00:30:34,190 She was a Oxford academic and was a terrific advocate for research that involves patients from the outset and is even led by them. 291 00:30:36,530 --> 00:30:45,320 So another number of things financial interest I've mentioned, and then I get to this campaign for the Stefan's data. 292 00:30:47,180 --> 00:30:52,340 So let me tell you, just before I finish down a little saga about Stefan. 293 00:30:54,920 --> 00:31:08,150 Back in 2013, the BMJ published a an analysis which was prompted by the fact that the Oxford Group that pulled together this systematic review, 294 00:31:08,180 --> 00:31:10,340 the trials on staff on statins, 295 00:31:11,240 --> 00:31:19,610 had produced a systematic review which seemed to suggest that as well as statens be useful for people at high risk, people who've had a heart attack, 296 00:31:19,880 --> 00:31:27,470 who had a stroke, people who have a family history and are therefore at high risk of heart disease and cardiovascular disease 297 00:31:27,530 --> 00:31:35,359 cerebrovascular disease that this treatment would also benefit people who have never had a stroke, 298 00:31:35,360 --> 00:31:41,240 never had a heart attack, do not have a family history, are low risk of cardiovascular and cerebrovascular disease. 299 00:31:42,260 --> 00:31:47,780 And this review published in The Lancet led to Cochrane, 300 00:31:48,440 --> 00:31:55,759 its own review coming up with a similar view that actually extending the statins to these low risk people wasn't 301 00:31:55,760 --> 00:32:02,690 was what was needed and subsequently nice clinical guidelines group coming up with the recommendation that yes, 302 00:32:02,810 --> 00:32:12,410 everyone over 50 would benefit from being on a staff and even if they were at low risk of heart come forward 2013 303 00:32:12,860 --> 00:32:19,670 to this time we've just recently had that indeed further evidence suggesting that people over the age of 75, 304 00:32:19,820 --> 00:32:23,060 pretty much everyone over the age 75 should be taking something. 305 00:32:23,540 --> 00:32:30,490 But in 2013 we published this critique which was really just saying the authors of the critique said, We just don't believe this is true. 306 00:32:30,500 --> 00:32:36,620 We've looked at the benefits and it doesn't hold true. And we've also looked at something that the other studies haven't really looked at, 307 00:32:36,620 --> 00:32:41,080 which is the harms, the small incremental, not necessarily life threatening. 308 00:32:41,370 --> 00:32:48,350 But quality of life threatening arms such as muscle, muscle, pain, fatigue, those sort of adverse effects. 309 00:32:49,640 --> 00:32:54,020 So we published this paper and got an immediate complaint from the Oxford Group. 310 00:32:54,530 --> 00:32:58,969 Rory Collins and his colleagues saying that we had made an error, that the paper contained an error, 311 00:32:58,970 --> 00:33:07,790 that the estimate of adverse effects from statens was overstated and this was going to kill people and a real worry that we had got this wrong. 312 00:33:09,500 --> 00:33:14,180 So I invited Rory Collins to write a rapid response, which is the response to everything. 313 00:33:14,180 --> 00:33:18,530 He declined to do that, and over a period of months we were in correspondence. 314 00:33:19,250 --> 00:33:24,379 And then at some point he went to the press and The Guardian published a thing saying The BMJ 315 00:33:24,380 --> 00:33:27,920 publish this terrible thing and people are going to die because they're not on this stuff. 316 00:33:27,920 --> 00:33:35,540 And by which time, I can't remember quite the chronology, but we published a correction of the article and the question was, should it be retracted? 317 00:33:36,440 --> 00:33:44,360 So we got an independent panel to come and assess for us whether we should retract the paper up to period of weeks. 318 00:33:44,360 --> 00:33:50,749 They came to the view that we should didn't we shouldn't do that. So to some extent, it felt like that was the end of the saga. 319 00:33:50,750 --> 00:33:57,110 But I think what had happened in the process was this had begun as a an issue about too much medicine. 320 00:33:57,350 --> 00:34:02,140 Should these people be on this drug? What was the outcome in terms of benefits? 321 00:34:02,150 --> 00:34:09,260 What would the expressed harms of extending this treatment to so many healthy people? 322 00:34:10,190 --> 00:34:18,319 But what it became in the process for me anyway, of this saga, which was very difficult and tricky and lots of people involved, 323 00:34:18,320 --> 00:34:23,270 and had we done it wrong, we got to write that sort of thing, feeling very much under scrutiny. 324 00:34:23,270 --> 00:34:27,260 What what it did for me was it made me realise this wasn't only a too much medicine story, 325 00:34:27,560 --> 00:34:33,590 it was an open data story because I had been completely unaware that this drug which is being 326 00:34:33,590 --> 00:34:39,350 prescribed is the most commonly prescribed prescribed drug in the developed world class of drugs. 327 00:34:40,430 --> 00:34:47,870 The data for those decisions were not available, and I just thought, oh, my goodness, this is it seems completely wrong. 328 00:34:48,920 --> 00:34:54,500 So so I realised I'm showing the wrong slide in telling all this, but okay, I'll go for it. 329 00:34:54,770 --> 00:34:55,790 I've got these in the wrong order. 330 00:34:56,480 --> 00:35:08,809 So as a result of that kind of moment of revelation, for me it became more of a fight for the data and also the sense that this Chinese wall, 331 00:35:08,810 --> 00:35:12,800 which the Oxford Group have created around their data, 332 00:35:13,820 --> 00:35:23,720 where they can all as chartists analyse each other's data, but that no one independent of those triallists is allowed to look at those data seemed, 333 00:35:24,740 --> 00:35:28,340 I think, wrong because there is no external third party scrutiny. 334 00:35:29,120 --> 00:35:34,070 So we began a long attempt to get the data and without much success, 335 00:35:34,610 --> 00:35:39,650 asking the Oxford Group to if they would share, asking individual trials that they would share. 336 00:35:39,860 --> 00:35:43,370 We got some who said they would. In this case we didn't have a Cochrane group. 337 00:35:43,370 --> 00:35:46,969 We didn't have the time, we didn't have a pizza gauci, we didn't have a car. 338 00:35:46,970 --> 00:35:49,940 We didn't have that. We didn't have people who would, if we got the data, take it. 339 00:35:50,330 --> 00:35:52,430 So there was a slightly different feeling about this, but we just thought, 340 00:35:52,430 --> 00:36:00,350 we've got to try and work out where these data are and I'm going in the completely wrong order anyway. 341 00:36:00,350 --> 00:36:01,760 We haven't really done very well. 342 00:36:01,940 --> 00:36:14,780 And so here we are back in now 2019 with these amazing transport systems in the press which are based on another one by the same group, 343 00:36:14,780 --> 00:36:27,680 the city, which seems to leap to quite a strong conclusion that if all these older people took statins, 8000 lives would be saved every year. 344 00:36:28,580 --> 00:36:32,690 And here's an editorial saying You're never too old to stop and treatment. 345 00:36:33,890 --> 00:36:39,200 But it turns out that actually, when you look at what's in the study and I can see this, 346 00:36:39,200 --> 00:36:45,950 that if you go to the study on the editorial that actually in the actual group of people aged over 75, 347 00:36:45,950 --> 00:36:53,030 there is very little evidence in that group and that a lot of this is extrapolation from studies that looked in different age groups. 348 00:36:53,900 --> 00:36:59,870 And statins have not been shown to reduce cardiovascular events in patients with cardiac or renal failure. 349 00:36:59,870 --> 00:37:06,950 Inclusion of older participants with these conditions in the trials might be an explanation for the failure to find a benefit in this group, 350 00:37:07,850 --> 00:37:11,450 which is borne out by an additional round of excluding trials conducted in these people. 351 00:37:11,450 --> 00:37:14,270 So what's been done is they didn't find the result they wanted. 352 00:37:14,540 --> 00:37:19,070 They then excluded people with cardiac or renal failure and found a slightly better result. 353 00:37:19,310 --> 00:37:23,180 And somehow or other that's interpreted the headline of 8000 Lives Saved. 354 00:37:23,900 --> 00:37:28,640 Carl kindly has done a look at the actual actual numbers here. 355 00:37:28,880 --> 00:37:32,959 So the question is, is life saved? A useful way of presenting? 356 00:37:32,960 --> 00:37:39,880 This is very dramatic. It goes for big headlines. But actually what we always say we want to do is to offer the number needed to treat number. 357 00:37:39,910 --> 00:37:42,800 These are. And this was Carl's. 358 00:37:43,160 --> 00:37:51,959 I think it's more work to go looking at the data and finding that in the 75 hours without backing this stuff and do not reduce major 359 00:37:51,960 --> 00:38:03,050 rescue events that the death or death may cause in over 70 years with vascular disease as a small reduction in vascular events. 360 00:38:03,350 --> 00:38:08,750 And so this 8000 lives saved. It's very hard to quite see where that comes from. 361 00:38:09,770 --> 00:38:16,010 And some of the other evidence suggests that actually there is really a great deal of benefit. 362 00:38:17,540 --> 00:38:21,290 So as I say, I'm rambling slightly here. 363 00:38:21,440 --> 00:38:22,730 Stefan's have that effect on me. 364 00:38:24,410 --> 00:38:31,070 What I wanted to try to clarify is that the too much medicine story and the hidden data story are like sister and brother. 365 00:38:31,490 --> 00:38:40,520 That actually when you look at whether it's too much medicine or too often, there's hidden data hiding behind it. 366 00:38:41,630 --> 00:38:44,900 And I think that's what's going on with Stefan's. 367 00:38:44,900 --> 00:38:46,160 And we do want to see the data. 368 00:38:46,940 --> 00:38:54,500 And as part of our campaign, we are now tracking the efforts of Tom and sounds like it sounds like a sort of Arctic Explorer, 369 00:38:55,280 --> 00:39:00,250 tone deaf and focusing on trying to get these clinical study reports. 370 00:39:00,260 --> 00:39:06,620 The thing we found out about because of Tamiflu, about statins to try to understand what is the evidence base. 371 00:39:07,790 --> 00:39:17,839 Can we be sure that large numbers of people in vast amounts of money are involved in this medicalized prevention, when in fact, 372 00:39:17,840 --> 00:39:29,690 we might be better off cleaning up our air, cleaning up our water, heating up our food to create healthier lives rather than putting people on pills. 373 00:39:31,040 --> 00:39:33,260 So you'll see. It says many requests are still pending. 374 00:39:33,650 --> 00:39:40,880 This comes out there writing to various regulatory bodies to try to get these data, but he might be willing to share some of these. 375 00:39:42,830 --> 00:39:56,440 So my final slide is one that Elizabeth Loder sent me, which is about what is what is the BMJ want to be and how we trying to compete, 376 00:39:56,450 --> 00:40:06,680 as I'm sure we are at one level with other journals or as in seven others, do we wish to be anything but Coca-Cola and I think we need. 377 00:40:08,530 --> 00:40:11,950 What we want to be the UN Journal. Thank you very much indeed.