1 00:00:00,330 --> 00:00:07,920 So hi everyone. I am Jamie Hartman Boice. I'm based at the Nuffield Department of Primary Care Health Sciences, just up the road in Jericho. 2 00:00:08,160 --> 00:00:12,360 And I also work closely with the Cochrane Tobacco Addiction Group and today I'm going to 3 00:00:12,360 --> 00:00:17,880 talk about why and when it all sides of the same question don't agree with each other, 4 00:00:18,240 --> 00:00:25,950 focusing on electronic cigarettes as an example. So I'm funded by the National Institute for Health Research and the British Heart Foundation. 5 00:00:25,950 --> 00:00:27,420 I don't have any complex to declare. 6 00:00:28,470 --> 00:00:34,770 So what I'll cover today is I'll talk very briefly about Cochrane, give a bit of an introduction to the issue of e-cigarettes. 7 00:00:34,770 --> 00:00:39,450 For some of you. You might know quite a lot of this, but for others you might not. And I think it's important context. 8 00:00:39,450 --> 00:00:47,400 Before we go into our Cochrane Review, some other meta analyses that set out to answer the same questions we did in our Cochrane Review. 9 00:00:47,670 --> 00:00:51,299 And then also looking into the wider literature for examples of where meta analysis might 10 00:00:51,300 --> 00:00:55,890 not agree with one another and end up talking about implications and potential next steps. 11 00:00:57,030 --> 00:01:02,459 So Cochrane is a global non-profit that primarily exists to do systematic reviews, 12 00:01:02,460 --> 00:01:08,130 and we aim to do this in a very robust, transparent, sometimes excessively thorough manner. 13 00:01:08,130 --> 00:01:11,430 So anyone who's read a Cochrane review knows that they are very long documents. 14 00:01:11,940 --> 00:01:16,379 But the reason we do this is we have very specific guidelines that we follow and we do this because we want 15 00:01:16,380 --> 00:01:20,970 to make sure that people making decisions about health care have the best available evidence to hand. 16 00:01:20,970 --> 00:01:25,020 And that includes patients, carers, clinicians and policymakers. 17 00:01:25,020 --> 00:01:29,969 And Cochrane is split into a number of different review groups. And those are subjects with this specific. 18 00:01:29,970 --> 00:01:33,390 And the Cochrane Tobacco Addiction Group is based here in Oxford. 19 00:01:34,920 --> 00:01:36,959 So now a little bit about e-cigarettes. 20 00:01:36,960 --> 00:01:42,180 E-cigarettes I think first came into the public consciousness in the mainstream way probably in the past ten years or so, 21 00:01:42,180 --> 00:01:44,400 but they've been around for a little bit longer than that. 22 00:01:44,820 --> 00:01:53,790 They were developed around 20 years ago now by a pharmacist in China who developed them for the purpose of them being a smoking cessation device. 23 00:01:54,180 --> 00:01:59,760 Within the field of smoking cessation, there are lots of evidence based things that can help people quit smoking. 24 00:02:00,030 --> 00:02:05,160 But unfortunately, even the best of those have success rates that really leave room for improvement. 25 00:02:05,490 --> 00:02:09,389 So in a best case scenario, let's say you're looking at doubling your quit rate. 26 00:02:09,390 --> 00:02:16,830 So you might be looking at a population who wants to quit smoking, doubling them from 10% without treatment to 20% within with treatment. 27 00:02:16,830 --> 00:02:20,489 And that's still leaving 80% of people who really want to quit smoking. 28 00:02:20,490 --> 00:02:27,719 We know the vast majority of adults who smoke want to quit, who just can't, because it is such a notoriously hard thing to do. 29 00:02:27,720 --> 00:02:31,230 And we also know that smoking is one of the worst things you can do for your health. 30 00:02:31,620 --> 00:02:34,799 And therefore the best thing a smoker can do for their health is quit. 31 00:02:34,800 --> 00:02:42,300 So e-cigarettes came into a market where we actually did need some better treatments to help people quit smoking and people were excited about them. 32 00:02:42,570 --> 00:02:44,309 And there's a number of reasons they were excited. 33 00:02:44,310 --> 00:02:52,320 So what they do is they heat a liquid into an aerosol for inhalation that usually comprises propylene glycol and glycerol, 34 00:02:52,320 --> 00:02:57,480 and it can or cannot contain flavours, counter, cannot contain nicotine, etc. 35 00:02:57,780 --> 00:03:01,049 And the key thing about e-cigarettes is that when they contain nicotine, 36 00:03:01,050 --> 00:03:05,459 they're delivering that nicotine without combustion because the harmful thing about smoking, 37 00:03:05,460 --> 00:03:09,300 the harmful thing about cigarettes is actually what happens when the product burns. 38 00:03:09,300 --> 00:03:13,590 It's not the nicotine itself. It's not the other chemicals on their own. It's what happens when they burn. 39 00:03:13,590 --> 00:03:18,569 And essentially what they were trying to do when they developed e-cigarettes was deliver a cigarette without the combustion. 40 00:03:18,570 --> 00:03:25,110 So make a less harmful alternative. Another really important thing about e-cigarettes is that they are not all the same. 41 00:03:25,110 --> 00:03:29,159 So it's really tempting when we talk about them to be like, Oh, they're dangerous or they're not dangerous. 42 00:03:29,160 --> 00:03:32,639 They do this or they do that. There's a huge range of e-cigarettes. 43 00:03:32,640 --> 00:03:37,770 So there are four generations, as they're commonly referred to, and that's what that picture is there. 44 00:03:38,220 --> 00:03:41,240 The first one, which is the first ones that really came out, are cigar like. 45 00:03:41,250 --> 00:03:45,390 So they look a lot like cigarettes. If you were 20 feet away, you'd probably think it was a cigarette. 46 00:03:45,690 --> 00:03:49,469 And over time, they've evolved to devices that look pretty much nothing like cigarettes. 47 00:03:49,470 --> 00:03:54,810 So there are vape pens which are second generation, those box models, which are those larger tank models. 48 00:03:55,170 --> 00:04:00,690 And in the last couple of years, we've seen a lot more vape pods, which look quite a lot like USB sticks. 49 00:04:00,990 --> 00:04:05,640 And probably the most popular brand of those are tool and take a seat. 50 00:04:07,110 --> 00:04:13,530 So in any cigarette, the e-liquid or the juice is stored in a refillable or disposable cartridge or reservoir. 51 00:04:13,530 --> 00:04:19,319 And that totally just depends on the type of e-cigarette, and it can vary dramatically in its nicotine content, 52 00:04:19,320 --> 00:04:24,120 so it can vary from absolutely no nicotine to here if it was a regulated product, 53 00:04:24,120 --> 00:04:28,319 not more than 20 milligrams per millilitre, that is specific to the EU. 54 00:04:28,320 --> 00:04:31,320 So that's a regulation we have here on our nicotine content. 55 00:04:31,320 --> 00:04:35,430 In the States, you might get concentrations up to, for example, 60 milligrams. 56 00:04:35,760 --> 00:04:40,739 They're very, very much in their flavours. And there's also an opportunity, especially with some of the newer devices, 57 00:04:40,740 --> 00:04:45,240 for people to put what they want in those devices and to tamper with them in some cases. 58 00:04:45,240 --> 00:04:51,900 So trying to make a guess at what an e-cigarette contains if you're not buying it from a reputable retailer, is a risky thing to do. 59 00:04:52,080 --> 00:04:55,739 And so here in the UK we're pretty well regulated and most people who use e-cigarettes 60 00:04:55,740 --> 00:04:59,640 buy them from a shop and they can trust that it's going to conform to all of these EU. 61 00:04:59,900 --> 00:05:02,950 Relations in the States. That's not necessarily the case. 62 00:05:03,550 --> 00:05:07,910 And e-cigarette users are sometimes described as vapers e-cigarette use as vaping. 63 00:05:07,930 --> 00:05:14,559 So if you hear me mention it, that's what I mean. And probably the other thing to know about e-cigarettes is that they cause a lot of controversy. 64 00:05:14,560 --> 00:05:17,770 So I don't know how many people have seen them in the news in the past year. 65 00:05:18,070 --> 00:05:24,879 I'm certainly very aware of it. And when I first started working in tobacco control research about eight years ago, 66 00:05:24,880 --> 00:05:28,330 you would go to a conference and generally everyone was on the same page about everything. 67 00:05:28,330 --> 00:05:33,340 We might have minor disagreements on analysis methods, but essentially we were all there with the same basic aim. 68 00:05:33,640 --> 00:05:40,360 That has dramatically changed. So these conferences have become a lot more exciting, but also a lot more conflicting. 69 00:05:41,410 --> 00:05:46,270 And people feel very, very strongly on that. Either pro e-cigarettes or anti e-cigarettes front. 70 00:05:46,270 --> 00:05:49,270 And broadly, those divisions go by countries. 71 00:05:49,540 --> 00:05:54,279 So in the U.K. here we are often criticised for being too favourable towards 72 00:05:54,280 --> 00:05:59,170 e-cigarettes and we in the UK often criticised our US colleagues for just the opposite. 73 00:05:59,620 --> 00:06:05,680 And what this means is that the public in particular get very, very conflicting messages. 74 00:06:06,070 --> 00:06:12,430 And there is a concern not only about this affecting the public's understanding of e-cigarettes and the relative risks and benefits, 75 00:06:12,730 --> 00:06:15,610 but also about possibly undermining public trust in scientists. 76 00:06:15,610 --> 00:06:21,610 Because when you have big scientists on either side kind of slinging insults at each other and saying dramatically different things, 77 00:06:21,610 --> 00:06:24,190 why does that put you in terms of trust in these experts? 78 00:06:25,840 --> 00:06:32,739 So as I said, researchers can't agree just a quick like I spent 5 minutes on Twitter to come up with some of these reasons why researchers can agree. 79 00:06:32,740 --> 00:06:36,010 And these are all people who are followed by thousands of people besides myself at the end. 80 00:06:36,010 --> 00:06:40,629 They don't have that many followers. But I stand glance at the University of California. 81 00:06:40,630 --> 00:06:47,380 San Francisco is a major name in tobacco control research, and he is majorly anti e-cigarettes. 82 00:06:47,380 --> 00:06:50,290 And we're going to look at some of his research further on down the line. 83 00:06:50,560 --> 00:06:54,670 And what he's arguing here is that e-cigarettes make it harder for smokers to quit. 84 00:06:55,090 --> 00:06:57,790 So that is an interesting and very strong message there. 85 00:06:57,790 --> 00:07:03,920 And he's also saying dual use, which means using both an e-cigarette and a cigarette at the same time is more dangerous than just smoking. 86 00:07:03,940 --> 00:07:09,969 So, again, very bold claim. But on the other hand, we have Linda Bauld is a professor up in Edinburgh. 87 00:07:09,970 --> 00:07:17,230 She's the Cancer Research UK is cancer prevention champion and this is her tweeting about a recent presentation which looked at the 88 00:07:17,230 --> 00:07:23,440 effects of e-cigarettes in the cardiovascular system and found significant risk reduction when people switched completely to vaping. 89 00:07:23,440 --> 00:07:29,590 So how do those things balance with that and have Martin Dockrell, who's the head of tobacco control for Public Health England, 90 00:07:29,800 --> 00:07:37,340 and he tends to be quite pro vaping and calls out some of the research saying that there are vaping epidemics that have Robert West at UCL, 91 00:07:37,340 --> 00:07:41,200 again, extremely senior professor who is getting extremely angry with the W.H.O. 92 00:07:41,200 --> 00:07:47,800 This was last week. The W.H.O. came out with another report on e-cigarettes saying they're harmful to health and are not safe, 93 00:07:48,100 --> 00:07:54,100 particularly risky when used by adolescents. Nicotine is highly addictive and young people's brains develop up to their mid twenties. 94 00:07:54,310 --> 00:07:57,130 Exposure to nicotine can have long lasting damaging effects. 95 00:07:57,490 --> 00:08:01,750 And Robert's response to this was to say this is propaganda that fuels distrust of experts 96 00:08:01,930 --> 00:08:05,799 and provides a spawning ground for conspiracy theories such as anti-vaccine movement. 97 00:08:05,800 --> 00:08:09,250 So these arguments are deep, people feel them strongly, 98 00:08:09,430 --> 00:08:13,990 and these are all relatively reputable people and reputable organisations saying very different things. 99 00:08:14,830 --> 00:08:20,830 Stern Glantz I think is right about at least one thing, which is that there's a war between scientists over the health risks of vaping. 100 00:08:20,830 --> 00:08:24,070 And that is one thing that we definitely agree on because I've written on it too. 101 00:08:25,600 --> 00:08:31,240 Ultimately, these conflicts boil down to a few key points and these key points drive policy, 102 00:08:31,450 --> 00:08:37,150 and they're related to how we interpret evidence, but they're also related to what questions were asking of the evidence to start with. 103 00:08:38,080 --> 00:08:41,709 So the first big question is, do they cause more kids to smoke? 104 00:08:41,710 --> 00:08:45,820 And I think it's fair to say when e-cigarettes first came on the market within tobacco control, 105 00:08:45,820 --> 00:08:52,120 that was probably the thing we were the most worried about because we knew that the tobacco industry was interested in e-cigarettes. 106 00:08:52,120 --> 00:08:57,320 They were buying up these companies and we definitely didn't trust them after, you know, 107 00:08:57,340 --> 00:09:05,050 decades and decades of horrible misconduct with are they going to use this as another route to advertise cigarettes, get people hooked, etc.? 108 00:09:05,830 --> 00:09:12,160 There are lots of cohort studies which show that kids who use e-cigarettes are more likely to go on and smoke. 109 00:09:12,470 --> 00:09:17,710 A lot of the time that is interpreted as e-cigarettes, leaving people to smoke and a gateway effect. 110 00:09:18,370 --> 00:09:24,250 That story changes slightly when we look at the prevalence of smoking in young people, that continues to decline. 111 00:09:25,510 --> 00:09:28,450 So there are some unanswered questions around that particular issue. 112 00:09:29,770 --> 00:09:35,020 Another issue also around young people is do young people vape who wouldn't otherwise have smoked? 113 00:09:35,350 --> 00:09:40,120 Because we know that e-cigarettes aren't completely safe, it's not a good idea to inhale anything other than fresh air. 114 00:09:40,630 --> 00:09:44,290 But compared to smoking, they do appear a lot safer. 115 00:09:44,290 --> 00:09:49,600 And there's a big debate here because actually, if these kids who are vaping would have otherwise ended up becoming addicted to cigarettes, 116 00:09:50,410 --> 00:09:54,040 we'd probably prefer them to be addicted to vaping compared to being addicted to cigarettes. 117 00:09:54,040 --> 00:09:58,270 But if they weren't going to be addicted to anything, then we don't want them to start vaping and become addicted. 118 00:09:58,600 --> 00:09:59,800 And this is something that's causing. 119 00:10:00,210 --> 00:10:07,980 A lot of concern, particularly in the US around this terminology, around a vaping epidemic and lots of teams starting to use e-cigarettes. 120 00:10:08,760 --> 00:10:15,240 And then what I'm going to focus on for the rest of the talk is really the issue about adult smokers and do they help adult smokers to quit? 121 00:10:15,540 --> 00:10:21,540 Public Health England has come up quite strongly on this. So every October they run a stop smoking campaign called Stop Tober, 122 00:10:21,540 --> 00:10:28,470 and this year they proactively encourage people to use e-cigarettes to switch and to stop smoking. 123 00:10:29,130 --> 00:10:32,220 And ultimately, of course, the main question here, too, is, are they safe? 124 00:10:32,430 --> 00:10:35,700 And here again, we have massively conflicting information. 125 00:10:36,120 --> 00:10:40,109 So the Centre for Disease Control in the US came out last year with quite a lot of 126 00:10:40,110 --> 00:10:46,260 advertisements targeting young people with quite scary messages about damaging their brains. 127 00:10:48,060 --> 00:10:53,070 Put aside that cancer research UK came out last year and kept on putting out infographics talking about 128 00:10:53,070 --> 00:10:58,910 their relative safety of e-cigarettes compared to smoking and saying pretty positive things about nicotine. 129 00:10:58,920 --> 00:11:03,750 So it's addictive, but it doesn't cause cancer. So that's a slightly different story you're seeing there. 130 00:11:04,140 --> 00:11:12,990 Public Health England has an often cited figure that e-cigarettes are not risk free but are estimated to be about 95% safer than regular cigarettes. 131 00:11:13,980 --> 00:11:20,879 And then we have the World Health Organisation last week answering the question are e-cigarettes more dangerous than regular cigarettes? 132 00:11:20,880 --> 00:11:26,220 And saying this depends on a range of factors, including the amount of nicotine and other toxicants in the heated liquids. 133 00:11:26,460 --> 00:11:30,360 But we know they pose clear health risks and are by no means safe. 134 00:11:30,750 --> 00:11:33,899 And all of this has become more of an issue because in the last year there 135 00:11:33,900 --> 00:11:37,590 have been a number of vaping related serious illnesses and deaths in the US. 136 00:11:38,220 --> 00:11:44,760 And one of the reasons why the conversations here get a bit confusing is that in the 137 00:11:44,760 --> 00:11:48,900 US it very much seems the case and the CDC has come out and said this is the case, 138 00:11:49,260 --> 00:11:53,400 that what is the risk here is an additive called vitamin E acetate. 139 00:11:53,700 --> 00:11:56,999 That is something that is not allowed in e-cigarettes in the EU because we 140 00:11:57,000 --> 00:12:00,210 absolutely knew from the start that if you vape that you're going to get sick. 141 00:12:01,350 --> 00:12:05,970 And it's used in e-cigarettes in the US, primarily ones that are sold on the black market, 142 00:12:06,000 --> 00:12:10,800 off the street that contain cannabis because it looks very similar to THC oils. 143 00:12:11,130 --> 00:12:17,580 So if you're buying something, you can basically cut it with vitamin E acetate and make more money that way. 144 00:12:17,850 --> 00:12:20,610 So we have all of that going on and no one quite knows what to think. 145 00:12:20,610 --> 00:12:25,829 And whenever me, with my own biases towards Cochrane systematic reviews see something like this, 146 00:12:25,830 --> 00:12:28,960 I'm like, Well, let's just do a systematic review and then we'll figure out the answer. 147 00:12:29,200 --> 00:12:33,690 It'll be unbiased, it will be transparent, everything will be great. So we did a Cochrane review. 148 00:12:33,780 --> 00:12:41,010 We first published our Cochrane Review in 2014. We updated it in 2016, and that's the version I'm going to be focusing on in this talk. 149 00:12:41,400 --> 00:12:46,260 And we're in the midst of updating again currently, so we're expecting a new version to be out later this year. 150 00:12:47,850 --> 00:12:53,520 In terms of what we included typically in our group, when we're looking at a smoking cessation therapy, 151 00:12:53,520 --> 00:12:59,639 we would only look at randomised controlled trials and here absolutely we preferred randomised controlled trials but we 152 00:12:59,640 --> 00:13:03,720 knew that there just weren't that many studies out there and indeed there still aren't that many studies out there. 153 00:13:04,080 --> 00:13:08,430 So we widened our inclusion criteria to also include uncontrolled intervention studies. 154 00:13:08,730 --> 00:13:12,959 So these were studies where everyone was given some sort of e-cigarette intervention and also to 155 00:13:12,960 --> 00:13:19,620 include observational studies of the type that basically survey a group of smokers at the beginning, 156 00:13:19,980 --> 00:13:22,110 ask them whether or not they're using e-cigarettes. 157 00:13:22,350 --> 00:13:29,580 Follow them up sometime later and see if there are differences in quit rates between those using e-cigarettes and those not using e-cigarettes. 158 00:13:30,000 --> 00:13:35,219 Again, as standard in our group, we are only interested in smoking cessation at six months or longer because unfortunately a 159 00:13:35,220 --> 00:13:39,210 lot of people who try to quit might do so successfully in the first week and then relapse, 160 00:13:39,450 --> 00:13:42,630 and that's not going to confer any significant long term health benefit. 161 00:13:42,640 --> 00:13:47,760 So we're really interested in lasting effects and we're interested in adverse effects after a week of use. 162 00:13:48,300 --> 00:13:54,450 And that third group of studies, which I will talk about quite a bit in a little while, we're not included in this current update, 163 00:13:54,720 --> 00:13:59,220 and that is because the nature of their design and serious risks of confounding just means 164 00:13:59,220 --> 00:14:02,730 we felt that actually we're getting at least a few more randomised controlled trials, 165 00:14:02,730 --> 00:14:07,410 a few more uncontrolled intervention studies. Let's leave this group of studies behind moving forward. 166 00:14:08,580 --> 00:14:12,930 So in terms of our outcomes, we're interested in cessation at six months or longer. 167 00:14:13,260 --> 00:14:19,170 We treat people who don't come to follow up as smoking in our analyses, and that's pretty standard across the field. 168 00:14:19,470 --> 00:14:25,980 And that's because if you're having contact with a researcher who's encouraging you to quit smoking and you have successfully quit smoking, 169 00:14:26,190 --> 00:14:28,950 it can be really lovely and quite an incentive to go back and tell them. 170 00:14:29,160 --> 00:14:34,740 And if you haven't quit smoking and they've invested time and effort in you, it can be quite a disincentive to go and tell them. 171 00:14:35,340 --> 00:14:38,490 And we use the strictest available definition of abstinence. 172 00:14:38,700 --> 00:14:42,510 We also looked at a range of adverse events and various safety profiles. 173 00:14:43,590 --> 00:14:48,239 In terms of our numerical analysis, we pooled data where appropriate and here again, 174 00:14:48,240 --> 00:14:51,860 we just used our standard Cochrane methods that we use for any cessation intervention. 175 00:14:52,260 --> 00:14:56,669 So this involves calculating risk ratios which we calculate as the number of people 176 00:14:56,670 --> 00:14:59,760 quit in the intervention group over the number of people in the intervention. 177 00:14:59,820 --> 00:15:05,340 And group. And then over that, the number of people quit the control group and the number of people in the control group. 178 00:15:05,340 --> 00:15:07,470 And so risk ratio because it's a ratio. 179 00:15:07,710 --> 00:15:13,080 If you end up with one, it means that you have the same proportion of people quitting in the intervention and control arms. 180 00:15:13,350 --> 00:15:18,989 If you end up with a risk ratio less than one, it means that more people quit in the control group than in the intervention group. 181 00:15:18,990 --> 00:15:24,389 And on the opposite side, if you have one greater than one, then you know that more people are quitting in the intervention group. 182 00:15:24,390 --> 00:15:28,950 And if an intervention is successful, that's what we're hoping to see is a risk ratio greater than one. 183 00:15:29,190 --> 00:15:35,370 And preferably if we want statistical significance, a risk ratio that doesn't confidence interval that doesn't encompass one. 184 00:15:36,510 --> 00:15:42,749 So unfortunately back in 2016 there were only two randomised controlled trials that looked at our question and even 185 00:15:42,750 --> 00:15:47,370 more unfortunately these are the same two randomised controlled trials that were included in our 2014 review. 186 00:15:47,820 --> 00:15:54,059 So these are both two relatively small studies, one conducted in Italy which followed up over 12 months, 187 00:15:54,060 --> 00:15:57,150 one conducted in New Zealand which followed up for six months. 188 00:15:57,570 --> 00:16:01,860 And the comparison we're interested in here are the comparison between people given 189 00:16:01,860 --> 00:16:05,790 e-cigarettes with nicotine and then those given e-cigarettes without nicotine. 190 00:16:06,060 --> 00:16:12,719 And the reason that's an interesting comparison is it's pretty much the closest we could get to a placebo controlled trial of this device. 191 00:16:12,720 --> 00:16:18,060 And there are more studies coming out now, which I think more interestingly are comparing e-cigarettes with nicotine to, for example, 192 00:16:18,060 --> 00:16:23,640 other forms of nicotine replacement therapy, which might be a more interesting question, but that's where we were then. 193 00:16:24,570 --> 00:16:28,680 And what we found is that neither of these studies on their own found a significant effect. 194 00:16:28,680 --> 00:16:34,620 But when we pulled them, all of a sudden we did have a statistically and clinically significant effect in favour of the intervention. 195 00:16:34,620 --> 00:16:37,199 So in favour of e-cigarettes with nicotine. 196 00:16:37,200 --> 00:16:44,250 As you'll note, the lower confidence interval is quite close to one, so it's just narrowly achieved statistical significance as it were. 197 00:16:44,640 --> 00:16:48,690 And as with all Cochrane reviews, what we do is we grade the certainty of our evidence. 198 00:16:48,690 --> 00:16:52,860 So we think about how much do we trust that, how much do we trust this effect estimate? 199 00:16:53,160 --> 00:16:56,879 And what we said was not very much. Essentially, our certainty in this is low. 200 00:16:56,880 --> 00:17:00,840 We think no studies are very likely to change our effect estimate. And there are two issues here. 201 00:17:01,200 --> 00:17:07,139 One was imprecision. So basically wide confidence intervals, and those are coming from very small numbers of events. 202 00:17:07,140 --> 00:17:15,600 So even though we have about 600 people in these analyses, we only have seven people in the control arm who quit in 43 in the intervention arm. 203 00:17:15,600 --> 00:17:24,509 So that's not very many. And another issue which is not as common is an issue of indirectness here. 204 00:17:24,510 --> 00:17:30,210 And this comes because e-cigarettes are relatively new. The technology is changing rapidly. 205 00:17:30,600 --> 00:17:34,049 It takes a very long time to get a randomised controlled trial off the ground. 206 00:17:34,050 --> 00:17:37,380 You apply for funding, you apply for ethics, you run the trial and then you need to publish it. 207 00:17:37,800 --> 00:17:41,880 And so the unfortunate reality in this field is that by the time most of these studies are published, 208 00:17:42,150 --> 00:17:45,780 the devices they tested are no longer on the market. And that's the case here. 209 00:17:45,780 --> 00:17:49,799 So both of the devices tested in these two studies by the time they published were 210 00:17:49,800 --> 00:17:52,500 pulled from the market because they were simply terrible at delivering nicotine. 211 00:17:53,040 --> 00:17:57,809 And bear in mind, the comparison we have is between a nicotine delivery device and a non nicotine delivering device. 212 00:17:57,810 --> 00:18:03,270 And so how relevant is this to someone who might show up to their clinician and ask, Should I go buy an e-cigarette? 213 00:18:03,870 --> 00:18:09,329 These studies aren't going to tell them quite as much about the e-cigarette they're going to buy now as what they would have bought back in 2014, 214 00:18:09,330 --> 00:18:15,600 let's say, in terms of adverse events. Again, we had low certainty in the evidence because of a small number of studies, 215 00:18:15,600 --> 00:18:19,230 but we didn't find anything particularly alarming in the studies that we looked at. 216 00:18:19,590 --> 00:18:22,620 There were no serious adverse events related to e-cigarette use. 217 00:18:22,920 --> 00:18:26,999 The non serious events didn't tend to vary between arms and the cohort studies. 218 00:18:27,000 --> 00:18:31,680 All told, a similar picture of mouth and throat irritation that seemed to dissipate over time. 219 00:18:31,680 --> 00:18:34,740 But major caveat. Our longest follow up here was two years. 220 00:18:35,070 --> 00:18:40,050 And of course, in most of the studies, what we're looking at are nicotine containing e-cigarettes that are regulated. 221 00:18:40,110 --> 00:18:42,720 So we are going to see a different picture from, for example, 222 00:18:42,750 --> 00:18:48,420 tempered products that someone might be buying off the street now, somewhat distressingly. 223 00:18:48,420 --> 00:18:55,530 So we published a review in 2014 with those two studies because we're Cochrane We update our review, so we published it again in 2016. 224 00:18:56,010 --> 00:19:03,629 We included the same two studies and in that time 20 other systematic reviews were published which looked at this exact same question. 225 00:19:03,630 --> 00:19:08,670 And I think you do have to then start to worry a little bit about whether or not that's a good use of time. 226 00:19:08,670 --> 00:19:15,180 There are so many unanswered research questions is this is what is this what all the researchers need to be looking at? 227 00:19:15,720 --> 00:19:23,400 I would argue possibly not all of them agreed that more evidence was needed, and I think that's a no brainer in this area. 228 00:19:23,400 --> 00:19:27,660 That's absolutely the case, particularly around long term safety and long term efficacy. 229 00:19:28,320 --> 00:19:30,600 And five of them conducted meta analyses. 230 00:19:30,600 --> 00:19:35,970 And in the next slides, I'm going to focus on this meta analysis and how they may be similar or different to ours. 231 00:19:37,140 --> 00:19:41,250 So my first one, and I'm not trying to pick on any of these because I understand why they did it. 232 00:19:41,550 --> 00:19:44,760 I published a second review that had the exact same two studies that my first review did. 233 00:19:44,760 --> 00:19:49,830 So I'm part of the problem too, arguably, but the first one was published in 2015. 234 00:19:50,070 --> 00:19:53,010 It included the exact same two randomised controlled trials. 235 00:19:53,010 --> 00:19:59,620 We did it use the exact same methods we did and it found the exact same thing and I think that is both trouble. 236 00:20:00,000 --> 00:20:03,920 And reassurance, certainly for me as the author and the one who did that analysis, 237 00:20:03,930 --> 00:20:07,470 it was wonderful that someone had reproduced it and found the exact same thing. 238 00:20:08,070 --> 00:20:11,670 However, was it really a useful thing for them to have spent their time doing? 239 00:20:11,790 --> 00:20:15,780 I'm not entirely sure, but we certainly found the same thing. 240 00:20:16,470 --> 00:20:24,930 Then there were three studies which came out in 2016 which had some similarities and some differences, and it's worth focusing on these. 241 00:20:24,960 --> 00:20:29,280 So just like us, they included the exact same two randomised controlled trials. 242 00:20:29,520 --> 00:20:37,770 Again, quite reassuring. We're not missing anything here. A big difference for them is that whereas we looked at results of the longest follow up, 243 00:20:37,770 --> 00:20:42,150 which in the case of that Copper Nettle study was 12 months and then followed with six months. 244 00:20:42,270 --> 00:20:49,020 They all pooled results at six months follow up losses were treated as continuing smokers just like for us. 245 00:20:49,440 --> 00:20:55,290 And the really key difference in their results was that they all found an effect size favour in the intervention, 246 00:20:55,530 --> 00:21:02,310 but none of their effect sizes were statistically significant. And what this comes down to is that difference between six and 12 months. 247 00:21:02,640 --> 00:21:05,250 And worryingly, what is driving that is one person. 248 00:21:05,520 --> 00:21:11,670 So in the cap, another study, which is the one that followed up at 12 months, there was one person in the control group had quit at six months. 249 00:21:12,000 --> 00:21:19,490 At 12 months, they'd relapse to smoking. And that difference in one person shifted the whole result to no longer being statistically significant. 250 00:21:19,550 --> 00:21:23,070 So I think that makes us feel better about downgrading for imprecision because 251 00:21:23,070 --> 00:21:27,450 clearly that is an issue and I think it also highlights some of the issues 252 00:21:27,450 --> 00:21:34,200 around overly relying on statistically significant is something seriously meaningful if one person starting to smoke again switches the whole thing? 253 00:21:34,440 --> 00:21:37,620 How much of our messaging should be focussed on that? 254 00:21:37,860 --> 00:21:41,669 You know, there are some minor variations to in their individual effect estimates and 255 00:21:41,670 --> 00:21:44,850 those are basically due to different models and random versus fixed effect, 256 00:21:45,120 --> 00:21:49,800 etc. But essentially they're all using six month data and they're all finding basically the same thing. 257 00:21:50,790 --> 00:21:57,239 Then the review will focus on the longest is one where the senior author is Dan Glantz, 258 00:21:57,240 --> 00:22:00,330 who I talked about a little bit earlier and promised I'd come back to. 259 00:22:01,260 --> 00:22:06,659 And this one is I'm focusing on because it found something dramatically different from ours and it came out the same year ours did. 260 00:22:06,660 --> 00:22:10,750 So we ended up having to do quite a lot of looking into why is this coming out so different? 261 00:22:10,770 --> 00:22:17,489 Where do we stand on this? And the main difference here and the reason that's driving their different results 262 00:22:17,490 --> 00:22:21,139 is that they included a much wider range of studies and their meta analysis. 263 00:22:21,140 --> 00:22:25,590 So rather than just restricting to the two controlled trials, which are the same two trials we had, 264 00:22:26,400 --> 00:22:31,380 they included 15 cohort studies and three cross-sectional studies. 265 00:22:31,650 --> 00:22:36,420 And they pooled all of these together and it is rare that you'll see lots of different study types combined 266 00:22:36,420 --> 00:22:41,190 in one analysis and hopefully I'll be able to convince you of some of the issues with doing that. 267 00:22:41,190 --> 00:22:47,250 I think sometimes it's legitimate, but they were heavily criticised for doing this and it is, I think, fair to say, quite unusual. 268 00:22:48,300 --> 00:22:54,420 We reported results at longest follow up. They reported whatever was in the original paper and didn't have any cut off based on length. 269 00:22:54,420 --> 00:23:00,719 So it could have been a month that could have been two years. They used whatever methods were used in the original paper for Lost to follow up, 270 00:23:00,720 --> 00:23:04,080 which I think is a problem because I think particularly if you're working in an area like 271 00:23:04,080 --> 00:23:08,700 this where you have lost the follow up that you absolutely know is not at all random. 272 00:23:08,910 --> 00:23:11,250 You probably want to be using the same method to impute. 273 00:23:11,250 --> 00:23:20,219 Otherwise, you're introducing spurious differences between studies and their effect estimate was 0.72 with confidence intervals that did not span one. 274 00:23:20,220 --> 00:23:23,610 So they found that e-cigarettes made it harder to quit smoking. 275 00:23:23,610 --> 00:23:28,050 And that is why stand keeps tweeting that and saying they're really stopping people from quitting. 276 00:23:28,290 --> 00:23:35,820 We found that e-cigarettes did help people quit smoking. And what is driving that absolutely is their inclusion of this observational data. 277 00:23:36,480 --> 00:23:39,690 And there's a number of reasons why we might expect the observational data in 278 00:23:39,690 --> 00:23:43,140 this area to show something different from the randomised controlled trials. 279 00:23:43,950 --> 00:23:48,209 The first one is that there's very good reasons to believe that the effectiveness of 280 00:23:48,210 --> 00:23:52,830 e-cigarettes for a smoking cessation aid is very dependent on the amount of support provided. 281 00:23:53,160 --> 00:23:56,250 So we know that even with something like nicotine replacement therapy, 282 00:23:56,250 --> 00:24:01,560 if you give everyone nicotine replacement therapy and randomised one arm to receive extra behavioural support, 283 00:24:01,770 --> 00:24:05,790 whether or not that's just talking to someone once, that will increase their chances of quitting. 284 00:24:06,000 --> 00:24:09,900 And then the trials of e-cigarettes, absolutely they were interacting with people. 285 00:24:10,200 --> 00:24:14,550 The other thing about e-cigarettes is that they are not necessarily the most intuitive to use. 286 00:24:15,480 --> 00:24:22,260 My in-laws use e-cigarettes. They're both in their late sixties and they needed a lot of guidance on how to use that e-cigarette. 287 00:24:22,530 --> 00:24:25,829 And if you're just buying one from a shop and you don't necessarily know what you're doing with it, 288 00:24:25,830 --> 00:24:28,950 you may well not be getting great nicotine delivery, for example. 289 00:24:30,060 --> 00:24:36,060 The second issue with these observational studies is that there are serious issues around the baseline definition of e-cigarette uses. 290 00:24:36,090 --> 00:24:39,419 So just to remind you, these are studies which go out, 291 00:24:39,420 --> 00:24:44,880 they get a group of smokers and they ask them at baseline if they use e-cigarettes and then they follow them up later. 292 00:24:45,210 --> 00:24:52,230 And these studies, some of them are saying, okay, we define e-cigarettes as using an e-cigarette at least once a week or at least once a day. 293 00:24:52,500 --> 00:24:56,010 Others are at any point in your entire life have you ever tried an e-cigarette? 294 00:24:56,310 --> 00:24:59,400 So someone who tried one once because their friend had one in the pub. 295 00:24:59,700 --> 00:25:04,589 And they took a path they decided it wasn't for them would also still be included in this group of e-cigarette users, 296 00:25:04,590 --> 00:25:07,620 which is again introducing quite a lot of variability. 297 00:25:07,620 --> 00:25:11,609 But the most to the two most important issues are just issues around design. 298 00:25:11,610 --> 00:25:15,750 And one of those is of course confounding. We're looking at observational studies. 299 00:25:16,200 --> 00:25:23,040 There are definitely some unexplored confounders here. And one of the reasons we know that these are going on is nicotine replacement therapy. 300 00:25:23,340 --> 00:25:27,360 It's been around for years. It's a first line treatment for smoking cessation. 301 00:25:27,600 --> 00:25:32,489 We have 113 randomised controlled trials with long term follow up that very clearly show 302 00:25:32,490 --> 00:25:36,600 with high certainty evidence that nicotine replacement therapy helps people quit smoking. 303 00:25:36,870 --> 00:25:42,299 But if you did the same study about nicotine replacement therapy and you surveyed smokers at baseline and said, 304 00:25:42,300 --> 00:25:45,690 are you using nicotine replacement therapy or not? And then you followed them up. 305 00:25:45,690 --> 00:25:53,340 Six months later, it would appear that nicotine replacement therapy was hindering their quit attempts, so fewer people using NRT would have quit. 306 00:25:53,610 --> 00:25:56,969 And the main driver for this is the level of addictiveness. 307 00:25:56,970 --> 00:26:01,500 So it's something that drives whether or not you use an R, T or e-cigarettes. 308 00:26:01,500 --> 00:26:06,569 And it's something that also makes it less likely for you to quit. So if you're not very addicted, you're someone who has a cigarette. 309 00:26:06,570 --> 00:26:12,330 Every once in a while you decide you're going to stop. You may well not try to switch to using it to use an e-cigarette. 310 00:26:12,540 --> 00:26:15,660 You may well not use NRT. You may find it very easy to quit smoking. 311 00:26:15,930 --> 00:26:21,030 If you are someone who has tried to quit smoking again and again and again, which most people are at this point, 312 00:26:21,480 --> 00:26:26,730 then you probably are going to try and use some product to help you, and you're also going to be less likely to quit if you are heavily addicted. 313 00:26:26,730 --> 00:26:33,030 So that is a very important confounder. And there are also some other confounders that we suspect are at play here. 314 00:26:33,720 --> 00:26:37,770 And then the other issue, which is very specific to this individual type of study, 315 00:26:39,000 --> 00:26:44,309 is that by their very definition, these studies are biased because if you imagine, 316 00:26:44,310 --> 00:26:47,670 let's say let's say we imagine a study that was conducted in 2016, 317 00:26:48,000 --> 00:26:53,490 and you imagine a group of people who in 2015 decided they were going to try to use an e-cigarette to quit smoking. 318 00:26:54,000 --> 00:26:58,440 All of those who use the e-cigarette to successfully quit smoking would not be 319 00:26:58,440 --> 00:27:01,950 included in our 2016 study because they would no longer be classed as smokers. 320 00:27:02,220 --> 00:27:05,940 So anyone who used them and it worked for them were ignoring them. 321 00:27:06,360 --> 00:27:12,750 The only people left in there are the people who are using an e-cigarette still smoking and haven't managed to switch completely to an e-cigarette. 322 00:27:13,050 --> 00:27:22,380 So we have a real issue there in terms of essentially only retaining treatment failures or people who have only just started to try and switch over. 323 00:27:22,710 --> 00:27:24,130 And so for all of those reasons, 324 00:27:24,150 --> 00:27:30,990 that's why we of course prefer randomised controlled trials and why that's what we focus on or look at cessation outcomes. 325 00:27:31,740 --> 00:27:39,900 Now it would be really nice to say, but this problem is pretty much just an issue because we don't have that many randomised controlled trials. 326 00:27:40,170 --> 00:27:42,660 If we had more than imprecision wouldn't be an issue. 327 00:27:42,840 --> 00:27:47,310 If we had more there would be much less valid reason for saying let's include observational studies. 328 00:27:47,880 --> 00:27:55,620 Unfortunately, it doesn't necessarily seem to be the case and what they've tried to defend themselves with here is that they did 329 00:27:55,620 --> 00:28:02,489 a sensitivity analysis to check that everything they did was fine and they did some kind of weird things here, 330 00:28:02,490 --> 00:28:03,780 which is probably worth talking about. 331 00:28:03,780 --> 00:28:09,630 It's probably also worth noting, for those of you who might be doing a systematic review, that what they didn't do was have a protocol. 332 00:28:09,930 --> 00:28:14,940 So there's no way for me to look back at that protocol and check that these were predefined sensitivity analysis. 333 00:28:15,330 --> 00:28:20,040 They seem weird to me because they did lots of sensitivity analysis, including this first one, 334 00:28:20,040 --> 00:28:22,829 my list, which is real world versus clinical and what they mean by that, 335 00:28:22,830 --> 00:28:29,940 it's observational versus randomised controlled trials and because they did lots of them, they controlled for multiple comparisons. 336 00:28:30,240 --> 00:28:34,650 And what that's essentially doing is it's making their P values quite a lot higher. 337 00:28:35,070 --> 00:28:39,479 And when you look at their table, which lists a very long list of sensitivity analysis, 338 00:28:39,480 --> 00:28:43,380 I've just taken out the main one that's relevant to us, which is the study type. 339 00:28:44,010 --> 00:28:48,060 We have a P value that's not at all statistically significant. It's 0.90. 340 00:28:48,480 --> 00:28:51,540 And yet when you look at the study types, it's a bit weird. 341 00:28:51,810 --> 00:28:59,640 So first of all, it seems kind of an underpowered sensitivity analysis if you're only taking out two studies from a meta analysis of 21. 342 00:29:00,090 --> 00:29:08,639 And secondly, as we would expect, we have quite estimates going in very different directions and we also have confidence intervals that don't overlap. 343 00:29:08,640 --> 00:29:14,460 So even though the P value here is not statistically significant, if I saw this as a sensitivity analysis, 344 00:29:14,700 --> 00:29:18,270 I don't think I'd find it reassuring, which is how they interpreted it. They said This is fine. 345 00:29:18,420 --> 00:29:22,950 It was totally appropriate to pull all the stuff. I would say that and think your confidence intervals don't overlap. 346 00:29:22,950 --> 00:29:27,750 Your point estimates going in completely the opposite direction. Maybe there's something more going on here. 347 00:29:28,950 --> 00:29:30,839 As I mentioned, and somewhat disappointingly, 348 00:29:30,840 --> 00:29:37,169 this is not just a problem for e-cigarettes and it's not just a problem because there aren't enough randomised controlled trials. 349 00:29:37,170 --> 00:29:43,829 So David Noonan, who's a colleague of ours based in the centre of evidence based medicine here in Oxford, and a colleague, 350 00:29:43,830 --> 00:29:51,389 Claudia Hack, did a review article that just came out early this year which looks at this issue outside of e-cigarettes. 351 00:29:51,390 --> 00:29:59,520 So it's looking at meta analyses which set out to answer the exact same clinical question in terms of their picos and why there might be different. 352 00:29:59,630 --> 00:30:05,660 Is between them. And what they did was they went out and they identified 24 match pairs. 353 00:30:05,960 --> 00:30:13,760 So 48 meta analysis where one of them was from a Cochrane review and one of them was from a non Cochrane review and they had the exact same picos, 354 00:30:13,760 --> 00:30:15,320 the exact same inclusion criteria. 355 00:30:15,590 --> 00:30:22,610 There was no reason you would think from reading them that they'd find something different, however, and only one pair were they the same. 356 00:30:22,850 --> 00:30:28,459 So that's a bit of a problem. In fact, it's a really, really big problem when we want to trust better analysis. 357 00:30:28,460 --> 00:30:32,300 And it turns out that everyone's saying they're doing the same thing and getting totally different results. 358 00:30:33,140 --> 00:30:33,830 On average, 359 00:30:33,830 --> 00:30:41,209 the non Cochrane Effect estimates were higher and in four of the pairs that was in the magnitude of at least two fold increase in the effect estimate, 360 00:30:41,210 --> 00:30:46,420 which is a pretty dramatic change for 14 pairs they issued. 361 00:30:46,940 --> 00:30:50,239 They looked at discrepancies in interpretation and the inclusion criteria. 362 00:30:50,240 --> 00:30:56,360 So even though they had the same inclusion criteria, I think this kind of spells out all the grey areas that even if we have a PCO, 363 00:30:56,750 --> 00:31:00,350 we can kind of work around or have differing opinions on. 364 00:31:00,560 --> 00:31:06,080 And I think this is why it's particularly important to nail down your protocol and exactly what you mean by your PCO early on. 365 00:31:06,770 --> 00:31:09,139 Another really big issue was the numerical data. 366 00:31:09,140 --> 00:31:15,860 So even if they agreed on the same studies, the same studies were in there, often there was not the same data in the analysis. 367 00:31:16,310 --> 00:31:20,450 So only two pairs agreed on the numerical data presented for the same studies. 368 00:31:20,450 --> 00:31:26,719 And what they did is they took a sample of 50% of the discordant pairs of the studies that didn't match up. 369 00:31:26,720 --> 00:31:33,810 So they looked at 45 of them out of 90, and in only 15 could they figure out why the data extracted was different? 370 00:31:33,830 --> 00:31:38,870 So in a minority, could they make sense of that? I think that's extremely worrying. 371 00:31:38,880 --> 00:31:43,670 Definitely points to a need for transparency, but also as a reader, what are you supposed to trust? 372 00:31:44,000 --> 00:31:49,920 In that case? They hypothesise that there are a number of reasons this is happening. 373 00:31:49,940 --> 00:31:55,879 Search strategies are of course one of them. Cochrane reviews tend to be quite extensive in their searching and one of the main things that 374 00:31:55,880 --> 00:32:00,310 can get missed in other reviews if they don't do thorough searches is the grey literature. 375 00:32:00,320 --> 00:32:04,160 So some of these Cochrane reviews were including unpublished studies that weren't showing up. 376 00:32:04,160 --> 00:32:12,140 In other reviews, which makes sense, this issue around the interpretation of the eligibility criteria was a reason, probably just some simple errors. 377 00:32:12,150 --> 00:32:15,830 We are humans and we are humans doing that analysis and we will make some errors. 378 00:32:16,280 --> 00:32:22,819 And those could come up in screening or in data extraction and then also probably some genuine differences in the extracted data. 379 00:32:22,820 --> 00:32:26,690 And a lot of the time what they speculate this might be is people going back to the authors. 380 00:32:27,020 --> 00:32:32,870 So if one review team thinks I can't actually quite make sense of this, if you do a lot of systematic review and I regret to tell you, 381 00:32:32,870 --> 00:32:37,880 there are many cases where you read the text and you read the table and the two things don't match up. 382 00:32:38,360 --> 00:32:41,179 And either you can say, I'm going to take the text or I take the table, 383 00:32:41,180 --> 00:32:43,880 or you can say I'm going to email these authors and figure out what's going on. 384 00:32:44,510 --> 00:32:48,820 And if you've emailed the authors and if the authors have gotten back to you, that is wonderful. 385 00:32:48,830 --> 00:32:52,130 That's the data that you use. But a lot of the time people aren't recording that. 386 00:32:52,400 --> 00:32:56,780 So it's like these reviews are being published with this data and that doesn't match up to anything in the published version. 387 00:32:57,170 --> 00:32:58,490 How do we know what's going on? 388 00:32:59,030 --> 00:33:06,649 And of course, I would argue there's a strong risk of bias here that comes into play both around what data we choose to use, 389 00:33:06,650 --> 00:33:08,540 around what studies we choose to include. 390 00:33:08,840 --> 00:33:14,360 I think we automatically think of financial conflicts of interest first, and of course some of those might be relevant, 391 00:33:14,630 --> 00:33:19,820 but there are also a lot of ideological conflicts of interest that come in here with e-cigarettes. 392 00:33:19,970 --> 00:33:25,129 Unfortunately, I'd say in about 80% of the papers I see, if you showed me the study authors, 393 00:33:25,130 --> 00:33:28,520 I could tell you what they found without reading the title or the abstract. 394 00:33:28,820 --> 00:33:34,160 And that's because it's almost all observational data. You can fiddle with confounders in one direction or another. 395 00:33:34,520 --> 00:33:39,860 There's no kind of standard guidelines for this, and therefore people end up finding things that fit their agenda. 396 00:33:40,370 --> 00:33:43,729 And I love tobacco control. It's the area I've worked in for a long time now. 397 00:33:43,730 --> 00:33:45,200 I'm super passionate about it. 398 00:33:46,220 --> 00:33:52,340 One of the reasons I'm passionate about it is that our research drives policy, but I think that's another reason why bias creeps in. 399 00:33:52,340 --> 00:33:56,150 Because if you have a strong policy agenda that's anti tobacco, 400 00:33:56,810 --> 00:34:01,639 you might try and be more inclined to do research that is going to support that agenda. 401 00:34:01,640 --> 00:34:08,030 And that's coming from a place that's not financially conflicted, that's coming from a moral place of saying, I've worked in this field for decades. 402 00:34:08,030 --> 00:34:11,180 I've seen all the harm that's done. I want to try and stop that. 403 00:34:11,480 --> 00:34:13,520 But it is an issue and it's one we have to be aware of. 404 00:34:13,520 --> 00:34:17,600 And of course, another issue here is that a lot of people who work in this field also do the primary research, 405 00:34:17,840 --> 00:34:22,250 which can be great because it means they know the area really well, they know the ins and outs of the studies. 406 00:34:22,670 --> 00:34:27,889 But it can also be an issue if what they're finding are studies that contradict their findings. 407 00:34:27,890 --> 00:34:29,600 So if they've come out with this big paper, 408 00:34:29,930 --> 00:34:35,150 a lot of the e-cigarette papers are published in very big journals now because it's a topical issue and then they do the 409 00:34:35,160 --> 00:34:39,770 systematic review and for other papers found something different and their meta analysis isn't agreeing with our study. 410 00:34:40,130 --> 00:34:44,330 That's going to be a bit of an issue for them, too. And all of these things are places where, of course, 411 00:34:44,330 --> 00:34:50,870 critical appraisal comes into play and something we want to be thinking about when we're looking at these reviews as readers. 412 00:34:52,130 --> 00:34:59,060 So what can we do? I mean, if you have time, the first thing I do is investigate and I do this in that. 413 00:34:59,430 --> 00:35:03,180 Two ways. One is actively search for other media analyses. 414 00:35:03,180 --> 00:35:07,890 So I do this whenever we do a Cochrane review. I try and do a search for all of the other analysis out there. 415 00:35:08,190 --> 00:35:14,459 And then because I have time and it's my job, I can actually spend time looking at them quite closely and trying to figure out the reasons. 416 00:35:14,460 --> 00:35:17,100 And if I can't figure out the reason, it drives me crazy. 417 00:35:17,100 --> 00:35:20,490 And I email the authors because I think, am I doing something wrong or are they doing something wrong? 418 00:35:20,850 --> 00:35:25,919 What's happening here? But as readers, we can definitely do that. And as readers we can also critically appraise. 419 00:35:25,920 --> 00:35:32,400 And I think if you have ten reviews and nine of them are saying vaguely the same thing, I would be inclined to trust those nine. 420 00:35:32,580 --> 00:35:36,000 What do you do when you have two reviews and they're saying something that's diametrically opposed? 421 00:35:36,240 --> 00:35:40,680 I think that's what we need to bring in. Our critical appraisal skills and more understanding of analysis. 422 00:35:40,680 --> 00:35:46,319 And where things can go wrong is really important as authors and I won't say it enough, 423 00:35:46,320 --> 00:35:50,220 I think it is incredibly important that we pre-register our protocols. 424 00:35:50,430 --> 00:35:56,549 That's partly so we can be transparent, but it's also partly because if I'm doing a review, I will search for protocols. 425 00:35:56,550 --> 00:35:59,250 I'll see if someone else is doing what I'm setting out to do. 426 00:35:59,820 --> 00:36:04,920 And if they are, I will ask myself some serious questions about whether or not it's a good use of my time, 427 00:36:05,190 --> 00:36:11,909 of the public's money to be doing the same review. I might get in touch with those authors and they might have published their protocols in journals. 428 00:36:11,910 --> 00:36:15,810 Or often what they do is they publish them on a register called Prospero. 429 00:36:16,080 --> 00:36:21,450 You can look at that. What I found from looking at that is often people say they're doing things and then you check in with them. 430 00:36:21,450 --> 00:36:25,320 They're like, Oh, no, I didn't actually do that, which is fine. I view that as a green light. I'm going to go ahead and do it. 431 00:36:25,560 --> 00:36:29,520 But if they say and this did happen to me once, yep, I'm doing exactly what you set out to do. 432 00:36:29,520 --> 00:36:33,240 I was a bit gutted because I'd been like, This is a clear area where we need a systematic review. 433 00:36:33,480 --> 00:36:36,960 This is great with part of my Dphil research. No one's done this before. 434 00:36:37,260 --> 00:36:43,110 Did a quick photo search. Someone else is doing exactly the same thing. I changed my review question because I contacted them. 435 00:36:43,290 --> 00:36:46,170 I knew they were going to do a good job. It wasn't the best thing for me to do. 436 00:36:46,770 --> 00:36:51,600 So people should really be checking these and thinking about research, wastage and duplication. 437 00:36:51,990 --> 00:36:56,010 Of course, another thing you can do, and I would say this because I'm part of Cochrane, is update your review. 438 00:36:56,370 --> 00:37:01,649 So every Cochrane review I do has a section on the on the other reviews published where I'll 439 00:37:01,650 --> 00:37:05,970 go through and I'll pick them apart and I'll see exactly what we might agree or disagree. 440 00:37:06,480 --> 00:37:13,200 Transparency, I think, is absolutely key. And of course, as a review author, you walk a fine line because if you report absolutely everything you do, 441 00:37:13,620 --> 00:37:16,770 your article is going to be longer than a journal will take. 442 00:37:17,040 --> 00:37:24,320 But that's why we have supplementary appendices. That's why we have online repositories where we can be very transparent about exactly what we did. 443 00:37:24,630 --> 00:37:27,810 And I think adhering to reporting guidelines really helps there. 444 00:37:28,500 --> 00:37:34,410 And finally, as authors, I think one of the things we might do when we see a meta analysis that disagree with each other is communicate, 445 00:37:34,710 --> 00:37:37,950 particularly if it's an area like this where we know people are actually looking 446 00:37:37,950 --> 00:37:41,910 at these merit analyses and making health care decisions on the back of them. 447 00:37:42,600 --> 00:37:50,520 And when it came to e-cigarettes, I was feeling pretty reticent about this particular issue because I thought, we don't know very much about them. 448 00:37:50,550 --> 00:37:55,500 It could be that in ten years we find that they're causing some horrible thing that none of us could have guessed. 449 00:37:55,500 --> 00:38:01,049 And Oh my God, out there saying, Yeah, okay. On the best available evidence, people should think about switching. 450 00:38:01,050 --> 00:38:07,380 But I have ended up staking my claim on that one because I thought about it and I thought, actually, 451 00:38:07,500 --> 00:38:10,710 once you've done a review, you probably are one of the people who knows the most about it. 452 00:38:11,010 --> 00:38:15,090 I actually don't have any vested interest in whether or not e-cigarettes work or not. 453 00:38:15,260 --> 00:38:21,450 I think it helps that I'm both American and British, so I kind of can sit firmly on the fence in that regard, 454 00:38:22,530 --> 00:38:24,299 and I would want the best available evidence out there. 455 00:38:24,300 --> 00:38:30,060 And I have family members who smoke and with them I've encouraged them to switch to e-cigarettes. 456 00:38:30,600 --> 00:38:35,159 And one of them said at a family dinner a few years ago, he said, you know, I debated switching, 457 00:38:35,160 --> 00:38:38,400 but you scientists can't agree, so I'm just not going to want to keep on smoking. 458 00:38:38,570 --> 00:38:41,280 I thought, this is terrible. We have to say more. You know, this is awful. 459 00:38:41,280 --> 00:38:46,550 If I'm feeling like I can tell the people I care about that they can switch, why can't I say it a little bit more loudly? 460 00:38:46,800 --> 00:38:52,230 So on a Light-hearted note, I wanted to talk about how I said that more loudly and to give a little bit of background. 461 00:38:52,230 --> 00:39:00,450 Back in 2013, when we first started our review of e-cigarettes, 7% of the British public thought that e-cigarettes were more or equally harmful. 462 00:39:00,450 --> 00:39:09,570 That's this orange bar here. Since then, all of the evidence that came out has very, very, very much supported e-cigarettes being safer. 463 00:39:09,720 --> 00:39:14,550 Right? So all of the studies we have of biomarkers of acute effects, of long term effects, 464 00:39:14,820 --> 00:39:19,590 all of them suggested that e-cigarettes were not completely safe but were much safer than smoking. 465 00:39:19,590 --> 00:39:22,739 And yet that orange bar is going in exactly the wrong direction. 466 00:39:22,740 --> 00:39:26,760 So from 7% in 2013 to 26% in 2019, 467 00:39:26,760 --> 00:39:33,390 of the British public who think e-cigarettes are more or equally harmful then literally the most dangerous legal product we have. 468 00:39:33,600 --> 00:39:37,260 Anything else that had the safety profile of an e-cigarette would not be legal. 469 00:39:37,560 --> 00:39:43,590 So they are uniquely deadly. They kill one in two people who use that product as they are intended in the States. 470 00:39:43,590 --> 00:39:47,249 Guns obviously are legal too, but here we're a little bit more sensible. 471 00:39:47,250 --> 00:39:52,110 So we just have the cigarette problem in terms of things that kill people that are easily available to them. 472 00:39:52,470 --> 00:39:58,680 So I've done lots of things. I've talked to journalists, I've written blogs about probably the most ridiculous thing I did. 473 00:39:59,210 --> 00:40:08,880 Was this? This is performed throughout the streets of Oxford, covered down deep in hills. 474 00:40:09,300 --> 00:40:13,820 There is a video to but you don't need to see. Don't want to graze if it's safe. 475 00:40:13,860 --> 00:40:17,280 Brian Not so you might as well smoke anyway. 476 00:40:18,480 --> 00:40:24,360 What your mate needs is a Cochrane review. All my friends have been checked at least twice. 477 00:40:24,930 --> 00:40:28,360 They find there's a lot that the experts agree on. 478 00:40:28,560 --> 00:40:33,020 I did get to. It's not a very good video, to be honest. 479 00:40:33,020 --> 00:40:43,040 So you're not missing too much. Now a cigarette burns rather horribly, wreaking all sorts of snakes. 480 00:40:43,040 --> 00:40:46,360 Cyanide. Which is why it will. 481 00:40:50,090 --> 00:40:54,410 This may look like personal preference so gonna say. 482 00:41:01,740 --> 00:41:07,020 Enforcing sections of. That's what the experts. 483 00:41:20,900 --> 00:41:26,360 And you leave this little bit of a concern. Checking evidence from the two killers, Ken. 484 00:41:30,630 --> 00:41:40,920 And thanks to many. Three have been around 1700 thousand. 485 00:41:41,940 --> 00:41:46,020 Participants in this training say that they are. 486 00:41:51,670 --> 00:41:54,610 That's out. Thank you, guys. And I'm here for questions.