1 00:00:00,210 --> 00:00:04,590 Good evening. Thank you very, very much for this evening's talk. 2 00:00:04,620 --> 00:00:12,000 I appreciate. It's a lovely sunny day outside, but you brought the cold rain and we're in for a treat. 3 00:00:12,360 --> 00:00:23,270 We've got more to love. Yes. Well, talking about fake surgeries and pills and the challenges facing the trial. 4 00:00:23,320 --> 00:00:26,850 Trial trials, they hand over to you. Thank you. 5 00:00:27,990 --> 00:00:35,100 Good evening. So I I'll be talking about control for for bias study design. 6 00:00:36,210 --> 00:00:42,300 So I'm a medic. And also I did the Dphil with Irene Tracey on pain imaging. 7 00:00:42,300 --> 00:00:48,510 So I've spent my dphil and my post-doc is trying to understand why certain 8 00:00:48,510 --> 00:00:54,090 people respond to pain and what happens in their brains after pain treatment. 9 00:00:54,570 --> 00:00:59,460 So you may ask what pain has to do with trial design. 10 00:01:00,030 --> 00:01:01,490 The thing is that pain is very common. 11 00:01:01,770 --> 00:01:10,560 So pain is the main reason patients seek medical attention, and pain accounts for about 22% of all GP consultations. 12 00:01:11,430 --> 00:01:18,480 It's estimated that during the life span, one in four adults in the UK will develop chronic pain condition. 13 00:01:18,810 --> 00:01:24,000 So it's highly likely that in one of your trials you'll have pain as an outcome. 14 00:01:24,690 --> 00:01:29,130 So what's pain? So according to the International Association for the Study of Pain. 15 00:01:29,160 --> 00:01:36,420 Pain is an unpleasant sensory and emotional experience associated with actual or potential tissue damage or described in terms of such damage. 16 00:01:36,900 --> 00:01:40,110 What's important about this, the finish physician is the pain is an experience. 17 00:01:40,350 --> 00:01:46,670 It's not just a sensation. It's a subjective experience that's very complex and multidimensional. 18 00:01:46,680 --> 00:01:50,390 So it's got the sensory aspect, emotional aspect, cognitive aspect. 19 00:01:50,730 --> 00:01:53,880 It's influenced by your past memories, your experience, 20 00:01:54,180 --> 00:02:02,190 but also by social and cultural aspect of your brother, but that you have to man up buried, no pain, no gain. 21 00:02:02,850 --> 00:02:09,990 You will behave differently. That people who are brought up and be allowed to express their pain, their pain and seek help. 22 00:02:11,490 --> 00:02:15,840 But in everyday practice and research, 23 00:02:15,990 --> 00:02:26,460 we usually just you say that pain is whatever the experiencing person says it is and it exists whether they say it does. 24 00:02:26,910 --> 00:02:30,660 So we cannot experience other people's pain. 25 00:02:32,070 --> 00:02:35,070 If this guy is in pain, I will not know it. 26 00:02:35,310 --> 00:02:41,340 I cannot measure it in any objective way. The same way I will be able to measure his blood pressure. 27 00:02:42,150 --> 00:02:51,750 The only way for me to ask him whether he is really in pain, whether he's trying to smile, but he's really suffering inside and cannot play his act. 28 00:02:51,750 --> 00:02:58,080 What is to ask? I can use questionnaires, but usually what we do, we use pain scales. 29 00:02:58,110 --> 00:03:03,239 So those are the two most popular pain skills. We've got a visual analogue scale, a numerical rating scale. 30 00:03:03,240 --> 00:03:11,069 So what we are what we do, we ask a patient or a researcher volunteer to rate this complex, 31 00:03:11,070 --> 00:03:16,530 subjective experience on this unique dimensional scale with relative anchors. 32 00:03:16,980 --> 00:03:24,090 But that's what we have and that's what we use. So in the trial, this will be our data. 33 00:03:24,180 --> 00:03:30,809 So we've got a baseline. We ask participants for pain ratings, then we give them treatment and we've got to follow up visits. 34 00:03:30,810 --> 00:03:34,650 And if the treatment works, the pain will go down. Awesome. 35 00:03:35,910 --> 00:03:39,540 If we ask them every day, we may see something like that. 36 00:03:39,540 --> 00:03:47,820 So that's a mean change in their pain score from zero down to improved by two points. 37 00:03:49,140 --> 00:03:56,160 But if we have a group of patients who received a dummy pill, a placebo or placebo pill, we may see something like that. 38 00:03:57,310 --> 00:04:03,850 So despite the fact that those people got just a sugar pill, their pain ratings went down as well. 39 00:04:05,410 --> 00:04:11,049 This is because what we observe as an improvement after treatment is not just 40 00:04:11,050 --> 00:04:14,950 a physiological effect related to the pharmacological properties of the drug. 41 00:04:15,640 --> 00:04:19,540 Some of it is just fluctuations of the symptoms. 42 00:04:19,660 --> 00:04:24,160 People have good days. People have bad days. Some of it is natural history of the disease. 43 00:04:24,190 --> 00:04:31,300 So, for example, lower back pain it will improve upon after a couple of days or weeks for some people. 44 00:04:32,020 --> 00:04:37,600 Some of it is just regression to the mean. So we've recruited our patients where they are really paid are really suffering. 45 00:04:37,900 --> 00:04:45,880 And then when we ask them of the follow up, the next rating may be kind of closer to the actual mean for this person. 46 00:04:46,360 --> 00:04:53,159 Some of it is going prevention. So we want our patients to only take our medication bath. 47 00:04:53,160 --> 00:05:00,200 They may be naughty, they may take paracetamol when they have a bad night, or they may change something about their lifestyle. 48 00:05:00,400 --> 00:05:06,250 Buy a new mattress, go to the gym more often because they're in the trial and what happens in the trial, 49 00:05:06,580 --> 00:05:12,370 behaviour of our participants will change because they know they are being observed and they know they're going to be assessed. 50 00:05:13,450 --> 00:05:16,359 Some of it is due to patient doctor interactions. 51 00:05:16,360 --> 00:05:26,860 So we know there is research that if doctor is really nice, shows empathy, gives a clear diagnosis, nice clear prognosis and a clear treatment plan. 52 00:05:27,310 --> 00:05:33,190 Patients are more likely to improve. Some of it may be just report bias because if the doctor is really nice. 53 00:05:33,460 --> 00:05:37,390 A patient may be slightly less willing to say that the treatment actually doesn't work. 54 00:05:37,840 --> 00:05:43,120 Patient may not want to upset the doctor. And some of it is a placebo effect. 55 00:05:43,540 --> 00:05:47,739 So the thing about placebo effect is that there isn't a very good definition and it's 56 00:05:47,740 --> 00:05:53,860 literally what's not specific to the treatment and what cannot be explained by any of those. 57 00:05:54,670 --> 00:06:02,860 So definition by definition, it's an improvement in clinical symptoms or patients well being in response to placebo manipulations such as 58 00:06:02,860 --> 00:06:09,580 in active substance of a procedure that only simulates an active therapy but itself has no specific effects. 59 00:06:10,090 --> 00:06:14,020 But the thing is, any pill or therapy is actually necessary. 60 00:06:14,620 --> 00:06:20,770 A good, positive consultation may also result in what we will call a placebo effect. 61 00:06:21,010 --> 00:06:24,830 People just don't want to say that. Oh, apples of consultation because it's a placebo effect. 62 00:06:24,850 --> 00:06:31,540 Well, positive consultation as part of a good patient doctor relationship sometimes. 63 00:06:31,540 --> 00:06:36,519 Also, just diagnostic tests result in improvement of symptoms. 64 00:06:36,520 --> 00:06:42,700 So people actually report that they feel better after an EKG or an MRI scan. 65 00:06:43,270 --> 00:06:52,810 So placebo effect is just this improvement that we cannot really explain by the actual specific effect of the treatment. 66 00:06:55,150 --> 00:06:58,000 So this is an experiment done by my colleagues in the previous group. 67 00:06:58,000 --> 00:07:07,060 So they've recruited healthy, pain free volunteers and they said, we are going to use this mode to give you painful stimulus, 68 00:07:07,330 --> 00:07:15,400 and we're also going to give you a very potent opioid and please rein the pain while you are on the drug. 69 00:07:15,760 --> 00:07:19,500 So first there was a baseline, so they gave them the stimulus. 70 00:07:19,580 --> 00:07:23,550 They could you please read the pain for us? And this is a visual analogue scale. 71 00:07:23,560 --> 00:07:28,990 This one is from not two 100 and the participants rated it as 65. 72 00:07:30,160 --> 00:07:37,240 And then without telling the participants, the drug infusion started through the IV line. 73 00:07:37,600 --> 00:07:46,810 But the participants did know and again, they received a painful stimulus and they were asked to give a pain rating and their pain rating went down. 74 00:07:47,170 --> 00:07:58,270 So what they have here, it's a hidden administration of an energy zinc, which we kind of assumed that this is just a pure effect of the drug. 75 00:07:59,080 --> 00:08:06,370 What happened next was a researcher walked into the scanner, said, well, now we are actually starting the experiment. 76 00:08:06,610 --> 00:08:13,209 We are going to turn on the drug. They lied because the drug was already on and this is what happened to the pain rating. 77 00:08:13,210 --> 00:08:18,670 So they went down even more. Nothing has changed except for the information given to the patient. 78 00:08:19,570 --> 00:08:25,270 But because my colleagues were really cheeky then they said, Well, now we are finishing the experiment. 79 00:08:25,270 --> 00:08:28,210 The drug is off. Could you please rate the pain again? 80 00:08:28,840 --> 00:08:37,060 So the participants were expecting no help from the opioids and they were just left there with their pain on their own. 81 00:08:37,570 --> 00:08:46,209 And this is the rating they're gave. So what was really interesting about this experiment is that being told that there is no medication, 82 00:08:46,210 --> 00:08:51,880 not only reversed the positive expectation, but it also reverse some of the effect of the drug. 83 00:08:52,480 --> 00:08:55,750 And this is the placebo effect that we can observe. 84 00:08:57,820 --> 00:09:09,370 So what we have to remember when we use subjective outcomes like pain is that patient patients are really affected by what they believe or well, 85 00:09:09,370 --> 00:09:15,640 they're told about the treatment, and patients believe that drugs are invasive. 86 00:09:15,790 --> 00:09:20,920 So, for example, injection or I.V. drip. Drugs that are new. 87 00:09:20,950 --> 00:09:27,580 So that's why they prefer the novel drug or the experimental drug or medication does expensive. 88 00:09:27,910 --> 00:09:29,740 That's why they, like, run the drugs. 89 00:09:30,010 --> 00:09:39,940 They believe that those types of treatment are more effective and they are kind of biased to give better response after those types of drugs. 90 00:09:42,190 --> 00:09:52,480 The funny thing is that the power of the pill in our culture is so strong that people respond even if they know that this is a placebo. 91 00:09:53,320 --> 00:10:01,750 Because in our culture, we go to a doctor, we get the diagnosis, we get the prognosis and walk out with a prescription. 92 00:10:02,320 --> 00:10:05,560 This is part of the ritual in the Greek culture. 93 00:10:05,800 --> 00:10:11,170 They make gold, a bottle of herbal remedy and the prayer to read. 94 00:10:11,650 --> 00:10:17,710 There is actually a snippet in a Greek text saying that you have to use this herb and say the prayer. 95 00:10:17,860 --> 00:10:24,940 Without the prayer, it will not work. So there is this cultural set up that influences what patients experience. 96 00:10:25,330 --> 00:10:30,730 And this is a trial done by KAPTCHUK in patients with irritable bowel syndrome. 97 00:10:31,210 --> 00:10:37,360 So one group got no treatment. They just had interaction with the clinicians and the research stuff. 98 00:10:37,750 --> 00:10:41,710 And the other group received placebo in an open label manner. 99 00:10:42,250 --> 00:10:51,190 This is placebo. This is a dummy pill. It may have some effect through the mind body interaction, but it has no pharmacological effect. 100 00:10:51,550 --> 00:10:58,240 And still, patients improved much better when they were given placebo than when they were receiving no actual treatment. 101 00:10:59,720 --> 00:11:02,770 And now let's get to the trial design. 102 00:11:02,780 --> 00:11:11,320 So the reason we do a trial is because we want to test efficacy and safety of a treatment in a controlled manner. 103 00:11:11,800 --> 00:11:19,300 So usually what we do, we know a nice cohort of patients here, very handsome and young. 104 00:11:19,570 --> 00:11:24,500 Not always, though. So what would we do? We randomise them into two groups to balance that out. 105 00:11:24,520 --> 00:11:33,790 So we've got men and women different eight different ethnicity, two to kind of balance out everything that may potentially bias our results. 106 00:11:34,300 --> 00:11:40,060 And then we can allocate them to treatment. So we've got interactions with the doctor and the pill and for example, no treatment. 107 00:11:40,480 --> 00:11:46,120 So this design will control for regression to the mean natural history of disease fluctuation, 108 00:11:46,630 --> 00:11:50,590 effect of being in the trial, so effect of being observe. 109 00:11:51,070 --> 00:12:00,650 But as we said, patients want the novel expensive treatment and this group will not get it, which will bias the results. 110 00:12:00,670 --> 00:12:03,730 So what we can do, we can give them placebo. 111 00:12:03,880 --> 00:12:07,700 So the power of pill, it's present in both halves. 112 00:12:08,140 --> 00:12:13,060 But this group will know that actually they are getting a dummy pill and this group is still getting 113 00:12:13,060 --> 00:12:18,580 this novel expensive new drug that will potentially completely change the treatment of this disease. 114 00:12:19,240 --> 00:12:26,860 So that's why we need blinding. So usually what happens, the drug is usually put in external capsule or coated, 115 00:12:26,860 --> 00:12:34,270 so so that it looks the same regardless whether inside is the active drug or a placebo. 116 00:12:34,480 --> 00:12:43,540 What we can do to improve the design even more is blind doctors as well, and we can blind people giving the drug. 117 00:12:43,960 --> 00:12:56,410 So by their behaviour or information they give or the way they give information they do not reveal the allocation to for the patients. 118 00:12:56,710 --> 00:13:04,720 Or we can also blind people who are assessing the outcomes because we know that by the way, people ask questions, oh, by the way they behave. 119 00:13:05,080 --> 00:13:10,740 They may bias patients responses. And also there is so-called Pygmalion effect. 120 00:13:10,750 --> 00:13:16,390 So people who are expecting a certain outcome are more likely to actually see it. 121 00:13:16,930 --> 00:13:26,290 So that's why blinding both patients and assessors and preferentially everybody interacting with a patient throughout the trial is so important. 122 00:13:27,490 --> 00:13:34,000 So we can kind of expand this design even more so we can test new treatable but standard treatment so it doesn't have to be placebo. 123 00:13:34,030 --> 00:13:40,750 We can test with whether one drug is better than another, whether it's safer than a novel, or we can use all three. 124 00:13:41,230 --> 00:13:45,309 So we can compare a new drug to all treatment and do placebo. 125 00:13:45,310 --> 00:13:48,670 And we make the pills identical. 126 00:13:48,970 --> 00:13:52,710 Patients will not know which treatment are they are receiving. 127 00:13:52,720 --> 00:13:56,530 And this is the trial I run as my. 128 00:13:57,050 --> 00:14:03,770 So this was a trial in which Pregabalin pretended to be the new novel Treatment for Pain. 129 00:14:04,190 --> 00:14:11,030 Tramadol was the standard treatment, so the less preferred, potentially worse. 130 00:14:11,330 --> 00:14:21,820 And then there was also a placebo drug. So this is a placebo controlled crossover three period randomised trials with only one group. 131 00:14:21,830 --> 00:14:26,150 So every participant, every patient was their own control. 132 00:14:26,570 --> 00:14:34,219 So everyone received placebo, pregabalin and Tramadol and there was a week of washout between. 133 00:14:34,220 --> 00:14:39,890 So there they came down their normal medication, they received placebo. 134 00:14:40,160 --> 00:14:43,670 Then there were some tablets with increasing dose of the drug. 135 00:14:43,880 --> 00:14:48,920 When the dose was stable, we assessed them, then we gave them a new blister medication. 136 00:14:49,160 --> 00:14:52,399 Then there was first there was placebo to bring down the dose of whatever drug 137 00:14:52,400 --> 00:14:56,150 they were on and then a new medication will start and that was repeated. 138 00:14:57,140 --> 00:15:07,610 So this is a brilliant design because you don't need a control group, you just have to recruit one group, but it's not always feasible. 139 00:15:07,610 --> 00:15:09,260 So for example, with your treatment or surgery, 140 00:15:09,350 --> 00:15:17,090 you cannot really randomise someone to surgery and then say, Well, now we'll cross over to physiotherapy. 141 00:15:17,300 --> 00:15:24,230 It doesn't really work like that. You cannot really reverse surgery. There is such a thing as placebo surgery. 142 00:15:24,230 --> 00:15:27,410 It really exists and this is another trial. 143 00:15:27,410 --> 00:15:32,900 So this is my second post. So this is a three arm surgical trial. 144 00:15:33,380 --> 00:15:35,750 All saw the Cromwell pain. 145 00:15:35,750 --> 00:15:44,480 So some people with age or because of the job they do for example they pain ceiling for a living they develop shoulder pain. 146 00:15:45,440 --> 00:15:54,290 And there was a very common surgery that involves a keyhole access into the joint and the removal, 147 00:15:54,300 --> 00:15:59,090 a bit of a bit of a bone to create more space for the shoulder tendons. 148 00:15:59,600 --> 00:16:03,740 And this is believed to cure the pain. 149 00:16:05,270 --> 00:16:15,980 So as a placebo control, a placebo surgery, those the patients in those in the control group only underwent the diagnostic arthroscopy. 150 00:16:16,280 --> 00:16:22,040 So the scope was put into the joints. A surgeon would take a look but wouldn't do anything. 151 00:16:22,760 --> 00:16:29,900 But a patient could not tell whether actually the anatomy was change or not, so whether they were in the surgical arm or the control arm, 152 00:16:30,380 --> 00:16:35,150 and there was also no treatment arm to control for the natural history of the disease. 153 00:16:35,420 --> 00:16:39,920 And as you can see, we could blind between the surgical or placebo arm, 154 00:16:40,490 --> 00:16:48,649 but people in the non interventional arm knew that they are not getting any surgery and those are the results of the trial. 155 00:16:48,650 --> 00:16:56,000 So this is on the Y axis, it's ox Oxford shoulder score, which is a composite score of pain and function. 156 00:16:56,210 --> 00:17:05,480 And the higher the score, the better the outcome. So you can see that at six, six months, surgery and placebo were better than no intervention. 157 00:17:05,510 --> 00:17:15,200 And at 12 months there was no difference between surgery and placebo and people in the non intervention group had better, worse outcomes. 158 00:17:15,440 --> 00:17:19,100 So we can interpret it that surgery is not better than placebo. 159 00:17:19,400 --> 00:17:22,850 Bob The intervention still seems better than no treatment. 160 00:17:23,570 --> 00:17:31,729 Or if we actually assume that people who did not receive a surgery never met those and some 161 00:17:31,730 --> 00:17:39,470 surgeons never went to the surgical theatre with the machines that go B felt like they missed out. 162 00:17:40,160 --> 00:17:44,450 We could also interpret that as a sea effect and just simply report bias. 163 00:17:45,800 --> 00:17:50,850 But this is a limitation of the trial trials. 164 00:17:50,880 --> 00:17:53,450 Surgical trials with a placebo arm are very rare. 165 00:17:54,050 --> 00:18:05,780 Surgical recipes are about 15% of all are published in the journals, and only a fraction of them actually uses a placebo control. 166 00:18:06,920 --> 00:18:13,190 So many people, when you mentioned placebo control, the thing that is absolutely ridiculous because when they think about surgery, 167 00:18:13,190 --> 00:18:22,520 they think about something like that, something really invasive, performed to save life and absolutely dumb as a last resort. 168 00:18:23,270 --> 00:18:30,800 But this was in the 18th century. Nowadays, many surgical procedures look like that. 169 00:18:31,010 --> 00:18:37,100 They are minimally invasive. We've got antiseptics. We've done it, we've got analgesia, we've got antibiotics. 170 00:18:37,100 --> 00:18:45,350 Surgery is less dangerous. The force performed more often and the indications for surgery have changed. 171 00:18:45,740 --> 00:18:56,270 So now instead of doing surgery just to save lives, we perform surgery to improve function, improve quality of life to help people. 172 00:18:56,770 --> 00:19:00,640 Who experience pay. So, for example, all the pharmacological treatment fail. 173 00:19:00,910 --> 00:19:05,920 So patients are offered surgery. Therefore, outcomes have changed. 174 00:19:05,920 --> 00:19:11,340 So very often we use pain in function and quality of life as an outcome of surgical procedures. 175 00:19:11,350 --> 00:19:19,570 So that's why we may need placebo control in a trial that assesses safety and efficacy of surgery. 176 00:19:20,230 --> 00:19:26,300 But when we discuss placebo control, people ask, Well, but what's the harm to benefit ratio can even do it? 177 00:19:26,320 --> 00:19:32,560 Is it feasible? Because it's not just the pill. You actually have to pretend that you are performing a surgery. 178 00:19:33,610 --> 00:19:38,140 Is there even a placebo effect? Do we really need a control? 179 00:19:39,040 --> 00:19:43,860 And finally, would surgeons be willing to do it? Would they approve of placebo control? 180 00:19:43,870 --> 00:19:48,460 And is it even ethical? So what we've done, that's what we've done. 181 00:19:48,850 --> 00:19:58,300 So we, we've reviewed all published randomised controlled trials on surgical procedures that have a placebo control. 182 00:19:59,830 --> 00:20:02,350 There were no the strict restrictions on conditions. 183 00:20:03,580 --> 00:20:12,970 An intervention had to be invasive, so it had to be a procedure that was believed to be therapeutic by nature of changing anatomy. 184 00:20:13,390 --> 00:20:23,500 So injecting something to deliver a drug without performing surgery to insert a stimulator or modulator was out. 185 00:20:24,220 --> 00:20:29,980 We had we needed a change of anatomy to be the crucial therapeutic element. 186 00:20:31,060 --> 00:20:36,370 A comparator was called either sham or placebo or imitation. 187 00:20:36,640 --> 00:20:40,870 But the key point was that the crucial therapeutic element was omitted. 188 00:20:41,230 --> 00:20:46,209 There were no restrictions on the outcomes. The study had to be a randomised controlled trials. 189 00:20:46,210 --> 00:20:54,040 We searched the Medline and based central but also clinical trials to the Gulf to look for recently completed trials. 190 00:20:54,460 --> 00:21:00,460 And then we looked into references of all D'identifier trials just in case we missed something. 191 00:21:01,420 --> 00:21:10,090 And we searched those databases since the beginning, till November 2013, and then we run the update two years later. 192 00:21:10,630 --> 00:21:21,910 So that's what we found. So most of the interventions trialled in that design were minimally invasive, so nobody really trialled an open surgery. 193 00:21:22,150 --> 00:21:27,729 The most invasive ones were the seminal trials on Parkinson's disease, 194 00:21:27,730 --> 00:21:36,640 in which a hole was drilled in some of the skull to insert foetal cells into the basal ganglia. 195 00:21:37,930 --> 00:21:48,610 Most of those trials used other medications, so there was either L-dopa in the Parkinson's trials, all, for example, if the outcome was pain. 196 00:21:48,610 --> 00:21:55,509 Many patients were given a rescue medication to begin with. So it's not true that people were completely left without any help. 197 00:21:55,510 --> 00:22:01,239 They just received the placebo surgery. Conditions were mostly non-life-threatening. 198 00:22:01,240 --> 00:22:13,060 So osteoarthritis, obesity. There were some trials on bleeding as a fragile viruses performed in the eighties when endoscopy was introduced. 199 00:22:13,600 --> 00:22:19,320 But since then, most of the trials were among life threatening conditions. 200 00:22:19,330 --> 00:22:21,880 Outcomes are mostly subjective pain, function, quality of life. 201 00:22:22,150 --> 00:22:32,020 Those trials were mainly small, with fewer than 100 patients, and the publication spanned from the late fifties to 2015, 202 00:22:32,170 --> 00:22:39,100 with a big spike after 2000, which may be related to popularity of the minimally invasive surgery. 203 00:22:40,480 --> 00:22:42,220 So are those trials risky? 204 00:22:43,390 --> 00:22:55,030 Generally, none of those trials reported any serious adverse events reported to anaesthesia, which was the biggest concerns in all the ethical papers. 205 00:22:55,240 --> 00:23:00,340 So there were many papers discussing those potential risks of placebo controlled surgical trials. 206 00:23:01,570 --> 00:23:04,930 43% of the review trials reported no serious adverse events. 207 00:23:05,800 --> 00:23:12,250 Six Never mentioned serious adverse events. But this is a problem with the reporting of trials, especially surgical trials. 208 00:23:12,700 --> 00:23:24,730 In 51% there were serious adverse events, but in two thirds of them there were in both study arms and in one six there were only in the surgical arm. 209 00:23:24,880 --> 00:23:36,280 Some of them didn't say in which arm they occurred in general, placebo arm, what was safer and associated with fewer and less severe adverse events. 210 00:23:37,030 --> 00:23:41,589 Some of some of those events were directly associated with the crucial surgical element. 211 00:23:41,590 --> 00:23:45,280 So for example, who are patients with endoscopy and ablation? 212 00:23:45,280 --> 00:23:55,660 So there was a change to the wall of the gut and in some of the trials there was a perforation caused by the ablation. 213 00:23:55,930 --> 00:24:02,870 So this is. A direct link between the crucial surgical element and the adverse event. 214 00:24:03,910 --> 00:24:08,290 In many cases, the adverse events were associated with the severity of the condition itself. 215 00:24:08,470 --> 00:24:13,930 So, for example, patients were admitted to hospital because of asthma, fog baths. 216 00:24:13,960 --> 00:24:20,440 All the patients in the trial had severe asthma, so some of them might have been admitted to the hospital anyway. 217 00:24:20,920 --> 00:24:24,870 So that's why it's kind of difficult to make a blanket generalisation. 218 00:24:26,140 --> 00:24:36,760 Are those trials too difficult to complete? So we looked at the strength of limitations, methods, sections and anaesthesia or problems with blinding. 219 00:24:36,760 --> 00:24:41,260 We're not reporters as obstacles in completing any of these trials. 220 00:24:42,550 --> 00:24:45,590 In general, if there was a general anaesthesia. 221 00:24:45,610 --> 00:24:50,140 Blinding was really easy because patients was asleep and they didn't know what was happening. 222 00:24:50,560 --> 00:24:55,480 But in about a quarter of the trials, they've used local anaesthesia, 223 00:24:55,660 --> 00:25:02,770 and in those trials they've used absolutely ingenious methods to blind patients and pretend they're actually doing the real surgery. 224 00:25:03,100 --> 00:25:07,800 So for example, in case of arthroscopy, they pulled my nipple and the limb splash saline, all that. 225 00:25:08,050 --> 00:25:14,200 Use a mechanical razor without the blade to pretend they're actually using devices and performing the surgery. 226 00:25:14,590 --> 00:25:21,580 They were asked to talk over the procedure as if they were really activating the laser, asking for devices. 227 00:25:21,940 --> 00:25:25,480 In the three trials, they actually tried to blind the surgeon. 228 00:25:25,960 --> 00:25:32,860 So those trials included an implant, and the producer of the implant prepared two different applicators, 229 00:25:32,890 --> 00:25:36,820 and one of them actually contained the implant, whereas the other did not. 230 00:25:37,600 --> 00:25:44,650 So the operator, the person actually performing the procedure, did not know whether the charm or the or the surgery. 231 00:25:44,860 --> 00:25:56,320 In other trial, a surgeon would do everything, placed the device, and then a technician would either activated or not. 232 00:25:57,040 --> 00:26:02,830 So the surgeon would not know whether actually the active element was bound or not. 233 00:26:03,700 --> 00:26:08,250 The biggest obstacle and absolutely underestimated by many researchers was recruitment. 234 00:26:09,580 --> 00:26:12,640 On average, 20% of screened patients were randomised. 235 00:26:13,600 --> 00:26:23,590 Only 20%. However, 75% of eligible patients were randomised into the trial and once recruited, the patients tended to stay and complete the trial. 236 00:26:23,890 --> 00:26:31,390 This was quite surprising. So because you are blinding patients and they don't know whether they've received the actual surgery or a placebo surgery, 237 00:26:31,600 --> 00:26:36,280 they tend to stay in the trial. They don't drop out because they're disappointed. 238 00:26:36,280 --> 00:26:39,220 Then look for treatment somewhere else. They tend to stay. 239 00:26:39,970 --> 00:26:48,340 But recruitment was a big obstacle and actually three trials had to be terminated early because the recruitment was as slow as one patient a month. 240 00:26:48,820 --> 00:26:55,360 So they said, sorry, we have to call it the day. We cannot continue the trial because we've run out of funding. 241 00:26:55,810 --> 00:27:03,730 So this is an example of a placebo controlled surgical trial on patients with knee problems. 242 00:27:04,300 --> 00:27:15,879 And you can see that of all 476 patients screened, 286 were not eligible because they either did not have the condition. 243 00:27:15,880 --> 00:27:23,320 All told, they were not eligible for the surgery. Out of 190 who are eligible, 102 declined participation. 244 00:27:23,470 --> 00:27:28,240 Some of them didn't like the idea of a trial. Some didn't like surgery, something like a placebo. 245 00:27:28,420 --> 00:27:35,530 So they refused. So 88 agreed to participate, but 48 were not included because it was a surgical trial. 246 00:27:35,530 --> 00:27:43,060 So there was an additional criterion that they had to have the particular change on the MRI and those who did not were not eligible. 247 00:27:43,210 --> 00:27:48,940 So out of 476 patients screened, only 40 were included in the trial. 248 00:27:49,210 --> 00:27:59,530 And this is the thing. Many people don't realise why when they design a trial that they may not have access to sufficient 249 00:27:59,530 --> 00:28:05,290 number of patients within the time of the that they have to run and complete the trial. 250 00:28:06,760 --> 00:28:11,049 So do we need them? Do we need those placebo controlled trial? 251 00:28:11,050 --> 00:28:20,800 Is there any placebo effect? So in the review trials, there was an improvement in 85% of them in the surgical arm and in the placebo arm in 74%. 252 00:28:21,020 --> 00:28:27,640 So in the majority, there was an improvement, despite the fact that it was only a fake surgery, not the real deal. 253 00:28:28,240 --> 00:28:37,420 Surgery was better than placebo only in 49% of these trials, which means that in over 50% there was surgery was actually not better. 254 00:28:38,590 --> 00:28:45,249 There was improvement only in the surgical arm in Southern Review trials, which was about 30% but important. 255 00:28:45,250 --> 00:28:53,170 Five of the five of these trials used objective outcomes, not pain and function, but actually something measurable in objective way. 256 00:28:53,860 --> 00:29:01,920 A placebo was better than surgery in one trial. No improvement was reported in any arm in six out of 53. 257 00:29:03,570 --> 00:29:11,610 This is part of the forest plot. So there are two a full pages of forest plot and this is, if I can say so, this is done. 258 00:29:11,610 --> 00:29:20,520 Of those difference between the surgery and the placebo arm, we did not calculate a pooled effect because of heterogeneity of the trials. 259 00:29:21,150 --> 00:29:24,629 We we group them by the condition above. 260 00:29:24,630 --> 00:29:28,200 Even within each condition, there were different procedures or different outcomes. 261 00:29:28,200 --> 00:29:35,690 So we simply could not pull them together. But overall, you can see that the effect size is small. 262 00:29:36,090 --> 00:29:40,799 This is a study by a different group also on placebo controlled surgical trials in which 263 00:29:40,800 --> 00:29:45,120 they actually calculated an effect size in the surgical and placebo arm separately. 264 00:29:45,390 --> 00:29:50,280 And if you look at those results, I'll in there will take them. 265 00:29:50,280 --> 00:29:54,509 So there are actually vertical you can see that the effect is generally large, 266 00:29:54,510 --> 00:29:59,400 but very often the difference between the surgical and the placebo arm is rather small. 267 00:29:59,490 --> 00:30:03,720 And this is really surprising because the active procedure here is a surgery. 268 00:30:03,870 --> 00:30:14,280 We are actually changing anatomy and we would expect actually a really large effect and that will be better than a fake surgery. 269 00:30:14,760 --> 00:30:17,729 So what we have here is efficacy paradox. 270 00:30:17,730 --> 00:30:26,340 So we've got a massive response after the real procedure and placebo, but the difference between them is very often non-significant. 271 00:30:27,240 --> 00:30:34,200 So what I was interested in is what's going on in the placebo arm, because all the trials were so heterogeneous, 272 00:30:34,200 --> 00:30:42,450 we analysed only the change in the placebo arm because then it was slightly more comparable because literally you are just comparing fake surgeries. 273 00:30:44,550 --> 00:30:51,780 So this is a meta analysis of responses between the baseline, the follow up in the placebo arm only, 274 00:30:52,590 --> 00:31:04,200 and this included all the review trials with continuous outcomes because we are interested in the magnitude of the effect and the effect of time. 275 00:31:04,200 --> 00:31:10,859 So you can see trials subdivided by the outcome type and there are also all the by 276 00:31:10,860 --> 00:31:16,439 the duration of the follow up from couple of days to two years at the very bottom. 277 00:31:16,440 --> 00:31:23,850 And this is the same here within each subgroup. So you can see that for subjective outcomes there was a large, a significant effect. 278 00:31:24,360 --> 00:31:30,360 And for assessed and objective outcomes, the pooled effect was not significant. 279 00:31:30,360 --> 00:31:34,020 And those whiskers are not confidence intervals, those are prediction intervals. 280 00:31:35,160 --> 00:31:46,770 So from this we can see that there doesn't seem to be a strong effect of time, but there seemed to be a really strong effect of the outcome type. 281 00:31:47,010 --> 00:31:51,930 So what we've done, we've looked only at trials with pain as an outcome. 282 00:31:51,930 --> 00:31:55,020 Okay, fine. Let's make it slightly more heterogeneous. Homogeneous. 283 00:31:55,020 --> 00:32:02,820 Let's look at subgroups. So those are trials only with pain as an outcome assessed as a primary outcome time point. 284 00:32:03,090 --> 00:32:07,890 And you can see big effect this point 70, 78, 285 00:32:08,080 --> 00:32:17,729 huge effect significant for all the trials and stays significant for 24 months and if the outcome is objective. 286 00:32:17,730 --> 00:32:24,660 So those are trials in which weight loss weight was used as an outcome, which is an absolute objective outcome. 287 00:32:24,660 --> 00:32:25,800 You put this step on the scale. 288 00:32:26,100 --> 00:32:33,450 There is a number whether you are happy or unhappy, whether you've been told the good news or bad news, it's just a number. 289 00:32:33,450 --> 00:32:39,329 You can look affected and you can see that the effect was not significant in any of the trials to pull. 290 00:32:39,330 --> 00:32:43,980 The fact is not significant and there seem to be a bit of an effect of time. 291 00:32:45,330 --> 00:32:47,490 So we've done a meta regression. 292 00:32:47,790 --> 00:33:00,450 So this is actually a beta coefficient showed as a forest plot for every factor that was reported in the literature as potentially affected, 293 00:33:00,720 --> 00:33:10,590 affecting the magnitude of placebo response. And there was no effect of time, no effect, no difference whether the outcome was assessed or objective. 294 00:33:10,770 --> 00:33:15,090 There was a large effect of subjective outcomes versus objective. 295 00:33:15,750 --> 00:33:19,860 It didn't matter whether there was a standard or rescue medication. 296 00:33:20,190 --> 00:33:26,610 It didn't matter whether the trial was done in the States or in Europe, and it didn't matter how. 297 00:33:26,880 --> 00:33:33,240 What was the chance of getting the active treatment, whether the RANDOMISATION ratio was 1 to 1 or 3 to 1. 298 00:33:35,790 --> 00:33:39,599 But we've got sports. We can do it even better. So what we've done, 299 00:33:39,600 --> 00:33:47,130 we've actually we've taken every single time point for every review trial with a continuous 300 00:33:47,850 --> 00:33:56,310 outcome and done the analysis across all time points and all visits because some in some trials. 301 00:33:56,360 --> 00:34:05,150 There were multiple follow up visits, and you can see that overall there was a significant placebo effect that seemed rather flat. 302 00:34:05,930 --> 00:34:07,540 It wasn't really what I was expecting, 303 00:34:07,550 --> 00:34:15,530 so I was expecting the placebo effect to decrease with time because you have to remember, this is not a pharmacological trial. 304 00:34:16,070 --> 00:34:20,090 In all those trials, there was only one surgery at the beginning. 305 00:34:20,480 --> 00:34:29,180 And then the only thing we do, we keep following patients up and asking them, Is your pain still better have improved? 306 00:34:29,330 --> 00:34:33,500 Could you please rate it for me? Could you please raise your symptoms? 307 00:34:34,850 --> 00:34:38,360 Because we knew that the type of outcome actually matters. 308 00:34:38,390 --> 00:34:43,100 We've done this analysis by subdividing trials, by the outcome type. 309 00:34:43,100 --> 00:34:47,959 So in red you can see trials with subjective outcome. In blue with assessed. 310 00:34:47,960 --> 00:34:55,000 We have very few trials. That's why the confidence intervals are massive and in green you can see trials with objective outcome. 311 00:34:55,310 --> 00:35:00,320 But because those are trialled mostly on obesity, the effect seems to decrease, 312 00:35:00,320 --> 00:35:05,270 which may be related to the fact that in obesity trials people tend to stick to diet 313 00:35:05,270 --> 00:35:10,910 and lifestyle changes at the beginning and then they slip back to the old habits. 314 00:35:11,300 --> 00:35:17,720 But for subjective outcomes like pain, function and quality of life, there is a big, 315 00:35:17,720 --> 00:35:23,720 highly significant effect that remains highly significant for 12 months. 316 00:35:25,220 --> 00:35:37,640 That's 12 and that's seven visits. So if we don't have a control arm, an effect like that will be interpreted as amazing success of our surgery. 317 00:35:39,290 --> 00:35:44,270 So just to finish up, so we've also asked surgeons what they think about placebo. 318 00:35:45,020 --> 00:35:53,510 So we gave a short survey to hundreds surgical members of the British Alban Shoulder Society at one of their conferences and ask them, 319 00:35:54,110 --> 00:35:58,820 how often do you use operations that you believe may have a significant placebo component? 320 00:35:58,850 --> 00:36:06,080 So surgeons are notorious for not admitting that what they do may not be actually specifically effective. 321 00:36:06,530 --> 00:36:13,970 This is well, this is not surprising because unlike doctors who just give a pill made by a big pharma company, 322 00:36:14,240 --> 00:36:18,020 surgeons actually have to go over and perform the surgery. 323 00:36:18,020 --> 00:36:22,700 So they have to believe they what they do is actually helpful and effective. 324 00:36:23,630 --> 00:36:28,910 So 42% said never. I've never done anything that wasn't directly helpful and effective, 325 00:36:29,930 --> 00:36:38,419 but 21% said really less than once a year, 24% more than once a year and 30% more than once a month. 326 00:36:38,420 --> 00:36:41,540 I do something that I believe may actually be placebo. 327 00:36:42,080 --> 00:36:48,260 So we've asked them what their position concerning the use of placebo and only 4% said it should be always prohibited. 328 00:36:49,100 --> 00:36:54,710 I disagree. It should not be used. I I'm I'm opposed. 329 00:36:55,130 --> 00:37:06,140 But 46% said it should be permitted only in clinical research, but 50% said, well, actually, it should be permitted in a clinical clinical practice. 330 00:37:06,380 --> 00:37:13,430 If research supported or it should be permitted in clinical practice, if experience in the departments supports its efficacy. 331 00:37:13,670 --> 00:37:17,900 So we have to remember that this is what people say, not what people do. 332 00:37:18,650 --> 00:37:25,640 But we were still surprised that the response was quite positive and the main concern that they had was deception. 333 00:37:26,360 --> 00:37:32,389 So surgeons were generally opposed lying to patients or deceiving them in any way. 334 00:37:32,390 --> 00:37:38,540 So as long as we actually disclose that this is a placebo surgery, they were quite okay with the concept. 335 00:37:40,640 --> 00:37:41,960 Finally, ethical issues. 336 00:37:42,410 --> 00:37:51,050 So with Professor Gillian Lesko, we really suggested that actually surgical trials are not that different from pharmacological trials. 337 00:37:51,470 --> 00:37:53,330 So as long as there is an equipoise. 338 00:37:53,630 --> 00:38:08,000 So lack of clear evidence that this treatment is truly effective and really superior than no treatment with physiotherapy, such a trial is justified. 339 00:38:08,930 --> 00:38:12,530 There should be some preliminary, preliminary evidence that the surgery is effective. 340 00:38:12,620 --> 00:38:19,400 So like those trials I mentioned earlier than they've done the placebo controlled surgical trial, and there was no improvement in neither of the arms. 341 00:38:20,480 --> 00:38:21,980 There was really no point doing that. 342 00:38:22,280 --> 00:38:30,920 But if we actually perform a surgery like the shoulder surgery and patients tend to improve, but they also improve after physiotherapy. 343 00:38:31,910 --> 00:38:38,450 So there is some evidence of efficacy, but we don't know whether it's actually due to surgery or placebo. 344 00:38:39,830 --> 00:38:43,160 There was definitely a sense to run the placebo controlled randomised trial. 345 00:38:43,820 --> 00:38:48,889 It's also not no point to run the trial if there is just a small improvement. 346 00:38:48,890 --> 00:38:55,370 So we want things like the shoulder surgery. There are hundreds of thousands of those surgeries performed every year. 347 00:38:56,390 --> 00:39:03,020 The number of surgeries of salt operated patients increased seven times between 2000 and 2010. 348 00:39:03,530 --> 00:39:13,100 So this is something performed very, very often. So if it's not necessary, we save money, we save time, we save NHS resources. 349 00:39:13,340 --> 00:39:17,330 And also instead of sending patients, patients to wait for surgery, 350 00:39:17,480 --> 00:39:23,150 we can just offer them physiotherapy and they'll go get equally effective treatment much sooner. 351 00:39:25,040 --> 00:39:28,030 We definitely propose that harms should be minimised. 352 00:39:28,040 --> 00:39:36,190 And like in the placebo controlled shoulder trial, if there is an option to offer direct benefits to patients in the placebo arm, 353 00:39:36,200 --> 00:39:39,890 for example, like diagnostic arthroscopy or laparoscopy, they should be offered. 354 00:39:40,160 --> 00:39:47,600 And finally, there should be no deception. And I would like to finish with a quote by Kolbe, 355 00:39:47,600 --> 00:39:58,070 who did a placebo controlled trial on the efficacy of increasing blood flow to the heart in the sixties. 356 00:39:58,370 --> 00:40:04,640 So he said that after observing some of the dramatic results afforded by only minor bilateral thoracic scale incision. 357 00:40:04,790 --> 00:40:12,079 So they've actually, to the point, cut one seriously questions how much of the reported clinical improvement after thoracotomy is actually 358 00:40:12,080 --> 00:40:19,190 dependent upon the patient's like reaction to surgery rather than an enhancement of coronary artery, 359 00:40:19,190 --> 00:40:24,320 blood flow or other physiologic alteration? Thank you very much for your attention.