Okay. Hi. Welcome, everybody. Welcome.
Some of you started the must've from stragglers from the department of wandered in to see how you might contribute to IPM.
For those of you I know, most of you. My name's Carl Hennigan.
I'm professor of evidence based medicine and director of the Centre for Evidence based Medicine, and I'm also a GP.
This talk, basically, I figured for those in the math as it applies to everybody,
if evidence based medicine is often seen to people like something that's really difficult to get involved in and actually actually can be quite scary.
So after a busy day in the masses, this is a talk about sort of some of the things I've learned over the last 20 years that have really
profoundly influenced me in the way I think and how I go about my contribution to evidence based practice.
And some of you might start to think about how you may contribute.
So I'm just going to ramble around from 50 years worth and feel free to interject.
Stop me any time you don't quite understand what I'm saying or you want to stop.
And maybe at the end we'll have a little discussion about how you might contribute.
So I think this is an important photo and this is an important landmark in how we
think because thalidomide with a really important issue in terms of regulation.
And this is John F Kennedy giving the gold medal or bronze medal at the time to Francis health.
Because in America, what they said is we're not going to allow you to have thalidomide unless it's in the context of a clinical trial.
And that was not for the pregnancy issue. That was because of the neuropathy problems.
That meant you were about 100 times more likely to have a child with fluid amide in the UK than America,
despite the fact they outnumber 6 to 1 in population. And at the time what happened is drugs were just basically brought about in the market,
released on the market, and people then said, let's see what they do.
And with this drug, they got spontaneous reports that actually it's really good in nausea and then said,
oh, well, there's a really good group to come to the fore. There's a really interesting group of people that this could apply to.
And it's pregnant women.
And at the time, John F Kennedy said, I'm going to put £2 million into the US FDA so that this problem would never occur again.
So that really is the birth of regulation around about the fifties today.
You've touched on this today. It's a really important interest in arena that you talk about the idea of antiarrhythmic and that's a real problem.
And it was the birth of in the eighties of evidence based practice, if you like.
This is an arena and area where people were pretty clear about clinical practice and
then along came this clinical trial and really turned over what we think we fall.
And often you should read stuff you've got to read if you're going to be influenced and think about the bigger, wider issue.
And this is a great book, actually. Thomas Moore, why tens of Thousands of Heart Patients Died in America's Worst Drug Disaster.
I would like to make my job participatory as well. He feels good about reading.
Feel good about reading. Can you read? Okay, good. We get the variety.
Students are not thrilled if they just read out. And I said loud and clear and slow and just read the first sort of ten lines for everybody.
This book tells the story of America's worst medical drug disaster over just a few years.
An estimated 50,000 people died from taking drugs intended to prevent cardiac arrest.
After hundreds of thousands of patients routinely took these drugs,
a definitive medical experiment proved that they did not prevent cardiac arrest, as doctors had believed.
Instead, the drugs caused cardiac arrest.
Often the effect was so sudden and unexpected that people literally dropped dead while going about their normal lives.
The result of this single medical misjudgement about the properties of these drugs produced
a death toll larger than the United States combat losses in wars such as Korea and Vietnam.
Okay. Thanks. Quite interesting, isn't it?
Quite powerful when you think about what's really going on,
the capacity of medicine to do benefit but great harm and that we all start in equipoise is really interesting.
So superb evidence. Both practices born all disappear, all the problems disappear around about just early late 1999, 2000, I become a House officer.
In fact, at the back there's Jeffrey Aronson, who now works in the department it with my consultant at the time and I oh yeah we practised.
It's all disappear but I want you to talk go forward to about 2000 and I'm going to talk to you about a particular drug in a minute.
But this is where the birth of evidence based practice comes from,
is the integration of breath research, evidence with clinical expertise and patient values,
trying to bring evidence to decisions, which seems like any time I ever speak to anybody who's not in health care,
they say, Well, that's obvious, isn't it? Everything should be based on evidence.
Why do we need a professor evidence based medicine? Surely you're out of a job, you're redundant.
And actually it seems nonsensical to patients and the public that medicine would be practice without evidence.
Yet it continues to be and continues to be in all sorts of ways influenced by a lack of evidence.
But I want you to take forward to 2000. So I'm a House officer.
I work in a team called Firm Fee,
and there's a particular drug called via a Cox two inhibitor that is being marketed as a pain reduction medication that's advantageous over current
non-steroidal because of its lack of gastric adverse events and is widely being used by people who I'm around because there are lunches going on,
there are people being taken out to dinner and I reckon about a thousand people within our care within a two year period were put on this medication.
So I want you to hold a thousand people. Many, Jeff, in the team would say about 5000 people.
It could be 30. Say we take a thousand. Now, what's interesting, when you come to this Vioxx, this is actually, again,
really shocking is results from the clinical trial in 2000 show five times the number of heart attacks a month,
patients taking Vioxx compared with those taking naproxen.
Merck say the reason was naproxen, strong protective effect on the heart.
So they argued that way. 2001, the company's internal intention to treat analysis,
which are all going to start to learn about that means the people, everybody, not the ones just on the drug.
If you dropped out, identified a significant increase in total mortality nearly five times four and a half times.
Okay. In one study. And the second study was 2.55, so nearly two and a half times more likely.
Overall, that meant there were more 54 death around amongst the 1069 compared to 12 deaths among the placebo.
So if you round that to about a thousand, that's around about 20 patients, give or take a few.
Do you see what we were doing? Anybody see the significance of of putting a thousand people on average on this treatment?
We're actually assigning people in the intervention to increase mortality thinking we're doing the right thing.
Carl, actually, what happened in the jump was that John Reynolds, who was chair,
said we should not have this struggle to formulate in 2000 yet all the people
we saw we're getting it outside were coming in the hospital in huge pressure.
At that time, pharmacy reps would go to the street to general practice, talk to people about this.
And that's an incredible just hold that point. Jeff said, let's think about that as a signal 2001.
They told them there was no increase in risk of death. Why?
They knew it was three times the FDA said, could you just send a letter to everybody saying, oh, it's it's a bit unsafe.
Think about it. Now, when you go to when you look at evidence generally is you don't need statisticians to interpret what's going on.
To be honest, when you look at a graph like this and you look at this is a cumulative death rate and say that the intervention in the placebo,
you don't need to be a statistician to work out what's going on. You can see there's an increased mortality.
Yeah. What happened with this is that this was systematically withheld from clinicians for around about three years.
And it only became in 2003 for apparent that this three times mortality effect was happening and they consistently withheld evidence.
And what was interesting at the time is this is from Catherine DeAngelis, who was editor of one of the big journals at the time.
This is her and this is her quote. This is occurred because the physicians allowed it to happen and it's time to stop.
And that's because sometimes commissions and everybody's complicit in not using evidence based medicine principles.
And what Jeff said to you with John Reynolds, who is a very smart physician, looked at the evidence and said, there's a problem here.
We should follow the evidence base as opposed to not using the evidence and saying, well,
there are all these pressures, all these guidelines, all these commercial interests telling you to do something.
And that's very profound to me in my thinking to say, gosh, you can really get it wrong very rapidly if you're not careful.
And then we come to the steps of EPM, which is about critical appraisal validity and thinking about these issues, about applicability.
And then this morning before this particular trial, which is rosiglitazone the dream trial.
Great name is the dream trial, the Lancet,
one of two journals in the world that have a profound impact on medications in practice, surgical interventions in practice.
Then two journals really do have a monopoly on these types of studies.
But what's interesting is when you do read this and you read the bottom two lines and I
can take people who've never practised before now nothing about evidence based medicine,
and I get them to read. And at the bottom they read this cardiovascular event rates were much the same in both groups,
although 14.5% participants in the glitches are group and 2.1% in the placebo group developed heart failure.
Well, when you think about that, that is nonsense.
Even as a GP, I know the heart is part the cardiovascular system and even my kids can tell you that from gxp biology.
So that statement can't be right. And if you look at what it is saying, is it saying it's five times more likely, 5 to 1 simple math?
And the P value is .01.
So 99 times out of 100, this is the effect you get. And one time in a hundred this could be my chancellor.
You likely if you expert. So if you just think about it, you repeat this experiment 100 times, you're going to get this effect 99 times out of 100.
That's not rocket time when I said that. So an oh my God. So when you see signals, you should start to take them seriously.
And that's been profound in the way we think.
And when I wrote this, I wrote this to The Lancet and they said, Well, we're not going to publish the editorial.
We don't want editorial from you. Hennigan So I rang up Fiona Godlee and said, This is a disgrace.
And we publish that immediately that the study came out. The number 82 harm free years is 250.
So for every 250 patients you put on there, you're going to have one case of heart failure.
And this is in really low risk people. This is not in high risk people.
So you say, well, that's pretty low, but think about it, this was a $3 billion drug at the time.
It's been hundreds of thousands. We're back to the corks to argument.
We're back to the Antiarrhythmic story again.
And what's interesting is when you look at the timeline and go back and look in the regulatory timeline about Rosiglitazone, you can take it back.
It was approved by the FDA in May 1999 and they asked for a doctor, the phase four trial.
It was 2000 by the European Commission record as a phase four.
So they wanted some further trial.
They said we thought they had all this final document on violated on the cardiac disease signal start to appear here from the firm's working group.
An Analysis of CV Events 2004 Summary First Meta Analysis Meta Analysis by Steve
Nissen And to have said there's a 43% increased risk again of heart problems in
this group and here 2010 we go down box warning here FDA gave a box warning in
2010 suspend sales and again if you start to dig around the information you find,
this is in the original 1999 medical review.
The increase in body weight and undesirable effects on serum lipid is a cause for concern.
Heart disease is a major cause of morbidity and mortality. It cannot be assumed that Rosiglitazone will decrease the CVD risk.
My concern about delivery theory with long term effects on the heart should be addressed,
but with the 11 years before it withdrew off, the market was the biggest selling diabetes drug around about 206.
So very interesting again about this idea that treatments have benefits and harms and out there is lots of information
around treatment and your job in evidence based practice is if you're smart enough is we'll get there in the end,
but you'll get there about four or five years before anybody else because like what you just said about John Reynolds is you go,
hey, I don't understand this. And it's the same for benefit and for harms.
It works both ways. People who are doing stuff beneficial get there, really.
They stop doing the harmful. I'll move on again then. So that's just so interesting and this is probably again really being quite enjoyed.
This is Margaret Chan in 2009, you'll remember the swine flu pandemic.
Anybody have swine flu? Yeah, I caught it when I came to my first class here.
Okay. And he's still alive to tell the tale.
That's the definition of a pandemic, which,
with change for this means it should cause serious an epidemiological infection that has widespread and causes serious morbidity.
The definition with change to suit swine flu at the time and at the time.
This is my girl from 2009. They're a bit bigger now. And tell me to shut up, dad, and go away because they're teenagers now.
But at the time they used to listen to me. Then the question is not this was my girlfriend.
And what happened is my sister also rang me. She said, I just rang up the doctors, we've got a fever.
And they said, you need to go to Northampton to pick up a prescription of Tamiflu.
And she said, What do we do? Call? And I thought, I've no idea at this time.
It's a systematic review that I thought had been done in adults. Oh, this is not that interesting.
So what we did is we'd been involved with a group who'd been doing the Cochrane Review.
We said, Let's prioritise doing the systematic review in children, and that's exactly what we did.
And that was the major headline that came out. The research is I was astonished that the lack of evidence base for trials in the world for children,
less than 1500 children had been in randomised trials for this treatment,
despite the fact influenza is hundreds of thousands, millions of people infected at any one time.
You could do this trial in a week or two. And what it showed was that basically it has a small reduction in symptoms,
no effect on complications, and it makes children have diarrhoea and vomit.
About one in 20 children have serious vomiting and diarrhoea with it.
And I was a bit shocked and we published that with major national news with all over the TV,
and it had no impact whatsoever on the prescribing of Tamiflu.
In terms of guidelines, policy, in terms of GP's, I started to get messages from individual clinicians going, Yeah,
this is really good, we think you're right and this is what we don't want to be pressured into prescribing this.
But there's been an interesting change in how the world is now pushing us with technologies.
And if you think drugs, devices, surgical, they're all technologies.
And why is this once it's come about in effect, why do we slightly think differently?
And I think there's been a profound change in about the last ten years around the business of medicine.
And this is 2005, and this is an article in The Independent about good old Donald Rumsfeld.
What was that? There's no known knowns or unknown unknowns.
But he may he was actually one of the shareholders in Gilead who had the licence for the drug.
That's why he made a lot of money. But one of the things is he has for a UN report on how many predicted death would bird
flu remember bird flu that's before swine flu court So if I asked you worldwide,
how many predicted death would you think in this model that the UN to put together the worldwide
number of deaths were to be predicted at that point and anybody would like to hazard a guess.
You can phone a friend 1000 thousand, 10,000.
You're going to go for 10,000. You're going to go for 10,000.
What about the third? How many predicted death? They looked at it. They said bird flu.
It's got a few cases. The been about 9000 deaths so far.
You've got ten. Hmm. You're going to go for 100,000.
So it was 150 million deaths is what the UN predicted and the number of deaths with 700,000 that might be expected in the UK alone.
That's when we pull our first order of 14.6 million courses of this drug started stockpiling the drug because we got particularly worried,
despite the fact that only 90 deaths so far. So the real commercial operation around some of these pushing you into and I could show you the
really interesting 205 the marketing material said flu is a disease that everybody stays at home.
We're going to make you all for your doctor. And when you see doctor, what happened?
You're going to get intervened. In fact, you're going to spread it more quickly and we're going to intervene.
We medications. And that's what the marketing material said. But I particularly like this.
See, this is the way you get in in in problematic thinking because of the way the news media now comes to the public.
And you and this is Hans Rosling,
who if you go and Ted has done some remarkably good talks about this whole rather and it's called the news to death ratio.
And so you can look at the number of death in the first 15 days from 24th of April, when we first learned about swine flu to the sixth May to oh nine.
And you can compare that to TB and you might want to think about that 13 days
in terms of now what's happened in Ebola and think about it in context of TB,
you could do the same for Ebola.
In fact, the number of deaths in that first 13 days with 31, the swine flu in the same period, the number of deaths for TB 63,066.
The number of news articles with 254,442, the swine flu and 6501 for TB.
That means if you see one, you have to see 8176 news articles for one death when it comes to flu.
However, if you see one article on TB, ten people have died.
When was the last time you ever thought to myself, Oh my gosh, people are really talking about TB as though it's an important issue.
We should really be solving this. So this is almost what we always see is the inverse care law,
which is if you want to read something interesting going back is inverse verse was a chap called
Choo to Heart who said health care is often disproportionate to the needs of the patient.
And he was a man in a mining village in Wales, in South Wales, and he said, look at all these people with chronic airway disease,
high blood pressure, they get zero care and all the healthy people get all the attention and the care.
And it's really interesting if you look at that, it's not sexy TV.
It seems to be in less deprived populations, lower socioeconomic, classic people are not interested in it.
But boy, this is a big problem globally. And if you think of that number, 63,000 in nine, in 13 days.
Ebola is really starting with that 4000 since about Christmas.
Important, yes, but nobody is worried about this.
So what happened is I was so amazed at the lack of evidence that I managed to meet with a call from Jefferson.
And this is Deborah Cohen, who we do a lot of work with.
And Jeff knows Deborah Cohen, I mean, started in the adult what's going on here in the adults.
And he said, well, we think that's a bit dodgy here and we're not happy.
And I'm just me. This is a whole group of us who ended up meeting, and this is it in Oxford.
There's Tom Jefferson there. There's a few of the people, Peter Doshi and Chris Del Mar from Australia, a few of the people.
But we ended up combining our efforts together because the Japanese shepherd sent
an email to Cochrane and if you send an email to Cochrane reviews to the authors,
you have to respond within six months and now we could go off into that, he said.
Of the ten trials you did, you published in the Adults and used in your Cochrane review, if you actually look at it, only eight.
Only two of them have ever been published.
The other eight were included in a meta analysis by a pharmaceutical employee who was not actually employed but was actually paid for with a doctor,
had been had his lab paid for, had included that data, but it never got published.
And he said the unpublished data has a far more greater effect than the published.
And it could go and startling. And I just thought, well, that's quite interesting, but.
This is what when Tom first said to me, we got our first ever clinical study report, which is the stuff that's not published the most to cite cited.
So these are the two sided trials. Nicholson 2000 and Trinity found that the only two published one FAI did not mention
any serious adverse events or stated they were no drug related serious adverse events.
And these findings were repeated by bodies such as the UK NHS.
No theory events were noted in the major trials and no significant changes were noted in laboratory parameters.
However, the first clinical study report we got, which is of that study,
describes ten theory adverse events, not in nine participants in the two trials.
Now, if I when I film everything I know, when you see that I cannot reconcile what's going on here with this.
And we have the clinical study reports, clinical study reports go into thousands of pages and they detail all the adverse events,
the serious adverse events, serious, unexpected, serious thing that adverse reactions.
And that's what it said. And at that point, I was shocked, to be honest about this.
Again, I'm in a position where we go back to 2000, all the other aspects.
And then it emerged that 60% of patient data from randomised trials had never been published,
including the largest ever treatment trial undertaken with just over 1400 people of all ages.
We've now already spent £450 million on this in the UK and this is the situation of what we spent.
I mean, and it's interesting, these things don't go away. We make a complete nuisance of myself and the team do still.
We've just published in The Lancet today a letter, opinion letter, because the Lancet,
the people who were all funded by the drug industry offered actually all the observational data,
say that this treatment is fantastic and you should all studies.
We started to say, well, if you basing treatment effects on observational data, you start to get worried.
It's on the W.H.O. Essential Medicines List on the basis of three observational trials that show a 70% reduction in mortality.
But when you think about it, well, people get Tamiflu, unwell people.
I will send you to a hospital without the Tamiflu. That's what happened.
If you really want. Well, I'm going to send you in and we're going to treat you appropriately.
And on Monday, next Monday, I met the has the parliament on Monday at the public accounts committee because they're
really pissed off because they're saying we've spent over £500 million on this treatment.
And if you look at what's happening currently,
a GP who says I prescribe this readily and we stockpile it and I'm not quite sure where and we still spend another £50 million a couple of
months ago because we're in a binding contract and then everybody says no money in health care can't quite understand and reconcile that.
But again, if you prepared to use the evidence, the evidence, it allows you to look and dig and think.
You need to have all the evidence around the benefits and harms.
I'm just going to finish that about well and then to talk about things that we may do differently.
This is some stuff, you know, evidence based practice.
If you prepared to be a bit of a nerd for me thing where you can go and this is about five years ago,
I got backed into a corner and said, we think there's a major problem with metal on metal hips.
And this is people at Channel four with Deborah Cohen at the BMJ.
But we've got this evidence and we can't really find any orthopaedic surgeons who are willing to look at it and give us a sort of impartial viewpoint.
And I looked at it and I said, again, it was a bit like the Vioxx.
If you looked at ceramic hips, the failure rate with about 3% in ten years, superb operation, had some problems.
A leg could dislocate it, put people on the bed, they get better and they go back to work.
But there were a few issues with it but not make it.
Metal hips came along and people said You can get back to work quicker, you can have a better range of movement and it doesn't dislocate.
And by about five years, the failure rate, 15, 20%.
I'm like, you don't need many statisticians to know we're in a really bad way.
And it turns out in devices it's like the Wild West compared to drugs because often a device can get on the market.
And we looked at it, we did this look to all the medical data.
I was shocked to learn about how devices are regulated, even in terms of implantable devices that you can get away with.
Just equivalence. My device is similar to somebody else's on the market.
I don't need a clinical trial. That's okay. You'll get CE marking and us FDA marking on what's called a 510 K license.
Well, I'm just alluding to that is I've ended up as a world expert in device regulation
almost by default on the basis I don't think anybody else read this stuff or Cath.
And if you go and read this stuff and say I'm really interested in it, you'll be amazed at what goes on in devices.
I can tell you it's unbelievable.
But actually understanding the evidence that's required to get drugs or devices onto the market puts you in a space where
actually you can be really influential in terms of thinking of how you might contribute to how health goes forward.
So that's a bit about some of the profound ideas and issues that have impacted on me.
And I went to the DEA who fund all of our research in January, and what they wanted to know is what should we think about differently?
And so I put these ideas, seven ideas to them.
I said, This is what's going wrong or what we might do differently.
I think there is a constant need to re-evaluate interventions that are already on the market.
And here's a classic example aspirin for everyone older than 50.
This is an editorial in the BMJ. In 2005, everybody at 50 was going to be like Smarties would be on aspirin by 209.
We're actually saying this is a bad idea, actually.
And when you look at it and think by 209, actually the benefits of aspirin in primary prevention are outweighed by the harm.
So by four years later, we're like, no, this is. And when you look at.
It's really interesting. If you go back to here about 1758, if you go up to Chipping Norton up the road,
you can make the Reverend Stone, who published here is the account of the success of the bulk of the Willow.
And he was a fellow. It was what him actually was a fellow at Wadham.
But when you look at this, it takes until 1974 for the first randomised trial of aspirin and my reported.
I mean only 40 years ago it seems ridiculous that only 40 years ago actually people didn't really have trial evidence to support aspirin in America.
So most of what we know is really young and early.
What's a puff of physiology is is hundreds of years old and all basic sciences, teaching pathology and physiology are coming from two 300.
But we're coming from a period where actually the evidence base is not that amazing.
And if you look at 1996, the FDA approved it for suspected MRI.
That's not that long ago. And I put this slide up, you'll see this slide.
Maybe people talk about this idea that cumulative meta analysis here,
the meta analysis and this is probability is 12 and that the overall effect favour treatment.
And what they do and I've done this for about ten years and then if I can't be right, this they go about here and they say,
well, everybody could have stopped doing the evidence here and it should have been implementing.
And that's correct, in a way. But actually, what's going on here?
What are all these trials doing right up to 1988?
They are confirming the finding in lots of different populations, generalising the results.
They're consistent. What happens now with Thrombolysis has become widespread and it's not something that people go.
I'm not quite sure about the evidence. What happens today is much of the evidence, the roundabout here.
And then we move into all sorts of other areas called knowledge translation, implementation, science.
But actually the problem is we're in 13 now.
When I went to the DEA and said, we don't we don't fund confirmatory trials.
And I said, that's why there's no other trial of Tamiflu outside of the manufacturers, because nobody said,
let's just confirm these results in a different population is sensible as opposed to spending 550 million.
I said to them, you could have spent in the telephone line 5000 people received the treatment in the first week.
You could have randomised them done the trial within 28 days and it cost you about £500,000 as opposed to 5 million.
Not interested.
Number two, insufficient evidence exists around most current intervention to determine if they are effective, which follows on from one.
And this is the classic example and we've been discussing this today.
I am amazed that in primary care around about 3 million people are on antidepressant.
And I'm even more amazed when you look at the amount of evidence that informs that decision.
This is in primary care, the systematic review 14 studies of which ten examined 58 to assess rise.
But forget that the number of participants again is only 1364 in the intervention and 919 in the placebo, and they're typically 6 to 8 weeks.
The discussion I was having with Kemal today said, Well,
how come the guidelines or the guidance is you've got to stay on the treatment for 18 months before you come off it,
if that's what the evidence says, no dose information and the offers were in Britain and the appropriate duration a three year treatment.
So it's a bit like the Wild West, but we've ended up with 3 million people on the treatment.
I can't get them off. I think that's very thought provoking. Third is this is actually now becoming a big problem for us is very difficult.
It's becoming very expensive to develop the evidence base. It's almost like an outcome to the old trials.
The front end regulation is so great that it takes about a year just to get off the ground.
And these trials now run rate of 20 like 2.5 million and you're like, we need more money just to get going.
If I came tomorrow and I said, I want to do a new intervention in pharmacy and we've discussed this for anticoagulation,
I want to do some educational well, the commissioners can give it to one pharmacist.
We can give it to every they can give it to half. You can give it to a third.
But if they decide to randomise, it, then inflated the cost by half a million pound.
So I think this is a real issue that has to be in there's a new thing called the Health Research Authority that's
trying to improve how we do clinical trials for this came up recently in a Twitter debate and got picked up.
Most interventions don't apply to the people we see in practice, and this was something I did with Jamie Oliver.
It's TV production said to many made of thin.
You're not the man who made it fat, the man who made it thin to deceive.
And he said, Well, look, this evidence, they all say this evidence for Weight Watchers works.
And I said, well, it does work, but it doesn't work to anybody we see in practice.
So if you take the evidence, this is based on the trial evidence if. And look at the evidence here.
You take a thousand people and invite them, 885 won't come.
So that's 88% a population are not interested in intervention.
So this is a public health. It's to both solve the problem of the hundred and 15 who do come take up the intervention,
fix well, attend the first class and 53 do not attend all the classes.
However, of the 62 attend all the classes, they will lose about 5.4 kilos at the end of them classes.
If you follow them up with two years, you down to 13 people who maintain the goal weight three and up by five you
that's about ten of which nine are women and one's a man maintain that go away.
So he started out with an intervention 4000 people and he managed to get to ten people.
Superb, fantastic public health intervention.
Do you think it's actually useless? It's useless if we think this is going to be our strategy for attempting to lose weight.
And if if you are one of the people who likes classes and is in this and you like going to Weight Watchers,
then you can keep going for five years and you've got a good friend and great it's a great thing to do.
But the health service is this what we should be spending our money on commissioning?
I think this is deeply and this is about thinking about these.
Who does this research apply to? Five.
I'm not going to march about this, but I did touch on this.
One of the HCA and John Breast is going to talk about this on Thursday that the current need to develop a rapid but also robust evidence base.
We need much more efficient, quicker information.
And when we decide that that information is important, we then need to think about doing a robust job.
The problem is, is if you seen today, if if I go to PubMed and I'm not wrong here,
but last time I checked it with 24 million articles and climbing and it's almost exponential.
PubMed counts for about 45% of the day's publications out there.
So the number of 50 million plus of which will bottom out about 50,000 clinical trials in about three or four years time.
It's massive. The explosion in evidence base and how do we get through efficiently?
What we want to know is the real problem. This you say, and this probably fits with the weight.
Here's my two important question for any intervention down here is the critical appraisal that you're all going to learn down here.
Yeah, but my two is the top. If you take a questionnaire, my two important questions are who does this apply to and does it make a difference?
And if you can't answer it, yeah. Applies to the people I see.
And it does make a difference. Why move on? Why even bother doing any appraisal?
And now I've been thinking about this and I might write this, that actually the real way we do it is the other way around.
You tend to ask a question and do the appraisal and say high quality evidence,
and then at the end of it run out of steam and go, Oh, who does it apply to?
But actually, sometimes the other way to what you find it, does it make a difference?
Are all these other issues like surrogates, composite end points, end points that don't mean anything.
6 to 8 week studies. Yeah, not long enough to even sufficiently say this is a useful end point and when you do it that way.
The problem on coming to in the way I'm thinking of this is we've got this mantra of cost effectiveness is what it's all about.
And people are not understanding that actually, where's the clinical effect?
What do they mean? So any time you look at an intervention, you should be able to say, what is the actual benefit here?
And when you do something like that in Tamiflu, you actually start to see that.
Actually, if you answer that question and anybody goes to me, well, there is a bit.
So I say, Well, okay, just imagine you are a GP and you're going to say to somebody, inform them, give them an informed decision.
What would you say about the benefit? What would you say at home and who does it apply to?
And when you do that, it breaks down the arguments.
Breakdown seven and then seven is this interesting issue is that there's a real problem supporting our regulatory bodies.
And this is just one example. This is an example of a drug called Rivaroxaban for acute coronary syndrome that came up at the FDA.
And this chap, Toby Matheny, is a real sceptic and does a really good job.
But he said we're in deep trouble.
He said the end point is about 1 to 1 and a half percent difference in the endpoint rates between river oxybate and placebo, however.
About 12% of the patients had an incomplete follow up.
And we would say, well, actually, if the loss of the follow up exceeds the endpoint, you've got a problem.
You back at the the Vioxx argument that you could have this intent.
What's happening to them patient. And when he did a simple plot of the year to the patients with missing data.
This is what it starting to look like over the last ten, 12 years.
In fact, what should be happening and this should be going this way because the end points a smaller here, the difference is smaller.
Three trials need to be larger, but they need to maximise following up patients because the really important what goes on in these study and
we're really interested in that stuff because this does make a difference to how you interpret the endpoint.
If we use this treatment, we could be right back at where we are with Vioxx because what are these 12% patients telling us?
So coming back to where you are, what you might think about these are the three ways when I think about anybody you worked with,
those contribute to the development of the evidence base,
contribute to the methods of EPM and contribute to the teaching dissemination and communication of VBA and then the free aspect.
So anybody work through this, I'm always pushing them a bit to say, Well, where are you?
So we, we do lots of here. So this is you could be the evidence base in terms of doing systematic reviews,
could be in doing primary studies, could be in implementation studies in practice.
So that's one aspect. They're interesting. They will make a difference in summary, but the other two areas contribute to the methods of EPM.
I told you that the first ever trial of aspirin was only for two years.
It's some information we know what we're doing yet we're only into the into some of these issues.
There's lots to be learned and thought about. There's so many areas you can look at.
It's almost it's almost exponential.
But you should be thinking, oh, I'm really interested in losses to follow up.
Just really interested in adverse events. Jeff, have we got a sensible system for measuring adverse events, would you say?
You know, Annette's really interested in diagnostics.
Would you know, do you think we've got a sensible way of assessing diagnostics in primary care?
No. So that's our contribution. But then the third is contribute to the teaching dissemination communication of VBA.
That's what I'm doing now. After I'm communicating, I'm trying to disseminate some way of teaching.
That's why we were on teaching courses this year.
We had a meeting in summer. We're going to start teaching IBM to schools because by the time you're at school, it's all too late.
And I can't believe that. And in fact, schools are starting to do a good job.
Actually, they are talking about drug development and clinical trials is on the collect curriculum, so it's already in there.
So we're going to work with schools a bit because we think there'll be interesting dissemination and communication.
There are many ways and I'll just finish it so you can think about them.
Here's here's a group that I did. And so thinking about communication and dissemination, you can participate in all fourth way.
This is a group of people who came to a three day workshop, practice workshop for three days.
And at data we looked at a systematic a paper that was published in the BMJ about the Association
of White RIF and Type two Diabetes and this group with so infuriated by how crap that paper was,
it said We have to do something about this. So I said, Well, why don't you get together and write a letter to the BMJ and when you write it,
so you really so we in the mini appraisal that we did it with like you're going to do tomorrow,
they spent 90 minutes to appraise the paper and they said, Look at all these problems.
I said, Well, why don't you go and write it? So they got together, wrote it up and communicated it.
That's me at the end, at the end and sent it to BMJ and it got published.
The communicating, they're participating. So if you develop these skills, you certainly should be using these skills in some way.
You shouldn't just put them in the closet.
That annoys me in a way, because I think if you've got something that you think to say in health care and you've seen something in a signal,
you should participate. Nobody's going to tell you head off. Because if you stick to your evidence and not your opinion, stick to the evidence.
This is what the evidence is saying. You'll be all right. Here's another. And we put all these and I just put here the left.
If you go to the CBM, you can put all the projects and here's your thoughts in all sorts of different ways.
This is a meeting that led to a publication about real birth is rubbish.
Even this is a load of people who are challenging us to think differently.
And some people are saying.
Trish Greenhalgh who's saying, look, there are loads of problems with the evidence base and some of it's rubbish and some of it,
and Jeremy's going, This is really bad and this is what the difference is.
And they published an article in the BMJ a couple of months ago that the biggest downloaded paper they've had this year,
because they were just saying, look, let's participate in thinking ideas.
There's no space where you can't contribute. I could spend hours talking about all the different ways you can contribute to different.
Project. Have a look at the website. Some of the different stuff we do and I wanted to finish and maybe this is a good point to finish.
So out of this, again, a contribution. What did we do after the Tamiflu story?
I was contacted by a couple of people, one with Ben Goldacre and the other with Fiona Godlee at the BMJ.
And what happened is they said, Oh, there's an MP in and I was amazed that people actually do this.
They actually really put their neck out and bingo, they could put your neck out.
And when that MP called Fair Wollaston,
who is prepared to talk to us about this issue because she'd read about the Tamiflu and they couldn't believe it.
Could you come down to London and talk to us about all this publication?
So we went down to London and basically out of that was born an initiative called All 12.
And what's been interesting and working with people like this is, is centred sense about things.
BMJ that Ben Goldacre James Lynn Library is most of the great projects I'm involved.
He'd never have a single penny of funding at all and all the stuff we get involved in,
often the question or the clinical issue is compelling because you've skilled yourself in some of these issues in evidence based medicine,
and often you don't need lots of funding to go and dig around, be involved.
520 organisations, 80,000 people have signed this position.
If you haven't signed it, could you go to trial? Because that's me telling you to sign it.
There'll be another 20 people help and it's a classic example of using the skills, the VBM and what we're trying to do.
We solve the problem that half of the clinical trials are never published or have not been published,
which prevents massive problems if we can understand the benefits and harms of treatment,
and that's a good example of how you might contribute or get involved.
Thank you very much. Does anybody want a couple of questions for you all?
How did you manage to get the Tamiflu people to tell you about the unpublished trials?
Because I tried to find out about trials and it's like, yeah, correct.
Totally correct. In fact, what happened is we ended up started with involving the BMJ,
then started involving the media and actually got channel forward involved and they did a TV programme.
Then we had this ridiculous where we'd write these emails to people like Roche and copy in 50 different media people, and they're all available to go.
And then we got to the position where we just kept ramping it and ramping up.
And then if you go on there, there's a BMJ Open Data campaign.
If you look at it, you can if you're a real nerd, you can see all the emails are published.
We even published all the emails. Instead, what we're going to focus on is because you keep saying you're going to give it and then you don't.
And that was about an 18 month movement.
And probably the bit that made a difference was to think the publication of Ben Goldacre is.
But, but, but pharma had a slight shift in it just resonated in some way.
And what it did really well with communicate. So we all I think I do all the work but not me.
But I know all the people who do all the work in this area. What he did is really communicate it to the wider public really well.
And that's what was exceptional. And then the old trials came and about four months after all trials.
Roche, GSK made a commitment that we're going to make this stuff available and they signed all trials.
And then it looked a bit odd because Roche were on the other side going, Well, we're not giving it here.
And then about three months in, they committed and retrospectively, well, they sent us everything on a they talked about privacy and all this.
And I was like and we had a discussion and they just posted me the CDs in a package to the department.
I was like, This is ridiculous.
After all this privacy and all these issues you just sent in the mail and I was just so I've got all these all this data.
We've now got it constituted for Tamiflu,
over 150,000 pages of data compared to about 12 pages that had been published in The Lancet and JAMA to that point,
to the bulk of the trials and not published. What are the excuses they have for not forcing trials?
Well, it's allowed if it's allowed, it's permitted. So you can get away with it.
I mean. Well, let's be clear. You have a drug, you're selling a drug.
You have a science department and a marketing department. A marketing department saying we've got $44 billion of sales.
And you're like, Yeah, but I'm fat on half these trials, which we want to publish that show with no effect.
That's publication bias.
Why would you then incentivise everybody to make that available if there's no regulations or laws that make you have to do that?
The FDA in 2007 brought in an FDA regulation that says for all drug trials they've come through,
FDA regulation is supposed to publish a summary of results within one year of the trial closing,
and people continually audit that and say nobody does it and not a single person has yet been fined.
So there's no regulation. Basically, there's no regulation, and that's part of it.
Last two weeks ago, the European Medicines Agency ruled that all out of 2016, in general,
any drug that comes to them, they are going to make the full clinical study report fully available on their website.
That's just all we do.
I mean, simple project I'm involved with with Alex Thomson in our department at the moment we're auditing the University of Oxford,
be our season, be all youth and saying because why a patient asked us to do it.
The patient representative said, I go to all trials and I'm really interested.
I'm on the board of the BRC and I quite like to know whether they're publishing their child or not.
And we thank you very much. Can't get the clinicians to say it. They all now have to do it.
It's been fantastic. We've now identified 115 trials.
We're going to audit them. Anybody can do that. This takes time, doesn't it?
But it's been great that a patient came because they force the issue, whereas everybody else is.
We don't quite don't want you to know what's going on, to be honest.
It's in the public good, isn't it, to make this stuff available? It's not there yet.
One more form and I'll let you go for a drink.
You think with the older drugs you know where normal practice is based on some it's one of the we've moved into midfielder I think that's available.
Well there should be because the aspirins are very sobering story because the aspirin,
if you make the individual data available which had been made available, if you go beyond 2009,
what happened is that the chap here called Peter O'Farrell is a neurologist who saw that individual data and was looking at it and said,
well, that missed something here.
And what that missed is the fact that aspirin might have a massive role in the prevention of cancer, particularly metastases.
And if aspirin came along now, they probably looking at it in cancer and not in heart attack.
And that actually it may be a really important drug in prolonging life in lots of cancers.
And that was because the data was made available. So there's probably lots of drugs out there that people could start to look at.
And we've got one antibiotics is one indication we're running out of antibiotics.
What if you made all the data available? You might say, well, we relooked at some of these antibiotics.
Actually, some of these may be useful in over here.
So, in fact, even from a beneficial point of view, we have to do it more so than the harms in some ways.
And if we don't do it, yeah,
my concern is we could spend 20 years here and actually clinical medicine has bottomed out and actually could start going the other way.
We'll get actually increase morbidity, reduce reduce mortality, life expectancy or shorten we're getting fatter, we exercise less.
Our children may not live along with the people in this room. And then the great thing is though, like the weight watch is perfect.
You could have a drug and an eye.
When we had a few weeks ago the overdiagnosis conference something else we got involved
in interesting the idea of overdiagnosis that actually you can find a disease
and diagnosed with the disease and diagnose more people with the disease but have no
effect whatsoever forever on the mortality or the morbidity is really interesting.
And there are also the TV that appearing like pre-diabetes which is superb for if you in the manufacturing pharmaceutical.
So yeah that's next on the agenda that we said is part of the fight is to say right you've made everything from 2016 available.
We now need to go back so that many of you people could say, I'm interested in I might look at this.
And then we need lots more people who are prepared to do the work. Okay.
All right. That's 7:00. You can have a drink. Thank you very much.