We want to thank you both for coming.
I want to thank the guys and the team that advertised this week that I think will ensure that our advertising and it's
so it's great to see you already to extend the numbers and get 45 bookings and then rolling out to have extended to 100.
So I'm sure that's for the whole of the of the of the speakers that we've got tonight, which I'm very,
very pleased to be able to to announce that we've got three great speakers who all, I'm sure will introduce themselves as they go along.
On the topic of same sex ed and what's the book about them?
The Ghost of Christmas, Christmas, Present, Christmas, The Past and the Future.
So you look under the Christmas carols reviews tonight.
Without further ado, I will I will introduce Ian Chalmers is going to start us off on the clock.
Well, I'm not Scrooge least I hope I'm not Scrooge.
I never thought of myself that way. But I'm going to talk about the past quite a long way into the past.
In fact, as you see from the title of the first slide,
I'm going to cover extremely fast from the beginning of the 18th century, the beginning of the 21st century.
So let me just start off with a sort of personal declaration.
It is. Why is evidence synthesis to assess treatment effects so important?
It's because without it, people suffer and die unnecessarily.
Quite a long time ago, an article that I've written was published in the BMJ asking the questions What do I want from health research and researchers?
When I'm a patient and obviously I want the evidence that's taken into account when I'm being treated to to be relevant and reliable.
And for me, that will usually mean evidence from systematic reviews of research designed
to minimise the effects of biases of various kinds and of the play of chance.
In fact,
measures to reduce the effects of bias in the play of chance when assessing the effects of treatment has been a concern since at least since 1722.
None of you in this room will have seen this disease. Does anyone not know what this disease is?
Okay, David, this is smallpox, and none of you will have seen it.
It's a horrible condition, has a high mortality.
So in essence, the general practitioner in Halifax, a man called Nettleton, wrote to the secretary of the Royal Society,
James Durant, giving some data that they had on the mortality associated with naturally acquired smallpox.
And he because there was dispute about whether inoculation that's actually giving smallpox scabs as the inoculum was being talked about in those days.
He said, what we need is a comparison between the mortality associated with the inoculation and the mortality associated with the natural disease.
And he collected data locally in Yorkshire.
And what you'll see already is that that bottom line, the total, he's actually, in essence, done a crude meta analysis.
He's taken these different studies from different areas and found an average effect describing the the totality.
What he also said was that not a single one of the 60 or so I think it was, who had had inoculation, had died of smallpox.
And no one needs any statistical expertise to know that that's likely to be it's highly unlikely to be a chance difference.
So he, for a start recognised the needs for comparisons and sufficient numbers of observations.
However, Isaac Masi was someone who I think I gather is it's mainly for religious reasons,
didn't like the idea of man intruding in God's plans for people.
But he made a good point, he said. What you've got to do is to make sure it like is being compared with like in any comparison.
To form a just comparison and calculate, right? In this case, the circumstances of the patients must and ought to be as near as may be on a par.
So he recognised the needs for fair comparisons.
James Durin, Secretary of the Royal Society, recognised the need to keep systematic reviews up to date.
So he and 1724 published an update of net Nettleton Yorkshire data,
but added to IT data that he had received from Chichester and Haverfordwest and the in.
I think there were 80 cases who had been inoculated up from 60.
And there again, there was not one case of smallpox associated with the inoculated group.
Now, a couple of years later, another fellow of the Royal Society, his ex, certainly His Excellency, custom ATA ambassador from Tripoli.
He produced some evidence from North Africa.
And what he said was out of 100 persons inoculated not to die.
In other words,
not not even as much as two people die out of 100 persons that are infected with the smallpox in the natural way that I commonly about 30.
So even higher mortality rate than had been shown in the British examples.
And he pointed out that inoculation is generally practised not only by the inhabitants of the times, but also by the Wild Arabs.
Now, how one takes that into account in an updated systematic review, one doesn't quite know,
but it certainly would want to be mentioned because it refers to a different part of the world and that might be relevant to know that.
So. Choice.
John Gasp A chaser actually updated James during this analysis after James Jones had died again.
1729 And you see the rather nice table he's produced there both for the this is for between 1721 and 1728, both for associated with natural smallpox.
And he shows you the proportion there, the poorer proportion, his calculated and then the those that have it by inoculation.
So this document issued June last year, one can only say about it is better 289 years late than never.
It's not just the Royal Society that, as far as I know, has not shown much of an interest in this issue at all.
Certainly, I'm confident in saying that the Academy of Medical Sciences has not only not shown any interest,
it has actually blocked attempts to generate interest within that organisation.
You would have thought it was a rather straightforward issue.
But clearly some people have felt challenged, let's put it that way, by the development of methodology for this type of research.
What about pioneering the development and reporting of methods used for research syntheses?
Well, here we can bring in this Scottish navel surgeon, James Lind,
who's usually known for the controlled trial that he did aboard a ship called HMS Salisbury,
comparing six different ways of treating scurvy and demonstrating really beyond any reasonable doubt that oranges,
lemons were a rather good idea compared with the other things that were being used.
For example, the College of Physicians favourite approach was sulphuric acid.
So there were a number of steps that he took to try and do a decent job of reviewing what was known.
And you'll see that on the title page of his book,
he mentions that it includes a critical and chronological view of what has been published on the subject.
He went on in the introduction to say, as it is, no easy matter to root out prejudices, biases.
It became requisite to exhibit a full and impartial view of what had hitherto been published on the scurvy.
And that in a chronological order by which the sources of these mistakes may be detected.
Indeed, before the subject could be set in the clear and proper light, it was necessary to remove a great deal of rubbish.
And this gives some idea of the care that he took to do this.
Now I've got a copy of a facsimile, in fact,
here of his book and mark the page where he describes the efforts that he went to to obtain the relevant information.
So you might like to pass that around during the course of the evening.
Now I'm going to jump forward quite a long way but hundred and 50 years because it was really.
A physicist. Who made it clear that scientists should accumulate scientifically because science is cumulative.
Sometimes there are big breakthroughs, in fact, where you ditch everything that has been thought of as important done then.
But more usually it's a cumulative process.
John Strutt. Lord Reilly was a physicist.
He discovered Argon and he was president of the Association for the Advancement of Science, and they had a two month meeting in Montreal in 1884.
Let's get proper attention to things. If you have met meetings that last couple of months and he said in his presidential address, if,
as is sometimes supposed science consisted in nothing but the laborious accumulation of facts, it would soon come to a standstill.
Crushed as it were, under its own weight.
The work which deserves, but I'm afraid, does not always receive the most credit is that in which discovery and explanation go hand in hand,
in which not only are new facts presented, but their relation to old ones is pointed out.
Austin, Bradford Hill. Probably Britain's greatest medical statistician approached it rather differently.
He said, What do readers want to know in the report of a study?
They want to know why. Why was it started? What did the authors do?
What answer did they get? And what does it mean?
And that implies that reports of new research should begin and end with systematic reviews of what is already known,
so that results are set in context. And failure to do it results in avoidable suffering and death.
So going back to this issue, we've spoken I've spoken about some of the biases that can enter in.
But what about reducing the player of chance?
Well, this is an early example published, as you see in 1904, which takes a number of studies of the effects of inoculation against time,
typhoid, enteric fever in the British army and calculates a mean value taken from the individual studies.
This one a little bit later on shows in the.
Data I outlined in red the cumulated results.
The metre analysed results of treatment serum treatment for lobar pneumonia in three separate hospitals in New York,
where alternation had been used to decide who was going to get the serum and who was not.
And as you will see, statistical estimates of variance start to get mentioned by then.
Toes. I've put Isabella Leach in there, not because she has, as far as I know, contributed to the ideas in that analysis.
But it seemed to me it was about time that a great woman and indeed she was featured in my slides.
This paper, which is available on the James Lynn Library,
is a beautiful description of the rationale for and the steps needed to be taken to do a decent job in systematic reviews.
Donald Reed used individual patient data from nine controlled trials to do a more powerful,
statistically powerful assessment of the role of anticoagulants in myocardial infarction.
Jan stands up in 1974.
He looked at controlled trials of adjuvant radiotherapy in women who had had surgery for breast cancer.
To the right of therapy. It was in general to the axilla.
And what you showed was that in the controlled trials that were done, these were all randomised trials.
There was a higher death rate in the women who had had the radiotherapy.
And this effect was particularly evident in women who had surgery for their left breast cancer.
Some of you may be thinking about how that could be.
I won't mention it. Some of you may have guessed already. Archie Cochrane said.
It's surely a great criticism of our profession that we haven't organised a critical summary.
That was the term that he used for systematic review adapted periodical periodically kept up to date of all relevant randomised trials.
The recognition came in the 1980s within medicine of the importance of global collaboration for evidence synthesis.
So you get this example published in the New England Journal of Medicine on the effects of adjuvant,
Tamoxifen and cytotoxic therapy for breast cancer.
This one published in the BMJ?
I think, yes, in the BMJ from a largely Oxford group on the effects of antiplatelet treatments with cardiovascular diseases.
And then this monstrous two volume book attempted to do a systematic review of basically the whole field of obstetrics and midwifery.
And perhaps what was new about this was all of the data were published electronically and have been updated ever since then, electronically.
Archie Cochrane very kindly gave that a fair wind.
He wrote in the forward that he thought it was good and that coming from him was a great compliment.
That's if you like. Pilot study led to the Cochrane Collaboration and the founding chair of the Cochrane
Collaboration was a predecessor of Karl's as director of the Centre for Evidence based Medicine.
He was the first director of the centre, in fact, and one should not assume that Cochrane reviews were particularly important in what
was being recognised by people who realised that this form of research was essential.
You'll see, sort of coursing along the bottom of these graphs is the numbers, numbers of Cochrane reviews.
Over time you'll see this growth in. There's an interest in these methods far beyond the Cochrane Collaboration.
I just want to give three examples for stopping of nice examples of the way systematic reviews can be developed and used for other purposes.
This one shows how, if you're trying to find out what might persuade teenagers to eat five fruits a day instead of Mars bars and whatever else,
then you can look and do a systematic review of the randomised trials that have been done to try and alter eating habits.
But you can also do a systematic review of the qualitative data that arises after
out of asking children themselves what they think might persuade them to change.
And you can look at these both together, and this is why I think this paper published in the BMJ was so beautiful because it actually
showed data from both different types of data coming from slightly different perspectives,
but addressing the same fundamental problem. I found that, in fact, you can use systematic reviews to detect plagiarism and fraud.
And this Croatian obstetrician, as in Korea, was demonstrated beyond any shadow of a doubt to have plagiarised other people and to.
Have been fraudulent as well. He has gone from strength to strength.
I might point out he now runs an extremely well-heeled operation in Dubrovnik and has paid no price for this at all.
This one is so important because it means it's a it's a really beautiful example of how you can address additional hypotheses.
To those that the studies were originally designed to address.
All of these people are basically cardiovascular disease epidemiologists and clinicians.
They thought that they might look at the to see because there was some evidence suggesting that
aspirin might actually have an impact on cancer incidence and cancer and cancer treatment.
And lo and behold,
they found that a daily aspirin source on massive numbers of daily aspirin reduced deaths due to several common cancers during and after the trials.
And it was it was about a 20% reduction. Big deal.
Then I'll end with two acknowledgements of the importance of the group based at the Centre for Evidence based Medicine.
This one is very important because and it may well be that that Carl refers to this, but I regard it as historically very important.
Indeed, it shows how one can address the worry of publication bias.
And lastly, this, I think is one of the most important articles written on this topic by Kim.
I'll be speaking in a moment. I think it's an extremely sensible injunction that all health researchers should
begin their training by preparing at least one systematic reviews when for review.
When I asked. My son, who works in education research to comment on a draft of this presentation, he said, is you can delete the word health.
All researchers should begin that training. Thank you very much indeed.
If it's all right, we'll take questions at the end. Yeah.
And then someone move straight on to, uh, let's call that again, talking to us about the presence of system activities.
Thank you. It's quite hard to define the president, isn't it?
So in turning the President, what I thought there is I just for how to go about it in mentioned a bit about the definition.
And I've been thinking about the definition quite a lot recently.
So I want you to keep in mind some things while I'm talking, the idea of a critical summer.
What do we mean by that? What did she mean by that? By speciality or subspecialty?
That's quite an interesting issue as well.
Now, in a world of ageing and multimorbidity adapted periodically, that's quite hard to do and all relevant randomised controlled trials.
So I'm in the present, I've been thinking a lot about that, so I thought I'd start about my journey.
So I'm going back is the present. I'm in 15 years, so I'm only going to jump 15 years.
Where is the jump to 150 years? And this is the first review I participated in.
So I started this journey about 2003 and published this with Paul Glazier, who many will know.
He's also a former director of the centre and Rafael PEREIRA.
Interesting to go back 15 years. I always thought, you know, it's interesting when you think about evidence and what's happening,
you think, well, surely this has been there for always and it's been there.
But actually this is at the period where a lot of the formulation of the ideas were coming in the first ten years.
Is that fair enough to say and look, the methods, that's what we did with no, no.
Right now, if you do a Cochrane review, you pretty much then said, these are my methods, here's my protocol.
And people said, get on with it. And people were using scales and the methods we use because that's how we approach the quality of trials.
And we sort of did it low, moderate and high risk.
That's pretty much what great looks like, isn't it, though,
when you think about grade and then bring in this on top structure actually was pretty well thought through and set out in the early days.
And I know I'm not sure if this helps you or hinders you,
but actually some of the structures are a bit more if you're not like prescriptive were easier to get your review through than this sort of approach.
Second as well, the basics of matter analysis of were and the systematic reviews were very clearly set out very early
on and they haven't changed very much within your single traditional review of an intervention,
a a statistical mechanism combining data with the forest plot, a type of effect model,
fixed or random heterogeneity assessment funnel plots, and then the interpretation with caution.
So that's what I'm calling the first generation of my review journey.
This is then the second generation of my review journey, which is expanding, realising that.
And we published something in Cochrane and this is about self-monitoring of oral anticoagulation.
But actually at the same time you could get things out there earlier in a journal.
And this is the same publication actually about a year early in The Lancet.
At the same time, in fact, actually previous predating Ian's Cochrane Review that I showed you in the BMJ publication in 2009,
we published this review on Tamiflu on the basis of the swine flu epidemic.
Remember when everybody was going to die? And actually, this is our first attempt at a rapid review.
And we did this in about 6 to 8 weeks.
And because of the emergence of the hotline and the need to produce evidence quickly,
the BMJ really got its act together and published this within about four weeks.
So from start to finish, we did this in about 12 weeks to journal publication.
So in fact, rapid reviews are possible and we did one really early on.
I actually have never articulated the story of why that happened, but it was cause it was a a policy priority also in my journey.
We started to apply it in touched on this that you could apply systematic review methods to any type of questions if it's appropriate.
And this was a visitor that had come to see me and Rafael.
And we've looked at the adequacy of reporting, monitoring regimes in risk factors for cardiovascular disease.
If you have hypertension, what do the guidelines say? And instead of looking at one guideline,
we looked at 114 guidelines and couldn't find a single one that agreed in a monitoring regimen of 114 different global guidelines.
And then finally, this one, as well as the idea that it was moving beyond just traditional treatment reviews,
that you could look at different questions. And this is a very influential one, which was quite interesting in the guideline in the health guidelines.
What it says is if you're assessing a child, it says under the age of two, he says a normal heart rate was above 120.
And then the difference, if you were over to the difference was then 20 beats per minute different.
So generally one day older, but the difference in your heart rate from normal to abnormal with 20 beats per minute.
Well, that can't be right. You can't jump 20 beats per minute in one day and then you go from abnormal to normal.
And as technology has emerged that we're better at measuring pulse, there was more of a need to go back and look at the evidence.
And what they showed is we managed to produce with the systematic review evidence,
smooth curve of heart rate and of respiratory rate in terms of child childhood infection.
And so that's that was our second that's my second generation in the third generation.
Moving on is the complexity of what we did changed and of what I did.
Learning new skills meant you could move questions beyond.
And this one is actually a systematic review of oral anticoagulation that we done the review for.
But there was a need to do an individual patient data analysis, and that's a much more complex thing to do,
much more costly, and you have to have a real specific need of why you want to do it.
But it is very helpful.
So we could answer all sorts of questions like we could do tying to event analysis, which weren't possible with just the published aggregated data.
We could also look at number needed to treat year one, year to year three, year four in year five.
And what we showed is the longer you had the intervention, the better you number needed to treat looked.
So that was individual patient data.
So in this increasing complexity, Ian pointed to this what we started to do use different ranges of techniques to answer questions.
So this is qualitative studies here, a systematic review, a diagnostic review.
I participated and here's another one looking at prognostic study.
So the range of study type was expanding in terms of what you could do.
And then finally, so this is what I call in my journey a fourth of first generation, second generation, third generation,
fifth generation was about looking at randomised controlled trials and thinking about single technology,
single intervention, comparing one with another, the second generation with actually expanding the ranges of study types.
And then in the third generation, thinking about the complexity of the type of reviews,
and then all the bets were off the table when we did this review.
Because it took us five years to get the data.
So in 2009, we published the Children's Review. I met Tom Jefferson for the first time in two nine because he was doing the adult review.
And the reason we got together is the BMJ sent me a paper to review, which was done by Tom Jefferson.
And it was the reanalysis of the Tamiflu data for the 2009 pandemic.
And I said, Well, we know the results. But I said, Oh, my God.
They found some incredibly important that eight of the ten trials were never published.
This is the story. Forget the results. There's some really odd going on here and I didn't realise that.
We didn't realise at the time we publish never even occurred to me that actually there was a huge issue of what we call under the waterline.
There's all this evidence that exists that is completely missing.
And that's been a very interesting journey. It's took is five years to get the data.
And it changed my approach to how we can trust the evidence in publications.
And then I'm just going to wind up with a couple of things that also made me think about the president.
This was an article by Richard Smith, which was the Corporate Cocoon Collaboration at 20.
It's published as a blog, but in their John Brodie we worked with.
He was very forthright. People who've met John Brodie will find him opinionated, but he is a genius.
He runs the trip database and in there they have a question and answering service
for which there's collected thick files and questions from stakeholders,
from professionals. There are questions.
And in this article he says out of 358 questions asked in dermatology, only three could be answered by a single systematic review.
So that makes you think about the complexity of the questions that we ask and with the evidence to date them questions.
It's quite an interesting issue.
So based on that, I'd been going back and forward with John quite a bit and asking him questions about how do we meet the agenda of Archie Cochrane?
Because that's what he said to me. How do you meet this agenda? So there are some important issues.
Periodically updating some people, including Cochrane,
have decided that periodic updating is too difficult to achieve because as you grow the number of reviews, how do you keep them all today?
And actually what they've gone for is a prioritisation strategy.
What about the range of questions and different study types I've showed you?
That's a problem also. And then we had in the clinical study reports, well, you can't do that or individual patient data for all reviews.
So here's where we are in our thinking.
Meeting the two agendas is that actually we could do a lot more in the Rapid Review arena, but use rapid reviews to update regular,
respond to new emerging evidence support practice, support policy, keep existing reviews of today and automate the components.
But I think there needs to be a middle bit, which I call the stakeholder prioritisation, where patients and people who are invested in the answer,
like clinicians, prioritise whether you do a more thorough job because we just can't do all of the work on all of these reviews.
So for instance, if the Rapid Review identifies there are two or three trials due to publish, why would you do the full systematic review?
What happens if you find that the rapid review actually makes no difference?
Why do all that work? If you answer this, will it make a difference to you?
Why bother? In the robust review? They're much more labour intensive.
They deal with much more data. There are 170,000 pages on our Cochrane Review in the in the CFR.
It's going to take you two years to do and analyse that data.
They should address publication and reporting bias, seek out important patient outcomes and analyse harms.
Most of the problems that exist in the current evidence base that we seek to have problems such as cholesterol lowering are
because of the problems with analysing harm and also they should seek to go beyond the globalisation using non-English studies.
So they should not just rely on English language. And so I put that into the present is about four things.
It's about how do we deal with reviews of single technologies.
I think if you started on your systematic review journal where you start, if I would start with doing a review in that area,
but then there's a movement to different study types which then go actually
the range of reviews that required ones to study go beyond randomised trials,
which differs from March's original definition.
I do think we need to then have a process for integrating complex reviews, but that can't be for everything and that's where the rapid both reviews.
And then the fourth is the sort of beyond journal approach, beyond the journal publication.
And we see this increasingly that evidence is not in journals that we require.
So before we move on to Kamaal, I want you to just share a story of what next?
About a year ago, I got approached by a lady called Marie Lyon,
who worked for the Association for the Childhood Health Damaged by Hormone Pregnancy Test.
Can't remember the acronym, the Expert Working Group for the MHRA.
It spent two years looking at hormone pregnancy test subjects called primitive,
and they come to a conclusion that there was no there was no association or causation between hormone pregnancy tests and congenital malformations,
and that they looked into the evidence and they come to the conclusion that it's very poor quality.
And when they looked at it, there's no reason they can find any association.
We got contacted by the patient group. Amory. He said, Could you look at this and tell us if the report is true?
Then we will go away and put the case to rest. These treatments we used 40 years ago, they were a pregnancy test.
Now you take two tablets of a pill of which one of the components contain progestogen.
That was 40 times more stronger than the contraceptive pill you take today, ten times stronger than their morning after pill.
It lasted for about 15 years and to me it's one of the stupidest ideas I've ever seen.
And there are loads of issues in the BMJ and publications where they said this is a silly idea
and we pick picking up a bit like Ian was showing lots of cases of congenital malformations.
You may have picked this up in the perinatal some time in the literature and most of
the arguments were about single studies that were either positive or not positive.
Immediately I read the report. I said they failed to do one thing.
Which I found incredibly odd. They failed to do a systematic review.
A government organisation in 2017.
Answering a question without doing a systematic review seemed autumn madness to me.
So what did we do? We did the systematic review. We published a systematic review.
Just been indexed on Pokemon. You can look at it, but interestingly, it doesn't make a difference for that.
Suspending this email to to my collaborators.
When you think about making a difference. The Department of Health and Social Care provided the following answer to your written
parliamentary question to ask the Secretary of State for Health and Social Care if it will
make an assessment of the metrics in the conclusions of the 2018 Report by Professor Karl
Honeywood on oral hormone pregnancy tests and the risk of congenital malformations.
Not only we ask the humans medicine is convening a new expert group to conduct an independent scientific review of our publication.
So now that we conduct a review of our systematic review, and they're also,
in addition asking the European Medicines Agency to actually conduct a review of our review.
The reason I want to share that is because I think in this modern era,
it's obvious that regulators should be doing systematic reviews on important questions like this are still failing to do that.
So I think also in what next? There's not only that, there's a sort of need to really hit home.
So it's interesting. I haven't actually seen the Academy of Medical Sciences policy paper.
I think actually I'm going to put as well I didn't realise it,
but I'm going to wonder what makes there's a real problem in regulatory science, not in our regulators, not using systematic reviews.
So we are going to have to point that out, the absurdity of and so I think I'm going to finish at that point to say systematic
reviews are just as important as they ever were and there's still a lot more work to do.
Hand over to my colleague at that point, Monaco. Okay.
So I'm going to chat for about 50 minutes or so in the future for this question.
There's a lot I can cover, but in 15 minutes I just want to focus on three things.
So as Carl said, I'm a commodity and I'm a GP.
I work in East Oxford and I'm also director of an interim master's program in systematic reviews.
So I may well be considered a bit biased about my love for this methodology, as would Ian and Carl.
But don't take it from me. Let's go to the very top. So this is Chris Whitty.
Chris Whitty is the UK chief scientific adviser for the Department of Health and Social Care.
He's also head of and I Chop. And a few years ago Chris Whitty published this paper.
What makes an academic paper useful to influence health policy?
And this is what he wrote way ahead of any other academic contribution to policymaking is rigorous and unbiased synthesis of current knowledge.
And he sort of threw down the gauntlet to everyone to see if the academic community
as a whole could do one thing to improve the pathway from research to policy.
It would be to improve the status, quality and availability of good synthesis.
So that's coming from the chief scientific adviser in the UK.
So it's not just us who's interested in this topic. Chris Whitty and his team as well.
So let me talk about three things that I feel around growing the field of evidence census.
I want to talk about embracing new methods and techniques, creating training opportunities and building capacity,
including what we're doing here in the CBN to start with new methods and techniques.
So why do we need new methods? Why do we need new techniques? There's a whole bunch of reasons.
I'm going to give you three and some of them have been alluded to already. So, number one, we've got this huge problem with publication bias.
I call talked about it already with regards to the Tamiflu review.
We know that a large proportion, estimated 50% of studies clinical trials are not published.
You're then going to do a systematic review of published studies.
How confident do you feel about the estimates that you come to knowing that a large proportion of data is missing?
So we've got this huge public problem about publication bias,
and it takes techniques like Carl alluded to with the Tamiflu review of spending years trying to get this data, these clinical study reports.
And we're doing a number of other reviews using this sort of data.
Real-World comparisons often. One of the challenges is that we usually trials where they're comparing a
treatment to a placebo or they're using outcomes which are surrogate outcomes.
And actually, what you really want to know in the real world is, is this treatment better than what this patient is currently being taken?
And also, is the outcome something that really matters to the patient rather than something that's a surrogate?
So we need better ways to synthesise data to try and get that sort of data out of it.
But find out on this list of three things complex, critical and policy.
So we no clinical questions. Clinical policy are getting more complex and therefore we need better techniques.
We need more advanced techniques to ask more than just does it work,
but actually does it work in the context and does it work in the contextualisation of other problems?
We've gotten issues like decision making. How do we make a decision with patients?
I'll give you an example. So here Will is a common headline from the BBC Health about our problem with obesity.
UK the most overweight country in Western. A Cochrane review.
This one, which is a pretty good review, will tell us that actually exercise and good diet will help you lose weight.
Right. And no one's update to that review because they're pretty confident about those results.
Let's take that into context. Now, this is deliberately small for a reason, as you'll see in a second.
So this is this is going to be a map of what contributes to the factors around obesity.
It's a very complex issue and came out with a nice diet and physical activity.
So those are the the components that we picked up in that review.
But actually look at the other factors that are thought to contribute to the problem of obesity.
So this is something from the full site report carried out in the mid 2000,
which mapped all the other issues around obesity ranging from physiology, physical activity, environment, as you see, social psychology and so on.
So we have to be able to synthesise a lot of this data and contextualise it to answer a very complex question.
You know, asking, you know, what? How are we going to, you know, improve the management of obesity is a complex question, as you can see.
So we need better techniques to answer questions, complex issues like obesity, for example.
Okay. So as I'm alluding to, we're pretty good at being able to answer questions around, does something work?
We've got good techniques. They've been honed mostly over a number of years.
So pretty good to be able to answer that question. But what's happened is, as Ben alluded to already, we've got an emerging range of techniques now.
So we now can explore questions around how accurate is a test we can explore and we can do
scoping reviews to explore what sort of evidence is out there before we do a more formal,
perhaps a more robust review. We've got things like really sciences which will use them will theory based approach
to synthesising data and often from other sources that are published domains,
qualitative senses, for example, to explore experience of using a treatment.
And we've got rapid or restrictive reviews as well,
whereby if you need a data or an evidence synthesis more rapidly, for example, you can use those methods.
And we've got things like network match analysis whereby you can infer indirect comparisons between treatment A,
treatment B when they weren't otherwise compared to each other in the original trials.
So there's a whole host of new methods emerging. And if you are really, really interested in this in this work,
I would recommend having a look at this paper published a couple of years ago by Andrea Tirico and her team in Canada.
And what they do is they scope the literature to say what sort of new techniques in evidence synthesis are emerging.
And they concluded they found at least 25 unique emerging methods in the evidence in this field, and probably by this time, there's going to be more.
So this is an evolving field, and it's something that is worth paying attention to because as I said,
it's changing as a result of a number of other factors. Carl alluded to in his final just fine a slide about the fact that we need complex reviews.
This is something that is of interest to us because, you know, as I said, the data is getting more complex.
We need more sophisticated methods.
But actually, what we found when we looked into it, no one's actually identified or described what a complex review is.
So what we did last year was we had a go, so we published a paper in the BMJ, UBL, and we defined the complex review as such.
I'll just read it. So a systematic review performed by a multi-disciplinary team consisting of multiple components,
large amounts of data from different sources or different perspectives,
collectively contributing more than would be expected from the individual contributions,
the individual components not being easily coordinated, analysed or disentangled.
It's a starting point, but we couldn't believe that. No one's actually just describe what a complex review is.
So we thought we'll have a code and we'll start the debate. Okay.
So number two, the need to try to create more training opportunities.
I said I direct the masters and you can see all three from inside.
I'm a bit biased. I believe everyone, all health researchers, as Ian's alluding to an issue leading to all researchers,
should consider doing this, this factory as part of the first part of their training.
But it's not just me who thinks that this is the research arm of the NHS, the nature, the fund.
A lot of our research every year they put out a fellowship call for systematic
review training and these would be fellowships specifically for these reasons,
to address the current shortage of systematic reviews in the UK.
So we're talking about Chris Whitty saying this is the most important study designed to change policy,
but we don't have enough systematic reviews in the UK. So now I'll put out these sorts of fellowships.
We also, as many of you or some of you may know,
we also run a master's in evidence based health care and a natural offshoot from that a couple of years ago was a
dedicated MSC in systematic reviews and as part of one of the multiple people is specifically on complex reviews.
And as a as I said, as we say here, developing a better understanding of the broader forms of evidence.
And we had our first cohort of students on there last year.
We do one day workshops as well. The CPM got one coming up in March and I was interested as well.
And then finally,
I just thought and by telling you a little bit more about some of the work we're doing to build capacity beyond some of the things I've said already.
So if you haven't visited our website, you'll see. Please do CBN dot net.
You'll see that we do a lot of systematic reviews and in fact, we promise this is an exit from our research strategy.
So from that point of. Now.
Okay, so CPM research is a global source of high quality evidence on which clinical decisions can be readily based.
And we prioritise.
If you look here, the first thing we raise, we will generate and synthesise high quality evidence that benefits patients and society.
That's that is part of our primary, part of our vision.
And we do that across a number of areas prevention, diagnostics, communicating evidence methods, therapeutics and health services.
And as I said, we produce a lot of systematic reviews.
So a natural offshoot is that is I'm pleased to announce that we will now have a dedicated EPM synthesis unit as part of the CBN.
And here's some of our ongoing reviews. So we've got David Noonan leading a review on the role of physical activity in chronic disease.
We've got some we've got a review on the role of paramedics in primary care.
But as part of also doing reviews, we're also developing and pushing forward the tools available for others as well.
And one of the things that we've been working on is a toolkit and evidence of this toolkit,
because, as I said, with this growing number of review designs and methods and as I said,
there's more coming, how are you going to keep abreast with the how are we going to how are we going to be
able to know which which type of synthesis design will be for which sort of question?
So that's what we're doing. We're developing a toolkit. And we are describing each review type, describing what it's for,
how we use the mathematical framework and how you can apply it in your synthesis design.
So that's just a short summary of what we're doing around new methods, training opportunities and building capacity as well.
And I'll just leave you with this final quote from Ben Goldacre, who also works closely with us at the centre of the Space Medicine.
The notion of the systematic review,
looking at the totality of evidence is currently one of the most important innovations in medicine over the past 30 years.
Thank you.