Over to you, Gordon. Thank you. Okay, great. Thanks very much.
So, people, it sounds like in this audience know me by what I've written about evidence based medicine.
What you may not know about me is that the process started in when I took over
as director of the Internal Medicine Residency Programme at McMaster in 1990.
And that's where my story of today will start.
And I still do clinical practise as a hospital based general internist, and I've just come off two weeks on clinical service, so I'm still at it.
So the topic is how should we teach evidence based medicine in the 21st century?
Conflict of interest. And you're going to hear about grade.
And I'm co-chair of the Great Working Group.
There may be something in this that indirectly refers to up to date.
Not directly, I don't think. But at any rate, I'm done a lot of consulting work with up to date the for any of you who want
to take a slower review of what I'm going to cover everything in this talk.
I think what works is in the article about understanding results, evidence, summaries,
applicability, not critical appraisal or the core skills of the medical curriculum.
That's the point. I'm going to try and make curry chicken and I published a paper making that case in BMJ IBN in 2020.
So in a relatively short period of time, I'm going to talk about critical appraisal before evidence based medicine, evidence based medicine.
At the start, it's every illusion and the implications for evidence based medicine education.
And I'm going to identify what I think are three challenges for evidence based medicine and education.
One, physicians must understand the results.
I'm convinced that that is the case. Conditions must understand grade.
I have confessed my conflict of interest as far as that is concerned.
But in a more general sense, perhaps they must understand the concept of quality and certainty of evidence and be able to apply it.
And thirdly, the underappreciated aspect of evidence based medicine,
even though it's more than 20 years since we've been highlighting the DBM, is all about patient values and preferences.
Some folks still don't get it and challenge three completely consistent.
In fact, at the heart of the beam is that we must teach physicians to understand the importance of patient values and preferences.
So this is Dave Sackett.
He was my mentor and in 1981 he published a very influential series that looked at critical appraisal of the medical literature,
which started with an article about how to read clinical journals, why to read them, and how to start reading the critically.
And this is what he started out a little later.
I'm not sure he would have well, I'm sure he wouldn't have agreed necessarily to all of this.
But he said, look at the title. Interesting or useful? That's probably a good one.
Review the authors. Good track record. That's probably not such a good one.
Read the summary. Evaluate with these results. Be useful.
Pretty good. Considered the site valid with these results apply to your practise?
Maybe. But then what was the focus after that?
Well, if it was a diagnostic test, look,
whether there was an independent blind comparison to learn about the prognosis was there in a sectional cohort,
aetiology or causation, which we've sort of abandoned as just the specific character characterisation, what are the basic methods?
And to distinguish useful from useless or even harmful therapy?
Was the assignment of patients really randomised risk of bias criteria?
So the focus was risk of bias criteria and then if the initial was met, read the patients and methods section.
Nothing here about the results.
The focus is on what we think of as risk of bias.
So that was critical appraisal. And then in 1990, when I took over the residency programme in Internal Medicine,
I had a vision of what I wanted to teach people and I thought, I need to call this something.
Come to McMaster to learn a particular approach to medicine, which is evidence based medicine.
And this is the first article that appeared about with using the term evidence based medicine in the medical literature in 1991.
And just to tell you, that's what I looked like in 1991 when that was published.
Sadly, very different from how I look now.
So I started this internal medicine residency programme and it was essentially a laboratory for IBM education.
And my initial goal was every graduate would be able to read the methods and results.
I was trained by Dave Sackett and I was very familiar with the initial reader's guides to the medical literature.
I was aware of the importance of what we then called validity, which we now call risk of bias.
However, at the end of seven years running this programme, I had observed some distressing findings.
Very few of the residents, despite my best efforts who finished my programme,
were able to appraise the methods and concluded that the advice in the CMAG series that they've led,
while a brilliant innovation at the time and crucial to the development of what eventually
became evidence based medicine in its focus on validity or risk of bias was misguided.
And so we wanted to get the word out.
So I finished the resume, my residency programme directorship in 1997 and in 2007, sorry.
In 2000 we published this article in the BMJ, which made the case that critical appraisal was not for everybody.
In fact, critical appraisal was for very few, but it was not realistic to ask physicians to find and appraise the results.
Learning even skills, if by what even skills one means critical appraisal and risk of bias is not the best way to achieve evidence based practise.
You will have the training. Those who do, including me, often won't have the time.
I still high, as you see, will do often what other clinicians do, which is not to look carefully at the literature myself,
or even critically appraise the literature summaries at the time.
In 2000, when we wrote this,
what was our alternative to learning Hebrew skills and being able to critical appraise it was pre appraised systematic reviews.
Well, maybe. But 22 years later, we actually make another suggestion about what the alternative is.
So the implications for education is that the goal is not critical appraisal of the primary studies,
nor is critical appraisal criteria for systematic reviews.
The structured clinical question. The comprehensive search. And so.
That's not the business of the clinicians either. We still teach it.
I still teach it, but I don't teach it with the expectation that anybody's going to remember any of those details.
And if you try it, I would predict that after a little while,
when you ask your your clinicians who've been through one of your courses about what they remember about concealment of application,
blinding and loss, to follow up, it will be very little.
So I don't teach that so that they can do it themselves.
They're not going to that I can do it only so that they have an appreciation that there is a process that is legitimate,
that is logical, that is trustworthy.
There is a process that allows the identification of high, moderate, low and very low quality evidence.
And so at the end, when somebody tells them this is high, moderate, low or very low quality evidence, they say, well, I can't do that myself.
But I remember that there is a structured and and trustworthy process of making these characterisations of the quality of the evidence.
But they still have to understand the results.
They have to understand the results because our interventions come with benefits and with burdens and harms.
And in particular, some of us believe that it should be the patient's values and preferences that bear on the issue.
And to do shared decision making, you need to understand.
So do clinicians currently understand the results?
We did a study in which we took a survey survey in eight countries of primary and internal medicine physicians in training who
came to their rounds and we presented them with six ways of presenting continuous variables which standardised mean difference.
Mean difference ratio means minimally important different units, relative and absolute effects.
So we start out continuous variables which can be appropriately modified to prevent present relative and absolute effects.
So they were attending teaching Secessions, Canada, Chile, Norway, Spain, Lebanon, US, Switzerland and Finland because we had a captive audience.
87% completed the survey.
So they were randomly assigned one of four versions of the questionnaire.
They either had small or large effects, and the six measures were either presented in order 1 to 6 or 6 to 1.
And then we tested their understanding of whether the effect was trivial, small, moderate or large.
Here was their understanding. Not surprisingly, we predicted this in advance.
Risk difference was the one they understood best. Relative risk.
Next. Next ratio of mean standardised mean difference that minimally important difference and the natural units of the instrument were less.
So that's what we anticipated. What we didn't anticipate is that even with risk difference, the one they did best, it was awful.
Only 40% of them got what we thought was the right answer in terms of the effect trivial, small, moderate or large.
This was the stri found this quite distressing and this hastened my I don't I was
already obviously since 2000 moving toward a focus on understanding results.
This shifted me further. This shifted me to say understanding results are what's important.
I should be spending even less time on the critical appraisal and more time helping clinicians learn to understand the results.
So. But. My experience today, we said it in 2000, is that most even teachers have not got the message.
They still think that clinicians are going to go out and do critical appraisal.
And our first challenge, I think my I perceive our first challenge is to get Hebrew teachers to accept reality.
Most of the nations will seldom or ever do critical appraisal.
I had. They will still be making decisions and hopefully doing that in.
The process of shared decision making with their patients.
They'll be counselling patients. And our studies suggest they can interpret the results to do that well.
So we need to get even teachers to understand the implications.
Markedly De-Emphasise critical appraisal markedly emphasise understanding the results.
When will we be successful? When we repeat that survey I told you about?
And at least with respect to relative risk and risk difference, maybe some of the others clinicians can actually understand the results.
That will be progress. So next thing I will tell you or remind you if you already know a little bit about grade.
So in 2004, we published a paper in the BMJ that presented this process of looking at quality of evidence and strength of recommendations.
And the missions now in my neck of the woods, actually worldwide,
I go worldwide and I say at least in academic centres who uses up to date and everybody extraordinary
number of people around the world are using up to date update has over 10,000 graded recommendations.
They present the quality of the evidence and the strength of recommendations.
A lot of people who use Uptodate don't understand, but up to date says at one or two C they don't understand what it means.
And they need to they need to understand that ABC is high, moderate and low quality evidence, and one and two is strong and weak recommendations.
So the next 2004,
we published the first presentation of grade in the BMJ and then we published the six part series about grade for clinicians if they see great use,
how to understand it. And we've been pretty successful.
So over 110 organisations have adopted grade include World Health Organisation, prominent American organisations, American Thoracic Society,
American College of Physicians, British NHS, Cochrane Collaboration and both the two leading electronic both up to date and that.
So what are grades, criteria, confidence or certainty or quality of evidence?
They're all synonyms. Can be high, moderate, low and very low.
Randomised trials start as high, but they may be rated down by risk of bias, inconsistency.
Indirectness Imprecision and publication bias.
Observational studies start as low. They may go to very low because of those problems, or typically because of large effects.
Things like insulin for diabetic ketoacidosis or hip replacement or dialysis do
not need randomised trials to show to provide high quality evidence of benefit.
So challenge number two, way back in 2000, we said clinicians look at systematic reviews.
We don't tell clinicians to look at systematic reviews anymore.
We tell them to look at trustworthy guidelines and we don't need to tell them that is what they in fact are looking at.
Good guidelines present the quality of the evidence. Most good ones will be using grade.
So our success with Challenge two will be met when clinicians understand the quality or certainty of evidence synonyms,
and they understand the basics of the great approach.
Now, in terms of the third challenge, I'm going to tell you a story of a study done by PJ Devereux,
who is now leading the world in terms of perioperative medicine, who way back did a study looking at values and preferences.
He took patients with atrial fibrillation at high risk of stroke.
Anticoagulation lessens the risk of stroke and the risk at the cost of more GI bleeds.
He presented that without treatment, 100 patients will suffer 12 strokes, six major and six minor anticoagulation decreased strokes to four.
And asked how many bleeds would you accept in 100 patients and still be willing to administer or take anticoagulants?
And if we were doing this interactively, I would ask you, how many bleeds would you accept to prevent eight strokes in 100 patients?
Five or fewer? 6 to 10, 11 to 15, six or 20 or more than 20.
Think for just a few seconds.
If you are finishing or even if you're not a clinician, how many bleeds would you be willing to accept to prevent eight strokes?
Here's what happens when Dr. Devereux asks physicians and patients with respect to the patient physician's flat distribution,
they went over right from. Very few under five.
Right up to a large number and across the whole range.
Physicians, values and preferences in terms of stroke or bleeding aversion or very variable.
The patients, the majority of patients overwhelmingly chose that they would be ready to accept 22 bleeds, two thirds of them to prevent eight strokes.
The patients were much more stroke averse and much less leaning averse than were clinicians, although a few of the patients were at the other end.
More like the clinicians. More bleeding averse messages is if you use the clinician's values and preferences, you're going to get it wrong,
at least in this case, if you're much better to use the values and preferences of the majority of patients.
But if you really want to get it right, you have to find out about individual patient values and preferences.
So great then says. And a strong recommendation would be when all or almost half of fully informed patients would make that same choice.
Interaction with the patient strong recommendation.
You can just inform the patient's week you need to do shared decision making.
Sure decision making can be aided by decision aid, and you're only going to use quality of care criteria for strong recommendations.
So challenge number three Achieve shared decision making when appropriate.
Clinicians and guideline developers need to understand two situations.
Just do it. The strong recommendations, weak recommendations,
or conditional that are value in preference sensitive success will be achieved when clinicians do shared decision making.
When appropriate, they understand the great, strong and weak recommendation.
And when every guideline tells you what their underlying values and preferences are that drive the direction and strength of their recommendation.
Thus, three challenges to teaching evidence based medicine.
First, get clinicians to understand the results, how to achieve it, teach guidelines, grades and results, not critical appraisal.
We will be successful when clinicians understand the guidelines that they are using.
Clinicians all over the world rely on guidelines in ways totally different than they did 20 years ago.
But to understand guidelines, they need to understand the concepts of quality of evidence and strength of recommendations.
We will be successful when clinical and guideline developers understand values and preferences,
and the clinicians who use those guidelines will understand it and applied in shared decision making.
So thanks very much for listening.