Hi, everybody. Well, my name is Carl Hannigan.
I know some people and I know some of you are on the diagnostic course and some of you have just walked in off the street.
Is that correct? Yeah. Are you in the right place? Yes.
Alright.
I've been doing diagnostic work for about 20 years ago and I put this title out and then you said on the Twittersphere and somebody last week went,
Oh, why do we need to talk about diagnostic reasoning?
We thought it all that out in the nineties, didn't we? And at that point, oh my gosh, that's really interesting.
Somebody believes that most of diagnostic reading is sorted in the NHS.
I very rarely meet somebody in clinical practice who would understand what the [INAUDIBLE] they're doing.
All right. And I'm going to explain to you why. All right.
I've got loads of examples, but I'm only going to give you a few.
I was in New York when I got an invite to an overdiagnosis conference and they said, We want you to do the keynote speech in Barcelona.
Oh, that's really neat. And I often thought when you were in New York, you know, the signposts were caution.
It was somewhat to do with the traffic.
But actually what it was really to do was, is this van, I think, where you can have a CT scan on the side of the road.
And if you go in New York, you can have the road to early detection and you can pay on the side of the road and get a CT scan.
Well, good idea. Scan the brain for free or for a few hundred dollars tomorrow.
And on the way home I started to realise all this stuff is emerging everywhere.
You get these adverts all the time on June £20 you can have a genetic test.
In fact, only a few weeks ago you said the NHS is going to do genetic tests for everybody and you're going to pay us.
Fantastic. And then when I got home I started to realise I'm at the right age.
So there are some people in the room who are a bit older than me and the young people.
This doesn't apply to you yet, but when you hear about 40, you start to get letters in the post.
Don't you like this? An urgent message about your eye examination.
It's two years since your last examination, and I have a family history of glaucoma.
Anybody else having family history? What does that entitles you to know?
ENTITLES you to a free annual NHS check? Does anybody realise that spectators is part of the NHS, the NHS organisation?
And I go in there and they send me letters like this.
You must come now. It's urgent and so everybody is over.
Once you hit 40, if you've got a family history of glaucoma, you will get an invite annually to come for an eye test.
Now, as Professor of VBM, what would you expect? I'll just go along and do my test or think I ask questions about is that appropriate?
Okay, now here's the evidence on this.
This is the health technology assessment for the cost effectiveness of screening for open angle glaucoma.
And here's what it says. So there are only two trials on this front.
What is it you think for a problem where you're going to go blind?
This would be a really important issue.
And these words I put in green because I think extrapolating and assuming can be translated in to we're making it up here.
So if you if you are the people at risk and you have a positive screening test, then basically you're going to be committed to the lifelong treatment.
And we think if you do that, on average, you'll go blind less by about ten years, 23, 25, as opposed to 35.
But look, prevalence would have to be about 3 to 4% in 40 rolls.
And if it was a screening interval of ten years, it would approach cost effectiveness.
So if I go at 40 and my risk is 3 to 4%, then it should have me test at 40 and 50.
I should get me in five, but then it would be cost effective.
But what it says not effective. So I think that's really interesting.
So we put in so if you go to the US Preventative Services Task Force, the people in America who do the guidelines,
this is what they say they have not recommended for or against routine glaucoma screening for adults based on the evidence that would make sense.
Anybody like to disagree with them? Anybody work for NHS choices in the room?
Because if you do, that's what it says.
It is important to have regular eye tests. So.
We in the NHS, when people say to me we need, we need more funds.
The first thing I say is more is not necessarily better. It doesn't translate into more health.
And what we mean by cost effective is do the benefits outweigh the harm?
Because if you pick up a disease early, if it's a false positive, you're committing somebody to 30 years of treatment.
But something that would not of they derived any outcomes whatsoever.
So you introduce you know, you have to think about the issues of medicalizing somebody for 30 plus years.
You walk in on the street, you're 40 years old. And some somebody goes, you have an eye problem.
That means you're going to go blind. Unless that's not true.
And so these statements are unhelpful and incorrect.
Put this on NHS charities.
So well, the thing is, if you look at the diagnostic evidence, okay, you have two terms, your sensitivity, annual specificity.
Yep. And all the people on the course have got them committed in their mind.
So by Thursday, when I come back to you, you're going to be like this.
The proportion of people with disease who test positive is the sensitivity.
And you want that to be really high in the proportion of people without the disease who test negative.
You also want that to be really high. So the perfect test has 100% sensitivity and 100% specificity, intraocular pressure.
That's when they put some in and test the pressure in your eyeball.
And if it go though, you've got you're at risk of glaucoma. If it gets too high, then you do the treatment.
So this is above a certain pressure. Okay.
So we go back to the 5% prevalence.
I'm going to walk you through this. This is a bit now where you go into a clinical session.
I've done this with radiologist, consultant, the lot that completely lost.
What this completely underpins your rational thinking. And if you assume a 5% prevalence, so somebody's at risk.
So if we assume as a 40 year old with a family history, you have about 5% chance of having glaucoma in your lifetime.
Okay. The first thing you can do is work out some very simple things like intifadas and people,
50 people will have the disease and 950 will not approve them.
Very simple. Now you can start to translate from very simple pieces of information.
If you take the sensitivity to be 90% okay, sensitivity is a proportion of people with the disease who test positive.
How many people? Well done, Cooper.
Well, it's not that difficult. Right. So it's pretty good, actually, if 50 people with the disease will test, but then 45 will test positive.
Well, and we also look at the specificity.
Amazing, 99%. So you also can say of the people without the disease, how many of them will test positive?
Well, actually, you can get this phone number. Okay.
And that's sort of like if you do the number, the math for it, that specificity is 90%.
It's not 99, 90%. But let me just tell you, if you do the best positive test result, 140 people will test positive, of which only 45 have the disease.
Yeah, that's the best result. So of everything, if you go to Specsavers have a positive test.
There's only a warning for each month of you actually having the disease empty.
All of the people will have a job for the next 30 years as a false positive.
But that's the best ever it can be that taking the best research, the biased of if you look at the stuff when you go for the worst case scenario.
Yeah. This is what it looks like for the worst case scenario.
Yeah. If you look at the research.
One in 30 chance of you actually having the disease. And so we accept this really poor quality research all the time.
And to me is understanding the reasoning suggests we have a problem.
The studies are no good.
Even if you take the best case scenario, you've got a one in three chance of it being correct and a two in three chance of it being incorrect.
So for every three people diagnosed, two people will be false positive.
Happy with that? If you assume and take the best possible study result, which that's what you do,
critical appraisal and you say there's a range of studies, so maybe we'll take the midpoint, you have to make some decision.
But if you look at the worth type of results, oh my God, he committed to 30,
40 years of treatment and which 97 people out of 100 won't have the disease and only free will and you'll never know the answer.
Nobody will know the answer because we don't do the research to follow people.
So it's not surprising. But this is why we do it, because you get these personal accounts all the time.
When I got my test just in time and I got the treatment and therefore I've been saved, you know, and that's the sort of where health care is.
So if you were me, what would you do? Would you go for your annual screening test if you knew the information like that?
What would your decision be? Would you go?
Never again. We every five years.
Would you go every ten years? Come on. This is evidence based practice.
What you. Egan, advise me based on the evidence, what the USPSTF said, what the evidence shows you.
We can't in practice. Evidence based practice. Go. I know if you want to go to Specsavers.
But do you mind if we do a study before we come? No, I'm telling you, I'm looking for you based on this information.
Well, for showing you that the best evidence assumes if I'm treated and I have to make it up because the trial is only 2 to 3 years,
if it's somewhere between 25 and 35 years in the delay in getting glaucoma.
So it's not it's not yes or no. It's just a delay because the pressures will go up and up and you delay in that process.
However, if I assume the best test, then there's a two in three chance, some false positive.
If I get a certain positive diagnosis, the US Preventive Task Force says Don't bother.
NHS says go regular and it looks like cost effective.
Says it's about ten years. If I have a four or 5% risk.
So what we're going to do, you might say I'll go ten you.
I've had my one test at 40, I've now been at 50.
So it's the same question you could ask yourself the evidence based question of going, What should I do in a screening world?
But what I'm putting to you guys right now is how little people ask these sorts of questions in health care,
whether it's people in the NHS, it's commissioners, whether it's doctors, clinicians, just go,
I'm going to go and have my test because I got a letter without understanding what the potential benefit and harms are.
Now when you think about it, this not everywhere is when I got another appointment invitation.
This is to a health screening clinic at the Kissan Stadium.
Wonderful. Look at this. Why have I been invited? We write to a small number of people.
How lucky am I? Yeah. We encourage you to compare the content and quality of our screening with BUPA.
Our screening is £129 cheap with chips.
Okay. However, if you go for the BUPA health check that includes all these essential health checks, it would be £424.
So this stuff is invading everywhere we go.
But going back some time, this is a paper which involved Brian Haines and Dave Sackett, which I referred to a book in 1978, which basically said,
we knew if you go into industrial settings in the seventies and can flim and explore people's diagnoses of hypertension,
you definitely will increase their absenteeism. You will make sure they're more anxious and depressed and they have more psychological problems.
So we don't understand what is the important outcome.
We have to have these trade off. Do you really want to know at 40 you might just die at some point on the fact you will die at some point?
Or would you like me to say, actually, we can tell you exactly the day and what it is and what you have to reduce your risk by all the time.
And in doing that won't harm through. We introduce into. On it for me to think about that.
So here's another one. I don't know anybody's come across that anybody ever done any depression screening?
You've done it as part of your job or have you done it on yourself?
Yeah, well, I'd recommend this. We do these scores all the time.
Q 93 is a health questionnaire for depression.
And what happens is these measures turn out of measures in research, because there are nice way of saying we did a treatment and we did the Q nine.
They've got some face validity and we showed the difference between Treatment A and B and then we go, Oh, actually we use it as a diagnostic criteria.
So look what happened. So here's the nine question.
So I put these questions to myself, okay, so little interest on pleasure in doing things.
On several days I was feeling a bit unwell when I did this.
I had the flu, I was feeling a bit down and it was like the week after Christmas.
It didn't say when it was, you know. So I scored one feeling.
I scored for trouble. Falling asleep, staying asleep or sleeping too much.
Nearly every day I have problems falling asleep and a problem staying asleep and it pretty much is quite common and myself afraid,
feeling tired or having little energy. Anybody in the room feel a bit like that?
I feel like now more than half the days you can see poor appetite or overeating.
I overeat a lot in the evenings and I have a poor appetite in the mornings and I
have trouble concentrating and things such as reading the newspaper or watching TV.
If that was my children, they'd be every day because they're always on their iPhone.
And so my score was about 5 to 9. I have minimal symptoms, but I need support and education to call it worth returning one month.
Okay, I have a provisional diagnosis. I'm nearly another point.
I've got mine and depression now is you've mentioned it like repeat the test.
I might do it again today. The score might be four. So there's no measurement about variation.
On another day I could score ten. I suspect everybody in this room on a day, on a bad day could score ten.
You get a rejection from the BMJ. You're like a ten, right?
But it's interesting when you look at the evidence of the false positives and the false negatives and looking at the diagnosis.
Yeah. And what's really interesting,
if you up the biggest factor in these studies that says what makes a difference to you getting a diagnosis of depression or not.
Yeah. So the you've got depression, you get a diagnosis of depression or you haven't got depression, but you do.
Yeah, it's familiarity that basically the biggest thing you can do is if you turn
up at your general practitioner enough and so you're fed up with the world,
you will get a label of depression irrespective of whether you've got it or not, because at some point the variation will.
[INAUDIBLE] ask you some of these questions. Do you feel tired? Yes.
Do you feel like lacking concentration? Yes. That's why I'm here.
Are you having poor appetite? Of course I am. You have depression.
And the biggest factor is how many times you turn up. Isn't that interesting?
And so we have huge number of people who are mislabelled.
And that's why we end up with issues like this of doubling in a decade and features like this, the numbers going through the roof.
And then this is open prescribing of what some people are doing and this huge variation in treatment and diagnoses across the whole juncture,
fourfold variation sometimes in diagnoses and and huge variation in prescriptions of about three fold variation in prescriptions because nobody
understands variation how many times you think you'd have to repeat that measure for it to become stable and you understand what's going on.
Now, we understand that in blood pressure, because if you do self-monitoring,
the blood pressure monitoring, they tell you you have to do it for seven days.
And the blood pressure will vary. Even if you need to go to the toilet, it will go before it will go up by 20 millimetres of mercury.
If you have a cup of coffee, it will go up. Oh, we think varies.
And so sometimes we say ambulatory blood pressure because we don't understand variation.
We give you a label on a single measure of depression, which is not what we should be doing.
So there's a whole wave of research to start to look at this variation in practice.
And at one thing, to have a read of which is available online is this atlas of variation in NHS diagnostic services.
Unbelievable. The variation that exist. Cannot be explained by patient need, completely lacking evidence based guidelines.
This type of variation of all imaging tests and blood tests.
So one of the things I've had is as a student working with me, Jack O'Sullivan,
who basically has come into the field and we've been looking at these things about rising number of tests.
And this is what it looks like, the temporal trends in total testing from UK primary care.
So look, this is test ordered per 10,000 person years.
Yes. So this is 10,000 people for one year? Yep. In 2000, it was about 14 to 15000 tests.
So one and a half test per person. Everybody happy with them?
Some get more, some get less. But where it is now, 50,000 per 10,000 years.
So on average, everybody gets about five tests per year.
Now, I don't know what you think about that, but that's a huge increase, isn't it?
Yeah. If you think about somewhere like Oxford City, that's about 170,000 people.
So it's 50,000 times about 70.
Yeah.
And they can go look at the paper and it shows you some people are getting an inordinate amount of tests, but actually it's been a huge increase in.
That's less than full of tests. Some tests are much greater.
I'll show you something like cholesterol has gone up,
but there are all sorts of old tests like vitamin B12 have gone through the roof and even in children increasing.
So across the board, it's going up. Now, anybody who is looking at that would go bloody, oh,
you better be preparing for them in the estimated time that it takes the GP just to look at the tests at this
point we reckon should take them 2 hours a day just in responding to the test they now get in practice,
which is impossible on top of all the other work they get.
So basically a lot of this stuff is never getting looked at and here's what it looks like just for imaging.
I'm just going to focus on imaging. Okay. Need to finish imaging so you all have to think.
Yeah, I'm going to give you. So there's been about 116% increase in youth, about doubling.
Right. So here's equate. All right. You write these down.
But you have to you have to be honest. Okay. Because we'll come back to the answer.
So, look, the five tests, you got to pick one of five going to go five different shapes.
Right. So you want to commit to memory. I could have made you write it down, but I haven't got a probably so with no chance.
Alright. So I'm going to come back. So that's sheet one, one of them.
Is that right? This is test ordering for imaging in primary care.
So this is c.t brae lumbar spine mri, chest X-ray, pelvic ultrasound, lumbar spine X-ray.
Happy with that? So two X-rays, an ultrasound, a spine, MRI, and a brain.
I'm going to come back and then I'm going to have you, but it's going to let you see the lot and then walk with it.
That's number two. That's number three.
Just a gradual rise. And before how much change and number five goes up and jumps down.
Okay. Right. You ready? Right here.
Who's going to go? Who's got anything sensible to think? What counts for this range?
Anybody go, what's up? Just that you are sorry for that C.T. brain.
Goes up a lot. Goes up a lot. Stop.
And we know this is no artefact and then continue thriving. So there's something happening, isn't there?
This is probably the advent of city with a real change.
When money came in to the NHS in 2000, people parity then huge increase stabilisation, but then it's going again.
So this is the things we're looking into. This is more money coming in, this is probably normality and then this is possibly guidelines coming back.
What's that one? Do you think this is really just a general increase all the time, going from about 200 to about 400.
So about doubling in numbers. Yeah.
Well, don't just x ray. Very interesting. So who said chest X-ray photo?
So that gentleman in that lady there. What? What for? Accounting for that.
Keep going up. Quite.
Remember, this is from primary care test order. Now, if did change their guidelines, they said basically in terms of the cancer referral guidelines,
if you've got a couple for three weeks, you should send people for a chest X-ray.
No evidence for that being a benefit but nice guideline said around about here.
People for because we're concerned that you might have an underlying malignancy it's
not needed in the it's not needed in the diagnosis and treatment of pneumonia.
So what's the accounting for this general? Right. We know nicely, but it can't account for everything.
We don't know whether it's people coming out trying different and thinking because the thing is, if you have pneumonia.
Yeah. And community acquired pneumonia. The chest x ray should alter my management.
The only thing if I'm suspecting something more suspicious. And that's just it right about that one.
Not this one. Yeah, well done. But it's a lot cheaper than it.
What should it look like? It should look like that.
Shouldn't it be coming down?
We know that from about 2000 onwards, all the evidence is telling you that lumbar spine X-rays are pretty much a useless waste of time in backpay.
And if you're going for serious pathology, you wouldn't be referring for an X-ray, would you, in the spine.
So we can I can explain that. But I thought this would be the one that when we did it, it's going to come down here.
But actually, as you can see, it's not much changed. Okay, sharp, rise up and down.
You're running out of options now if you've remembered her pelvic ultrasound.
Huge increase. Yeah, huge increase.
And then some it's happened here where the guidelines will have changed.
Guidelines have a huge impact. And then this is the final one.
So there are multiple reasons,
but isn't it worth thinking about the consequences of what we do all the time in terms of the data on diagnostic testing?
What are we doing, what we achieving? One of the things I think I hope you can say is, oh my God, look at the amount of work we created.
Because every time you do a test, it's not just the test is the individual.
We have to communicate them results. Then they have to come in and have a discussion with you and then you have to do something further about it.
I mean, any business would be going, Oh my God, this is amazing amount of work.
We're going to expand primary care. We're going to be bigger and better.
We're going to sell you the world. But actually what's happened is we've had less GP's in less time and this is why we've got a real strain.
But I think this is lumbar spine is because we got direct access referral, so that's quite easy to explain,
but it does create a huge amount of work for people who are not really that thoughtful in terms of diagnostic reasoning.
So how did you do it? Interesting, isn't it?
I'm doing this with a group of radiologists who they're the people who look at all these tests.
Nobody got more than three out of five.
And it's quite interesting because they never understood in their own organisation we should look at what we're doing,
thinking about what what the test looked like, what's going up.
Because as we do more tests, what happens to the, the, the, the ability of the test to determine the positive predictive value?
What happens? It gets worse, performance gets worse because if you do more tests, the prevalence gets left,
doesn't it, that the prevalence gets less, it gets harder to rule in and rule out.
Disease gets much harder.
So we have a huge problem if you're doing more and this is what what really interests me and I think later on the way you do it,
you get something like this. Yeah, the incidence of thyroid cancer in the UK is going up and up and up and then mortality is more or less flatlining.
So we have situations where we pick it up more and more.
We're trying to say to people, stop sending people for ultrasound of the head and that and nothing is changing here.
So there's a huge business in health care, in diagnostic services and technology that will offer no benefits,
but will offer benefits to industry and the economy,
because selling people stuff that's of no benefit to them is really important for keeping the economy going.
And America is very good at that. But this this flat line is really interested.
And so looking at the. So if you look at the diagnostic atlas, if you go to the Atlas of Variation,
there currently is a 4.6 fold difference between clinical commissioning group in terms of the level of activity for CTE.
Yeah. Nobody has a clue what should be the optimal level.
Now in some parts of the world. Overuse is a big issue in America.
Last time I looked at this, it was about one in every five people have a c t on an annual basis, probably about one in every four now.
And that contributes about a radiation dose equivalent to about 1% of the cancer risk.
So it's a huge amount of radiation.
So not only that we talk about now trying to reduce the radiation doses about three fold variation in the radiation dose using CT scanning.
And you've got this whole body sky screening, as I showed you, where we started, where it was emerging.
And I think all of that's quite interesting. So I consider there are some options now for evidence based action.
Yeah, we need pathways for diagnostics.
It's not just you having a test. You need to know the benefits and harms.
And the results are very simple language that explains to you if you have this test, here's what it means to you.
And the follow on treatment is this. And I'm all right with people making decisions to have screening or not.
That's not what I'm here about. I'm here about saying that when you do that, you should be informed.
We need a much clearer understanding of the benefits and harms resulting from different rates of investigation.
What does it mean in these areas where you're having more or less tests?
What's the implications on the population? And I do think we've lost our way.
If you go back to the literature in the seventies and eighties, you'll see audits were published in The Lancet.
People would think they're actually doing a really good old. It was a good way of understanding what's happening in practice.
And we've lost our way with the ability to benchmark, look at practice and consider what it is we're doing and the implications.
And it's been downgraded of just somewhat. You do just put on your CV and once you've done it, that's it.
But actually auditing and benchmarking is really helpful for understanding what's going on in practice.
And then I'm going to finish just to say, thinking about what it is we communicate when we're thinking to the public about tests.
You see this a lot. You know, we're saving lives. Put it on your list.
The emotive messages that we put out. Yeah, that's how we sell.
We don't tell you, actually, is the numbers and the false positive and true and explain to you what's really going on.
And, you know, you can just have this emotive and I guess I was on the London Tube just recently
and this was my favourite I think is I can get my screening started at just £23.50.
It is that's on the London tube with the full set of tests that I can have today which
have no evidence underpinning the benefits of harm for me as an asymptomatic person.
And I think I'm going to finish here because I think when you next see an advert like this, the question you want to ask yourself is,
do I understand or have I got any possibility of understanding the benefit and the harm for me or the individual I give this test to?
Now, that should be mandatory in the training and in any organisation.
If we do this test, what does it mean? How does it change our management and all of that's missing now?
And the problem is in some organisations I go into, we have also lost our way because the health system won't back people up.
So they just say, I'm testing just in case. Well, if you're doing that, you don't need doctors.
You just test everybody to come through the door to screen a CT scan and you walk through the door you're in.
So I think I ask, why is it so challenging to get reasoning and diagnostics over to people?
I'm not sure. But it's one of these things in the advent of knowledge that we should have a revolution in understanding.
Otherwise, we are going to blow everybody's budget in health care really quickly, and we're already doing that.
Thank you very much.