I am thrilled to be here with you and to share with, you know,
for go and what we've been doing with no forgo in the last four maybe five years now in the hospital setting.
And I'm going to focus first on the hospital settings because that's where the
rubber meets the road and that's where the decision making really comes to a crux.
And then we'll talk about maybe how this can be applied regionally, even from the government perspective, which we're working on in Ontario.
So I'm from Canada, from Ontario, from the City of London, where we have a Piccadilly street in the Thames River,
and our university looks a little bit like Oxford in areas.
So it's quite nice to be there. And we're 2 hours from Toronto and 2 hours from Detroit.
So no go is a tool that we've developed over the last five years to improve decision making, and it's a passion of mine.
I took some time off about four years ago to do a master's in Health Technology Assessment and thought I would find
some answers there on how to make decision making better and was somewhat disappointed that it didn't get me there.
So I developed this thing called No. Four Go, and then our senior administration in the hospital were very kind,
very excited about it when they heard about it and have implemented it across the hospital.
And now there's interest from the region as well as from our Ministry of Health.
And I'll tell you why.
Now, I don't want to speak too highly about nor for go and give you the false impression that it's perfect because it's far from that.
And I really, really want to stress that if you have any feedback or thoughts about how it could be better,
I'm more than willing and open to hear that. And what we'll do first is a presentation.
I think in this format it will work best and then we'll have some time at the end for reflection,
comments, feedback, rebuttals, whatever you have, that's great.
I always learn a lot from my audiences and have presented this in a variety of places in Canada and around the world.
And because of that, it's been refined based on the differing perspectives and experiences that people have kindly shared.
So I do need to thank the people at my hospital and the people at CIA of the Canadian Agency for Research,
as well as our National Agency for Health Technology Assessment, called Karath for their support.
They have funded part of this, and then our High Impact Technology Evaluation Committee in the hospital and our EPCOR team,
which is our evidence based Perioperative Clinical Outcomes Research Group.
All right. So what are we going to do in the next few minutes? Well, we're going to talk again from the perspective of the hospital.
I'm going to talk about the importance of evidence based health technology assessment in the hospital setting.
And I'm going to start out by setting the context in what are the traditional challenges around decision making and why it's not working in general,
and then how we try to answer to those particular challenges using Novago as a solution and
what has been our experience and then future directions where we'd like to go from here.
So hospitals in Canada are costly. I know they are here too.
Hospitals account for about 30% of our overall health expenditures,
and then drugs beyond that also account for another about 17% of our total health care costs.
So a lot of people ask me, why would you do this? Why would you start this type of a project in the hospital setting?
Well, the answer is hospitals in Canada, in Ontario is really where a lot of the action is, and it is really where the rubber meets the road.
Technological innovations are often introduced first in the hospital setting.
That's where we have often the back door to new technologies entering the door at the hospital level.
And these technology technological innovations are restructuring health care in profoundly and somewhat unsettling ways.
The demand for innovative and new technologies, devices, drugs, procedures has clearly outpaced our capacity to address them in a systematic way.
And this is all concentrated very much at the hospital setting.
And it to me, it created the perfect petri dish just to implement a new tool or a new process
for decision making that tries to bring in evidence based decision making,
that tries to bring in health technology assessment formally, but that is also accountable to the resources available in Canada.
We have a multilayered set of programs in health technology assessment,
and I think it's quite similar to the U.K. in that we consider drugs and devices.
And machines more commonly than we consider procedures which may or may not involve devices, drugs or machines.
We have a national agency for Health Agency for Health Technology Assessment cameras, which does assessments and they make recommendations.
But these are in no way binding.
Same with at our provincial level, we have Otec, which is a health technology assessment committee, which makes recommendations.
But again, in no way are those recommendations binding.
So what happens in the hospital is that decisions are made and perhaps informed
by the health technology assessment that has gone on in the recommendations,
but perhaps not. So we needed something that addressed these gaps and the ability for things to walk in the back door.
I consult part time for the Canadian agency, as well as our provincial agencies for drug assessment and non-drug technologies assessment.
But when it really comes down to it, the decision making at the hospital level really still needs a lot of help, despite these external supports.
When a health technology assessment is completed,
we get a report that outlines the evidence as well as the expanded considerations such as ethics and feasibility, etc.
But what we found in the hospital setting is that really wasn't enough to make the decision for us.
External editors do beautiful health technology assessment reports, but in the end those reports do not tell us how to prioritise.
These reports need to be localised with considerations of infrastructure,
existing technologies that we've already got in practice, our patient population,
our internal needs, our external needs in the region, our health professionals skills that are on hand,
our learning curves, competing priorities, etc., etc.
I really like health technology assessment because in the hospital setting, again, because it's fast paced, it has to be real time.
There's not a lot of time to wait for assessments to become available.
Sometimes the train will already leave the station or the horse will be out of the barn.
If we wait for an assessment to be completed and in the hospital setting, we have our end users.
The very target of the recommendations are with us when we're putting together
recommendations and we can contextualise it to the local health setting.
And it also makes us entirely and extremely accountable to whatever predictions we put into
our recommendations or our models around what is going to be the resource requirement,
what likely benefits will accrue from implementing this new device or drug or other technology.
And every decision that we make in the hospital setting is felt poignantly.
It really is because we're working within a box.
We have four walls around us, we have a budget, and as soon as we make a decision for something, it means that we feel whatever we've given up.
And I'll tell you a little bit more about that. So I call it HD in a box, and it becomes something that we have to do truly, madly and very deeply.
It does go very deep because you're faced with the decision makers at the time that you're making decisions.
It's not always comfortable, that's for sure. And yes, it is where the rubber meets the road.
It's very easy to do health technology assessment when you're not the one who has to make the trade-off decisions.
These decisions become moral dilemmas. No matter how we make the decisions, somebody's going to get hurt, i.e. for every decision to do something.
It's a decision not to do something else, and you are going to hurt some part of the population.
The hope is that you can maximise the good that you've done for your population with the range of decisions you've made.
So in order to tell you what we're doing today, I need to tell you what we've done in the past ten years ago now.
We started a program called the Evidence based Prescribing Initiative in the hospital setting.
Senior leadership in the hospital recruited me to come in and start a program
that tried to get evidence into practice around the area of drug prescribing.
And that was a lot of fun.
At that time, we tried to get the evidence straight, but we had a lot of rebuttals and a lot of pushback and rebuttals, such as evidence is biased.
Full stop. I don't believe you. Numbers are tortured.
Statistics lie like evidence from evidence.
Let's just get on with it. I don't have time to wait for your assessment. It works and I want it.
There's no time for assessments here. And of course, the ubiquitous blind to the evidence.
That's not the way I see it. And the inherent new is always better if it's better.
It's the presumption that it must always be worth it.
Okay? Or that new must always be better. Therefore it's worth it.
And what we started to do over time was calling these particular rebuttals because they really were showstoppers to getting evidence into practice.
So we called them symptoms of compromise, decision making,
and the most common categories of rebuttals were the evidence doesn't apply to my patient, and my patient is just exceptional or unique.
You can't say when that particular evidence applies to my patient.
And what we found was that patients, according to the prescribing doctor or the treat the care provider, the exceptions were becoming the norm.
Nobody wanted to admit that their patients are actually normal.
Obviously the evidence is wrong because everyone else is still doing it was the other common rebuttal.
And we have examples here around which some of these rebuttals were launched.
And again, commonly this place is archaic.
It's only about the bottom line about cost, the dirty word cost.
Don't bring that into my decision making when I'm responsible for a patient here.
These are lives. And of course, we have this key challenge of too fast or too slow to adopt things we have in our hospital.
And I don't think that we're unique compared to the U.K. We have this thing called gizmo idolatry.
Technology is fun, it's shiny, it's new, comes with purported advantages compared to the old.
And they just want it. And they feel that sooner is always better than later.
And in the teaching centre attached to the universities, we have the responsibility to be leading edge.
So please just let us try everything out and be the leaders and such.
And I need to be able to teach my residents, my fellows, my guys about everything, about all the options out there.
And this leads to what we call the technology hype cycle, where we overshoot implementation of new things.
So we have this peak of inflated expectations. But then when we try it out and we see that it doesn't always work or in fact it caused some harms,
or we're not adept enough to actually take it up into practice,
then we fall into this trough of disillusionment before we figure out what the evidence really says about where it's true place in practice should be.
But the problem is here, if we have this peak of inflated expectations,
followed by a trough of disillusionment, we've got two major areas of inefficiency here.
If we've over taken up or under implemented any particular technology.
So what we wanted to do was come up with something that smooths this particular curve in order to get good things into practice sooner or earlier,
smoother without overshooting, but not have to be in this constant cycle of trying to always implement the new when it's not proven.
And we have multiple, multiple opposing agendas in our hospital setting, just as you do when you're setting high.
We have skills deficits, we have people that are territorial.
They want just to be able to practice their particular pet surgeries.
I don't care what the evidence says.
And we have a lot of different agendas that are covered politically in our governments as well as in our frontline practitioners.
And of course, the average man's judgement is so poor he runs a risk every time he uses it.
This this is true, but most of us are not aware of that.
So what can we do? We got a lot of evidence out there. We've got some good health economics out there.
How do we pull this all together? So when we tried to do this evidence based prescribing initiative, we really kind of felt deflated.
We went in with absolute excitement, passionate about getting evidence into practice, and we thought everyone would come to the party.
Well, they did, but only to try to poke holes in the whole process and to dig in their heels and to say, I'm sorry, but my patients are exceptional.
The exception is the norm. So we kind of felt like we're here on that curve in getting evidence into practice.
But rather than just throw out, throw in the towel and collectively throw up our hands and give up, we decided to have another go.
What we were doing at that time was pretty good.
I think considering it was ten years ago. I'm but not entirely unique.
What we did was synthesised systematically through systematic review,
the benefits versus risks for a new drug or health technology assessment or we used what others had done, if it was recent enough.
And then we combine that with our local resource considerations, synthesised it through formal economic analysis.
And you've been exposed to that today, I hear.
And in that synthesis, we made recommendations for our local setting, often through made analysis and then from a cost effectiveness analysis.
But after doing all of that and for many years, we found that systematic review does not equal the decision.
Economic analysis does not equal a decision.
Evidence as own is one consideration, but not the only consideration for decision making.
This was startling to us because the promise of evidence based medicine, as the title almost infers, is that evidence should get you there.
It should be fairly robust until you want to do an economic analysis.
I mean, that's, you know, even further along the path, it should tell you what to do, but it didn't quite get us there.
And we found that in all of our decisions that we were informing systematically by the evidence and through economic analysis,
we found that yes was always perceived to be better than a no, and there was this propensity to say yes.
And there's pressure for us as the community to say yes to everything, and that there should almost never be the opportunity to say no.
And it's unfair when you do. So they said. And so we moved on.
We said, okay, evidence is essential, but we need a better way.
Something's not working out here. So now we say evidence is this.
Unsure, but insufficient. And we developed this thing called AlphaGo.
And this was based on the premise that the application of evidence to decisions was too technical in and of itself.
It was too linear. It was too blunt in the decisions that it was trying to inform or make.
Evidence didn't come in the shape necessarily of our particular hospital.
So we said evidence from expanded domains of influence also needs to be brought into the each of the decisions that we make.
Evidence from these expanded domains of influence include social, legal, ethical, environmental, etc., types of considerations.
And we said we really need to somehow enumerate the upper opportunity costs.
We need to be making explicit what is the opportunity cost before decisions can be made comfortably.
And in doing so, we decided we have to be able to go where the evidence does not.
And we need to actually bring in the concept of sleepers.
And we we created this particular acronym because it's a good one.
It's memorable when we're sitting around the table trying to assess what are the expanded domains of influence on this particular decision.
The sleeper is a social, legal, ethical or considerations of equity, environmental, political and entrepreneurial or research and stickiness factors.
So all of these expanded domains of influence, we said, have to be explicitly considered before we can make the decision otherwise.
These very factors in and of themselves may trump the evidence prematurely.
A political factor may come in and just make the decision for you if you're not
systematic in actually getting these particular factors onto the decision making table.
The last one here, I think most of the categories speak for themselves.
When we talk about when we look at social considerations that are unique to our particular environment,
we look at legal considerations, we look at ethical considerations, and we look at equity across our hospital in our region.
Now we look at environmental and organisational factors, very important when you're talking about taking up a new surgery technique or a new device.
We look at the political factors as well as it relates to our hospital or our region.
And we also look at what entrepreneurial and research opportunities also need to be considered.
We'll talk more about that later because we're a teaching centre.
There are some opportunities in research and developing information that are worth the value of putting more resources in.
Perhaps this last factor, the stickiness factor, confuses a lot of people and it's not intuitive when you see that first.
But the concept here is that whenever you have something already taken up into practice,
it's a lot harder to take it back out of practice than it would have been to stop it from getting into practice in the first place.
So a lot of our clinicians in our hospital and even our senior leadership team will say, well,
why don't we just give this new technology or this new drug a try in our hospital and just see what it's like and then we'll decide in a year.
Trouble is, if we don't explicitly enumerate the stickiness factor,
we run into the problem of presuming that we could just as easily undo that practice once it's already got into practice.
This very concept is very well enumerated by the late and great Bernie O'Brien in his paper.
Is there a kink in the curve? And Bernie wrote about this, I think, very eloquently, where he says the willingness to pay curve is really kinked in,
that people are willing to pay more to keep something that they've already had and enjoyed than
they would have been to pay for it in the first place before they even had ownership of it.
So it's the sense of ownership,
the sense of tearing something away from you once you've already had it makes it more valuable to to you once you've already enjoyed it.
So I need to tell you and show you what we do.
How do we go here and how do we go where the evidence dares not to better inform our decisions to smooth that particular technology hype curve?
Well, we need to first understand what we're talking about when we're talking about true cost.
Has anyone seen this definition before? True costs of cost is the sacrifice of consequences in the next best alternative use of resources.
That's the real cost. It's not the dollar value on the drug or the device or the surgery that we're considering.
It's actually the value of the next best alternative use of those resources.
And this concept is very well espoused and is related to Newton's third law,
where the every decision to do one thing is a decision not to do another or every action causes an equal and opposite reaction.
And we use this as an analogy to the fact that we need to consider the benefits, the risks, the costs.
But we also have to somehow enumerate what is foregone if we're going to be able to determine what is the
true cost of taking up this new technology instead of all the other options that we had on the table,
and we call this no forego because you have to know when to go for the decision or whether you have to know whether it's a no go.
And these decisions have to be based on the four factors systematic review, objective evidence of the benefit versus risks as well as the costs,
as well as those expanded domains of influence, the sleepers, as well as what is foregone.
And in doing this, what we do in our hospital is we work around a trade-off table.
We bring the stakeholders for the decision around the table after we've looked at the benefit risk cost sleepers.
And we work with them to further enumerate or calibrate the sleepers and figure out what is the foregone benefit.
So it's not an easy task, but what we have done is we've created the Trade-off table to be like a pool table or a billiards table.
And essentially what is built on is the cost effectiveness quadrant where we
have the benefit index along the x axis and the budget impact on the Y axis.
To get us started, what are we plotting? So for the budget impact, it's the incremental cost of taking up this new technology,
drug or device or procedure per patient in our real hospital setting,
real world setting times the number of eligible patients that we would anticipate.
And then we look at the benefit index. We're talking about the trade off table.
And what we plot is something that's very practical but it's very much localised.
So our budget impact got that there.
And the benefit index, which is the number of eligible patients divided by the number needed to treat to benefit really.
Okay. It's not rocket science but it is local localised and it fully engages the best available evidence about benefits, risks and costs.
And it also brings in the foregone opportunities, which I'll show you in a moment.
So when we're sitting around this particular trade-off table, what we do is we we draw in our go no go line.
Okay. We'll come back to how we define that in the future.
But to orient yourself, any decisions that come below this particular line, because now that you're familiar with the cost effectiveness curve,
you know that for increasing levels of benefit,
you're willing to pay increasing numbers of dollars or devote increasing amounts of resources makes common sense.
But if we talk about it, the corollary to that for infinitesimally smaller benefits, we're willing to pay much less, if anything at all.
Okay. So that's based on this premise.
So anything coming in below the line means that we're likely willing to put out that amount of resource for that amount of predicted benefit.
And so if we have, let's say, five different decisions that we need to plot on this particular trade-off table,
a decision right at the at the intersection of these two axes would be a decision as in the green ball,
where we have really no budget impact, but really no known benefit either.
If we have the blue ball and we have loads of resource impact,
but really no proven benefit or risk if we have the yellow ball on the right on the horizontal axis,
we have lots and lots of benefit but really no known resource impact.
So maybe there are some savings that come along with implementation,
but most commonly we have the pink in the orange ball whereby there's some sort of incremental
benefit for some sort of increment in cost and we've got to make some pretty difficult choices.
So over time, we've been plotting decisions on this particular trade-off table and making real decisions,
and we now have over 40 decisions that are plotted. And it's it's on the increment in terms of the number of decisions that are here.
I say 40 because those are 40 formal. We've done a number of informs too.
But if I put them all on here, it gets quite busy. And so what we do is we simply plot the benefit risk.
So that benefit index. So how many of our patient population would we expect to benefit for the amount of resource outlay that we have to put out in?
Our hospital and we colour these balls based on the focus and the width of the ball is,
is pro-rated based on the confidence intervals or, or the amount of uncertainty.
And I mentioned that the colour of the balls, if they are actually coloured, it means that the decision's already been made.
If the ball is grey, as in B and C on the horizontal axis are on the vertical axis at the top,
it means that those are decisions that we're considering today.
If the ball is white as ADR, it means that those are decisions that have not been taken up into practice,
but which exist below the line for which we should have said go.
But for some reason we haven't gone and said yes to those decisions yet.
So you can imagine in this particular case, if we are sitting down and considering whether or not we should take up B and C and
we'll call B inhaled nitric oxide for patients post cardiac surgery and we'll call
C a special enzyme replacement therapy for a patient with a very rare disorder for
whom we want to perhaps bridge over until they can get a bone marrow transplant,
perhaps as definitive therapy for their disease, their very rare disease.
So you can see that C and B have no proven overall benefit.
We have we have no proven benefit for either of these. We have huge budget impact.
And this graph has been crunched down so that it fits on a page. But the budget impact is in the range of half a million dollars for both of these.
Okay. Now, when we're sitting down and making these particular decisions, it's it's not always easy.
There's a lot of uncertainties. And you can see that C has a lot of uncertainty with it, which is the enzyme replacement therapy,
because the evidence only showed one randomised controlled trial with very large confidence
intervals around the one benefit that was measured which wasn't even really clinically relevant.
So in this particular case, what we would do then is make a table of the different options that we have on the table today.
And if we are considering inhaled nitric oxide, for example,
for a variety of different indications in this case and the enzyme replacement therapy
and we have a few other things that are labelled here just to orient yourselves,
in particular the white balls, which are the things that we haven't taken up yet,
which have a benefit, proven benefit, and are under that go no go line.
Then what we would do is consider all of these decisions at once.
What should we take up? There was a lot of pressure to take up the enzyme replacement therapy,
and there was a lot of pressure for human pressure to take up inhaled nitric oxide,
especially for post cardiac surgery and post heart transplant surgery.
Once we have these balls graphed or on the trade off table,
then we start to talk about the evidence generally as well as the budget impact generally, and then we move into the expanded domains of influence.
So I haven't showed you yet how we bring in the sleepers. So we have to decide which of these are we going to go or say no, go to.
So to bring in the sleepers now, again, reminding you what those sleepers are, we actually have to do surveys of the relevant stakeholders.
What are the social, legal, ethical, environmental,
political and research opportunities and innovation opportunities with each of these options before us,
for those white balls, we had already done them in the past. Okay.
We now would just if we're sitting at the table with these other life decisions,
we now have to do it for whatever, for the B and C decisions at the top on the grey.
So what we do is we we we survey our stakeholders using radial plots with the sleepers on each one of the axes of us of a star plot, for example.
And we survey them, we survey the physicians, we survey the patient groups, we survey our senior leadership team.
We survey others who are in the hospital,
other practitioners who are not necessarily experts or even proponents of the particular technology in question.
And then we come back and we have data on those sleepers.
And what is the purported weight of those sleepers on the decision?
And we put a price tag on the ball if there's a significant sleeper issue.
So in this particular case with B,
there were some significant issues that the physicians working in the area felt should trump the evidence or the lack of evidence in this case.
And those issues were such as well as last ditch rescue efforts.
And we hope the inhaled nitric oxide will save somebody who can't get off the pump when they're coming out of surgery and or will get their
pulmonary pressures down enough for us to bridge over at least a day until we can get them on to something orally once they're activated.
And how could you say no to that, despite the fact that there's no proven benefit?
So that was the significant sleeper there. And these graphs here are hypertext.
So we can actually get back in and find out what are the significant sleepers.
First. See, you can imagine what the sleeper issues could be for an enzyme replacement therapy for
an exceedingly rare disease for which there's only one randomised controlled trial.
Sleepers ended up being such as things such as?
We think it would be unethical to withhold this despite the fact that the single rc t showed no benefit.
We think it's underpowered to have been able to show that particular difference.
And what's $500,000 for the shot at a child's life?
The other underlying issues were political, and it's interesting.
The decisions that were made in this particular case were that we would actually say yes to the enzyme replacement therapy.
And that was despite the fact that we presented the evidence and the lack of proven benefit and the huge impact on resources.
The yes was based entirely on a political issue.
And because we had done this sleepers analysis,
that particular issue became explicit and there was some feedback that went across the hospital and
the region because this was made explicit and because this was early in our no formal process,
it really helped to shape our future decisions. The political issue was ramped up because a number of us got telephone calls personally to say that
your face will be on the newspaper tomorrow if you don't say yes to this particular technology.
And the patient in this case was was the child of very influential parents in our community.
And our hospital was undergoing a very sensitive political opportunity at the time,
whereby we were undergoing examination to become the Centre for Innovation around a particular topic.
And I can't really give all the details because I don't want to put anybody in jeopardy,
but it gives you a flavour for the underlying things that occur.
But in this case at least, it was made explicit and we found ways to better address these political trumped cards over the evidence,
which, with our future decisions with B, the decision was to say no for inhaled nitric oxide, not outside of the research setting.
Now, how does this help us to actually enumerate the foregone benefit?
Well, in this particular case, we have an equation which is essentially written out in words with numbers if they want it.
But the equation tells them if we take a B or if we take up C or if we take up both,
then those price tags are those sleepers which are marked by the price tags on B plus C better be more than or B value to you more than the benefits
that a plus de plus E could have brought us and decisions a plus de plus e were for technologies for which we know we could have saved some lives,
prevented the need for some surgery, reduced length of stay, and in some cases, improved quality of life.
In this case, our senior leadership decided for the enzyme replacement therapy that it was worth more than a plus de plus E,
but it doesn't sit comfortably with all of us. Beauty of this is that a plus de plus e are always staring us in the face.
Whenever we bring another decision to the table, those opportunity costs are always there and we have to wear new decisions,
plus the sleeper's price tag against the opportunities that we're foregoing which are now enumerated.
And we have to decide explicitly whether or not those sleeper issues really are and should trump over a plus de plus e,
which our lives saved, surgeries prevented. Quality of life improved.
And days of stay prevented. So that's how we do it has called no forego.
And again, we have to bring these four dimensions of influence into the decision making,
explicitly the benefit for evidence or the evidence for benefit versus risk.
Our local resource considerations, the sleepers and then the foregone benefit, the feedback from our particular institution has been very positive.
It has really taken us forward in terms of our level of decision making.
It's made us a lot more comfortable in our decision making,
a lot more comfortable to say no or to change something that would have been a sleeper trump card into
something that becomes not at all rated highly anymore compared with our opportunities foregone.
It works.
It tends to resonate very well across the levels of decision making in our hospital, from senior leadership down to the frontline practitioners.
It's multidimensional and nuanced in that it does bring in the sleepers systematically,
and it also brings in our very real local estimates of benefit and cost.
So what is it that we're really buying if we take up this new technology and we think that it's honest and accountable because we we use,
of course, objective evidence based review, systematic reviews,
but we also have to be accountable to our own particular limits in resources as well as our own particular social contexts, organisational context,
etc. It also keeps us accountable because every time we sit down to the decision making table, those decisions are staring us in the face.
So if you remember from our graph up here, when we sit down to the table next time and let's say now we're considering ball number eight or 28 or six,
and in that case the balls would be grey. But again, we'd have to say,
are these three worth more than eight plus de plus e a plus de plus e which would cost less an E which even would save us money.
The overall resource impact would be almost zero if we took up eight plus de plus eight.
Are they really worth more than that?
Quite often once you make it this explicit form for decision makers, even though they were Rahim and Ali advocating for a new technology or procedure,
they tend to lay down their weapons after they've been through the process and they say,
Oh my gosh, I had no idea that we were not yet implementing A-plus de plus E or I had no idea
that A-plus plus E would give us that much tangible benefit to this hospital.
I'm willing to say no. I'm willing to set down my armour.
I'm willing to set down my badges and forego this particular opportunity because I see the value very tangible.
It also embeds the decision in the context of the past, the present and the future.
What do I mean by that? Well, again, back to that graph.
When we're sitting down to this trade off table, the billiards table, our past decisions are plotted here.
They're coloured balls, but they're not permanently set in stone.
If we're now deciding on, let's say, ball number ten, whereby we'd get some benefit but save some money,
perhaps we should forego something that was already implemented, such as 12 plus 15.
Okay. So those it embeds our current decisions in the decisions that we've made in the past.
This really helps orient us contextually in terms of past decisions that we've made and hopefully
helps us get our house in order more and more into the future because we can't do it all at once.
It also means that our future decisions, things that we know coming down, we'll plot them sort of in the background to show people what's coming up.
And it means that decisions once made are not always live.
They're not just a single event once a decision is made. If it is a no, it doesn't mean a forever no.
It just means for now that ball needs to be placed on the graph as a no.
And in the future, if there's something better to trade off for, we'll make it a yes.
And that is where the extra comfort has come in from the decision makers.
So the feedback you remember some of the rebuttals that I brought to you in the beginning of the presentation.
Now, the feedback has been, you know, huge sighs of relief from decision makers, especially front line decision makers,
but also our senior leadership team, our VP's, where they say, oh, now I understand I didn't kind of get the cost effectiveness thing before.
It kind of seemed ephemeral to me, not something very tangible.
I didn't quite get the uncertainty around the evidence before, but now I understand you've helped me to bring it all together.
It makes sense. I can live with that. One particular physician about a year ago said he's a surgeon.
He said, I'm not happy with the decision, but I will accept it, knowing what I've seen now and things like.
Finally, you've shown me explicitly what I've tried to understand for years and then we need more of this.
And now the hospital wants to put the whole capital planning process through this newfangled tool as well,
which gives me a bit of the shudders, but we'll see how it goes. So we think that the tool might be dextrous.
We've we started out in the world of drug therapies because that's where my expertise originally lies.
And then we've now also applied it to devices and surgical procedures.
Increasingly,
there's been some asks around how could we use this now to evaluate programs which are embedded with a number of technologies and opportunities, etc.?
I think there's opportunities there as well, though we haven't yet done it for programs.
What are some of the surprising findings as we close up?
Well, I think you'll find this interesting.
There's been a lot of surprising findings, including by, from my own perspective, where before we always thought more technology is better.
You know, more drug options on the formulary are better, more devices are, you know,
at least having the option makes us more able to meet the individual needs of patients.
But it's interesting, more and more we've come to experience that less is more and that the more options that we have out there,
the more complex and confusing it becomes, the more errors that occur,
and the more likely that practitioners and patients are unsettled because as soon as they choose one option,
they feel that they might be missing out on the other options that they had before them.
So that's been a surprising finding. We've also found that even though in the past we sort of considered decisions to
be more likely dichotomous a yes or a no or yes with conditions in some patients,
but not all. For example, now we consider that there is also the opportunity to say yes,
but only in the context of a bonafide research trial, often a randomised controlled trial in our own setting.
And we've also found that our clinicians are more and more accepting that yes is not better than no, yes is also no.
So here's the issue about moral hazard that we talked about just a few minutes ago.
Whenever we say yes to something, we're trading it off against other opportunities that we could have spent our money on.
And because of that, a yes to one thing is a notice something else. So we better be very comfortable that we've made the right.
Yes. And again, decisions always being life has really made clinicians more comfortable to say no today.
But we'll see in the future. Maybe it becomes maybe a go no go line will change.
And again, it causes us to be accountable to the evidence, to our own resource requirements, to the sleepers and systematically so.
And it causes us to enumerate what is the foregone benefit.
A lot of people think that we're about the bottom line when we talk about money in the context of decision making, health care, decision making.
But to me, opportunity to me, money is not the bottom line.
It's actually opportunities. Moneys are just the proxy for opportunities.
Whenever we make a decision that's going to save us money, we immediately put that money into another opportunity.
It's not that we just have more money in the bank and opportunity itself may present more than once, but you can only spend the same dollar once.
So opportunity might present twice, but you can only spend the same dollar once.
So you better make sure you take up the right opportunity. And the other surprising finding was that we really need to subtract before we add.
Meaning that if a clinician or a group is asking for something new and it's going to be an increment in cost,
then we have to ask them, well, what are you going to give up?
Because we can't implement something without extra dollars in our bank and we truly don't have extra dollars in our bank.
So we actually have to get them to subtract something before we actually add another new thing.
Future efforts we'd like to further develop, know for go in other settings in other contexts,
because we've learned so much more about its capabilities when doing so, and it gets further refined when other people are using it.
We'd like to further develop it also for considering research opportunities and our research wing is very open,
is very interested in doing so because a number of decisions really should be yes,
but only in the context of further research if the value of ascertaining more information is worth it.
Once we graph it on the map, we'd also like to get better at our decommissioning opportunities.
A number of the coloured balls that you saw on our graph, you can see they're well be lying below the line of go no go because they were decisions
that might have preceded our ability to address those decisions systematically.
So now we need to look back at those and get better at disinvesting now.
And the sleepers become a huge issue. And that s factor on the sleepers, the stickiness factor.
You have to be very careful to enumerate what it's going to take to undo a practice, and you have to be very honest about that.
Our region is interested in using it, as well as our ministry and other provincial ministers and other provinces.
Ministry has contact us, contacted us to use it as well.
We'll see how that goes. And now we're moving more and more into the area of less tangible technologies,
such as procedures in and of themselves, which brings a whole new raft of sleepers with them.
So in summary, I hope I've shown you that Hospital HCA is a good petri dish for trying something like this out before we try it.
In the more broader range of contexts, traditional decision making failed us, has failed us in the past.
So we moved on. Traditional decision making, evidence based decision making, again,
can be very linear and doesn't necessarily take on the nuances of the decision at hand systematically.
So we created this to do just that, to answer to that gap.
And we found that if we're to engage rather than alienate our decision makers, whether it be the VP's in our hospital or frontline practitioners,
we need to make to an evidence based medicine relevant, explicit and tangible.
That's why we do the graph. The trade off table provides a framework to explicate the evidence and its uncertainty
by toggling the width of the balls while acknowledging the sleepers and enumerating,
explicitly enumerating what's the foregone benefit.
We found that if we don't make this explicit, then the sleepers often trump very important decisions.
They often trump the evidence, even when it's a very clear evidence that something does not work.
Sometimes the politics just takes over.
Making the sleepers explicit and the forgo explicit has been our saving grace in terms of comfort in decision making.
And you'll notice that there's a lot of things that know Fargo espouses and incorporates and tries to bring together to the decision making table.
Yes, evidence based decision making is there to. Is there a medical economics?
Is there. And then there's some other flavour of game theory and program budget.
PBM and accountability for reasonableness,
which you may want to read more about to make us accountable and robust in our decision making
that makes us live within our means and that makes us smoother on this technology hype cycle.
So we have a way to address the overhype and the under hype explicitly.
So with that, we're close. We're drowning in information. While starving for wisdom, we hope that this process has gotten us closer to wisdom.
But again, it is not perfect. And many thanks for your attention and for your patience with my my voice.
Now, I'd like to take some time to have some feedback and some discussion and hear your thoughts,
comments and your feedback here is most welcome and by email as well.
If. If you'd like to do that over time. Well, thank you very much indeed.