So that's a welcome again to everybody else in the room. And welcome to everybody online.
We've had an amazing response to this lecture. Um, around 200 250 people, which is really just a marker of how digital the world is becoming.
But also lots of questions around why digital interventions fail.
Um, so this focus of this session has really struck a chord.
Why digital health and care interventions fail, and the part that you're probably all waiting for is what we can do about it.
Um, and on, um, confidence that you're not about to come up with, um, very clear problems and very clear solutions.
But we're in in Catherine's hands today. And I'm really delighted to welcome, uh, Professor Catherine Creswell.
Um, Catherine is another social scientist and a colleague that we've never actually met before until today.
There's a few of us in the room.
Um, and Catherine brings extensive experience, uh, evaluating digitally enabled change and improvement programs across health and, uh, care.
She's got numerous titles, gongs and publications, which I'm not going to read out, but if you do a quick search, you can find all of the detail.
And, um, she's amazing basically. And we're very lucky to have her with us today.
Now. Catherine's research the use of a variety of health information technologies in context.
And I think that's really key over many,
many years and often involving bringing together stakeholders with varying perspectives and voices, which I'm sure you'll touch on today.
Um, and we've invited Catherine to speak as part of our MSC programme that we run here on Translational Health Science.
My name is Sarah Shaw. I'm the module lead for the first module that runs on that program, uh,
which is about the social science of innovation, but it's also about the methods that go along with that.
We're training a bunch of new social scientists, if you like.
Um, and we focus not only but partly on how digital technologies are adopted, implemented, used, evolved, adapted spread scales and lots of course.
And that's, I think, what we're particularly interested in today.
So I'm going to shut up, um, and let's hear more from Catherine about why digital health interventions fail.
Um, for us, we might turn them over to you.
Catherine. There we go. Um. Um, thanks for the lovely introduction and thanks for inviting me.
Um, I'm delighted to be here. I hope I'm not disappointing you with a with a snazzy title and a boring presentation, so I'll do my best.
Um, I heard I'm in between fresh green, hot and dinner, so.
It's so nice to be, I have to say, but, um.
Yeah. So I, um. I'm from the University of Edinburgh. I'm, uh, professor of digital innovations in health and care.
I've mainly looked at health care because, um, the opportunities to study care are just coming up.
Really? And, um, if there's any, um, then if there's any that you're involved in, um, then I'm very keen to get more into that area.
Um, I'm a social scientist, as Sarah's already said.
Um, and the medical school, I've been there for 20 years working with medics, um, and, and other, um,
other, um, uh, professions that are interested in digital systems and look at them from different angles.
Um, and the biggest, um, underlying assumption as a social scientist, um, um,
my biggest underlying assumption is that, um, I start with a second bullet point.
Technological systems have social consequences always.
You can't look at technology and social dimensions in isolation.
We always need to look at them together. So how do technological systems have social consequences?
Well, they can change or they do always pretty much change, um, the way people work.
Um, and they also change the way organisations function. And I get into that a little bit.
But but similarly social systems also have technological consequences.
So um, well, that abundance lot technologies, um, that they fail like we'd like to say, although I think it's not that easy,
it's not that simple to to frame these things with its successes and failures, because it's success to somebody is a failure to another person.
So it's slightly more, um, nuanced than that. Um, and also, um, they can, um, they can lead to workarounds.
And I'm going to talk a bit about that. Um, and so if it's one thing that you're going to take away from this, um,
technological systems have social consequences and social systems have technological consequences.
Um, my expertise is and formative evaluation.
And I will explain what that is later on and the kind of the biggest things I've looked into so far, um,
have been the national programme for it, which went down in history as the biggest public sector, actually failure, um, uh, ever.
Um, and it was very interesting to research. Um, and we looked at the national procurement of electronic health records as part of that.
And I read my PhD on that very, um, very interesting, um, piece of work.
And I'll talk about that a little bit later.
Um, and then I co-led the Global Digital Exemplar programme evaluation, which was an attempt, um, by NHS England to,
um, to create a learning health ecosystem, so to pass on implementation knowledge from hospital to hospital.
And actually that was quite successful. Um, but the lessons weren't learned.
And again, that wasn't due to technology, that was due to social and social aspects about the technology.
Um, and, um, yeah, I can point you to papers that, uh, we look at that, um, but, but,
but they're pretty much the social aspects around the technology are always pretty pretty much the same.
So, um, but and I'm going to get into that at the moment, I'm doing, um, the, um, leading the independent evaluation of the initial AI lab,
um, which, uh, I can't talk about, unfortunately, um, because it's a half underway.
But, um, our results will come out in April next year.
And, um, again, it's a national programme and, um, very important lessons that, uh, we need to be learning.
Um. Okay. So, um, just for context, I mean, you know, all this, we're all in health care,
but we are getting fatter and older and sicker for longer, unfortunately.
So I've looked at some stats, but basically in 25 years from now, uh, 1 in 4 of us, uh, will be over 60 and 1 in 2 will be overweight.
So it doesn't look too great. Um, and actually health care expenditure is increasing.
Um, and we basically need to achieve, uh, more with less.
Um, now, the solution, like policymakers like to tell you, um, which, which might which, which might be true.
Uh, but I think it's often an overstated assumption is, uh, the potential of health and care it which is actually, um, in some instances,
um, uh, actually, well, um, well evidenced, uh, in others, not so, um, so there's a lot of hype around some aspects of technology.
Uh, I mentioned earlier.
Um, so, um, the assumption is that we can, we can deliver more systematic and high quality care decision support of clinicians who,
um, who will then make better decisions, uh, more proactive and targeted care through data analytics.
Precision medicine is one thing that I'm really interested in at the moment, but again, it's not.
It exists and everybody is really excited about it, but it doesn't work in practice as yet.
Um, due to some of the issues I'm going to talk about, um, better coordination of care.
So integration of primary, secondary and tertiary care.
Um, do. I know something.
Other, um, technology.
Um, this is one of my vivo, though. Um, so there you go.
Coordination of care. Um, improved access to specialist expertise.
We can do it via telehealth. Now. People don't have to travel anymore. Um, greater patient engagement.
I mean, that was an announcement by Wes Streeting. Uh, people will have access to the NHS app.
Sounds all great. Um, people having access to their own records and having more involvement in their of care, improved resource management such as,
um, using analytics for improving patient flow, let's say organisational aspects of change.
And, um, the holy grail of, uh, creating a learning health system, um, and data is playing a huge part in that.
Um, uh, but, uh, we need to think about, um, what what what these changes bring with it.
Yeah. And in order to achieve them, we need to develop new competencies.
I mean, I know, um, that in England, um, there's a huge skills gaps in procuring technologies.
People procure technologies that don't know much about technology at all. Um, so that's one thing.
Um, but also, uh, we need, uh, to, to understand how we can best exploit data,
that we create the huge amounts of data that we create through data analytics and,
um, and, and, and, um, the clinical interface, we need to understand how we use data and what data we use.
I mean, the biggest problem now is not to get the data, but good quality data.
And um, and one of the biggest questions is how do we make data actionable and what data is actionable and what data results and what actions.
Um, and um, and they are changes the way decisions are made.
So like I said, us looking into, um, into precision medicine.
So we are increasingly creating predictive inputs that that's tough and issues about the future.
Um, so it's it's, uh, it's it's a different level of evidence that they're going to work with.
So skills have to change. Um, we also, as clinicians, uh, need to develop new ways of interacting with patients from face to face to,
uh, remote and potentially, um, uh, um, yeah, it's a distance, um, through telehealth, let's say.
And that might have adverse impacts on to on, on, on relationships.
Um, uh, it might also lead to digital inclusion. I mean, I have simplified this whole area.
It's so complex. Um, but these are just examples.
Um, I'm trying to make this kind of digestible in a way that I think it's Lewis Carroll said this, you know,
if you have a map that's the size of the scale of the Earth, then what's the use of having a map?
So I'm completely simplifying this whole, uh, map.
Um, but but it will give you some ideas of how to navigate, um, so, um, you know, competencies, new ways of interacting with patients,
changing organisation and professional boundaries, uh, through increasingly complex health information,
healthcare information infrastructures, um, we building systems upon systems.
And the new systems are depending on the old systems. Sometimes the old systems just exist to, uh, maintain the new systems.
Um, and the professional boundaries of, of care systems such as, um,
used to working together suddenly have to work together because they share common pathways and common dataflows.
So, um, yeah, this is still active.
Um, I'm getting this. So, uh. Yeah, the problem is, I mean, this this is an estimation,
but this is the numbers to find between 79% of Digital Health Start-Ups fail in the first five years.
That was I think it was Ipsos Mori. And the first, uh, the biggest reason was no market needs.
Well, that could have been resolved relatively easily, but most of them actually.
Um, uh, the so the problem is that, um, developing and implementing um, technology in health and care is not an easy thing.
Yeah. If it was easy, then everybody would do it right. Because we all, you know, we all want to improve safety, quality and efficiency.
Um, and, and it should be relatively straightforward for some people to make money off this, which is fine.
It's completely fine. Actually. That has to be ignored. I think it's okay.
Um, because we need to align, uh, different, uh, stakeholder motivations and incentives.
Um, but it's not it's not a technical problem with an easy fix.
There's no recipe for success. And, um, I like this quote from Harvard Business Review.
As a result, executives leading difficult change initiatives are often blissfully ignorant of an approaching threat until it's too late to respond.
So, um, people understand the problem, but only when it's too late.
So I'm taking a health system perspective here. So this is this is for the good of um, of the health system.
So, um, this this should say and or but what what do we want?
What what are we trying to do? Yes. At the most at the most simple level.
We're trying to develop technology not just developed, but implement and get it used as well.
Um, that improve safety, quality and efficiency of care, if at all.
Efficiency of care. Uh, if it doesn't, we should stop developing it.
You know, the problem is that it's quite simple.
Yeah, it seems quite, uh, but a lot of technologies are developed that don't actually fulfil the basic criteria.
Um, then we want people to use this technology again.
Um, developers focus on developing and technology, and they think that once it's developed, it's succeeded.
No it hasn't. There's so many technologies that exist and that that seem like a great idea, again,
to strategic decision makers who don't actually know very much about technology or social aspects of technology.
Um, we need people to use it. Otherwise it has. Yeah.
Um, and then then we also want technology to embed within existing work practices and within organisational practices,
because if it doesn't there's duplication of work. And that's gives an example later.
Um, uh, and, and and ultimately the ultimate aim is to actually sustain a technology use, um, over time so that it gets used over time.
It's not to be reimplemented and costs and creates disruption, but it but but it keeps being used and refined but optimised um, and um across space.
So we want, we want technologies to talk to each other in different settings for example.
Yeah. Because otherwise we have technological silos and multiple redundant systems and limited interoperability.
I'll come back to that. So that's my general. That's my, um, my my, uh, argument base of my argument.
And these should come in one by one. Um, but I didn't I don't know why, but, uh, let's start with the technology.
So these, these factors, uh, you can and and you can apply them to any technology.
They match up any such thing. Really.
Uh, but if you think through these four factors, then, um, it would really it will really help you to make things work.
Um, and we've, we've done a whole framework and I'll just give you some examples of these dimensions.
But there's more, um, there's a more holistic framework around this that I'm sure very, very happily.
So let's start with the technology.
Um, a technology might perform, like I said earlier, might solve very well in the lab, might be super, but then actually,
when it's implemented, um, if people don't use it, um, and zero is one, one thing that was uh, but specific monitoring now.
Um, and that basically failed because people it worked well in the lab.
It's rolled over the fence into the real world and people didn't want to, um,
didn't want to record their sleep patterns before that because what they because they weren't worried.
Well, um, and, um, so that's the technology.
And again, they overlap as well, which makes it even more complicated.
Um, so adoption is another dimension.
So issues surrounding usability of technology and usefulness of technology and that that always increases the risk of abandonment.
If something isn't useful to a person using it, then why would they use it?
Um, and and one of the, one of the biggest, uh, that was, I think in 2011, um,
IBM Watson, uh, beat, uh, some, some people, some humans in jeopardy game jeopardy!
And um, they then developed it had a huge high but great idea to put it into oncology settings, um, and develop its oncology system.
Well, people didn't trust the system, and, um, it was of no benefit to them.
So actually they thought that it didn't. It was abandoned.
Um, unfortunately, um, organisational issues.
So organisations might have different priorities.
Um, we see which um, uh, the our national program experience, uh, illustrates that support from our paper here.
Um, because the government basically procured electronic health record systems for NHS hospitals and that they even though they got it for free,
they didn't want it. They didn't want they didn't want an American system that was based on billing.
Um, you know, because it didn't fit with them and they didn't they didn't.
It's sometimes it's just the process of having the power of feeling and power to make the decision that, um, that didn't happen.
And then the, you know, the whole program fell down. Um, and what we're seeing increasingly, uh, with AI technology and I have some examples,
data is that it doesn't align with existing business models a lot of the time.
So organisations are important, uh, stakeholders. And lastly, and we never think about this really a lot.
Not enough. Um, it's a health system dimension. So a technology needs to be a national strategic priority.
Flavour of the month. It's good. Um, in a way, because it can help adoption, because leaders get interested.
And then when you have an interested leader, they. Will, you know, push the organisation towards it.
Um, if it's not in line with existing health system structures and cultures, really important.
Um, and in some instances, um systems, uh um, uh, yeah.
Systems are too complex. They need to sit on an existing health information infrastructure.
And if that doesn't exist, then advanced systems don't actually bring major benefits.
So an example here is Babylon. It was again hailed as some, um, basic system that could triage, um, back to triaging.
Primary care implemented in London, didn't fit in with existing payment structures basically, and went bankrupt.
Um, they think they should have thought about that, but they should have thought about that earlier.
Basically, I'm trying to tell you think about these things before you develop a technology.
Um, unfortunately, uh, all these things are interrelated and it becomes really complex.
So I acknowledge that. Um, so now what can we do about it?
Um, so there are two types of evaluation. Um, one is summative.
Yeah. You basically you cook a meal, um, and you test it after, um, you tell.
You say, well, this test is great or it's not great. So that's that's basically what summative evaluation,
assessing whether something's worked and that's um, that is assessing a quantitative kind of impact,
um, been done through quantitative means such as, um, impact on quality and safety of care or economic impact.
And that's important.
We need to know whether something might be really important, um, especially for making business cases, for making investment decisions.
Um, but what I focus on and ideally you do both um, is formative evaluation.
So and that that basically means you test the dish early on and you put some salt.
But if it means that you put some pepper and you make it taste good.
Um, what's working on it on the way? Um, so it seeks to answer the question of how can we make this work?
Yeah. Um, some I think some industry term might be something, um, like a premature.
Um, if you ever thought that imagining that something has gone wrong and then mitigating, um, mitigating against that with with with a, yeah.
With, um, with, with certain methods and, and those methods, uh, designed to understand processes.
So, um, why did something what how can we learn from it?
Why didn't it work? And how can we then apply that learning to other settings.
And, and I usually draw on social theory. I mentioned, um, health information systems, uh, um, uh, health information infrastructures earlier.
I'd like to look a little bit into the, um,
science and technology studies literature because I find it quite intuitive and not very theoretical, but I do look into that, uh, literature.
So I highly recommend you look. Um, and, uh, we use qualitative methods.
We actually, we talk to people. That's, that's pretty much quite common sense, actually.
But when you when you're excited about the technology, you forget that other people might have different views.
Um, and especially aligning these different views becomes secondary to you developing that system that you always dreamt about.
Um, so, um, so formative evaluation is important to account for this complexity.
It's not simple, these things. We have different stakeholders.
As I've mentioned, we have evolving interventions, um, technologies change needs change technologies, like I said, on technologies change.
Um, and we have different contexts. So a system might work really well in one context and it might completely not work in another.
Um, so, so so those are the kind of underlying reasons for this complexity.
But, um, another really important thing is that I, uh, encounter again and again is we need to identify risks and potential unintended consequences.
Early and unintended consequences are not only negative.
Um, so introduction of new threats. We know that, um, that that is a threat.
But actually, um, that can be positive. So that can be look at them as a source of innovation.
Yeah. Uh, turn it on its head. So it doesn't necessarily just have to be negative unintended consequences.
And and key is we want to learn. Yeah. We don't want to repeat same mistakes.
And and we are a lot at the time. And that's why um yeah we talk about the same things again and again and um, across different implementations.
And um, and it's quite frustrating because we, we don't learn.
So we're trying our best, um, to change that. Um, so we want to have future implementations.
And so that means that one context has learned something.
The other the other organisation doesn't have to go through all these bad experiences.
And patients are often suffering from these bad experiences. Um, and it also works across technologies.
I mean I've written papers on I, I yeah, I is slightly different.
Um, but actually it's also a technology. I mean, that's just the dimensions that apply.
You know, we need to look backwards as well at the evidence, uh, when we,
when we have a new system because there's that there's lessons that apply across technologies.
So this is, um, this is my. Um.
It's this clock, right? Yes.
Oh, is it okay? Oh, I can slow down. That's good. Oh, excellent.
Uh, so this is my framework. Um, but I will take you through.
Yeah. And that's why I think we need to look and what we can do, um, about avoiding failure.
Um. Uh, so technological development is the first one.
So we need to actively involved different stakeholders in system design and also the development,
um, of the office system to ensure that that technology meets the needs of adopters.
Um, then we need to do implementation research to ensure that the technology embeds within the context of use.
And then we need to move to optimisation. So once the technology is in we want to optimise it to the context of use.
And we can do that again through um adjusting implementation strategies, let's say training.
That's uh one that uh, managers always mention but it's not.
Oh you can't, you know, if the software is unusable, you can't solve that with training either.
So, um, yes. Um, uh, but it's the kind of continuous optimisation over time and that will never end.
Um, and then we need to think about how we can, um, uh,
scale and sustain technology so how we can transfer it across contexts, i.e. transfer to different groups of adopters.
I think that's something that about pathways as well, and how information flows from one organisation to another and how people,
um, contribute to that and draw on that information across different settings.
Um, but also different organisations and different health systems.
Um, like I said, some dimensions are quite similar and relevant.
Um, and also sustainability over time. So um, we want this.
We don't want to re-implement constantly because, um, yeah, that's a waste of money.
Um, because then we would have optimised the system to work perfectly and then, um, uh, well, within the constraints of the system, of course.
Um, but, um, yeah, it's about sustainability of time, maybe, and making systems work.
So if we start with the first one. So the risk here.
Yeah. Uh, technology development, um, the risk is that technology, it's not needed.
Um, and not used or not used and, um, and that that's the most basic one.
So before you, um, before you do anything, um, you need to figure out, uh, how a technology could meet the needs of a lot of its users.
And one study we've done, um, is, uh, surrounding artificial intelligence based decision support.
Um, there are many, many early stage AI innovations that you will be aware of.
There's huge visions and huge expectations and huge enthusiasm around this and this one.
This system didn't actually exist when we, um, when we researched it, which is excellent.
I mean, that doesn't you know, that doesn't happen very often.
So, um, usually people have an idea, so people have an idea and you kind of get caught in too late when,
uh, when, when things exist and they can't be changed anymore.
So with this one, um, it was the vision of a system to transform care by predicting, actually, a fibrillation from EKGs.
And we did a qualitative study speaking to people. We gave them, we showed them the system.
We said, you know, this is simple. Do this. What do you think about it?
Um, and you got it. You got you got to junior people a lot.
Um, we also need to speak to the senior people.
But the junior people are really important because they often take a lot of the work so that they're the ones to actually really focus on.
Um, the powerless. Yeah. Uh, important, um, uh, consultants don't do data input that to, uh, transfer to their, um, junior doctor.
So junior doctors are the ones that I would want to speak to first.
Always. Um, so, um. Yeah.
So we thought we got so much. What? What would you. You know, what would you do with the system?
How would it change the way you work? And I said, oh, yeah, I is great.
And then said, how? But how would you what. Uh, it wouldn't.
And then we thought, okay, well then we need to, you know, we need to rethink the system if, if it doesn't do anything.
And now, um, now we have to go back to the drawing board, and, um, and then we left the field, which is another problem.
Um, if you do formative evaluation. So I don't actually know, um, what's happening.
I think it's been it's been rethought at the moment, but the developers hadn't invested a lot of effort.
And, um, so that was an easy lesson to learn right at the beginning.
Um, so something that I call that something that, uh,
needs to be done early on then to understand these needs and align them ideally is a co-creation, we call it.
Um, so you actively involve people, um, in system design and,
and potentially also implementation strategies, and you ensure that the requirements of it.
And that's not that's not always easy. As you see in the last point um, they uh requirements will conflict.
You know, sometimes it's, it's, it's a least worst option and discussion to have.
But um, involving people actively in that and being able to justify why certain design decisions have been made is really, really important.
Um, so and then, um, you use this information and you need to make certain changes to, like I say, always think system design or strategy.
Yeah. The two um, and um, but the problem during that process is already like I said, it's aligning different requirements.
So for example, um,
we see a lot that organisations implement or introduce systems that are good for the organisation but create more work for the clinician.
Um, you know, that's if the if it's a problem if.
There's no immediate benefit to the person putting in the work. Um, so that needs to be that needs to be somehow, uh, that circle needs to be squared.
Um, and, um, I'd also use this, um, very well equipped to rethink processes.
So it's that kind of, um, hadn't you thought Henry Ford? Yeah.
The statement. Um, if I would have asked people what they wanted, they would have told me, first of all.
So, you know,
you don't want to present them with a final solution because you don't want to stifle their creativity and impose your vision of a system onto them.
Uh, but you also, um, you also, um, you so it's a it's a balance.
Yeah. Um, and, um, of course, we are dependent on developers, especially with a big we call them mega systems.
Um, but but it's always a problem. Uh, change requests land and massive.
Well, you know this better than me. Uh, the conditions they land landed because.
And some of them are never actions. Uh, people make a big fuss a lot.
If a lot of people make a big fuss, there might be action tomorrow. But. Yeah, so that's a problem.
Um, later on, when the system's more established, especially, um.
So co-creation is one. Uh, how are we? How we can avoid failure?
Um, the other one is implementation research.
Um, and again, it overlaps with the other stages.
But, um, we want technology to embed within contexts of use, because if it doesn't, then we, um, we duplicate work.
And um, we see that again and again.
So, um, one of the studies we have done here was with, um, electronics health Records, that was the national programme, actually, that was my PhD.
Um, and, uh, yeah, clinicians were doing more data entry.
Um, they had more work, uh, which we see commonly with electronic health records.
Um. That's okay.
I think it's okay that there's more work as long as, um, there's as long as they're not sold on the basis of, um, this is going to change.
Your life is going to make everything easier, which is what organisational strategic decision makers tend to do.
So I think we just need to be we need to be upfront to people and saying, look, this is going to be worth it, you know.
Um, it's going to improve patient safety and um, it's going to be exquisite.
But it might, you know, you might you're likely to have some more work and some areas and actually more work.
I find this applies across all complex systems. So there's always more work for somebody somewhere.
You just have to find that person while you do the social science.
Um, we also see a lot I saw a lot, um, that people develop workarounds.
You know, if a system doesn't fit in with what they do within their workflows, um,
then they will create the most amazing ways to get around, um, fitting into the system.
And again, I think it's really, really important to understand some of the creative ways people do that,
um, because they can really help you streamline the processes.
Um, so we saw people taking notes on paper and then putting them in at the end of the day.
And that then has implications for data quality,
because people who were accessing the electronic system weren't working with the most up to date information.
Um, that was electronic health records. The second one, um, we did is we looked very recently at an AI based decision support system in radiology,
and, um, radiologists loved it and loved it.
They said it makes my life easier. You know, there were clear drivers for the for this for the system because we don't have enough radiologists.
Um, so, um, yeah, they loved it.
But then we spoke to organisations who had gotten the system for free as part of the trial that we were evaluating, and they said, well, you know,
I don't know how to justify buying the system going forward because the basically it looked, you know,
I have different priorities and it looks like one little thing, one little aspect of detected cancer.
It might have, um, you know, what about increased referrals?
It might create some work somewhere further down the line. Do I even want to know about this?
Um, so there were challenges for establishing a business model.
Um, of, of this very, very specific, um, system that we were looking at here.
So, uh, implications for research, Sutton. Which is what?
You know, that's the stuff we can do about it. And I always say some evaluation is better than none.
So even if we think about these things, you don't have to do the research.
Just think about them. Um, that already helps. Um, so, um, so we we need to do implementation research.
Ideally. I mean, this is the, you know, you need somebody like me coming in and and helping you,
um, to, to to make sure that systems embed within that context of use.
And that involves exploring implementation and adoption processes.
We do that qualitatively by doing observations and interviews, but um, documentary analysis as well.
And um, yeah, ideally we um, uh, come back to that.
Actually we, we need to understand how these systems work, how these systems work, and how they transform and change the way people operate.
Right. Um, I mentioned that other stuff before.
Um, and what always helps me and again, something that to take away and, um, this is far from realist evaluation,
but, um, it's about, you know, um, what works for whom and under what circumstances.
So always ask yourself that there will always be different views.
There's not one that's not one thing.
And those different views of what works for whom and under what circumstances and helps you have the right conversations.
Now, um, these two things. Um. Three things.
Optimisation, scaling and sustainability are very, very difficult.
Um, so the so looking at, you know, co-creating something.
Um, asking users what they want, kind of making changes to system design,
then putting it into putting a system into processes and planning how it changes organisations.
It's relatively easy. Um, when you compare it to, um, to, to these next stages.
Um, but they only become important. And that's probably why we don't know much about them when, uh, system is actually used in one context.
Um, and when it has been shown to embed relatively successfully.
Um, um, and also, um, the stages are only consider when things go wrong.
Yeah. And then they call social scientists and um, try and how can we make people use the system like, um,
but then it's this Harvard Business Review saying, you know, you're too late, um, because people are probably disillusioned already.
Um, so, um, yeah, it's it's it's I'm basically telling you, you need to think about these things early on, as early on as possible.
Um, and, um, because people don't focus on, on all these stages, it also means that there's relatively little,
uh, research on them because we just don't get funded to study these things.
Um, so, so most of the stuff, most of the cost of the empirical, uh, research focuses on the early development and implementation.
Um, and yeah, it's I mean, this quote is just to just to show that, um, a simple solution is appealing, but it's not always the right thing to do.
It might make things very simplistic. Um.
So. But but we have looked at, um. We have looked at ways to, to address that problem.
And I did mention optimisation earlier. So, um, uh, the risk here is that, uh, technology is not sufficiently exploited here and,
and it can't keep up with, with changing technologies and changing needs.
So, um, so we need to take that iterative refinement of systems and strategies, uh, forward, um, across time never ending.
That shows a never ending. It will become, uh, continuous improvement.
And I've stolen this, uh, from my my partner in crime, Robin Williams from science and Technology studies.
Um, but yeah. So optimisation is, um, we've done a study on this in, in, uh, electronic prescribing work in hospitals.
And actually we do some work now with and adjusting that to look at optimisation and electronic health records.
Um, but optimisation of system refinement based on different user needs and, and, and the problem is that some of them only emerge in use.
So you can't you can plan and it's important to plan for optimisation in advance.
Yeah people don't believe it. But again they put the system in and then they oh yeah we could you know we could use XYZ that data.
But if you have a strategy it really helps um some kind of strategy.
So um there is optimisation activity that needs to be done at the pre implementation stages.
Um, and um uh but but others can only be done once the system is being used.
Um, and uh and also the change over time. So it's a never ending journey.
It's just like digital maturity. Um which I have another lecture about.
Um, it's not something that we can achieve. Um, and that's okay.
We just need to acknowledge that. Um, so, um, we found in the e-prescribing work that organisations employed a different strategies to optimise.
And that was misaligned, resolving misalignments between the technology and clinical practices, um,
enhancing the systems or putting in um, different, um, uh, different features and also streamlining some of the activities,
uh, some of the pathways, um, and also, um, skills, basically uh, training and also, um,
how training, how people could optimal, how they could use data coming out of the system.
And um, this optimisation. So we were in the field for 45 years, five years, I'd say actually, you know,
we did the research before because the first year always takes ethical approval, takes a year.
But unfortunately that's the that's the difficulty about research.
So we need different methods here. Um, but um, so we um, yeah, it continued.
And then we stopped because we weren't funded to study anymore.
And that was annoying. Uh, because actually, that's really important, um, to figure out the processes around optimisation.
So now I'm hopeful that this new project will shed some light. And it has already actually, um, and watch out for the paper that's coming.
Um, on what people, you know, what the optimisation activities are and which optimisation activities leads to, um, improve perceived improvements.
Um, and uh, um, optimisation of the system actually comes back and systems, um, um, again,
this is like this, this is the idea of and actually, uh, the idea is beyond that as well.
But um, we ideally we want to look at, uh, systems implementation before.
Yeah. So, you know, dimension is crucial before we want to have great baselines because otherwise if we just focus on the implementation,
we don't know what difference the system is made. And that is a huge problem in digital health that we just don't have proper baselines.
And that's quantitative baselines and also qualitative baselines.
So by qualitative baselines I mean what are the existing processes are that you know what process maps.
For example they could be FODMAPs. But any kind of idea of what is going on before it's really,
really important to plan the changes and then also figure out whether the system had the, the intended impact.
Um, and it can help you how? Yeah. Plan. I said that um, and it can also help you tailor the system.
Um, so before implementation is super important. Always neglected.
Um, you can't go back once you implement it.
Um, during implementation, um, when things are going I mean, so most of my work's been in electronic health records,
so complex systems, um, everything is the organisation is a huge upheaval trying to make this thing work.
Um, but that's an important point to look at how people make these things work and how the result, what tensions come up, how to resolve them.
And that's a huge source of learnings. Um, and also, um, that's when I anticipated an unanticipated, uh, consequences.
Come, come up and we can, we can, uh, look at these and then that can be addressed further on that one.
And also, uh, workarounds, I think workarounds. People tend to keep them secret.
But I think the top tip is please get people to share the workarounds that are really important for the organisation to know,
because they can really mess up anything that comes up on the on the other side of the system.
Um, so, um, and then so before implementation, during implementation and after implementation,
when, when people in the organisation feel that they can't do without it anymore.
Um, ideally uh, or when it's been abandoned, in which case you don't need to cut back.
Um, but uh, when, when they've established new routines.
Um. A system has become embedded, we call it.
So so those are this is the optimal design.
Uh, in a within the time frame, we usually after implementation would be we always say between 6 to 12 months after implementation.
Um, but with if we want to study optimisation activity. So this is a project.
Um, but if we want to study optimisation activities, that means to be much longer.
And actually it probably shouldn't ever stop. Um, so yeah.
Uh, so some, some idea of how things are developing over time.
Longitudinal dimension is really important. Um, now, uh, sustainability and sustainability.
This is, uh, a difficult one. Um, because. So I make that claim, and I don't know.
I've been thinking about this a lot, actually, uh, walking the dog. So when, um.
Uh, so the risk is that we create technology silos.
Yeah, we we want especially now we integrating care across increasingly large, um, uh, locales and different organisations.
We want a degree of interoperability. Now, we don't want seamless interoperability because that makes it quite difficult for
organisations to systems that are suited to their individual needs and circumstances.
So that but but I would argue that if you want a sustainable and scalable system, um, you need a degree of interoperability.
Um, and, and the problem is that these strategic dimensions around scaling sustainability,
uh, in the interest of, um, of the health healthcare system.
Yeah. Strategic decision makers, they are they are stuff that isn't really studied unless you do a big national program evaluation.
Which we have done so. So we got some insights into that.
But actually I think policymakers are currently trying to figure out how we scale and sustain systems.
And because there's no research it's quite difficult. You know, these things have to be tried out in the real world.
Um, and again, um, that's that's also part of the AI lab work that we're doing because they have really they have tried,
um, to scale and start to scale these systems, uh, across, across a sort, across the whole NHS.
Um, and, um, so one example here from the past is the national programme for it, um, where we had, um, a top down,
politically driven programme, you know, and that's important because you can align stakeholders and we can do that in the NHS.
So that's one of the benefits here. Um, we um, you have one kind of strategic vision.
And um, and that that was good because that helped, uh, to secure leadership and support.
But the problem is that you limit local involvement in decision making,
and you have a risk of disengaged disengagement of stakeholders and eventually abandonment, which is actually, um, what's happened?
This is from Lorenzo, which is the system I studied.
Um, which again sounded like a great idea, but they're still working on decommissioning it in the NHS now,
which is for 15 years on something like that.
Um, so, um, so there's this tension between, you know, focusing on these nationally and locally led initiatives.
You can have a perfectly working local system that suits you, suits, um, the local organisation.
But but you can't share information with others with other systems and that compromise large scale interoperability.
So, um, there has been a push here in the NHS to implement, um, uh, mega systems, that big Cerner epic.
Um, because they and if you have one instance of Cerner, which they have in Queensland, I think they have it across 16 different hospitals.
Information can flow completely seamlessly. Um, so that's something that, um, strategic decision makers want.
But organisations sometimes find it quite difficult because it doesn't align with their local needs.
So somehow what we need to strategically do is we need to align organisational drivers with national strategy.
And there will always be some kind of trade off. That's just how it is.
Um. Um, and there's a risk of this, uh, what I mentioned earlier between the clinician doing the data entry and the organisation reaping the benefit,
the same thing applies between organisations and, um, and national, uh, national um, uh, health system benefits.
Yeah. Organisation health system benefits. So we have this federated data platform, uh, being implemented at the moment,
which you might have heard about, but I think that's really showing that tension, um,
that organisations find that quite difficult to sign up for because they can't see the immediate benefits for them,
but they somehow need to be aligned for these systems to work. Um.
So, um, studying, scaling. Um, so what can we do?
What what have we done and what has worked? So that's a phrase that like that.
So we do case studies. Um, and that means looking at, um, at one organisation implementing a system and,
um, comparing it to other organisations implementing the system, a system, a system.
So that. Yeah. All right. This is from then over our quarter two.
We start up here. Perfect. Okay, good. You know, technology was a good one.
Um, so, um. Yeah. No, I'll be finished in two slides.
So, um. Yeah, we need to study. Um, we need to study different contexts,
and we can do that by studying different organisations like we used usually conceptualise these as case studies.
Um, and um, we, uh, I think it's crucially important that we study both macro environments.
And by that I mean the health system dimensions, um, and also the, the micro factors.
So organisational dimensions, but also um, adoption dimensions of the individual interacting with the technology.
And um, we've seen um, yeah.
And the interrelated. So the problem is that all these things are interrelated.
And I've mentioned that a few times, and that makes it very, very complex.
But this is where you need to look if you find any problems.
Um, between what tensions between these dimensions that are usually things that need to be resolved for things to work, I thought.
Makes sense. Um, and ideally we want longitudinal components.
So we want to we want to look at short medium that should say medium and long term transformations that will never end.
Um, and I think, um, yeah.
And we need to plan how systems will integrate within wider health and care delivery infrastructures.
Pathways are really, really important because none of these things are going to exist in isolation anymore.
Um, and uh, another another issue to look at is how to, to some degree, intervention needs to be generalisable to work in different settings.
Um, but also it needs to address, uh, local needs. Um, yeah.
So, I mean, I'm of course this is what I do. I, you know, I'm flying flag performance evaluation.
I think it's really, really important. Um, because it can help us, uh, to to to avert risks.
But it's never done. We get called into called in too late. Um, uh, people don't like to, um, you know, people like to celebrate successes.
And I think that's really, really important. But you can learn most from failures.
Really. Um, and we just need to be slightly more. It's easy for me to say, well, isn't it?
Because it's, you know, I just jet in and criticise people.
But actually, um, I acknowledge that it's really, really difficult to do these things and they're not easy decisions that have to be made.
Um, and it's, uh, we need to think about technological adoption,
organisation and health system factors as early on as possible throughout the technology lifecycle.
Um, problem is that efforts are often invested at the front end and during development.
And we are not thinking early enough about integration and optimisation and scaling.
And, um, I think that's it. I made this with I thought, um, that's me.
Yeah. So if you have any if you have any questions or anything, um, please email me.
Um, you know, I like my job, so I'm very happy to, to get back to you.