Hi, everybody. Welcome to our house and this evening's talk.
I'm really pleased to be able to introduce to you Dr. Fisher, who's associate professor within psychology from the University of Southampton.
And she's going to be our speaking for this evening.
And her topic is going to be looking at her experience of using mixed methods research in health psychology.
So there will be a little bit of time for questions at the end.
Okay. So without further ado, do you think. Okay, so welcome back, everyone.
I hear you've had a full day of mixed methods already.
Most of you so really pleased you've made it back to hear about my experiences.
And I'm hoping to give you give us some of that,
some of the insights I've gained really by doing mixed methods over the past 15 years or so and some of the kind of lessons that I've learned.
Um, I'll start by giving you a brief introduction to why I think we need to use mixed methods in our research and the value I think it has.
And then I'm going to conceptualise mixed methods in terms of technical approaches and philosophical approaches.
And I'm going to argue that we need to conceptualise mixed methods in both of these ways in order to really reap the benefits of
qualitative and quantitative approaches and combining the two in studies or larger scale programmes of research that poses some problems.
So particularly when we take a philosophical approach to thinking about combining methods.
So I'll talk to then about pragmatism as one way to address some of those philosophical challenges of combining quality methods.
Then I'll move on to look at the technical questions in mixing methods.
I'll talk to you a bit about some design typologies that have been presented in the lecture,
just different ways in which we can think about designing mixed method studies,
and then I'll illustrate some of those from the research that I've done.
The idea is primarily looking at the use of complementary medicines, but also looking at placebo effects and web based interventions as well.
So I'll draw on a range of different substantive topics in my examples for you.
And as Margaret said, I hope we'll have time with five or 10 minutes at the end of the questions.
But if there's anything you don't understand as I'm going through, please do ask as I as we go as well.
Okay. So just say we're on the same page here.
Tell me some quantitative methods. I'll take you up.
Survey. Survey. Case control.
Or cohort studies. Yeah. Anything else?
Yeah. Yeah. Good and experiments as well.
We take the kind of non-clinical version of the RCC, so quite a few diverse quantitative approaches we can think about.
What about qualitative methods? When I tell you qualitative, what springs to mind?
Interviews, interviews, focus groups.
Phenomenology could say that we're moving away from the techniques of collecting data to more of that kind of holistic view of qualitative work.
Yeah, that's another big approach to qualitative research and ethnography, right?
And we're thinking about how how way is embedding ourselves as researchers into our topic.
Yeah. Anything else? One more.
I'm thinking of narrative. Narrative?
Yep. Close to narrative. Another one similar to narrative analogy.
And I'm not saying, oh, look, that was great case study, good documents, document, good.
Now it's still one more in my mind, and now I'm surprised that hasn't come up sooner.
Slate Normally thematic is one of the first framework.
Yeah, that's another one. Yes. Conversation analysis and discourse analysis.
They were the other two I was thinking of that are quite different, aren't they?
They're quite different from the other qualitative approaches.
They entail a different view of language and and real focus on how we say things and the linguistic resources we're drawing on,
more so than the kind of interpretive approach that really focuses on the content and the substantive meaning of what we're saying.
Okay, brilliant. So I think we got there.
Yes. Action research. Okay.
So that's another one that traditionally uses qualitative approaches, but actually can be quite a good overarching approach using mixed methods.
So I just want to I like doing this at the start of this session because it's important
to think broadly about what we mean by qualitative and quantitative methods,
I think,
and bear this diversity in mind within each class of methods when we're moving forwards and thinking about how to combine these different approaches.
So. Yeah. What? We don't sleep well between those benefits.
Is that deliberately? That's.
Yeah. My focus is on methods more than the type of data that they produce.
But certainly we can say that in general, these methods produce numerical data and these produce and analyse textual data.
Okay. I'm very happy to make that distinction.
Okay. So why is there even an issue?
We have this diverse smorgasbord of methods available to us as researchers.
Why should we? Why is mixed methods even a thing?
Well, in the past, there's been a real dominance of quantitative methods in medicine and in psychology, which is where I was trained.
But more recently, there's been a growing acceptance of qualitative methods.
Although the recent letters in the BMJ might lead us to doubt that among certain medical groups.
But in general, that's led to more people being interested in combining these approaches,
like more acceptance of qualitative methods in fields traditionally dominated by quantitative approaches.
People are thinking, okay, well, how can we make the most of this? Let's not have two distinct research traditions.
Let's see what we can do to bring them together. Let's see what can be gained from doing that.
And I think it's not really contentious now to say that.
A generally accepted statement is that these methods are necessary to gain a complete understanding of human behaviour,
whether that's in the context of health or in a different context.
And they have complementary sets of strengths and limitations.
So. In mixed methods research, qualitative and quantitative approaches,
or we might call them components or utilise together in a single study or series of related studies.
So like all research, mixed methods, research entails both philosophical assumptions and technical methods of inquiry.
One can conceptualise mixed methods research as methodology.
And when we do that, we think of it as a general approach to research that flows from underlying philosophical assumptions.
Or we can define and conceptualise mixed methods research as a method.
And if we do that, that entails a focus on specific techniques for collecting and then analysing research data.
I'm going to argue that we need to do both and that we can't focus only on the technical
methods of inquiry without also attending to the philosophical questions around methodology.
So let's look a bit deeper at a technical approach and what happens if we take
that and a philosophical approach and what challenges that can throw up.
So if we take a technical approach, we would see quantitative methods simply as techniques for collecting and analysing numerical data.
And we would say qualitative methods like in or semi-structured interviews are focus
groups simply as techniques for collecting and analysing non numerical often,
but not only textual forms of data.
And if we take that approach, that kind of technical approach to mixed methods,
the only things that we're really going to be thinking about are technical questions about procedures.
Yeah. Which method do we do first? Which order do we recruit for the studies?
Which participants should be included? In which component?
Or should we include the same participants in both components?
How are we practically going to go about integrating the findings from our interview study on the one hand with the results of our survey.
On the other hand. So these are undoubtedly important questions.
And I'm sure as you've been working in your groups and developing your projects over the course of this week,
there is a really key questions to actually be able to produce the mixed methods research.
And we are going to come back to them.
But if we focus on them from the start, we run the risk of taking it quite a superficial and unsatisfactory approach to mixed methods research.
So the pitfalls of this technical approach like these happen,
whether you're a qualitative researcher wanting to do a bit of quantitative or a quantitative research, wanting to do a bit of qualitative risk.
If we start off thinking from a quantitative perspective, I think, okay,
I would quite like to do some interviews to collect some more in-depth information about patients experience of illness.
Okay. But. I don't have a good understanding of qualitative methods.
I want to do this to supplement a quantitatively oriented survey that I've already completed.
So I might I might do a decent job of asking open ended questions because know that's we know about that.
That's a pretty clear thing that we have to do if we're using qualitative methods.
But I might not do such a good job at really interpreting the data.
Probing for more detail in the interviews,
using my topic guide in a very flexible way and really interpreting people's
narratives in terms of the subjective meaning of illness to those individuals,
or perhaps taking a more discursive perspective in terms of the discursive resources that people
draw on in order to construct particular experiences of illness in a particular place and time.
So this is the failure to realise the potential of of qualitative methods.
I might also be tempted to try and draw some causal inferences from my data because
that's the way that we tend to think when we're doing quantitative research,
isn't it? We're thinking about causality and universal laws.
I might also think that reliability and validity are absolutely essential and
generalisability because if I'm a kind of fully fledged quantitative research,
I've had that drummed into me from the word go right?
I'm going to be all about validity and reliability and generalisability.
So I might try and obtain a representative sample for my qualitative study or a very large sample, for example.
And then if you take a large sample, you really are going to fail to realise the interpretive potential of your qualitative data,
because you can't do good in that qualitative analysis on very large samples of qualitative interviews.
So. If we fail to think about the underlying assumptions and aims and purposes of qualitative methods
and just treat them as means of collecting and analysing data of a slightly different type,
then we really run the risk of failing to get the most out of these approaches.
The same, of course, is true from the other perspective.
So. If I'm a qualitative researcher and I've been trained to think about purposive sampling and engaging with my
participants in a very open and non-threatening manner and share in control of the research with my participants.
If I then decide to follow up the findings from a qualitative interview study with a survey, I might prioritise patients language.
I might. If it's a telephone survey, I might ask questions in different orders.
I might use my survey tool very flexibly.
I might encourage people to give me their answers in their own words rather than in fixed response categories.
I'm not going to attend to key issues that make quantitative surveys and quantitative research credible.
Okay? Like the reliability and the validity of my survey tools and the need for a large
representative sample if I'm going to draw conclusions that are meaningful for myself,
I think so. Pitfalls come from both sides here.
So how can we avoid these pitfalls? I think you've got this message by now already.
An appreciation of the deep differences between qualitative and quantitative methodologies can help avoid them,
and it can better maximise the potential of qualitative and quantitative components and the combination.
So technical approach lends us to say these different methods are easily compatible, doesn't it?
What's what's the problem with collecting some numerical data and some textual data?
Yeah, seems absolutely fine. But when we start to take a philosophical approach.
We see that the methodologies actually and their underlying philosophical assumptions can start pulling us in opposite directions.
And that's where the challenges come in with mixed methods research when viewed from a philosophical perspective.
And this is because quantitative approaches are traditionally associated with post positive epistemology and
qualitative approaches are traditionally associated with interpretative or constructionist epistemology.
So those epistemology pillars in opposing directions in terms of how we think about conduct and interpret our research.
So if we look in a bit more detail how a philosophical approach embeds methods within quite a broad set of assumptions.
A quick recap for those of you who aren't familiar with those terms.
These are really basic, fundamental assumptions about what it is that we're researching,
how we can go about discovering or generating knowledge, what our value systems are and what counts as knowledge.
Okay. So these are really fundamental assumptions.
And in the seventies and eighties, when this was the kind of dominant approach to thinking about qualitative and quantitative methods,
they pulled people so far apart that we had the paradigm wars in social sciences where
really you would have qualitative researchers who wouldn't talk to quantitative research.
You'd have whole departments of qualitative researchers and other departments of quantitative researchers,
and it was often very little communication between the two.
This table just summarises what we can.
What you used to see in research methods, textbooks.
Okay. That highlights the differences between these underpinning sets of assumptions and the ways in which they pull us in opposing directions.
So post positivist epistemology is typically instead a realist belief in an independent reality that we can come to know.
And knowledge is limited only by our technologies of knowing.
So ultimately, while we might not ever be able to attain it,
it is a reasonable goal to try to obtain bias free objective measurement from this perspective.
And we do that in order to try to discover universal laws,
causal relationships between variables that govern behaviour or health processes or outcomes.
On the other hand, a constructionist or an interpretive tive epistemology typically entails a relativist
belief in the world that is only knowable through our conceptual frameworks,
through our language, through our cultural upbringing, through a lens, our own unique lens on the world, if you like.
And so those conceptual frameworks can differ and do differ between individuals and between cultures.
So from this perspective, knowledge is also inescapably embedded in these frameworks,
in values and in cultures, including the research process itself and the culture of research itself.
So rather than striving to achieve some kind of objective or universal law, which really doesn't make sense from that perspective, instead,
a constructionist approach would strive to develop locally situated and contextualised
understandings of behaviour of human health through inherently subjective means.
Indeed, this subjectivity is values. Okay.
And when you draw out those differences in that way, people come to think can come to think that they are irreconcilable.
Okay. And this slide just gives you some examples of how a post positivist might think about a constructionist approach and vice versa.
Now. I don't think these approaches are irreconcilable.
Otherwise I wouldn't have done any research right.
But what I do think is that in order to obtain and conduct good quality, mixed methods research, we need to attend to these underlying assumptions.
And we need to maintain the integrity of each of our components within mixed methods research.
So we need to appreciate that qualitative methods come from a different set of
assumptions than those that guide us when we're doing quantitative research.
And one way in which we can legitimately then combine them,
combine these approaches into one program or one project of research is by drawing on pragmatist philosophies.
Now. Sometimes when you say pragmatism, that immediately sparks off a kind of, oh, that sounds like you're just doing whatever works.
Yeah, that's not what pragmatist approaches to mixed methods or about pragmatist approaches to mixed methods,
or about drawing on the pragmatist philosophers like Dewey,
like William James, like Richard Rorty to try to develop a philosophically grounded approach to mixed methods research.
There are diverse philosophical approaches to pragmatism.
If you want an introduction to that, there's a great article on the Stanford Encyclopaedia of Philosophy, which is a really excellent source.
And because of the diversity in philosophical positions,
there is understandably some diversity in what methodologies have then taken
from that literature and try to apply and develop to mixed methods research.
So there are subtly different pragmatist positions that you'll find in the mixed methods literature.
But essentially pragmatist approaches acknowledge the epistemological differences between qualitative and quantitative methods,
but don't see these different forms of inquiry as incommensurable.
So instead, pragmatism advocates a shared aim for all research to produce positive change in the world.
Pragmatism typically entails the rejection of this subjective,
objective dualism that we saw in those comparison slides on the philosophical assumptions of interpretive and post positivist work.
So pragmatism rejects that distinction.
It views scientific theories as provisional and achieved through both experience and experimentation in the world.
Sees knowledge as being both constructed and grounded in the world.
So you can see from that description that it's actually.
Compatible in some ways with a constructionist set of assumptions and in some ways with a post positivist set of assumptions.
So interpreted, interpretation is essential and unavoidable, but needs to be grounded in experience.
Okay. And experimentation can be valuable.
But only if it produces positive change in the world.
It's not valuable because it reaches some kind of ideal truth.
That's not why we would value experimentation. So.
This means that we're not asking whether knowledge produced by research accurately represents reality,
but we need to ask whether it has valuable external consequences in the context of the researchers own time and place.
For example, we'd want to know whether our research leads to improved quality of life for patients with a particular
illness or leads to more effective public health services targeting a particular health behaviour.
From this perspective,
then all research should be evaluated according to the extent to which it produces its own pre-specified desired external consequences.
But what those consequences are and how we evaluate whether they've been achieved or not is going to differ depending on what message you face.
Okay. And addressing quality criteria for qualitative components and addressing quality criteria for quantitative components when
you're using quantitative methods can help to ensure that your research does achieve its desired external consequences.
Okay,
so this approach directly argues for the need to maintain the integrity of qualitative components and quantitative opponents in a mixed method study.
So we'll be evaluating qualitative research according to criteria such as the commitment and rigour that the researchers showed to their topic.
We'll be evaluating quantitative research according to traditional criteria, like the reliability and validity, the internal validity of the study.
A slightly different approach,
but a really nice what I think is a really nice summary of the implications of pragmatism comes from Tachikawa and Tetley,
who talk about the dictatorship of the research question.
Study what interests you and is of value to you. Study it in the different ways that you deem appropriate and utilise the results
in ways that can bring about positive consequences within your value system.
I really like that as a kind of guide to how we do research.
And it applies clearly beyond the realm of mixed methods as well.
Okay. So that's the philosophy stuff over and done with.
Let's move on to some more kind of concrete issues.
Let's think about, okay, we've decided we can mix methods.
We've decided we've got some kind of philosophical grounding for doing that.
How can we go about doing it? One approach has been to develop typologies of mixed methods designs and these have been proliferated over the years.
Creswell and Clay in 2011 documented over 15 of these topologies.
There's been plenty more since then as well, and this can be daunting for someone coming to mixed methods research.
Not only do you have to decide on a design, but you have to decide on a framework for choosing your design in the first place.
It can also be constraining, though, for people wanting to do really kind of big elaborate programmes of mixed methods research.
Most of the design typologies are actually quite simple sometimes that have been called
a bit simplistic and don't really cater for these programmatic approaches to research.
So these were the kind of four main designs that were offered by Creswell in Plano,
Clark ten years ago in one of the earlier textbooks on mixed methods research.
They differ in terms of these two designs here at.
Involves the components being done in sequence in order.
These designs are the concomitant designs when the two components are done together at the same time,
or very close together in time, and in three of them we have one component emphasised or prioritised.
That means it's given more resources to it and the interpretation is more strongly weighted towards that component.
But then in the triangulation design, we have a design where we can equally weight our two components and interpret
our overall research on the basis of both the Qual and the client findings.
In 2011, they added two more designs in the revision of their textbook.
The transformative design was. A little bit empty.
And that basically they they describe that as an alternative to pragmatism.
It's okay if you take a transformative lens to your research, then.
Anything goes. I may be paraphrasing too much there.
That's my view on it.
And the other addition they made was this multiphase design, which addresses this issue of actually, if we're doing programs of research,
there are going to be more complexities involved than we could really fit into any one of these designs.
Okay. So the multiphase design acknowledges that you might have multiple different mixed methods designs
within a larger scale program of work more some may house take a slightly different approach.
Differences here. Again, we've got some sequential, some simultaneous.
So pacing or relative timing is again an important issue for them.
Here we always have a core and a supplemental component.
So in their framework, there isn't an option for equal weighting of the different components.
And you might also spotted we've got to put down the bottom here.
And a couple in here that don't actually meet the definition of mixed methods that I work on.
But that's an interesting one because actually it relates back to kind of our very first slide, doesn't it,
about the diversity of approaches within qualitative methods and then the veracity of approaches within quantitative methods.
Because you could argue that actually a discourse analysis has very little in common with a thematic analysis,
and a thematic analysis might have more in common with a cohort study or survey than it does with a discourse analysis.
And indeed, the underlying assumptions might be closer together as well.
So this is an interesting innovation, I think, incorporating qualitative and qualitative studies and calling that mixed methods.
Yeah. That's all I wanted to say about that one. Okay.
And then we have Johnson and Onward Busey,
who really kind of completes what I think was the implicit model in the Creswell and Plano Clark original taxonomy or framework,
where we have every possibility filled in in this grid and.
And again,
you can see that the differentiating factors of the order in which we use the different components and the emphasis that we give each of them.
Ultimately, though, we can say that designing mixed methods research is like designing any other form of research,
and design shouldn't be driven by what?
Like the look of that one or two, that it should be driven by our research question or in mixed methods research often at research questions.
Okay. So to move on now to give you some examples of how I've combined and methods in some of my research over the years,
and I'm going to use the kind of basic Creswell and Plano Clark typology to, to frame this,
partly because actually that's what I was using when I designed a lot of these studies.
So the set that was one of the earlier ones that became available.
So we'll start with this sequential exploratory design, where we start with a qualitative study.
We emphasise the qualitative study. We finish that, then we move on into a quantitative follow up study.
So we're exploring to start with, and then we're following up with some quantitative work.
So we used this approach when we were developing using a qualitative study to then develop a quantitative questionnaire measure.
And that's really common, right? So many questionnaires are developed.
Having done some preliminary qualitative research first, but they're not often conceptualised as mixed methods research.
We went into this project,
Conceptualising admits mixed methods research because we wanted to really actually get something out of the qualitative work.
We didn't want to just do focus groups to generate items for our questionnaire.
We wanted to understand in much more depth the constructs that we might want to be measuring on that questionnaire,
as well as just finding out what language we should be using to measure those constructs.
So we carried out focus groups we sampled purposefully for our participants with a range of different types of back pain,
duration of back pain, age, gender, occupation, these kinds of criteria we used in our sampling.
And we did that because we wanted to understand the variation.
And we wanted to be able to identify key ways in which people common ways in
which people from diverse backgrounds think about treatments for low back pain.
And we were able to do that. We identified four domains along which participants evaluated different treatments for back pain,
and they discussed a whole range of therapies in these focus groups.
They talked about painkillers, they talked about surgery.
They talked about Alexander's technique, about acupuncture, about all sorts of different things.
But it didn't matter what therapy they were talking about, and it didn't matter whether they had it or not.
These were the things they were interested in. Is it a credible therapy?
Is it effective? Has it been effective for me? Is it likely to be effective for me if I haven't tried it?
What concerns does it raise me? Is it going to hurt? Do the needles feel funny in acupuncture?
If I go to a chiropractor, what does the clicking really mean?
What are they doing to make an individual fit?
Is it right for me? Does this therapy feel right for me?
Is it consistent with my beliefs? Does it fit with my lifestyle?
We also found that these specific domains were contextualised within a kind of
broader experience of seeking treatment in which patients were often disappointed,
felt like they were being dismissed by GP's who say, okay, we've tried that, let's try this, let's try this next with no rationale, no diagnosis.
So patients felt like they were experiencing a process of trial and error when they were seeking treatment and they didn't like that.
So we then set out onto a quantitative phase where we try to measure these constructs and to establish whether, first of all, they could be measured,
and then to set out on further research to look at whether they had any relationships
with important outcomes like adherence to treatment or satisfaction with treatment.
So we developed our questionnaire.
We based the wording on the qualitative work, and we surveyed a large number of back pain patients from primary care.
We got some quite sophisticated stats to look at the results, and we found that our survey measured these specific domains quite nicely.
They came out in the factor analysis and the item response theory analysis,
but we really weren't very good at measuring these more contextual domains.
So. That was quite interesting for us.
And we thought that, okay, on reflection,
these were very well defined and actually having very well defined constructs is what you need when you're doing quantitative
research and this contextual stuff around the process of treatment seeking was much harder to pin down into questionnaire items.
And with hindsight, okay, well, that maybe that's not something we want to be measuring, actually.
Maybe that is something that has to be explored qualitatively with people.
Maybe quantitative methods aren't appropriate for all aspects of this process of seeking treatment for low back pain.
And we published these two pieces of work separately, which allowed us to really do justice to each of them.
And it meant because of the sequential design, it meant that this this paper could cite and draw on the earlier qualitative papers.
So we could really explain where our constructs and wording had come from.
But what it didn't really encourage us to do was, was make those links back again and think about like,
well, what does that mean for our qualitative data? And she think about feeding that back into the process.
Questionnaire design isn't the only way to use this quote and design.
I also used it in a small study to explicitly tests and ideas derived from a piece of qualitative work.
So we used qualitative work to identify an issue of importance to patients.
We interviewed people about their experiences of acupuncture.
One of the issues that came up was how they found an acupuncturist and how they
decided who to go and consult once they made a decision to try acupuncture.
And the we found that this choice is important and that patients consider how is this person qualified?
Are they going to be good? Can they do the job? Are they safe?
But also, are they friendly? Am I going to get on with them?
Can I relate to them? Do I trust them? So the personal qualities were also important.
So. So, okay, let's see if we can validate that and let's see if that applies to a large sample of people in a quantitative study.
So we used an experimental design.
We asked people to rate fictional acupuncturists, and we varied descriptions of those according to gender training, their qualifications.
And we also looked at patient gender as well. And what we found is that in general, our patients preferred the female acupuncturists,
and they preferred acupuncturists who were both medically qualified as well as being qualified in acupuncture.
So this provided us with a bit of kind of broader insight into the factors that might actually influence the decision.
So here we were able to think about some causal relationships that qualitative work had suggested,
but of course would in no way actually be able to test or or prove.
We published this in one paper in a medical journal where the word limits 3000 words is difficult to publish the qualitative study in 3000 words,
let alone a qualitative study and an experiment together.
We didn't do justice to it. We really did not do justice to the qualitative findings.
But we were able to clearly show how the different components related to each other.
And it did help us to then look back at the qualitative data and think about using that again to help us interpret the statistical results.
Okay. So this design flips and we use the quantitative component first and we follow up with a qualitative component.
Essentially, the idea here is that you use the qualitative component to explain or contextualise the results of the quantitative work.
And this design can be a little bit challenging to organise,
particularly because if you're using the qualitative work to try and explain the quantitative,
oftentimes you're going to want the same participants involved in both these components,
but you won't know until you finish the quantitative what you're actually going to want to talk to the qualitative participants about.
So in terms of getting funding or getting ethical approval,
this design can sometimes be a little bit challenging and can involve going back for revisions and things.
So this is kind of a mirror study to the acupuncturists study.
In this study, we looked at how patients choose osteopath.
Oh, okay. This time let's start with our quantitative study.
There's actually a lot of information about gender effects and choice of medical practitioner.
So let's go with that existing literature and test some of these hypotheses in our context of osteopath.
Again, the same idea. We had fictional vignettes describing osteopath.
Patients who received the survey had to say which ones they would choose.
Would you choose this one? Definitely not. Definitely, yes.
Kind of set it up like a Yellow Pages advert kind of thing. We had a simple random sample from the general population in this study,
and similar to the acupuncture study, it was qualifications that the respondents cared about.
We also had a bit of a problem with the quantitative results in that 16% of the questionnaires that came back to us were actually unusable.
People had refused to endorse or not endorse any use.
The osteopath that we gave them and the kind of that we felt like that was quite a lot.
We expect to the low response rate. We didn't have any incentives to take part.
We didn't have any follow up emails, but we didn't expect people to bother sending the questionnaire back but not complete that.
So that was an old one. So we thought, okay, actually that's something we want to follow up.
And as is often the case with these small scale studies, we didn't have any funding to do any more work on it.
So we've got some we've got some interviews already with some osteopathy patients.
Let's see if there's anything in there.
So we did a secondary analysis of some existing qualitative interviews which didn't really tell us anything about this.
It didn't help us to explain any further the impact of qualifications or the lack of impact of gender or health care sector that we found.
But it did help us to explain the missing data, because in the interviews,
people talked so much about the importance of a personal recommendation for their osteopath.
And that was what the osteopathy patients that we talked to really cared about.
So that helped us to explain our missing data.
Clearly in hindsight we would have preferred or even at the time we would have preferred to do new qualitative
interviews rather than rely on existing data that didn't quite ask the questions we wanted to be asking.
But I think given the real world of resource constraints, actually, you can do secondary data analysis and incorporate that into mixed methods design.
And it can help. It can help you generate some explanations for puzzling findings.
So moving on to the triangulation design. This is about using different components to analyse and investigate different facets of the same topic.
This was work that I did with Jeremy Howick, who Station Primary Care Department here at Oxford.
We did an online survey, a 783 British GP's about their use of placebos in clinical practice.
We wanted to know do TPS use placebos? Did they use placebo effects?
What do they think these things are? What are their attitudes towards them?
And so we had some open ended and some close ended questions.
So within the same data collection instrument, we had a qualitative and quantitative component,
the quantitative measured, the prevalence of use and the prevalence of certain attitudes about using placebos.
The qualitative component. We took all of the answers to the open ended questions and we analysed them descriptively.
And we looked at and were able to describe various different definitions that Tpz had in mind of placebos and placebo effects.
We found that the ethical issues were key to GDP and societal and regulatory issues.
They wanted guidance about this issue. And we found some really interesting information about why they thought doctors might use placebos.
Some of this was range from doctors saying, well,
it's feebleminded doctors who can't do any better and shouldn't be practising any more on the one hand, too.
On the other hand we had GP saying things like, well, placebo effects are inherent in good quality general practice,
they're inescapable and they're a power that we should be harnessing.
So we had a huge diversity of views that we really hadn't captured in the quantitative component at all.
And here, the qualitative aspect really added some depth and insight into the reasons why people were and weren't using placebos in practice.
The final design I want to illustrate for you is this embedded design where we have one component that's much larger,
the real kind of focus of the research and a smaller component embedded in it.
And the classical example of this design is embedding a small scale qualitative study within a large scale major ask.
And this is work that Sarah did with me back when she was in Southampton with Hazel Everett,
where we were working on a trial of web based CBT for patients with IBS.
And the concept of trial was one of these really complex,
multifactorial trials with patients with randomised to one of three different medication arms and one of three self-management arms.
The qualitative work we sampled from all of those different groups in the trial.
So we had that trial randomisation as a sampling frame for the qualitative work,
and we wanted primarily to explore patients experiences in order to improve the intervention
and the trial design for a future trial that is actually now just coming to a close.
So the findings from this qualitative component were used to improve the intervention that has now been trialled in a large scale major trial.
We found the patients could be classified into different types of engagement with the website.
And an interesting thing when we related the qualitative findings here back to the quantitative data we had on the same participants,
was that some of our participants, who really from their own descriptions, had very limited or even no engagement with the website,
had actually been classified quantitatively as adhering to the intervention.
Okay.
So that gave us really convincing evidence that we needed better measures of adherence and use of the website in the main trial that we have now use.
Okay. So just to summarise some of this for you, I think we can think about the strengths and limitations of sequential and concurrent designs.
That's a nice way to think about it,
rather than having to think about the strengths and limitations of the multiple different designs in the typologies.
I find it a lot easier to retain the integrity of components in a sequential design because you do one study,
you evaluate it on its own terms, you think about validity in those terms.
Then you can switch mindsets and think about the next study and become fully engrossed in that paradigm.
And as a single researcher, that's certainly the case.
When you've got multiple studies of two components going on at the same time,
it's actually quite easy to get a bit muddled, especially if you're involved centrally in both of those components.
But then that's where teamwork comes in and that's absolutely vital, I think for the current study designs.
I think it's very difficult as a lone researcher, as a PhD student, for example, to do a concurrent mixed method study.
Basically the model for the sequential design is kind of long and thin, takes a long time,
but for the concurrent design, shorter and fatter, more resources, more intensive.
But it's over and done with much quicker. You finish it quicker.
I think sequential studies lend themselves more to single publication, 2 to 2 being published separately,
but actually that's you can publish them together and the same goes for concurrent.
They tend to lend themselves to being published together because you've got the
you've often got the same participants and they're often a bit more closely linked.
But actually both of the concurrent designs I've just the examples I've just given you, we publish them separately.
So sometimes they do come out separately. As I mentioned when we were going through,
it can be difficult to fully specify the second component in a sequential design in advance for funders or for regulatory bodies.
Whereas in concurrent designs, it's not only doable, but you kind of have to do it.
Some of the challenges I've encountered over the years.
And when you find discrepancies between your two components, you say, Oh, hang on a minute, that contradicts that.
Which one do I believe actually turns out?
That's a really great opportunity to look more critically at each piece of work.
So I'll give you an example of that. We did a mixed methods trial where we wanted to look at whether acupuncturists being more empathetic with
their patients would be more effective than acupuncturists who were just very kind of matter of fact,
no chitchat, none of the how are you?
How are the grandkids kind of thing?
The qualitative work suggested that empathy was really important and patients felt cared for and they really appreciate that the quantitative.
The. Empathy had no effect on the outcome.
So. So initially we're like, okay, there's something funny going on here.
What is that? We look deeper into the qualitative work and the empathy that some of the patients were describing was not actually coming from the US.
It was coming from everyone else in the trial.
From that, even people very tangentially involved, like the receptionist at the medical facility where the trial was being run.
They were a clinical trials unit. The receptionist was lovely and friendly, so the patients felt cared for and treated as individuals,
even though the acupuncturist was just absolutely matter of fact and adhered to that protocol.
So actually, the apparent discrepancies forced us to go back and look much more critically at our data, which is very helpful.
It's difficult to do justice to each component and the mixed methods component as well.
No matter how you publish, I think at the end of the day,
although one model seems to be potentially publishing separate papers and then publishing a mixed methods paper where you link
the two components more and team working as in any big team project can be challenging on mixed methods projects particularly.
So if you've got team members who are very staunch advocates of one method or another,
and that's where it's really helpful to have someone who can bridge across and and can really open up discussions amongst the team.
I think in future we need more training as a support for for mixed methods research.
I love the fact that you have a whole module devoted to it here.
We're quite good at training people to do quantitative research and qualitative research, but mixed methods is a whole other ball game.
So we need to be skilling people up to do these methods research.
We need to be encouraging publications so that we've got examples to look at in the literature as a good practice in mixed methods research.
And I think as as we're publishing in open access journals and word limits it to an extent,
in some cases becoming a thing of the past that gives us a bit more flexibility and scope for publishing.
Mixed methods work, and I think it's increasing.
It's really important that when we publish mixed methods research to actually describe it as such and to provide a
rationale for why we're combining the different components that we're combining and how we make decisions about that.
Okay. So mixed methods designs can help us to think through the technical challenges of mixed methods research.
But if we find ways of resolving those which we can do,
that really shouldn't obfuscate the need to address the philosophical challenges and the underlying assumptions as well.
And pragmatism offers this just one approach to doing that.
And thank you very much for your attention. I hope that's made you think more and maybe in a slightly different way about mixed methods.
And I'm very happy to stay for some questions.