1 00:00:01,290 --> 00:00:06,580 Hi, everybody. Welcome to our house and this evening's talk. 2 00:00:06,600 --> 00:00:17,069 I'm really pleased to be able to introduce to you Dr. Fisher, who's associate professor within psychology from the University of Southampton. 3 00:00:17,070 --> 00:00:19,710 And she's going to be our speaking for this evening. 4 00:00:20,100 --> 00:00:26,460 And her topic is going to be looking at her experience of using mixed methods research in health psychology. 5 00:00:26,820 --> 00:00:31,050 So there will be a little bit of time for questions at the end. 6 00:00:31,320 --> 00:00:36,959 Okay. So without further ado, do you think. Okay, so welcome back, everyone. 7 00:00:36,960 --> 00:00:39,960 I hear you've had a full day of mixed methods already. 8 00:00:39,960 --> 00:00:45,480 Most of you so really pleased you've made it back to hear about my experiences. 9 00:00:45,480 --> 00:00:50,250 And I'm hoping to give you give us some of that, 10 00:00:50,250 --> 00:01:00,790 some of the insights I've gained really by doing mixed methods over the past 15 years or so and some of the kind of lessons that I've learned. 11 00:01:01,020 --> 00:01:11,220 Um, I'll start by giving you a brief introduction to why I think we need to use mixed methods in our research and the value I think it has. 12 00:01:12,630 --> 00:01:19,170 And then I'm going to conceptualise mixed methods in terms of technical approaches and philosophical approaches. 13 00:01:19,740 --> 00:01:27,809 And I'm going to argue that we need to conceptualise mixed methods in both of these ways in order to really reap the benefits of 14 00:01:27,810 --> 00:01:38,260 qualitative and quantitative approaches and combining the two in studies or larger scale programmes of research that poses some problems. 15 00:01:38,260 --> 00:01:42,540 So particularly when we take a philosophical approach to thinking about combining methods. 16 00:01:42,960 --> 00:01:51,780 So I'll talk to then about pragmatism as one way to address some of those philosophical challenges of combining quality methods. 17 00:01:52,860 --> 00:01:57,360 Then I'll move on to look at the technical questions in mixing methods. 18 00:01:58,080 --> 00:02:02,510 I'll talk to you a bit about some design typologies that have been presented in the lecture, 19 00:02:02,560 --> 00:02:07,800 just different ways in which we can think about designing mixed method studies, 20 00:02:08,250 --> 00:02:12,909 and then I'll illustrate some of those from the research that I've done. 21 00:02:12,910 --> 00:02:20,879 The idea is primarily looking at the use of complementary medicines, but also looking at placebo effects and web based interventions as well. 22 00:02:20,880 --> 00:02:25,890 So I'll draw on a range of different substantive topics in my examples for you. 23 00:02:28,000 --> 00:02:32,440 And as Margaret said, I hope we'll have time with five or 10 minutes at the end of the questions. 24 00:02:32,710 --> 00:02:38,270 But if there's anything you don't understand as I'm going through, please do ask as I as we go as well. 25 00:02:38,300 --> 00:02:42,370 Okay. So just say we're on the same page here. 26 00:02:43,810 --> 00:02:49,310 Tell me some quantitative methods. I'll take you up. 27 00:02:51,650 --> 00:02:55,800 Survey. Survey. Case control. 28 00:02:57,290 --> 00:03:01,010 Or cohort studies. Yeah. Anything else? 29 00:03:06,820 --> 00:03:11,020 Yeah. Yeah. Good and experiments as well. 30 00:03:11,020 --> 00:03:19,280 We take the kind of non-clinical version of the RCC, so quite a few diverse quantitative approaches we can think about. 31 00:03:19,300 --> 00:03:23,860 What about qualitative methods? When I tell you qualitative, what springs to mind? 32 00:03:25,470 --> 00:03:28,950 Interviews, interviews, focus groups. 33 00:03:30,570 --> 00:03:40,050 Phenomenology could say that we're moving away from the techniques of collecting data to more of that kind of holistic view of qualitative work. 34 00:03:41,200 --> 00:03:49,700 Yeah, that's another big approach to qualitative research and ethnography, right? 35 00:03:49,710 --> 00:03:57,360 And we're thinking about how how way is embedding ourselves as researchers into our topic. 36 00:03:57,390 --> 00:04:01,790 Yeah. Anything else? One more. 37 00:04:01,800 --> 00:04:04,810 I'm thinking of narrative. Narrative? 38 00:04:04,840 --> 00:04:10,290 Yep. Close to narrative. Another one similar to narrative analogy. 39 00:04:10,560 --> 00:04:20,040 And I'm not saying, oh, look, that was great case study, good documents, document, good. 40 00:04:20,040 --> 00:04:25,980 Now it's still one more in my mind, and now I'm surprised that hasn't come up sooner. 41 00:04:26,190 --> 00:04:30,149 Slate Normally thematic is one of the first framework. 42 00:04:30,150 --> 00:04:35,610 Yeah, that's another one. Yes. Conversation analysis and discourse analysis. 43 00:04:36,120 --> 00:04:39,720 They were the other two I was thinking of that are quite different, aren't they? 44 00:04:40,050 --> 00:04:42,870 They're quite different from the other qualitative approaches. 45 00:04:42,870 --> 00:04:50,940 They entail a different view of language and and real focus on how we say things and the linguistic resources we're drawing on, 46 00:04:51,750 --> 00:04:59,610 more so than the kind of interpretive approach that really focuses on the content and the substantive meaning of what we're saying. 47 00:04:59,670 --> 00:05:05,100 Okay, brilliant. So I think we got there. 48 00:05:05,640 --> 00:05:08,190 Yes. Action research. Okay. 49 00:05:08,190 --> 00:05:16,380 So that's another one that traditionally uses qualitative approaches, but actually can be quite a good overarching approach using mixed methods. 50 00:05:17,850 --> 00:05:22,620 So I just want to I like doing this at the start of this session because it's important 51 00:05:22,620 --> 00:05:27,530 to think broadly about what we mean by qualitative and quantitative methods, 52 00:05:27,540 --> 00:05:27,960 I think, 53 00:05:28,200 --> 00:05:39,360 and bear this diversity in mind within each class of methods when we're moving forwards and thinking about how to combine these different approaches. 54 00:05:41,620 --> 00:05:46,510 So. Yeah. What? We don't sleep well between those benefits. 55 00:05:47,020 --> 00:05:50,240 Is that deliberately? That's. 56 00:05:50,930 --> 00:05:56,990 Yeah. My focus is on methods more than the type of data that they produce. 57 00:05:57,230 --> 00:06:06,110 But certainly we can say that in general, these methods produce numerical data and these produce and analyse textual data. 58 00:06:06,680 --> 00:06:10,040 Okay. I'm very happy to make that distinction. 59 00:06:12,650 --> 00:06:15,870 Okay. So why is there even an issue? 60 00:06:15,890 --> 00:06:20,690 We have this diverse smorgasbord of methods available to us as researchers. 61 00:06:21,050 --> 00:06:24,140 Why should we? Why is mixed methods even a thing? 62 00:06:25,580 --> 00:06:35,060 Well, in the past, there's been a real dominance of quantitative methods in medicine and in psychology, which is where I was trained. 63 00:06:36,770 --> 00:06:40,790 But more recently, there's been a growing acceptance of qualitative methods. 64 00:06:42,410 --> 00:06:52,190 Although the recent letters in the BMJ might lead us to doubt that among certain medical groups. 65 00:06:54,520 --> 00:07:00,430 But in general, that's led to more people being interested in combining these approaches, 66 00:07:00,480 --> 00:07:06,850 like more acceptance of qualitative methods in fields traditionally dominated by quantitative approaches. 67 00:07:07,330 --> 00:07:12,970 People are thinking, okay, well, how can we make the most of this? Let's not have two distinct research traditions. 68 00:07:12,980 --> 00:07:17,650 Let's see what we can do to bring them together. Let's see what can be gained from doing that. 69 00:07:19,840 --> 00:07:24,850 And I think it's not really contentious now to say that. 70 00:07:27,130 --> 00:07:34,930 A generally accepted statement is that these methods are necessary to gain a complete understanding of human behaviour, 71 00:07:36,040 --> 00:07:39,700 whether that's in the context of health or in a different context. 72 00:07:41,080 --> 00:07:46,450 And they have complementary sets of strengths and limitations. 73 00:07:51,930 --> 00:07:58,080 So. In mixed methods research, qualitative and quantitative approaches, 74 00:07:58,440 --> 00:08:05,910 or we might call them components or utilise together in a single study or series of related studies. 75 00:08:06,780 --> 00:08:16,710 So like all research, mixed methods, research entails both philosophical assumptions and technical methods of inquiry. 76 00:08:18,330 --> 00:08:22,660 One can conceptualise mixed methods research as methodology. 77 00:08:23,520 --> 00:08:31,530 And when we do that, we think of it as a general approach to research that flows from underlying philosophical assumptions. 78 00:08:33,020 --> 00:08:38,720 Or we can define and conceptualise mixed methods research as a method. 79 00:08:39,910 --> 00:08:50,710 And if we do that, that entails a focus on specific techniques for collecting and then analysing research data. 80 00:08:55,660 --> 00:09:02,290 I'm going to argue that we need to do both and that we can't focus only on the technical 81 00:09:02,290 --> 00:09:09,250 methods of inquiry without also attending to the philosophical questions around methodology. 82 00:09:10,770 --> 00:09:15,209 So let's look a bit deeper at a technical approach and what happens if we take 83 00:09:15,210 --> 00:09:20,100 that and a philosophical approach and what challenges that can throw up. 84 00:09:23,330 --> 00:09:33,440 So if we take a technical approach, we would see quantitative methods simply as techniques for collecting and analysing numerical data. 85 00:09:34,310 --> 00:09:40,879 And we would say qualitative methods like in or semi-structured interviews are focus 86 00:09:40,880 --> 00:09:47,660 groups simply as techniques for collecting and analysing non numerical often, 87 00:09:47,660 --> 00:09:51,540 but not only textual forms of data. 88 00:09:53,520 --> 00:09:57,870 And if we take that approach, that kind of technical approach to mixed methods, 89 00:09:59,340 --> 00:10:05,700 the only things that we're really going to be thinking about are technical questions about procedures. 90 00:10:06,520 --> 00:10:16,090 Yeah. Which method do we do first? Which order do we recruit for the studies? 91 00:10:16,930 --> 00:10:20,650 Which participants should be included? In which component? 92 00:10:21,010 --> 00:10:24,040 Or should we include the same participants in both components? 93 00:10:24,790 --> 00:10:34,180 How are we practically going to go about integrating the findings from our interview study on the one hand with the results of our survey. 94 00:10:34,480 --> 00:10:40,120 On the other hand. So these are undoubtedly important questions. 95 00:10:40,120 --> 00:10:46,240 And I'm sure as you've been working in your groups and developing your projects over the course of this week, 96 00:10:46,570 --> 00:10:51,580 there is a really key questions to actually be able to produce the mixed methods research. 97 00:10:52,690 --> 00:10:54,280 And we are going to come back to them. 98 00:10:55,090 --> 00:11:05,440 But if we focus on them from the start, we run the risk of taking it quite a superficial and unsatisfactory approach to mixed methods research. 99 00:11:07,990 --> 00:11:13,810 So the pitfalls of this technical approach like these happen, 100 00:11:14,980 --> 00:11:25,510 whether you're a qualitative researcher wanting to do a bit of quantitative or a quantitative research, wanting to do a bit of qualitative risk. 101 00:11:30,620 --> 00:11:35,410 If we start off thinking from a quantitative perspective, I think, okay, 102 00:11:35,420 --> 00:11:45,620 I would quite like to do some interviews to collect some more in-depth information about patients experience of illness. 103 00:11:46,430 --> 00:11:52,040 Okay. But. I don't have a good understanding of qualitative methods. 104 00:11:52,050 --> 00:11:57,540 I want to do this to supplement a quantitatively oriented survey that I've already completed. 105 00:11:59,280 --> 00:12:06,809 So I might I might do a decent job of asking open ended questions because know that's we know about that. 106 00:12:06,810 --> 00:12:10,800 That's a pretty clear thing that we have to do if we're using qualitative methods. 107 00:12:12,510 --> 00:12:19,500 But I might not do such a good job at really interpreting the data. 108 00:12:20,400 --> 00:12:24,450 Probing for more detail in the interviews, 109 00:12:24,810 --> 00:12:32,070 using my topic guide in a very flexible way and really interpreting people's 110 00:12:32,370 --> 00:12:39,300 narratives in terms of the subjective meaning of illness to those individuals, 111 00:12:39,810 --> 00:12:45,629 or perhaps taking a more discursive perspective in terms of the discursive resources that people 112 00:12:45,630 --> 00:12:53,190 draw on in order to construct particular experiences of illness in a particular place and time. 113 00:12:56,700 --> 00:13:02,430 So this is the failure to realise the potential of of qualitative methods. 114 00:13:02,640 --> 00:13:08,490 I might also be tempted to try and draw some causal inferences from my data because 115 00:13:08,490 --> 00:13:12,840 that's the way that we tend to think when we're doing quantitative research, 116 00:13:12,840 --> 00:13:16,230 isn't it? We're thinking about causality and universal laws. 117 00:13:17,370 --> 00:13:24,089 I might also think that reliability and validity are absolutely essential and 118 00:13:24,090 --> 00:13:29,430 generalisability because if I'm a kind of fully fledged quantitative research, 119 00:13:29,490 --> 00:13:32,610 I've had that drummed into me from the word go right? 120 00:13:32,850 --> 00:13:37,770 I'm going to be all about validity and reliability and generalisability. 121 00:13:38,070 --> 00:13:45,270 So I might try and obtain a representative sample for my qualitative study or a very large sample, for example. 122 00:13:45,720 --> 00:13:52,110 And then if you take a large sample, you really are going to fail to realise the interpretive potential of your qualitative data, 123 00:13:52,110 --> 00:14:00,660 because you can't do good in that qualitative analysis on very large samples of qualitative interviews. 124 00:14:02,700 --> 00:14:11,849 So. If we fail to think about the underlying assumptions and aims and purposes of qualitative methods 125 00:14:11,850 --> 00:14:17,040 and just treat them as means of collecting and analysing data of a slightly different type, 126 00:14:17,820 --> 00:14:23,070 then we really run the risk of failing to get the most out of these approaches. 127 00:14:25,520 --> 00:14:29,510 The same, of course, is true from the other perspective. 128 00:14:30,110 --> 00:14:43,319 So. If I'm a qualitative researcher and I've been trained to think about purposive sampling and engaging with my 129 00:14:43,320 --> 00:14:51,000 participants in a very open and non-threatening manner and share in control of the research with my participants. 130 00:14:51,630 --> 00:15:05,070 If I then decide to follow up the findings from a qualitative interview study with a survey, I might prioritise patients language. 131 00:15:05,070 --> 00:15:09,060 I might. If it's a telephone survey, I might ask questions in different orders. 132 00:15:10,020 --> 00:15:12,720 I might use my survey tool very flexibly. 133 00:15:14,040 --> 00:15:21,300 I might encourage people to give me their answers in their own words rather than in fixed response categories. 134 00:15:22,110 --> 00:15:30,030 I'm not going to attend to key issues that make quantitative surveys and quantitative research credible. 135 00:15:30,660 --> 00:15:38,190 Okay? Like the reliability and the validity of my survey tools and the need for a large 136 00:15:38,520 --> 00:15:45,510 representative sample if I'm going to draw conclusions that are meaningful for myself, 137 00:15:45,530 --> 00:15:50,670 I think so. Pitfalls come from both sides here. 138 00:15:52,230 --> 00:15:56,550 So how can we avoid these pitfalls? I think you've got this message by now already. 139 00:15:57,240 --> 00:16:03,240 An appreciation of the deep differences between qualitative and quantitative methodologies can help avoid them, 140 00:16:03,660 --> 00:16:10,200 and it can better maximise the potential of qualitative and quantitative components and the combination. 141 00:16:13,280 --> 00:16:20,600 So technical approach lends us to say these different methods are easily compatible, doesn't it? 142 00:16:21,500 --> 00:16:26,300 What's what's the problem with collecting some numerical data and some textual data? 143 00:16:27,140 --> 00:16:32,150 Yeah, seems absolutely fine. But when we start to take a philosophical approach. 144 00:16:33,940 --> 00:16:41,800 We see that the methodologies actually and their underlying philosophical assumptions can start pulling us in opposite directions. 145 00:16:42,960 --> 00:16:51,870 And that's where the challenges come in with mixed methods research when viewed from a philosophical perspective. 146 00:16:52,920 --> 00:17:01,469 And this is because quantitative approaches are traditionally associated with post positive epistemology and 147 00:17:01,470 --> 00:17:09,530 qualitative approaches are traditionally associated with interpretative or constructionist epistemology. 148 00:17:10,260 --> 00:17:18,810 So those epistemology pillars in opposing directions in terms of how we think about conduct and interpret our research. 149 00:17:21,590 --> 00:17:31,220 So if we look in a bit more detail how a philosophical approach embeds methods within quite a broad set of assumptions. 150 00:17:33,600 --> 00:17:37,020 A quick recap for those of you who aren't familiar with those terms. 151 00:17:37,050 --> 00:17:42,810 These are really basic, fundamental assumptions about what it is that we're researching, 152 00:17:42,810 --> 00:17:53,850 how we can go about discovering or generating knowledge, what our value systems are and what counts as knowledge. 153 00:17:54,840 --> 00:17:57,900 Okay. So these are really fundamental assumptions. 154 00:17:59,310 --> 00:18:07,920 And in the seventies and eighties, when this was the kind of dominant approach to thinking about qualitative and quantitative methods, 155 00:18:08,370 --> 00:18:13,469 they pulled people so far apart that we had the paradigm wars in social sciences where 156 00:18:13,470 --> 00:18:17,690 really you would have qualitative researchers who wouldn't talk to quantitative research. 157 00:18:18,420 --> 00:18:24,700 You'd have whole departments of qualitative researchers and other departments of quantitative researchers, 158 00:18:25,290 --> 00:18:29,040 and it was often very little communication between the two. 159 00:18:33,500 --> 00:18:38,240 This table just summarises what we can. 160 00:18:38,990 --> 00:18:43,250 What you used to see in research methods, textbooks. 161 00:18:43,940 --> 00:18:56,030 Okay. That highlights the differences between these underpinning sets of assumptions and the ways in which they pull us in opposing directions. 162 00:18:56,810 --> 00:19:05,240 So post positivist epistemology is typically instead a realist belief in an independent reality that we can come to know. 163 00:19:05,960 --> 00:19:10,130 And knowledge is limited only by our technologies of knowing. 164 00:19:11,730 --> 00:19:17,310 So ultimately, while we might not ever be able to attain it, 165 00:19:17,640 --> 00:19:26,850 it is a reasonable goal to try to obtain bias free objective measurement from this perspective. 166 00:19:27,690 --> 00:19:33,479 And we do that in order to try to discover universal laws, 167 00:19:33,480 --> 00:19:41,160 causal relationships between variables that govern behaviour or health processes or outcomes. 168 00:19:42,660 --> 00:19:49,560 On the other hand, a constructionist or an interpretive tive epistemology typically entails a relativist 169 00:19:49,710 --> 00:19:54,750 belief in the world that is only knowable through our conceptual frameworks, 170 00:19:56,040 --> 00:20:04,860 through our language, through our cultural upbringing, through a lens, our own unique lens on the world, if you like. 171 00:20:05,280 --> 00:20:11,670 And so those conceptual frameworks can differ and do differ between individuals and between cultures. 172 00:20:12,960 --> 00:20:21,150 So from this perspective, knowledge is also inescapably embedded in these frameworks, 173 00:20:21,150 --> 00:20:28,740 in values and in cultures, including the research process itself and the culture of research itself. 174 00:20:29,640 --> 00:20:41,670 So rather than striving to achieve some kind of objective or universal law, which really doesn't make sense from that perspective, instead, 175 00:20:41,970 --> 00:20:48,810 a constructionist approach would strive to develop locally situated and contextualised 176 00:20:48,810 --> 00:20:56,460 understandings of behaviour of human health through inherently subjective means. 177 00:20:56,760 --> 00:21:00,810 Indeed, this subjectivity is values. Okay. 178 00:21:06,210 --> 00:21:16,230 And when you draw out those differences in that way, people come to think can come to think that they are irreconcilable. 179 00:21:17,080 --> 00:21:28,170 Okay. And this slide just gives you some examples of how a post positivist might think about a constructionist approach and vice versa. 180 00:21:34,450 --> 00:21:40,269 Now. I don't think these approaches are irreconcilable. 181 00:21:40,270 --> 00:21:42,670 Otherwise I wouldn't have done any research right. 182 00:21:43,690 --> 00:21:56,620 But what I do think is that in order to obtain and conduct good quality, mixed methods research, we need to attend to these underlying assumptions. 183 00:21:56,620 --> 00:22:04,090 And we need to maintain the integrity of each of our components within mixed methods research. 184 00:22:04,630 --> 00:22:09,640 So we need to appreciate that qualitative methods come from a different set of 185 00:22:09,640 --> 00:22:15,220 assumptions than those that guide us when we're doing quantitative research. 186 00:22:16,980 --> 00:22:21,690 And one way in which we can legitimately then combine them, 187 00:22:21,780 --> 00:22:32,470 combine these approaches into one program or one project of research is by drawing on pragmatist philosophies. 188 00:22:34,200 --> 00:22:43,490 Now. Sometimes when you say pragmatism, that immediately sparks off a kind of, oh, that sounds like you're just doing whatever works. 189 00:22:44,210 --> 00:22:52,000 Yeah, that's not what pragmatist approaches to mixed methods or about pragmatist approaches to mixed methods, 190 00:22:52,010 --> 00:22:56,059 or about drawing on the pragmatist philosophers like Dewey, 191 00:22:56,060 --> 00:23:05,660 like William James, like Richard Rorty to try to develop a philosophically grounded approach to mixed methods research. 192 00:23:06,770 --> 00:23:12,170 There are diverse philosophical approaches to pragmatism. 193 00:23:13,190 --> 00:23:22,940 If you want an introduction to that, there's a great article on the Stanford Encyclopaedia of Philosophy, which is a really excellent source. 194 00:23:24,560 --> 00:23:27,830 And because of the diversity in philosophical positions, 195 00:23:28,100 --> 00:23:33,709 there is understandably some diversity in what methodologies have then taken 196 00:23:33,710 --> 00:23:40,460 from that literature and try to apply and develop to mixed methods research. 197 00:23:40,910 --> 00:23:46,700 So there are subtly different pragmatist positions that you'll find in the mixed methods literature. 198 00:23:48,870 --> 00:23:57,690 But essentially pragmatist approaches acknowledge the epistemological differences between qualitative and quantitative methods, 199 00:23:58,020 --> 00:24:03,330 but don't see these different forms of inquiry as incommensurable. 200 00:24:07,950 --> 00:24:16,260 So instead, pragmatism advocates a shared aim for all research to produce positive change in the world. 201 00:24:18,690 --> 00:24:22,499 Pragmatism typically entails the rejection of this subjective, 202 00:24:22,500 --> 00:24:32,700 objective dualism that we saw in those comparison slides on the philosophical assumptions of interpretive and post positivist work. 203 00:24:33,780 --> 00:24:37,050 So pragmatism rejects that distinction. 204 00:24:37,410 --> 00:24:46,740 It views scientific theories as provisional and achieved through both experience and experimentation in the world. 205 00:24:48,290 --> 00:24:54,290 Sees knowledge as being both constructed and grounded in the world. 206 00:24:57,070 --> 00:25:01,660 So you can see from that description that it's actually. 207 00:25:02,700 --> 00:25:12,090 Compatible in some ways with a constructionist set of assumptions and in some ways with a post positivist set of assumptions. 208 00:25:12,900 --> 00:25:21,090 So interpreted, interpretation is essential and unavoidable, but needs to be grounded in experience. 209 00:25:21,960 --> 00:25:28,010 Okay. And experimentation can be valuable. 210 00:25:29,740 --> 00:25:35,350 But only if it produces positive change in the world. 211 00:25:35,500 --> 00:25:40,150 It's not valuable because it reaches some kind of ideal truth. 212 00:25:41,200 --> 00:25:47,020 That's not why we would value experimentation. So. 213 00:25:48,440 --> 00:25:55,940 This means that we're not asking whether knowledge produced by research accurately represents reality, 214 00:25:56,570 --> 00:26:04,970 but we need to ask whether it has valuable external consequences in the context of the researchers own time and place. 215 00:26:05,930 --> 00:26:14,540 For example, we'd want to know whether our research leads to improved quality of life for patients with a particular 216 00:26:14,540 --> 00:26:22,220 illness or leads to more effective public health services targeting a particular health behaviour. 217 00:26:24,400 --> 00:26:25,660 From this perspective, 218 00:26:25,660 --> 00:26:37,060 then all research should be evaluated according to the extent to which it produces its own pre-specified desired external consequences. 219 00:26:37,900 --> 00:26:49,080 But what those consequences are and how we evaluate whether they've been achieved or not is going to differ depending on what message you face. 220 00:26:50,280 --> 00:27:01,870 Okay. And addressing quality criteria for qualitative components and addressing quality criteria for quantitative components when 221 00:27:01,870 --> 00:27:10,180 you're using quantitative methods can help to ensure that your research does achieve its desired external consequences. 222 00:27:10,630 --> 00:27:10,840 Okay, 223 00:27:11,350 --> 00:27:21,220 so this approach directly argues for the need to maintain the integrity of qualitative components and quantitative opponents in a mixed method study. 224 00:27:22,150 --> 00:27:35,530 So we'll be evaluating qualitative research according to criteria such as the commitment and rigour that the researchers showed to their topic. 225 00:27:36,190 --> 00:27:44,920 We'll be evaluating quantitative research according to traditional criteria, like the reliability and validity, the internal validity of the study. 226 00:27:49,180 --> 00:27:50,839 A slightly different approach, 227 00:27:50,840 --> 00:27:59,300 but a really nice what I think is a really nice summary of the implications of pragmatism comes from Tachikawa and Tetley, 228 00:27:59,600 --> 00:28:03,740 who talk about the dictatorship of the research question. 229 00:28:04,490 --> 00:28:12,649 Study what interests you and is of value to you. Study it in the different ways that you deem appropriate and utilise the results 230 00:28:12,650 --> 00:28:17,360 in ways that can bring about positive consequences within your value system. 231 00:28:21,720 --> 00:28:27,270 I really like that as a kind of guide to how we do research. 232 00:28:27,270 --> 00:28:31,770 And it applies clearly beyond the realm of mixed methods as well. 233 00:28:34,250 --> 00:28:38,300 Okay. So that's the philosophy stuff over and done with. 234 00:28:38,330 --> 00:28:43,060 Let's move on to some more kind of concrete issues. 235 00:28:43,070 --> 00:28:46,879 Let's think about, okay, we've decided we can mix methods. 236 00:28:46,880 --> 00:28:50,480 We've decided we've got some kind of philosophical grounding for doing that. 237 00:28:50,900 --> 00:29:03,190 How can we go about doing it? One approach has been to develop typologies of mixed methods designs and these have been proliferated over the years. 238 00:29:03,940 --> 00:29:09,280 Creswell and Clay in 2011 documented over 15 of these topologies. 239 00:29:09,640 --> 00:29:18,880 There's been plenty more since then as well, and this can be daunting for someone coming to mixed methods research. 240 00:29:19,120 --> 00:29:25,300 Not only do you have to decide on a design, but you have to decide on a framework for choosing your design in the first place. 241 00:29:26,620 --> 00:29:34,960 It can also be constraining, though, for people wanting to do really kind of big elaborate programmes of mixed methods research. 242 00:29:35,650 --> 00:29:42,100 Most of the design typologies are actually quite simple sometimes that have been called 243 00:29:42,100 --> 00:29:48,010 a bit simplistic and don't really cater for these programmatic approaches to research. 244 00:29:51,180 --> 00:29:57,450 So these were the kind of four main designs that were offered by Creswell in Plano, 245 00:29:57,450 --> 00:30:02,400 Clark ten years ago in one of the earlier textbooks on mixed methods research. 246 00:30:03,090 --> 00:30:09,120 They differ in terms of these two designs here at. 247 00:30:10,750 --> 00:30:14,020 Involves the components being done in sequence in order. 248 00:30:14,590 --> 00:30:21,309 These designs are the concomitant designs when the two components are done together at the same time, 249 00:30:21,310 --> 00:30:32,200 or very close together in time, and in three of them we have one component emphasised or prioritised. 250 00:30:32,470 --> 00:30:41,350 That means it's given more resources to it and the interpretation is more strongly weighted towards that component. 251 00:30:42,340 --> 00:30:52,240 But then in the triangulation design, we have a design where we can equally weight our two components and interpret 252 00:30:52,990 --> 00:30:58,780 our overall research on the basis of both the Qual and the client findings. 253 00:31:03,250 --> 00:31:11,770 In 2011, they added two more designs in the revision of their textbook. 254 00:31:12,370 --> 00:31:18,480 The transformative design was. A little bit empty. 255 00:31:18,490 --> 00:31:25,330 And that basically they they describe that as an alternative to pragmatism. 256 00:31:25,810 --> 00:31:29,920 It's okay if you take a transformative lens to your research, then. 257 00:31:31,360 --> 00:31:36,950 Anything goes. I may be paraphrasing too much there. 258 00:31:37,270 --> 00:31:38,140 That's my view on it. 259 00:31:39,160 --> 00:31:48,940 And the other addition they made was this multiphase design, which addresses this issue of actually, if we're doing programs of research, 260 00:31:49,300 --> 00:31:57,790 there are going to be more complexities involved than we could really fit into any one of these designs. 261 00:31:58,390 --> 00:32:04,870 Okay. So the multiphase design acknowledges that you might have multiple different mixed methods designs 262 00:32:04,870 --> 00:32:15,490 within a larger scale program of work more some may house take a slightly different approach. 263 00:32:17,890 --> 00:32:23,500 Differences here. Again, we've got some sequential, some simultaneous. 264 00:32:23,500 --> 00:32:28,060 So pacing or relative timing is again an important issue for them. 265 00:32:30,100 --> 00:32:34,210 Here we always have a core and a supplemental component. 266 00:32:35,140 --> 00:32:44,560 So in their framework, there isn't an option for equal weighting of the different components. 267 00:32:45,640 --> 00:32:52,960 And you might also spotted we've got to put down the bottom here. 268 00:32:54,280 --> 00:33:02,650 And a couple in here that don't actually meet the definition of mixed methods that I work on. 269 00:33:04,720 --> 00:33:11,350 But that's an interesting one because actually it relates back to kind of our very first slide, doesn't it, 270 00:33:11,350 --> 00:33:19,420 about the diversity of approaches within qualitative methods and then the veracity of approaches within quantitative methods. 271 00:33:20,110 --> 00:33:30,400 Because you could argue that actually a discourse analysis has very little in common with a thematic analysis, 272 00:33:31,000 --> 00:33:38,680 and a thematic analysis might have more in common with a cohort study or survey than it does with a discourse analysis. 273 00:33:39,310 --> 00:33:43,870 And indeed, the underlying assumptions might be closer together as well. 274 00:33:44,650 --> 00:33:55,690 So this is an interesting innovation, I think, incorporating qualitative and qualitative studies and calling that mixed methods. 275 00:34:00,430 --> 00:34:04,460 Yeah. That's all I wanted to say about that one. Okay. 276 00:34:04,480 --> 00:34:07,780 And then we have Johnson and Onward Busey, 277 00:34:08,260 --> 00:34:20,710 who really kind of completes what I think was the implicit model in the Creswell and Plano Clark original taxonomy or framework, 278 00:34:22,240 --> 00:34:29,470 where we have every possibility filled in in this grid and. 279 00:34:30,990 --> 00:34:31,680 And again, 280 00:34:31,710 --> 00:34:41,460 you can see that the differentiating factors of the order in which we use the different components and the emphasis that we give each of them. 281 00:34:48,950 --> 00:34:56,810 Ultimately, though, we can say that designing mixed methods research is like designing any other form of research, 282 00:34:57,260 --> 00:35:00,830 and design shouldn't be driven by what? 283 00:35:00,950 --> 00:35:09,710 Like the look of that one or two, that it should be driven by our research question or in mixed methods research often at research questions. 284 00:35:12,590 --> 00:35:23,090 Okay. So to move on now to give you some examples of how I've combined and methods in some of my research over the years, 285 00:35:24,680 --> 00:35:31,490 and I'm going to use the kind of basic Creswell and Plano Clark typology to, to frame this, 286 00:35:32,240 --> 00:35:37,160 partly because actually that's what I was using when I designed a lot of these studies. 287 00:35:38,870 --> 00:35:42,050 So the set that was one of the earlier ones that became available. 288 00:35:43,850 --> 00:35:50,060 So we'll start with this sequential exploratory design, where we start with a qualitative study. 289 00:35:50,360 --> 00:35:56,780 We emphasise the qualitative study. We finish that, then we move on into a quantitative follow up study. 290 00:35:58,010 --> 00:36:02,570 So we're exploring to start with, and then we're following up with some quantitative work. 291 00:36:05,360 --> 00:36:17,690 So we used this approach when we were developing using a qualitative study to then develop a quantitative questionnaire measure. 292 00:36:18,320 --> 00:36:22,130 And that's really common, right? So many questionnaires are developed. 293 00:36:22,430 --> 00:36:29,910 Having done some preliminary qualitative research first, but they're not often conceptualised as mixed methods research. 294 00:36:30,950 --> 00:36:32,209 We went into this project, 295 00:36:32,210 --> 00:36:41,060 Conceptualising admits mixed methods research because we wanted to really actually get something out of the qualitative work. 296 00:36:41,090 --> 00:36:46,490 We didn't want to just do focus groups to generate items for our questionnaire. 297 00:36:46,730 --> 00:36:52,790 We wanted to understand in much more depth the constructs that we might want to be measuring on that questionnaire, 298 00:36:53,090 --> 00:36:58,640 as well as just finding out what language we should be using to measure those constructs. 299 00:36:59,660 --> 00:37:09,739 So we carried out focus groups we sampled purposefully for our participants with a range of different types of back pain, 300 00:37:09,740 --> 00:37:16,910 duration of back pain, age, gender, occupation, these kinds of criteria we used in our sampling. 301 00:37:17,930 --> 00:37:21,060 And we did that because we wanted to understand the variation. 302 00:37:21,500 --> 00:37:27,020 And we wanted to be able to identify key ways in which people common ways in 303 00:37:27,020 --> 00:37:33,140 which people from diverse backgrounds think about treatments for low back pain. 304 00:37:34,970 --> 00:37:45,620 And we were able to do that. We identified four domains along which participants evaluated different treatments for back pain, 305 00:37:45,620 --> 00:37:48,700 and they discussed a whole range of therapies in these focus groups. 306 00:37:48,710 --> 00:37:52,190 They talked about painkillers, they talked about surgery. 307 00:37:52,190 --> 00:37:57,560 They talked about Alexander's technique, about acupuncture, about all sorts of different things. 308 00:37:58,550 --> 00:38:04,460 But it didn't matter what therapy they were talking about, and it didn't matter whether they had it or not. 309 00:38:04,970 --> 00:38:08,900 These were the things they were interested in. Is it a credible therapy? 310 00:38:09,440 --> 00:38:14,840 Is it effective? Has it been effective for me? Is it likely to be effective for me if I haven't tried it? 311 00:38:15,860 --> 00:38:22,070 What concerns does it raise me? Is it going to hurt? Do the needles feel funny in acupuncture? 312 00:38:22,400 --> 00:38:25,650 If I go to a chiropractor, what does the clicking really mean? 313 00:38:25,670 --> 00:38:29,720 What are they doing to make an individual fit? 314 00:38:29,750 --> 00:38:33,080 Is it right for me? Does this therapy feel right for me? 315 00:38:33,140 --> 00:38:36,320 Is it consistent with my beliefs? Does it fit with my lifestyle? 316 00:38:37,700 --> 00:38:45,049 We also found that these specific domains were contextualised within a kind of 317 00:38:45,050 --> 00:38:50,570 broader experience of seeking treatment in which patients were often disappointed, 318 00:38:50,570 --> 00:39:02,150 felt like they were being dismissed by GP's who say, okay, we've tried that, let's try this, let's try this next with no rationale, no diagnosis. 319 00:39:03,050 --> 00:39:10,220 So patients felt like they were experiencing a process of trial and error when they were seeking treatment and they didn't like that. 320 00:39:12,610 --> 00:39:23,709 So we then set out onto a quantitative phase where we try to measure these constructs and to establish whether, first of all, they could be measured, 321 00:39:23,710 --> 00:39:29,710 and then to set out on further research to look at whether they had any relationships 322 00:39:29,710 --> 00:39:34,060 with important outcomes like adherence to treatment or satisfaction with treatment. 323 00:39:35,560 --> 00:39:37,209 So we developed our questionnaire. 324 00:39:37,210 --> 00:39:46,570 We based the wording on the qualitative work, and we surveyed a large number of back pain patients from primary care. 325 00:39:47,230 --> 00:39:58,150 We got some quite sophisticated stats to look at the results, and we found that our survey measured these specific domains quite nicely. 326 00:39:58,150 --> 00:40:01,870 They came out in the factor analysis and the item response theory analysis, 327 00:40:02,620 --> 00:40:09,130 but we really weren't very good at measuring these more contextual domains. 328 00:40:10,450 --> 00:40:16,370 So. That was quite interesting for us. 329 00:40:16,370 --> 00:40:19,130 And we thought that, okay, on reflection, 330 00:40:19,490 --> 00:40:27,319 these were very well defined and actually having very well defined constructs is what you need when you're doing quantitative 331 00:40:27,320 --> 00:40:39,410 research and this contextual stuff around the process of treatment seeking was much harder to pin down into questionnaire items. 332 00:40:40,250 --> 00:40:46,129 And with hindsight, okay, well, that maybe that's not something we want to be measuring, actually. 333 00:40:46,130 --> 00:40:49,880 Maybe that is something that has to be explored qualitatively with people. 334 00:40:50,210 --> 00:40:57,920 Maybe quantitative methods aren't appropriate for all aspects of this process of seeking treatment for low back pain. 335 00:40:59,850 --> 00:41:07,230 And we published these two pieces of work separately, which allowed us to really do justice to each of them. 336 00:41:08,610 --> 00:41:17,669 And it meant because of the sequential design, it meant that this this paper could cite and draw on the earlier qualitative papers. 337 00:41:17,670 --> 00:41:21,720 So we could really explain where our constructs and wording had come from. 338 00:41:22,530 --> 00:41:28,670 But what it didn't really encourage us to do was, was make those links back again and think about like, 339 00:41:28,680 --> 00:41:35,640 well, what does that mean for our qualitative data? And she think about feeding that back into the process. 340 00:41:39,370 --> 00:41:46,270 Questionnaire design isn't the only way to use this quote and design. 341 00:41:47,220 --> 00:41:57,880 I also used it in a small study to explicitly tests and ideas derived from a piece of qualitative work. 342 00:41:58,720 --> 00:42:04,990 So we used qualitative work to identify an issue of importance to patients. 343 00:42:05,500 --> 00:42:09,250 We interviewed people about their experiences of acupuncture. 344 00:42:09,880 --> 00:42:15,400 One of the issues that came up was how they found an acupuncturist and how they 345 00:42:16,060 --> 00:42:20,530 decided who to go and consult once they made a decision to try acupuncture. 346 00:42:21,430 --> 00:42:29,049 And the we found that this choice is important and that patients consider how is this person qualified? 347 00:42:29,050 --> 00:42:32,890 Are they going to be good? Can they do the job? Are they safe? 348 00:42:33,370 --> 00:42:36,910 But also, are they friendly? Am I going to get on with them? 349 00:42:36,940 --> 00:42:42,790 Can I relate to them? Do I trust them? So the personal qualities were also important. 350 00:42:43,540 --> 00:42:54,340 So. So, okay, let's see if we can validate that and let's see if that applies to a large sample of people in a quantitative study. 351 00:42:54,790 --> 00:42:56,739 So we used an experimental design. 352 00:42:56,740 --> 00:43:07,030 We asked people to rate fictional acupuncturists, and we varied descriptions of those according to gender training, their qualifications. 353 00:43:07,450 --> 00:43:17,650 And we also looked at patient gender as well. And what we found is that in general, our patients preferred the female acupuncturists, 354 00:43:18,970 --> 00:43:26,050 and they preferred acupuncturists who were both medically qualified as well as being qualified in acupuncture. 355 00:43:27,280 --> 00:43:39,220 So this provided us with a bit of kind of broader insight into the factors that might actually influence the decision. 356 00:43:39,280 --> 00:43:47,650 So here we were able to think about some causal relationships that qualitative work had suggested, 357 00:43:48,160 --> 00:43:52,930 but of course would in no way actually be able to test or or prove. 358 00:43:59,490 --> 00:44:10,650 We published this in one paper in a medical journal where the word limits 3000 words is difficult to publish the qualitative study in 3000 words, 359 00:44:10,680 --> 00:44:14,070 let alone a qualitative study and an experiment together. 360 00:44:14,550 --> 00:44:20,250 We didn't do justice to it. We really did not do justice to the qualitative findings. 361 00:44:22,020 --> 00:44:27,570 But we were able to clearly show how the different components related to each other. 362 00:44:27,900 --> 00:44:38,220 And it did help us to then look back at the qualitative data and think about using that again to help us interpret the statistical results. 363 00:44:40,960 --> 00:44:51,490 Okay. So this design flips and we use the quantitative component first and we follow up with a qualitative component. 364 00:44:53,350 --> 00:45:03,160 Essentially, the idea here is that you use the qualitative component to explain or contextualise the results of the quantitative work. 365 00:45:03,910 --> 00:45:11,710 And this design can be a little bit challenging to organise, 366 00:45:12,340 --> 00:45:17,830 particularly because if you're using the qualitative work to try and explain the quantitative, 367 00:45:18,430 --> 00:45:23,320 oftentimes you're going to want the same participants involved in both these components, 368 00:45:23,890 --> 00:45:30,670 but you won't know until you finish the quantitative what you're actually going to want to talk to the qualitative participants about. 369 00:45:31,150 --> 00:45:34,420 So in terms of getting funding or getting ethical approval, 370 00:45:35,110 --> 00:45:41,770 this design can sometimes be a little bit challenging and can involve going back for revisions and things. 371 00:45:44,460 --> 00:45:47,670 So this is kind of a mirror study to the acupuncturists study. 372 00:45:47,910 --> 00:45:51,510 In this study, we looked at how patients choose osteopath. 373 00:45:52,050 --> 00:45:56,310 Oh, okay. This time let's start with our quantitative study. 374 00:45:56,700 --> 00:46:02,880 There's actually a lot of information about gender effects and choice of medical practitioner. 375 00:46:03,210 --> 00:46:12,600 So let's go with that existing literature and test some of these hypotheses in our context of osteopath. 376 00:46:14,160 --> 00:46:18,390 Again, the same idea. We had fictional vignettes describing osteopath. 377 00:46:18,780 --> 00:46:23,610 Patients who received the survey had to say which ones they would choose. 378 00:46:24,300 --> 00:46:27,540 Would you choose this one? Definitely not. Definitely, yes. 379 00:46:28,080 --> 00:46:38,070 Kind of set it up like a Yellow Pages advert kind of thing. We had a simple random sample from the general population in this study, 380 00:46:38,670 --> 00:46:45,750 and similar to the acupuncture study, it was qualifications that the respondents cared about. 381 00:46:46,980 --> 00:46:57,750 We also had a bit of a problem with the quantitative results in that 16% of the questionnaires that came back to us were actually unusable. 382 00:46:58,260 --> 00:47:03,839 People had refused to endorse or not endorse any use. 383 00:47:03,840 --> 00:47:10,620 The osteopath that we gave them and the kind of that we felt like that was quite a lot. 384 00:47:11,940 --> 00:47:16,409 We expect to the low response rate. We didn't have any incentives to take part. 385 00:47:16,410 --> 00:47:23,130 We didn't have any follow up emails, but we didn't expect people to bother sending the questionnaire back but not complete that. 386 00:47:23,940 --> 00:47:29,550 So that was an old one. So we thought, okay, actually that's something we want to follow up. 387 00:47:30,330 --> 00:47:37,160 And as is often the case with these small scale studies, we didn't have any funding to do any more work on it. 388 00:47:37,780 --> 00:47:42,480 So we've got some we've got some interviews already with some osteopathy patients. 389 00:47:42,490 --> 00:47:43,830 Let's see if there's anything in there. 390 00:47:44,430 --> 00:47:54,690 So we did a secondary analysis of some existing qualitative interviews which didn't really tell us anything about this. 391 00:47:54,690 --> 00:48:03,929 It didn't help us to explain any further the impact of qualifications or the lack of impact of gender or health care sector that we found. 392 00:48:03,930 --> 00:48:09,239 But it did help us to explain the missing data, because in the interviews, 393 00:48:09,240 --> 00:48:15,420 people talked so much about the importance of a personal recommendation for their osteopath. 394 00:48:15,840 --> 00:48:21,480 And that was what the osteopathy patients that we talked to really cared about. 395 00:48:23,220 --> 00:48:25,620 So that helped us to explain our missing data. 396 00:48:27,940 --> 00:48:35,229 Clearly in hindsight we would have preferred or even at the time we would have preferred to do new qualitative 397 00:48:35,230 --> 00:48:42,100 interviews rather than rely on existing data that didn't quite ask the questions we wanted to be asking. 398 00:48:42,760 --> 00:48:51,100 But I think given the real world of resource constraints, actually, you can do secondary data analysis and incorporate that into mixed methods design. 399 00:48:51,430 --> 00:48:57,700 And it can help. It can help you generate some explanations for puzzling findings. 400 00:48:59,250 --> 00:49:09,720 So moving on to the triangulation design. This is about using different components to analyse and investigate different facets of the same topic. 401 00:49:10,800 --> 00:49:16,140 This was work that I did with Jeremy Howick, who Station Primary Care Department here at Oxford. 402 00:49:16,890 --> 00:49:23,880 We did an online survey, a 783 British GP's about their use of placebos in clinical practice. 403 00:49:24,270 --> 00:49:29,190 We wanted to know do TPS use placebos? Did they use placebo effects? 404 00:49:29,670 --> 00:49:33,360 What do they think these things are? What are their attitudes towards them? 405 00:49:34,560 --> 00:49:38,100 And so we had some open ended and some close ended questions. 406 00:49:39,150 --> 00:49:44,340 So within the same data collection instrument, we had a qualitative and quantitative component, 407 00:49:45,330 --> 00:49:53,220 the quantitative measured, the prevalence of use and the prevalence of certain attitudes about using placebos. 408 00:49:54,030 --> 00:50:02,200 The qualitative component. We took all of the answers to the open ended questions and we analysed them descriptively. 409 00:50:03,540 --> 00:50:14,520 And we looked at and were able to describe various different definitions that Tpz had in mind of placebos and placebo effects. 410 00:50:15,450 --> 00:50:22,349 We found that the ethical issues were key to GDP and societal and regulatory issues. 411 00:50:22,350 --> 00:50:32,130 They wanted guidance about this issue. And we found some really interesting information about why they thought doctors might use placebos. 412 00:50:32,220 --> 00:50:36,450 Some of this was range from doctors saying, well, 413 00:50:36,450 --> 00:50:43,530 it's feebleminded doctors who can't do any better and shouldn't be practising any more on the one hand, too. 414 00:50:43,560 --> 00:50:50,310 On the other hand we had GP saying things like, well, placebo effects are inherent in good quality general practice, 415 00:50:50,310 --> 00:50:53,190 they're inescapable and they're a power that we should be harnessing. 416 00:50:53,910 --> 00:51:01,320 So we had a huge diversity of views that we really hadn't captured in the quantitative component at all. 417 00:51:02,160 --> 00:51:12,900 And here, the qualitative aspect really added some depth and insight into the reasons why people were and weren't using placebos in practice. 418 00:51:15,680 --> 00:51:22,520 The final design I want to illustrate for you is this embedded design where we have one component that's much larger, 419 00:51:22,940 --> 00:51:27,800 the real kind of focus of the research and a smaller component embedded in it. 420 00:51:28,610 --> 00:51:38,690 And the classical example of this design is embedding a small scale qualitative study within a large scale major ask. 421 00:51:38,960 --> 00:51:45,890 And this is work that Sarah did with me back when she was in Southampton with Hazel Everett, 422 00:51:46,640 --> 00:51:53,270 where we were working on a trial of web based CBT for patients with IBS. 423 00:51:54,020 --> 00:51:58,429 And the concept of trial was one of these really complex, 424 00:51:58,430 --> 00:52:06,470 multifactorial trials with patients with randomised to one of three different medication arms and one of three self-management arms. 425 00:52:07,910 --> 00:52:13,820 The qualitative work we sampled from all of those different groups in the trial. 426 00:52:14,090 --> 00:52:18,710 So we had that trial randomisation as a sampling frame for the qualitative work, 427 00:52:19,250 --> 00:52:25,909 and we wanted primarily to explore patients experiences in order to improve the intervention 428 00:52:25,910 --> 00:52:30,680 and the trial design for a future trial that is actually now just coming to a close. 429 00:52:31,430 --> 00:52:40,880 So the findings from this qualitative component were used to improve the intervention that has now been trialled in a large scale major trial. 430 00:52:42,020 --> 00:52:48,710 We found the patients could be classified into different types of engagement with the website. 431 00:52:49,310 --> 00:52:58,310 And an interesting thing when we related the qualitative findings here back to the quantitative data we had on the same participants, 432 00:52:59,030 --> 00:53:07,580 was that some of our participants, who really from their own descriptions, had very limited or even no engagement with the website, 433 00:53:08,150 --> 00:53:14,510 had actually been classified quantitatively as adhering to the intervention. 434 00:53:15,640 --> 00:53:16,210 Okay. 435 00:53:16,840 --> 00:53:29,830 So that gave us really convincing evidence that we needed better measures of adherence and use of the website in the main trial that we have now use. 436 00:53:31,450 --> 00:53:44,280 Okay. So just to summarise some of this for you, I think we can think about the strengths and limitations of sequential and concurrent designs. 437 00:53:46,370 --> 00:53:49,549 That's a nice way to think about it, 438 00:53:49,550 --> 00:53:55,850 rather than having to think about the strengths and limitations of the multiple different designs in the typologies. 439 00:53:58,260 --> 00:54:07,350 I find it a lot easier to retain the integrity of components in a sequential design because you do one study, 440 00:54:08,250 --> 00:54:12,240 you evaluate it on its own terms, you think about validity in those terms. 441 00:54:12,510 --> 00:54:19,080 Then you can switch mindsets and think about the next study and become fully engrossed in that paradigm. 442 00:54:19,920 --> 00:54:23,970 And as a single researcher, that's certainly the case. 443 00:54:24,510 --> 00:54:29,550 When you've got multiple studies of two components going on at the same time, 444 00:54:30,300 --> 00:54:37,710 it's actually quite easy to get a bit muddled, especially if you're involved centrally in both of those components. 445 00:54:38,040 --> 00:54:44,840 But then that's where teamwork comes in and that's absolutely vital, I think for the current study designs. 446 00:54:44,850 --> 00:54:53,040 I think it's very difficult as a lone researcher, as a PhD student, for example, to do a concurrent mixed method study. 447 00:54:55,860 --> 00:55:01,650 Basically the model for the sequential design is kind of long and thin, takes a long time, 448 00:55:02,700 --> 00:55:07,340 but for the concurrent design, shorter and fatter, more resources, more intensive. 449 00:55:07,350 --> 00:55:10,590 But it's over and done with much quicker. You finish it quicker. 450 00:55:14,370 --> 00:55:23,220 I think sequential studies lend themselves more to single publication, 2 to 2 being published separately, 451 00:55:23,520 --> 00:55:29,310 but actually that's you can publish them together and the same goes for concurrent. 452 00:55:30,300 --> 00:55:33,780 They tend to lend themselves to being published together because you've got the 453 00:55:34,000 --> 00:55:38,160 you've often got the same participants and they're often a bit more closely linked. 454 00:55:38,550 --> 00:55:43,950 But actually both of the concurrent designs I've just the examples I've just given you, we publish them separately. 455 00:55:45,030 --> 00:55:51,179 So sometimes they do come out separately. As I mentioned when we were going through, 456 00:55:51,180 --> 00:55:59,910 it can be difficult to fully specify the second component in a sequential design in advance for funders or for regulatory bodies. 457 00:56:00,300 --> 00:56:05,160 Whereas in concurrent designs, it's not only doable, but you kind of have to do it. 458 00:56:10,570 --> 00:56:13,660 Some of the challenges I've encountered over the years. 459 00:56:13,690 --> 00:56:20,120 And when you find discrepancies between your two components, you say, Oh, hang on a minute, that contradicts that. 460 00:56:20,140 --> 00:56:23,930 Which one do I believe actually turns out? 461 00:56:23,950 --> 00:56:29,770 That's a really great opportunity to look more critically at each piece of work. 462 00:56:30,280 --> 00:56:42,519 So I'll give you an example of that. We did a mixed methods trial where we wanted to look at whether acupuncturists being more empathetic with 463 00:56:42,520 --> 00:56:48,400 their patients would be more effective than acupuncturists who were just very kind of matter of fact, 464 00:56:48,430 --> 00:56:51,459 no chitchat, none of the how are you? 465 00:56:51,460 --> 00:56:52,870 How are the grandkids kind of thing? 466 00:56:54,700 --> 00:57:04,540 The qualitative work suggested that empathy was really important and patients felt cared for and they really appreciate that the quantitative. 467 00:57:04,810 --> 00:57:07,840 The. Empathy had no effect on the outcome. 468 00:57:08,200 --> 00:57:12,930 So. So initially we're like, okay, there's something funny going on here. 469 00:57:12,940 --> 00:57:23,710 What is that? We look deeper into the qualitative work and the empathy that some of the patients were describing was not actually coming from the US. 470 00:57:24,280 --> 00:57:27,580 It was coming from everyone else in the trial. 471 00:57:29,080 --> 00:57:35,740 From that, even people very tangentially involved, like the receptionist at the medical facility where the trial was being run. 472 00:57:36,550 --> 00:57:44,050 They were a clinical trials unit. The receptionist was lovely and friendly, so the patients felt cared for and treated as individuals, 473 00:57:44,410 --> 00:57:49,750 even though the acupuncturist was just absolutely matter of fact and adhered to that protocol. 474 00:57:50,800 --> 00:57:58,210 So actually, the apparent discrepancies forced us to go back and look much more critically at our data, which is very helpful. 475 00:58:02,050 --> 00:58:08,530 It's difficult to do justice to each component and the mixed methods component as well. 476 00:58:09,130 --> 00:58:12,490 No matter how you publish, I think at the end of the day, 477 00:58:13,330 --> 00:58:21,430 although one model seems to be potentially publishing separate papers and then publishing a mixed methods paper where you link 478 00:58:21,430 --> 00:58:33,729 the two components more and team working as in any big team project can be challenging on mixed methods projects particularly. 479 00:58:33,730 --> 00:58:42,580 So if you've got team members who are very staunch advocates of one method or another, 480 00:58:42,580 --> 00:58:52,120 and that's where it's really helpful to have someone who can bridge across and and can really open up discussions amongst the team. 481 00:58:55,270 --> 00:58:59,600 I think in future we need more training as a support for for mixed methods research. 482 00:58:59,620 --> 00:59:02,800 I love the fact that you have a whole module devoted to it here. 483 00:59:04,450 --> 00:59:10,640 We're quite good at training people to do quantitative research and qualitative research, but mixed methods is a whole other ball game. 484 00:59:10,930 --> 00:59:14,380 So we need to be skilling people up to do these methods research. 485 00:59:15,580 --> 00:59:23,620 We need to be encouraging publications so that we've got examples to look at in the literature as a good practice in mixed methods research. 486 00:59:24,790 --> 00:59:32,049 And I think as as we're publishing in open access journals and word limits it to an extent, 487 00:59:32,050 --> 00:59:38,290 in some cases becoming a thing of the past that gives us a bit more flexibility and scope for publishing. 488 00:59:39,010 --> 00:59:43,150 Mixed methods work, and I think it's increasing. 489 00:59:43,300 --> 00:59:49,900 It's really important that when we publish mixed methods research to actually describe it as such and to provide a 490 00:59:49,900 --> 00:59:57,850 rationale for why we're combining the different components that we're combining and how we make decisions about that. 491 01:00:01,340 --> 01:00:09,200 Okay. So mixed methods designs can help us to think through the technical challenges of mixed methods research. 492 01:00:09,770 --> 01:00:13,790 But if we find ways of resolving those which we can do, 493 01:00:14,330 --> 01:00:21,020 that really shouldn't obfuscate the need to address the philosophical challenges and the underlying assumptions as well. 494 01:00:21,560 --> 01:00:25,670 And pragmatism offers this just one approach to doing that. 495 01:00:28,130 --> 01:00:34,460 And thank you very much for your attention. I hope that's made you think more and maybe in a slightly different way about mixed methods. 496 01:00:34,850 --> 01:00:37,130 And I'm very happy to stay for some questions.