1
00:00:00,480 --> 00:00:03,000
So. Good morning, everyone. I'm Paul Glacier.

2
00:00:03,030 --> 00:00:08,470
I'm the director for the Centre for Evidence based Medicine and I'm a part time general practitioner as well.

3
00:00:08,490 --> 00:00:13,110
I worked in Beaumont St opposite the Ashmolean Museum, a very privileged place to work.

4
00:00:13,830 --> 00:00:16,920
I'm glad you all managed to make it through the snow to get here.

5
00:00:18,030 --> 00:00:25,080
I think we've had most people turn up, so that's great news. And I'm sorry we're a little bit late, but a few people do have difficulty coming in.

6
00:00:26,160 --> 00:00:32,730
I'm I'm going to talk in a moment about an introduction to evidence based medicine, but just a couple of little practicalities before we start.

7
00:00:33,030 --> 00:00:36,329
You should all have a pack, which has two components.

8
00:00:36,330 --> 00:00:40,230
There's a workbook and there's a set of readings. We've all got those.

9
00:00:41,550 --> 00:00:48,720
The other really important thing is you should you should have got a list of the group that you're in, and they'll be a room on it.

10
00:00:49,050 --> 00:00:53,880
But you'll have to find that's your first task after this lecture is to find your room.

11
00:00:54,210 --> 00:00:57,300
That's the room that you'll be in for the entire three days.

12
00:00:57,750 --> 00:01:06,660
Most of this workshop is about getting you guys to do things to get, practice hands on, work at critical appraisal, searching, etcetera.

13
00:01:06,930 --> 00:01:13,410
So develop the skills in this area. We'll talk at you a little bit, but that's mostly for orientation to different areas.

14
00:01:13,680 --> 00:01:16,650
Most of the real work is you actually practising things.

15
00:01:17,310 --> 00:01:23,550
So find out where that room is because that's the place where you'll do the real work and we'll get you there straight after this lecture.

16
00:01:24,210 --> 00:01:31,620
The other thing is that you are sent out a programme and you, some of your managers probably brought along with you on this.

17
00:01:31,620 --> 00:01:36,719
A couple of small modifications to that, and they're printing all those off at the moment.

18
00:01:36,720 --> 00:01:42,990
So before we finish this session, we'll get you a new copy of the programme as well with those couple of small modifications.

19
00:01:45,280 --> 00:01:50,560
Okay. I'm going to talk about the basics of evidence based practice.

20
00:01:51,220 --> 00:01:53,410
And in a sense, I'm going to talk about two things.

21
00:01:53,740 --> 00:02:01,960
One is about the sort of skills that you need to process the traditional four steps of evidence based medicine about formulating questions,

22
00:02:02,320 --> 00:02:07,170
searching skills, appraising the evidence, and applying it to individual patients.

23
00:02:07,180 --> 00:02:10,630
And in particular, I want to talk about the formulation of questions this morning.

24
00:02:11,650 --> 00:02:15,370
But the other thing we're trying to get you to learn is about some of the practicalities.

25
00:02:15,370 --> 00:02:20,049
How does this work in practice? That's where I'd actually like to start this morning,

26
00:02:20,050 --> 00:02:26,980
is to give you a sort of snapshot of when you go back to wherever you're working, how would this look when you're actually doing it?

27
00:02:27,550 --> 00:02:31,930
So there's a set of skills that you need, but there's also a set of processes that you need to learn about.

28
00:02:31,930 --> 00:02:35,430
How would you put this into practice? And I want to give you a picture.

29
00:02:35,440 --> 00:02:42,510
So I'm actually, in a sense, going to start at the end. Before I do that, though, I want to know why you're here.

30
00:02:45,030 --> 00:02:48,569
And you'll do this in your small groups will start with asking you what you want

31
00:02:48,570 --> 00:02:52,260
to get out of this process because it's not a completely fixed program for this,

32
00:02:52,260 --> 00:03:00,660
particularly in the small group sessions. In a sense, you can make up your own workshop, at least persuade your tutor to modify it to fit your needs.

33
00:03:01,770 --> 00:03:06,470
But I want to know a sort of big picture thing, first of all, and I want to find out why you're here.

34
00:03:06,480 --> 00:03:14,490
You're here to learn ABM because and it can be all five if you like, but I'd like you to put your hand up for each one that you think you're here for.

35
00:03:16,300 --> 00:03:22,180
So he is who's here primarily because they're working in clinical practice and they want to use this in clinical practice.

36
00:03:22,180 --> 00:03:26,319
Hands up. Okay. That's about half of you.

37
00:03:26,320 --> 00:03:34,320
That's great. Here because you're working on evidence resources such as reviews, guidelines, evidence reports, those sorts of things.

38
00:03:35,190 --> 00:03:41,880
Okay. It's probably about a third of you and some doubling up obviously here because you want to help others use evidence.

39
00:03:41,940 --> 00:03:47,440
You're okay. Here because you plan to teach him.

40
00:03:48,590 --> 00:03:52,669
My goodness. Quite a lot of you. That's great. And here.

41
00:03:52,670 --> 00:03:59,180
Because your boss said you had to come. Okay, that's a few of you, too.

42
00:04:01,100 --> 00:04:05,960
I noticed some people from the centre put up their hands too. I didn't tell you you had to come.

43
00:04:06,560 --> 00:04:12,350
So just to orient you, what we're going to do now is I'm going to talk, as I said, about the basics of evidence based practice.

44
00:04:12,440 --> 00:04:16,490
Then you'll go to your small group for an important thing you've got to find after this session.

45
00:04:16,850 --> 00:04:21,320
And we'll have a tutorial about about the question formulation component.

46
00:04:21,770 --> 00:04:28,040
Then we'll have morning tea come back, and Rafael's going to do a session on rapid critical appraisal where we'll teach you the

47
00:04:28,040 --> 00:04:34,070
basic our simplified version of critical appraisal based on Rod Jackson's gait process.

48
00:04:34,400 --> 00:04:38,090
And there are just two acronyms you have to learn, and that's Pico and Rambo.

49
00:04:38,630 --> 00:04:46,400
We'll teach you those two acronyms. If you know those, you know, about two thirds of what you need to know about IB and then in the afternoon,

50
00:04:46,400 --> 00:04:55,250
after and after lunch will be sending you off to a computer lab about 100 yards down the road here so that you can do some hands on searching.

51
00:04:55,550 --> 00:04:58,780
We'll look at you to formulate some questions before that, hopefully,

52
00:04:59,060 --> 00:05:05,090
but we'll have one of the librarians teach you about hands on searching, and then you'll have your final session this afternoon.

53
00:05:05,600 --> 00:05:08,660
That's a relatively fixed program with some flexibility in it.

54
00:05:08,990 --> 00:05:13,910
There are two days following it tomorrow and Wednesday. Actually, the program is much more flexible.

55
00:05:14,180 --> 00:05:19,910
And so part of what you'll need to do today in your small groups is just decide which bits you want to emphasise,

56
00:05:20,240 --> 00:05:23,390
which bits do you need to know more about that and discuss that with your tutor.

57
00:05:24,740 --> 00:05:28,940
Okay. What I'm going to talk about in this session is two things, as I said.

58
00:05:29,360 --> 00:05:32,889
One is what is evidence based medicine? What does it look like in practice?

59
00:05:32,890 --> 00:05:39,799
So I want to give you some sort of picture of that. And then I'm going to talk about mostly about the first of those four steps,

60
00:05:39,800 --> 00:05:46,520
about the formulation of questions and the different styles of question that you can have and what research would fit them best.

61
00:05:48,140 --> 00:05:55,400
Okay. In order to warm up for this, what I'd like you to do is you should all have a sheet that doesn't have my full set of slides on it,

62
00:05:55,400 --> 00:06:03,360
but it's got the critical ones on the back of that sheet. Is a set of changes in practice.

63
00:06:04,050 --> 00:06:10,230
What I'd like I'm going to give you about 103 seconds to write on that.

64
00:06:11,010 --> 00:06:15,090
Things that you have changed in your practice, whatever your practice is.

65
00:06:16,800 --> 00:06:19,920
So what things have you changed in the last 12 months?

66
00:06:24,100 --> 00:06:27,200
And then I'd like you to think, okay, how did that change come about?

67
00:06:27,220 --> 00:06:30,940
Where did you learn it from? And what was the evidence behind it?

68
00:07:50,550 --> 00:07:53,860
Since. It's.

69
00:08:01,320 --> 00:08:04,590
Okay. I can see many of you racking your brains to think of more.

70
00:08:06,780 --> 00:08:10,560
I'm sure if I gave you a longer period that you'd actually think of many more.

71
00:08:11,480 --> 00:08:16,440
Well, I just want to get some sense of how many changes people have made over the last 12 months.

72
00:08:16,740 --> 00:08:22,200
Anyone with more than ten hands up. Between six and ten.

73
00:08:24,540 --> 00:08:31,780
Okay. Let's do the countdown. Five. That's two for.

74
00:08:34,930 --> 00:08:40,740
Six three. It's about ten to.

75
00:08:42,590 --> 00:08:49,240
That's about ten one. 12345670.

76
00:08:53,010 --> 00:08:59,280
Okay. So that looks like somewhere between two and three is the most common number of changes.

77
00:08:59,790 --> 00:09:01,920
Now, I put that up for a couple of reasons.

78
00:09:02,880 --> 00:09:09,300
One is just to illustrate that we feel like we've been that there's a lot of information coming through and that we're reading lots of stuff.

79
00:09:10,050 --> 00:09:15,030
But actually the number of changes that we make in practice is actually fairly small.

80
00:09:15,960 --> 00:09:21,750
And what we'd like to do is actually make your life easier by trying to find more things that actually change your practice,

81
00:09:21,750 --> 00:09:28,410
that influence patient care, but without increasing and maybe even decreasing the amount of work you need to do to do that.

82
00:09:30,420 --> 00:09:34,280
Okay. So we'd like to get these numbers up more towards this end of things.

83
00:09:34,280 --> 00:09:39,930
So next year, when you come back and answer this question and again, you can probably write down ten, but you've actually been reading less.

84
00:09:41,070 --> 00:09:45,120
The only thing I'd like to ask you, though, is what drove those changes.

85
00:09:45,180 --> 00:09:47,580
Maybe you want to discuss this for a moment with your neighbour.

86
00:09:47,790 --> 00:09:54,360
But can I just ask how many how many people had a change that was driven by a systematic review of randomised trials?

87
00:09:57,460 --> 00:10:00,670
Nobody. Not that I could remember anyone.

88
00:10:00,700 --> 00:10:07,720
Okay, so you just want to confer with your neighbour about what your changes were and where did they come from?

89
00:11:22,160 --> 00:11:25,400
Okay. Can I have your attention again?

90
00:11:27,800 --> 00:11:30,890
I can see we're going to have no problem getting you guys to work in tutorials.

91
00:11:30,890 --> 00:11:39,680
That's fantastic. Talkative Bunch. Can I just ask for examples of where you learn things from?

92
00:11:40,750 --> 00:11:44,950
What sorts of places did you actually get the learnings from the change your practice?

93
00:11:45,760 --> 00:11:48,930
Sorry, I'm embarrassed to say it's not everything.

94
00:11:49,280 --> 00:11:52,540
No, no. So please, please be truthful.

95
00:11:52,550 --> 00:11:55,960
I'm not expecting you to say, Oh, you read the BMJ Journal and you were really.

96
00:11:56,800 --> 00:12:00,110
You know, you don't have to say that except you.

97
00:12:01,660 --> 00:12:08,799
So it was a team that you learned it from sorry, regulations from about regulations from above.

98
00:12:08,800 --> 00:12:12,970
So it was a guideline that you read that said this is what you've got to do. You're also from a team.

99
00:12:14,180 --> 00:12:17,400
From research. From the research.

100
00:12:28,390 --> 00:12:32,990
Was. This is your own research, is it? No, it's not my. Okay.

101
00:12:32,990 --> 00:12:38,270
So you read research, so that's an even process. Fantastic. But I think there's a variety of means, isn't there?

102
00:12:38,660 --> 00:12:45,080
You might get a directive from from the National Institute for Clinical Excellence or some guideline from colleagues.

103
00:12:45,080 --> 00:12:49,490
Anyone else have something different? Yeah, I'm very comfortable.

104
00:12:49,760 --> 00:12:55,010
Okay. So your boss said no, you've got to do it this way, or this is new way of doing things.

105
00:12:55,280 --> 00:13:01,909
Anyone else from a course like you went to a session specifically to learn about something, you went, Oh, that's a good idea.

106
00:13:01,910 --> 00:13:05,760
I'll do that in practice. Discussion with colleagues. Okay.

107
00:13:05,780 --> 00:13:10,850
Discussion with colleagues just sitting around the tea room or something or saying, I have this problem.

108
00:13:11,090 --> 00:13:14,120
We're meeting to have a meeting like other stuff.

109
00:13:14,510 --> 00:13:19,070
Uh huh. Well, what we have is to support that.

110
00:13:19,640 --> 00:13:31,690
Okay, fantastic. Sorry. From email group, we have an email group and someone sent in a response and you thought, oh, that's not interesting.

111
00:13:32,840 --> 00:13:39,060
Okay. Like fantastic. Any other reflection on your own practice from reflection on.

112
00:13:39,150 --> 00:13:42,450
Okay, just thinking about it. Oh, okay. I should probably do that differently.

113
00:13:43,430 --> 00:13:47,810
Lessons in that experience. Mm hmm. Okay.

114
00:13:50,270 --> 00:13:53,830
Okay. Sorry. Anyone else?

115
00:13:54,580 --> 00:13:58,000
Project orders like you didn't order.

116
00:13:58,130 --> 00:14:03,140
Oh, my goodness. I know. And you want.

117
00:14:03,520 --> 00:14:08,620
Sorry. Experts in the field. So personal communication with them.

118
00:14:09,430 --> 00:14:12,460
You went to a lecture or you were talking to them in the corridor or something?

119
00:14:13,280 --> 00:14:18,960
Okay. Anyone get it from a patient? Oh, yes.

120
00:14:19,630 --> 00:14:23,610
Email the Daily Mail. Okay. That the media as well.

121
00:14:23,620 --> 00:14:27,710
Yeah. Yeah. Yeah.

122
00:14:27,850 --> 00:14:36,730
So what was the one for them? You remember one from the media. A country at a country.

123
00:14:36,990 --> 00:14:40,340
The one recent one for us was the example of the antidepressants,

124
00:14:40,340 --> 00:14:49,820
which in the media a couple of weeks ago there had been a recent systematic review in PLOS Medicine based on FDA submitted data.

125
00:14:49,850 --> 00:14:53,960
So when we actually did in our own journal called because we expected patients to come in asking about that.

126
00:14:54,470 --> 00:14:59,020
So we wanted to look at the paper. Okay, great. So I wanted to illustrate that.

127
00:14:59,230 --> 00:15:03,820
First of all, we don't make a lot of changes. And actually at the moment they come from a variety of sources.

128
00:15:04,870 --> 00:15:08,049
How many of those do you think were explicitly evidence based?

129
00:15:08,050 --> 00:15:14,440
That is, that you knew what the evidence was, that that somebody was suggesting a change to you and you knew what their evidence was,

130
00:15:15,010 --> 00:15:19,240
hands up, who had announced they felt pretty secure about the evidence.

131
00:15:19,960 --> 00:15:24,970
So I'm mostly not okay. Now, I'm I think we should be honest about this.

132
00:15:24,970 --> 00:15:30,910
We do learn things from a variety of places. Sometimes we're in a hurry and we just ask a colleague, What would you do about this?

133
00:15:30,910 --> 00:15:37,480
And you go, Oh, that sounds pretty good. I'll go and do that. And we have to recognise that's where our changes come from.

134
00:15:38,230 --> 00:15:43,600
The one thing I'd suggest is, okay, if you think this is going to be something, I'm a problem, I'm going to come across repeatedly.

135
00:15:43,600 --> 00:15:46,910
Maybe you want to think, maybe I should check the evidence on that as well.

136
00:15:46,930 --> 00:15:50,710
Ask them that. Maybe do a search yourself. Okay.

137
00:15:51,400 --> 00:15:58,450
So a few interesting learnings from that. I want to contrast that with our feeling of huge amounts of information coming in.

138
00:15:58,870 --> 00:16:03,160
So I'm going to assess your Jasper score, which has five components to it.

139
00:16:03,790 --> 00:16:06,880
Do you feel ambivalent about renewing your journal subscriptions?

140
00:16:07,090 --> 00:16:12,309
Do you feel anger towards prolific authors like me? Do you ever use journals to help you sleep?

141
00:16:12,310 --> 00:16:16,480
Are you surrounded by piles of periodicals? Do you feel anxious when journals arrive?

142
00:16:17,350 --> 00:16:20,979
You get one point for each. Yes, that you might.

143
00:16:20,980 --> 00:16:24,010
To that I want you to add up your score from 0 to 5.

144
00:16:25,170 --> 00:16:33,670
How many yeses? Okay.

145
00:16:33,700 --> 00:16:37,520
Hands up the zeros. One, two, three.

146
00:16:39,580 --> 00:16:44,130
Four or five. Okay. You're pretty normal sort of audience then.

147
00:16:45,280 --> 00:16:51,130
They've done some standardisation and it's good to see we have no liars in the group.

148
00:16:52,690 --> 00:17:00,879
Most of you are in what's called the normal range. That may not be the healthy range that I like, but that is said to be the normal range.

149
00:17:00,880 --> 00:17:04,360
And the few of you are in trouble and need to go and see an information specialist.

150
00:17:06,040 --> 00:17:14,710
Okay. Why do we feel that way? One is just the amount of research that's being published and that can come through us for a variety of means.

151
00:17:15,370 --> 00:17:19,540
This is just one illustration of the number of medical articles being published per year.

152
00:17:19,930 --> 00:17:23,700
Actually, this is slightly out of date. Medline is now over half a million per year.

153
00:17:23,710 --> 00:17:30,560
I think last year it was about 600,000. Articles in Medline, of course, is very restrictive.

154
00:17:30,560 --> 00:17:35,240
It's just part of the world literature, which is probably more like a couple of million articles per year.

155
00:17:35,660 --> 00:17:39,140
And even if you just want to just confine yourself to the randomised trials,

156
00:17:39,560 --> 00:17:44,570
there are about 30,000 randomised trials being published every year as well.

157
00:17:45,740 --> 00:17:51,110
So the title this comes from one of the few practical books I read going through medicine called Kill

158
00:17:51,110 --> 00:17:57,410
As Few Patients as Possible by Oscar London and Rule 31 was review the World Literature Fortnightly,

159
00:17:57,980 --> 00:18:06,110
which turns out to be a pretty tough task. In fact, if you put it just in Daily Times, we're talking about 55 trials per day,

160
00:18:06,500 --> 00:18:14,570
101,500 Medline articles per day, and probably about 5000 biomedical articles per day.

161
00:18:15,700 --> 00:18:18,819
The literature is huge and try to find your way through.

162
00:18:18,820 --> 00:18:23,590
That is very difficult, even though you may have a narrow speciality area that you're working in.

163
00:18:23,950 --> 00:18:27,140
Often that literature is really scattered. It might.

164
00:18:27,160 --> 00:18:31,320
Some of it will be maybe about a third of it in your in your in all the speciality journals.

165
00:18:32,110 --> 00:18:34,899
But then another third of it will be in general medical journals.

166
00:18:34,900 --> 00:18:41,260
In fact, the the big stuff will probably be in the major general medical journals, even if it's a speciality topic.

167
00:18:42,390 --> 00:18:47,340
So even specialists have to read their own subspecialty journals, plus all the general medical literature.

168
00:18:47,670 --> 00:18:52,830
The remaining third will be scattered across a huge array of different areas.

169
00:18:53,520 --> 00:18:56,670
So it's very difficult to find. Okay.

170
00:18:57,270 --> 00:19:05,160
So the key problem that evidence based medicine addresses is the the issue of trying to sift out

171
00:19:05,820 --> 00:19:10,649
the best evidence that's coming out through research and using it to improve our patient care,

172
00:19:10,650 --> 00:19:18,570
to change the way that we practice. And it's there isn't one solution to this problem, and we may invent new solutions during this workshop.

173
00:19:19,110 --> 00:19:24,420
This is an evolving science, if you like. So there isn't one magic answer to this.

174
00:19:24,810 --> 00:19:31,050
But the definition is evidence based. Medicine is the integration of best research, evidence with clinical expertise and patient values.

175
00:19:31,560 --> 00:19:36,030
And for us two of those and motherhood, we wouldn't expect to do this without clinical expertise.

176
00:19:36,480 --> 00:19:42,450
We wouldn't expect to do it without involving the patient in the decision making process in some way and

177
00:19:42,450 --> 00:19:46,889
finding out what their objectives are and what they want and the magic ingredient that we'd like to add.

178
00:19:46,890 --> 00:19:51,930
The extra thing that Eben wants is to add the best research evidence and patients.

179
00:19:51,930 --> 00:19:54,120
If you tell them that we don't do that at the moment,

180
00:19:54,510 --> 00:20:00,540
I'm really surprised you try and explain evidence based medicine to patients and they go, You mean you aren't doing that already?

181
00:20:02,010 --> 00:20:05,310
You mean you don't use the best research evidence when you make decisions about me?

182
00:20:06,750 --> 00:20:10,829
And we go, Well, that's almost impossible on the other side of it.

183
00:20:10,830 --> 00:20:15,420
So DBM is the sort of trying to get those the nexus between those three things.

184
00:20:16,140 --> 00:20:20,310
What I want to do is just talk about a couple of models of how this might work,

185
00:20:20,310 --> 00:20:24,330
the practicality, what would it look like if you were going to do this in practice?

186
00:20:24,870 --> 00:20:28,139
And I'm beginning I'm going to begin with the impossible version of it.

187
00:20:28,140 --> 00:20:34,380
So bear that in mind, this is the impossible version of it, which is why have the the rings down the bottom here?

188
00:20:34,740 --> 00:20:37,800
I'm going to talk about the Olympic champions of evidence based medicine.

189
00:20:38,520 --> 00:20:41,729
Dave Sackett was the founder of the Centre for Evidence based Medicine.

190
00:20:41,730 --> 00:20:48,420
And when he was here in the 1990s, he was an internal medicine physician working up at the John Radcliffe Hospital

191
00:20:49,170 --> 00:20:53,100
and on the ward rounds they used to take a thing called the evidence cart,

192
00:20:53,580 --> 00:20:56,940
which had a data projector and a laptop computer,

193
00:20:56,940 --> 00:21:03,030
and on the laptop they had Medline, the beginnings of the Cochrane Library of Best Evidence,

194
00:21:03,030 --> 00:21:09,749
which is of all the things from the ICP Journal Club, etc., and things that they had previously appraised.

195
00:21:09,750 --> 00:21:15,730
And they're all set up on the laptop and during the ward round. They would ask questions now.

196
00:21:15,800 --> 00:21:20,350
Gee, I wonder if we should use X or y? Should we give this patient with heart failure a beta blocker?

197
00:21:20,360 --> 00:21:24,500
What does should we use, etc.? And they would look that up during the ward round.

198
00:21:25,730 --> 00:21:32,900
They'd look up two or three questions per patient. It took about 15 to 90 seconds.

199
00:21:32,900 --> 00:21:38,570
And I'll show you the data in a moment to find the piece of evidence that they needed to answer that question.

200
00:21:38,570 --> 00:21:46,700
And it changed about one third of their decisions. So in most in most patients, they are actually making some decisional change.

201
00:21:46,730 --> 00:21:50,250
Every patient that they were seeing. I can't.

202
00:21:50,250 --> 00:21:55,500
You think that's impossible, isn't it? Well, one fact is that the ward rounds took a little bit longer.

203
00:21:56,790 --> 00:22:02,850
Is Carl here? No, he's actually John. Carl was a medical student during the so he actually experienced some of these ward races.

204
00:22:03,420 --> 00:22:07,110
I hear rumours that the ward rounds could take anything up to 5 hours.

205
00:22:07,110 --> 00:22:12,560
Carl. Yeah. He could actually sleep all day. Going left with that many people with down clothing, too.

206
00:22:13,020 --> 00:22:20,700
Yeah. Carl. So you may say this is this is not feasible and and not possible.

207
00:22:20,790 --> 00:22:22,630
So two caveats that I'll make about that.

208
00:22:22,650 --> 00:22:28,710
One is that most people do several hours of continuing medical education per week when I've assessed this in workshops.

209
00:22:29,010 --> 00:22:31,420
Most people on average are doing that 3 hours a week.

210
00:22:31,440 --> 00:22:39,270
When you add up all the journals, reading the lectures that you go to, etc., if you put it all together, it's about 3 hours per week.

211
00:22:40,230 --> 00:22:47,490
What Sackett was doing was moving that 3 hours of continuing medical education and putting it adding it into the ward round.

212
00:22:48,120 --> 00:22:52,410
So the ward ran was lengthened because that's where he was going to put the continuing medical education,

213
00:22:52,920 --> 00:22:56,040
where it was going to be most effective with individual patient care.

214
00:22:56,340 --> 00:23:03,570
They were learning on the fly just in time learning. So when I put up the top here, a sort of different approach to education.

215
00:23:04,170 --> 00:23:07,860
The other thing is being able to answer things that quickly.

216
00:23:07,890 --> 00:23:10,530
I don't even expect you to do that at the end of this workshop. Right.

217
00:23:11,220 --> 00:23:18,150
Sackett had been doing this for 20 years by the time he got here and Sharon Strauss and that's why I say these are the Olympic champions.

218
00:23:19,060 --> 00:23:24,700
At the end of this workshop, you might be able to find the best piece of evidence for something in maybe 15 minutes,

219
00:23:24,700 --> 00:23:28,900
but the first time can take an hour to do, and you get progressively faster at it.

220
00:23:30,000 --> 00:23:33,870
One thing you'll learn is that trial finding no evidence.

221
00:23:33,870 --> 00:23:38,260
When there's no evidence there, it actually takes you longer because you're never sure.

222
00:23:38,280 --> 00:23:46,020
Have I exhausted all possible search terms? So learning when to stop is actually a difficult thing as well, but it takes a while.

223
00:23:46,050 --> 00:23:50,850
Lots of practice, hundreds of questions before you can get down to this sort of pace of being able to

224
00:23:50,850 --> 00:23:55,500
find the information of realising where to search and what the search term should be,

225
00:23:55,860 --> 00:24:01,740
and then being able to appraise it rapidly as well. That just takes time in practice and I don't expect you to start there.

226
00:24:02,820 --> 00:24:08,340
This, by the way, is the data. When they looked at previously appraised topics, it took them about 15 seconds.

227
00:24:08,730 --> 00:24:12,389
If they use the back issues of the Journal and ACP Journal Club,

228
00:24:12,390 --> 00:24:18,690
it took them about 30 seconds and going to Medline took them about 90 seconds on average.

229
00:24:19,200 --> 00:24:23,670
Sharon Strauss, who was his registrar at the time, took the took the data.

230
00:24:25,180 --> 00:24:32,620
Okay. So I'm going to suggest instead that what we're aiming to learn here is three other practical things that you may be able to do.

231
00:24:33,580 --> 00:24:43,389
And these are the suggestions that I'd make. One is to read some sort of evidence based journal, some practical journal, such as for primary care.

232
00:24:43,390 --> 00:24:49,900
Internal Medicine is the ABM Journal. The ACP Journal clobbers evidence based, nursing evidence based mental health.

233
00:24:50,470 --> 00:24:52,990
Evidence based child health, I think has just come out.

234
00:24:53,380 --> 00:24:59,740
So there are a series of abstraction services and it's useful to read those rather than trying to keep up with the literature yourself,

235
00:25:00,100 --> 00:25:03,579
get somebody else to do it for you to find the best things.

236
00:25:03,580 --> 00:25:07,420
This is the sort of alerting service and that's what we might call push.

237
00:25:07,810 --> 00:25:11,590
One of the new things I need to know. The other is your own questions.

238
00:25:11,590 --> 00:25:14,980
The things that arise out of practice that you're going to go and ask somebody about.

239
00:25:15,460 --> 00:25:20,080
And what we suggest is you just keep a logbook of those and we'll give you out a logbook card.

240
00:25:20,080 --> 00:25:26,680
You've got them. They're going to hand them out now. So we'll give you out a little logbook that you can keep in practice of your own questions.

241
00:25:27,850 --> 00:25:33,370
You don't have to be ambitious here. Remember, you've probably made only less than ten changes in the year.

242
00:25:33,760 --> 00:25:38,409
If you could make an answer one question a week and every second week make a change,

243
00:25:38,410 --> 00:25:43,899
you would actually at least double the number of changes you're making. Okay, so just don't be too ambitious.

244
00:25:43,900 --> 00:25:48,280
You start off with an aim of answering one question per week would be sufficient.

245
00:25:49,750 --> 00:25:55,420
Okay. And the other thing to do is to share this. Both things can feed into a team based discussion.

246
00:25:55,750 --> 00:26:00,520
Let's talk about this as a practice for half an hour, a week or an hour a week, if you can spare it.

247
00:26:01,180 --> 00:26:09,190
Shifting this sort of content, don't do more stuff, but change the way you do your continuing medical education to run a case focussed journal club.

248
00:26:10,570 --> 00:26:15,470
Using eBay materials. So three suggestions.

249
00:26:15,830 --> 00:26:19,159
There are probably others. And as I say, this is an evolving thing.

250
00:26:19,160 --> 00:26:22,360
We don't know the best way of doing evidence based practice,

251
00:26:22,820 --> 00:26:29,570
but we know there's a lot of good research out there that isn't being used in practice, and we're trying to work out how to bridge that gap.

252
00:26:30,170 --> 00:26:35,180
And if you guys come up with otherwise, we'd love to share them during this workshop or after the workshop.

253
00:26:36,750 --> 00:26:42,540
So these are just three suggestions. I'm just going to show you one slot on each of those to tell you how they work.

254
00:26:42,540 --> 00:26:52,320
The process for the IBN journalists that we scan about 120 journals, including the major general medical journals such as JAMA, Lancet, BMJ, etc.

255
00:26:52,890 --> 00:26:59,550
And check the 60,000 articles in those to see whether they passed some simple validity criteria.

256
00:26:59,940 --> 00:27:03,810
If it's therapy, then it should be a randomised trial with at least 80% follow up.

257
00:27:04,260 --> 00:27:05,909
If it's a prognostic question though,

258
00:27:05,910 --> 00:27:13,140
we just want what's called an inception cohort and you'll go through all of these study types in your in your workshop.

259
00:27:14,310 --> 00:27:20,430
Okay. About 5% of the material passes that. So that filters out 95% of it.

260
00:27:21,240 --> 00:27:24,840
So the number needed to read defined one valid articles about 20.

261
00:27:26,600 --> 00:27:30,720
Then we ask, is it relevant? We get various people to say, Would this change your practice?

262
00:27:30,740 --> 00:27:34,610
Is it newsworthy? Is it relevant? Is it going to be a major change in practice?

263
00:27:34,940 --> 00:27:44,570
And we actually select a much smaller percentage then. So the number needed to read there is about 200 in order to get one relevant and valid article.

264
00:27:46,260 --> 00:27:48,180
There's a message about that for journal clubs.

265
00:27:48,180 --> 00:27:54,540
By the way, if you're running a journal club, if you just haphazardly search the literature and then appraise it at the journal club,

266
00:27:55,140 --> 00:27:58,950
mostly you'll come up with things that you trash and don't change your practice using.

267
00:28:00,210 --> 00:28:02,460
And it gets very disappointing then to run the journal.

268
00:28:02,610 --> 00:28:07,680
And every time you just trash the article and say, we're not going to change our practice on the basis of that.

269
00:28:08,160 --> 00:28:13,440
Actually, a journal club you'd like to get down to these mostly the things that are going to change your practice,

270
00:28:13,740 --> 00:28:20,370
the positive hits so that each time you journal club, you're actually learning about what practice change you should make.

271
00:28:20,790 --> 00:28:24,090
Mostly, sometimes it'll be something where there's pressure on you to change.

272
00:28:24,420 --> 00:28:28,570
But when you look at the article, you say, Well, actually the evidence for that is pretty weak.

273
00:28:28,590 --> 00:28:32,030
We're not going to go with that fashion. So that's one thing.

274
00:28:32,060 --> 00:28:36,200
The second thing is just to keep a simple log book of questions like the one you've got,

275
00:28:36,470 --> 00:28:41,330
it's spiral bound so you can keep it open on your desk, use it any way you like.

276
00:28:41,450 --> 00:28:46,310
Answer a couple of important questions. You can discuss the evidence with colleagues in a journal hub.

277
00:28:46,850 --> 00:28:51,560
You don't have to do a search. Maybe you should ask somebody, but also do a search and compare the two.

278
00:28:51,570 --> 00:28:55,220
What did my best colleagues tell me compared with what Medline came up with?

279
00:28:56,060 --> 00:29:02,330
Use it in all sorts of ways, but start out with just not trying to record every question because that will be too much.

280
00:29:02,390 --> 00:29:06,459
But aim to answer one important question of which. Okay.

281
00:29:06,460 --> 00:29:10,300
And the final thing is to run an ABM Journal club. This should be fun.

282
00:29:10,840 --> 00:29:15,310
One of the principles is to have fun and to have food. This is why I want to buy Mont St.

283
00:29:15,940 --> 00:29:21,280
You probably can't pick it here, but this is actually a blind tasting of different types of margarines and butters.

284
00:29:22,690 --> 00:29:29,680
So they label with a post-it note here to say which ones they coded so we could break the code afterwards.

285
00:29:30,010 --> 00:29:33,880
Because we're interested in the cholesterol lowering margarines. Did they taste any different?

286
00:29:35,160 --> 00:29:40,840
Okay, so it's a bit of fun as well. We've had yoghurt tastings and all sorts of things, demonstrations of the Epley manoeuvre.

287
00:29:40,840 --> 00:29:45,040
We do all sorts of things at the Journal Club and these are some of the questions that we've done.

288
00:29:46,120 --> 00:29:54,310
Antidepressants in adolescence are ten long for hypertension, etc. and of bold, the ones that have actually come from the ABM Journal.

289
00:29:54,520 --> 00:29:56,739
So what we do is when the ABM Journal comes out,

290
00:29:56,740 --> 00:30:03,250
I get people to vote on which things they want to look at in the journal club, which are the practice changing things.

291
00:30:03,790 --> 00:30:07,270
So that's the sort of push side of things, things that we've been alerted to.

292
00:30:07,630 --> 00:30:13,870
The others were specific questions that came up in practice because a patient had alerted us to it and we wanted to discuss that.

293
00:30:13,870 --> 00:30:21,069
And we did the search and found the evidence. And you can see that there's actually a mix of the two we use both for these discussions.

294
00:30:21,070 --> 00:30:28,470
Sometimes it's driven by a patient question, sometimes it's driven because new evidence has come along about our combined hail

295
00:30:28,490 --> 00:30:32,920
inhalers better for asthma or pelvic floor exercises for erectile dysfunction,

296
00:30:33,490 --> 00:30:41,300
which you probably never would have thought of as a question. But you say, well, this is an important problem in for us in primary care.

297
00:30:41,570 --> 00:30:44,840
Let's actually learn about that. Okay.

298
00:30:44,870 --> 00:30:50,330
That's useful in the journal clubs as you'll have in the little room to have some tools like a flip chart,

299
00:30:50,630 --> 00:30:52,820
a whiteboard where we used to keep our questions,

300
00:30:52,820 --> 00:30:58,100
we now keep them on an intranet, some good sources of evidence and plenty of coffee to wait people like this up.

301
00:31:01,250 --> 00:31:06,190
Okay. So I'm going to move on to part two.

302
00:31:06,200 --> 00:31:11,839
I said I'd talk to you a little bit about the skills, and I'm going to talk to you particularly about the formulating the question.

303
00:31:11,840 --> 00:31:19,610
But these are the the usual four steps of ABM. This is all in your workbooks in the book that you've got, so you don't need to take notes.

304
00:31:19,610 --> 00:31:29,329
But this is the skill part here, and I give you the example of a stethoscope as a sort of illustration of the amount of work that it takes,

305
00:31:29,330 --> 00:31:33,860
I think, to learn the process of critical appraisal. It's a bit like learning to use a stethoscope.

306
00:31:35,000 --> 00:31:42,530
You actually need a bit of practice there to actually be able to hear the heart sounds even, and then for the abnormalities.

307
00:31:42,530 --> 00:31:49,580
And some one clinician was telling me the other day it wasn't until his second year as a registrar that actually heard his first third heart sound.

308
00:31:50,090 --> 00:31:52,190
So it's a skill you'll gradually develop.

309
00:31:53,250 --> 00:31:58,260
You might have to fake it at first, like he'd obviously done for many years, saying, Oh, yes, I can hear that.

310
00:31:58,260 --> 00:32:01,640
Third heart's in. Okay.

311
00:32:03,380 --> 00:32:06,530
So let's just talk about formulating answer more questions.

312
00:32:06,620 --> 00:32:10,010
The big messages are that there are different types of questions.

313
00:32:10,730 --> 00:32:15,200
We need to be able to structure them with a pinecone. Can I ask who's heard of Paco before?

314
00:32:16,950 --> 00:32:21,630
Okay. Who's heard of variants of pinecone supposed questions, for example?

315
00:32:21,690 --> 00:32:26,020
Okay. A few people. So let me go through them.

316
00:32:27,460 --> 00:32:35,110
Pica was a very useful way of breaking down the questions, and it's particularly useful for an intervention question, a therapy question.

317
00:32:35,120 --> 00:32:39,520
But I'm going to show you that we can think about other types of questions using the same structure.

318
00:32:40,240 --> 00:32:43,660
Sometimes we're just interested in the outcome itself.

319
00:32:43,660 --> 00:32:49,600
What's important to patients and this on a pre question mark question, we don't know what the outcomes are.

320
00:32:50,710 --> 00:32:57,370
That are important to patients. Let me give you one specific example of this patient with rheumatoid arthritis.

321
00:32:57,760 --> 00:33:03,700
What do you think the most important symptom or problem for a patient with rheumatoid arthritis is?

322
00:33:05,560 --> 00:33:10,520
PYNE Okay, that's number two. Sorry.

323
00:33:11,360 --> 00:33:15,950
Function. Number three, reform, cosmetics, deformity.

324
00:33:15,980 --> 00:33:19,430
I think that's about number five yet. Anyone know what number one is?

325
00:33:21,200 --> 00:33:28,220
Stiffness, pi. Now, that's probably number four. I think going to the moon, going to the loo could be number six or seven.

326
00:33:29,420 --> 00:33:32,460
Amanda, as a patient with rheumatoid arthritis, do you want to tell us what?

327
00:33:32,480 --> 00:33:36,340
Number one is exhaustion, tiredness, fatigue?

328
00:33:36,360 --> 00:33:45,099
Yes. And the way this is now part of the measures that they use in rheumatoid arthritis.

329
00:33:45,100 --> 00:33:47,350
But it actually took the rheumatologists a long time.

330
00:33:47,360 --> 00:33:54,970
It's a group called Omer Act, which has been trying to standardise the measures that they use for clinical trials of rheumatoid arthritis.

331
00:33:55,480 --> 00:33:59,380
And I think it was in their third or fourth meeting that they decided to include patients.

332
00:33:59,860 --> 00:34:05,980
And the patients said, Oh, your tools are great, but they leave out our worst symptom, our biggest problem.

333
00:34:06,310 --> 00:34:07,300
And I said, What's that?

334
00:34:07,720 --> 00:34:15,600
And I said, Fatigue, exhaustion, knowing, oh, my goodness, why then did a survey of patients to find out the frequency of that problem?

335
00:34:15,610 --> 00:34:22,750
Indeed, the patients were right. It was the number one problem. So the second type of question you might have is about the prevalence or incidence.

336
00:34:23,110 --> 00:34:27,790
You need to know what the phenomena are first. And often that's qualitative research.

337
00:34:27,790 --> 00:34:31,780
You need to find out what's going on. Then you can start to quantify it.

338
00:34:32,650 --> 00:34:41,260
So we could have asked what's the prevalence of fatigue, pain, dysfunction, etc. in patients with rheumatoid arthritis either as a snapshot?

339
00:34:41,260 --> 00:34:44,860
What's it like it now that's a prevalence snapshot or incidence.

340
00:34:44,860 --> 00:34:56,290
How does it develop over time? So they're both PO type questions, prevalence being just a simple pill and the incidence when you add the T of the go.

341
00:34:58,040 --> 00:35:02,150
Okay. So we might ask the question, how common is an Eli Friess CEO?

342
00:35:02,690 --> 00:35:11,450
Eli Christakis check your neighbour to see if I've got one. It shows you a picture of one here.

343
00:35:13,400 --> 00:35:17,760
Normal. Eli. Eli. Chris. Okay.

344
00:35:17,770 --> 00:35:25,180
They actually increased with age. They've been said to be a risk factor for cardiovascular disease.

345
00:35:25,390 --> 00:35:31,630
But actually the confounder here is age that I just become more common with age and of course, cardiovascular disease does as well.

346
00:35:33,190 --> 00:35:36,490
So it's a potential confounder. So sorting out that is a risk factor.

347
00:35:36,700 --> 00:35:40,910
We need to know this prevalence by age as well. Okay.

348
00:35:42,140 --> 00:35:46,490
The next type of question is a PI code question. And here we've got the AC in.

349
00:35:46,490 --> 00:35:51,350
So I want to emphasise that not all questions have all four elements of the pi go in them.

350
00:35:51,740 --> 00:35:54,770
They can just be simple po questions or even the PO question mark.

351
00:35:54,770 --> 00:36:01,010
One's next type is a risk factor. Do patients with rheumatoid arthritis have a higher mortality?

352
00:36:01,580 --> 00:36:05,150
How do we tell that? How would you know with a risk?

353
00:36:05,480 --> 00:36:08,630
Rheumatoid arthritis patients. What study would you do to do that?

354
00:36:16,300 --> 00:36:20,560
Like like I can, you know, not using the word cohort.

355
00:36:20,570 --> 00:36:30,940
Can you just describe what you'd actually do for the patient from the time of the census and see how many of those people could.

356
00:36:33,300 --> 00:36:36,510
Compared to another group? Compared to another group.

357
00:36:36,870 --> 00:36:42,480
Okay. So the patient group would be patients with the initial diagnosis that's the inception cohort

358
00:36:43,110 --> 00:36:48,389
of rheumatoid arthritis would be the indicator here versus a group that don't have that,

359
00:36:48,390 --> 00:36:56,730
no rheumatoid arthritis. And we'd have to argue about what the appropriate controls are and the outcome would be more death, mortality, death.

360
00:36:57,600 --> 00:37:02,700
Okay. No randomisation involved in that, by the way, because here we're just interested in the natural history.

361
00:37:02,700 --> 00:37:05,850
Do they have a higher mortality rate if you've got rheumatoid arthritis?

362
00:37:06,150 --> 00:37:09,630
If they did, we could start to say, what's the causation of that?

363
00:37:09,660 --> 00:37:14,490
Think about that. But the first thing we want to know is, is this true? Do they have a higher mortality?

364
00:37:15,510 --> 00:37:23,160
So that's a prognostic factor. Treatment might be do patients with rheumatoid arthritis benefit from methotrexate?

365
00:37:23,640 --> 00:37:28,670
So what's the population? Maybe rheumatoid arthritis?

366
00:37:28,680 --> 00:37:34,770
What's the intervention comparator? Not methotrexate and the outcome.

367
00:37:37,020 --> 00:37:40,620
Pain and fatigue?

368
00:37:41,760 --> 00:37:47,850
Yes. Fatigue, dysfunction, all of those things that we said we want to measure all them.

369
00:37:47,850 --> 00:37:56,070
Which things does it actually change? It may reduce pain, but not change the incidence of the fatigue or the flu like episodes that patients get.

370
00:37:56,340 --> 00:38:00,540
And that would be of interest to patients. Okay. So it's the same structure.

371
00:38:00,750 --> 00:38:07,200
But here we'd probably like to have a randomised trial as our best evidence patients randomised to methotrexate or no methotrexate.

372
00:38:09,780 --> 00:38:15,460
Okay. So one important message there is the best evidence will depend upon the type of question.

373
00:38:16,000 --> 00:38:21,610
If we're just interested in the phenomena, what if patients with rheumatoid arthritis experience, what troubles them?

374
00:38:22,450 --> 00:38:25,890
What outcomes are they interested in? That's qualitative research.

375
00:38:25,900 --> 00:38:29,690
We sit down with a bunch of people and ask them. Okay.

376
00:38:29,690 --> 00:38:31,700
If we want to know the frequency of that problem,

377
00:38:32,000 --> 00:38:37,490
then we want to get a sample of patients with the condition and ask them about them and preferably a representative sample.

378
00:38:37,890 --> 00:38:44,130
My way of getting that is usually a random sample. I didn't talk about this one, but this is also a picture of structure.

379
00:38:44,610 --> 00:38:48,299
How does the person have the problem? How do we diagnose rheumatoid arthritis?

380
00:38:48,300 --> 00:38:55,620
And we can look at CERP or other new techniques of diagnosing rheumatoid arthritis and compare that with a gold standard.

381
00:38:56,590 --> 00:39:04,660
That's a Tyco structure as well. The prognosis we said that's a follow up or inception cohort, the mortality from rheumatoid arthritis.

382
00:39:05,260 --> 00:39:10,030
And finally, we want to fix the problem and this is probably our most common question in practice,

383
00:39:10,030 --> 00:39:15,040
by the way, when people have kept log books or looked at questions sent into services,

384
00:39:15,400 --> 00:39:19,270
about 70% of our questions actually turn out to be about therapy,

385
00:39:19,660 --> 00:39:25,090
which is why there's a special emphasis in evidence based medicine on randomised trials.

386
00:39:25,420 --> 00:39:27,969
But for all these others you don't need randomised trials.

387
00:39:27,970 --> 00:39:34,390
In fact they wouldn't be as good an answering the question for this, the randomised trials, the ideal thing.

388
00:39:36,400 --> 00:39:39,070
Which is why we have these hierarchies of evidence.

389
00:39:39,490 --> 00:39:44,950
The one that you mostly see is for treatment where randomised trial is the so-called level two evidence.

390
00:39:45,970 --> 00:39:50,890
Right. But if we're talking about prognosis, the level two evidence would actually be an inception cohort.

391
00:39:51,340 --> 00:39:56,260
A randomised trial may not be as good because of the selection process into it.

392
00:39:56,620 --> 00:40:03,159
You'll get a limited number of people in. With an inception cohort, you usually get a broader representation of folks,

393
00:40:03,160 --> 00:40:11,200
so it's actually a better thing to use and randomised trials would be under inception cohort as a lesser thing for prognosis and diagnosis.

394
00:40:11,200 --> 00:40:14,900
You need a simple cross-sectional study. Okay.

395
00:40:14,910 --> 00:40:21,150
And the ideal thing, the level one evidence for all of those would be a systematic review of whatever the level two evidence is.

396
00:40:21,570 --> 00:40:26,460
Systematic review of inception, cohorts for rheumatoid arthritis, mortality, etc.

397
00:40:27,630 --> 00:40:31,530
So it's important. One thing you'll you'll start to notice when you look at levels of evidence,

398
00:40:31,860 --> 00:40:36,630
people fail to mention that when they're talking about that, they're really talking only about treatment.

399
00:40:37,440 --> 00:40:46,750
It's an important misconception. Like this afternoon you'll find out how to filter down to those things using PubMed

400
00:40:46,760 --> 00:40:51,440
in a special thing called pubmed clinical queries whose use pubmed clinical queries.

401
00:40:52,780 --> 00:40:59,739
About a third of you. That's great. One of the special features is that it has you can click on diagnosis, therapy, prognosis,

402
00:40:59,740 --> 00:41:07,540
etc. There are a number of categories here so that when you search you get some filters that will filter down to the randomised trials for therapy.

403
00:41:07,900 --> 00:41:13,450
The inception cohorts for prognosis. ET cetera. And I filter by about an order of magnitude.

404
00:41:13,450 --> 00:41:17,200
So it makes your searching much, much easier to use those filters.

405
00:41:17,590 --> 00:41:21,460
But remember, you've got to use the right filter depending on the question that you're asking.

406
00:41:22,770 --> 00:41:28,559
And that's a table of the filters developed by Brian Haines by using what the filtering process that we

407
00:41:28,560 --> 00:41:35,129
use for the ABM Journal to try and find out what the most sensitive and specific filter terms would be.

408
00:41:35,130 --> 00:41:42,710
And you can see they're quite complex. On the right hand side, they're the best filters for each of those types of article.

409
00:41:43,500 --> 00:41:48,060
You don't have to type in. Anything you want to do is click on a little button and it types everything for you.

410
00:41:48,360 --> 00:41:55,410
It's really nice of it. Okay. I just wanted to finish by saying there are other hierarchies of evidence.

411
00:41:55,420 --> 00:42:02,700
There are lots of different levels of evidence that you'll see. The Grey Group is trying to standardise this whole process of the levels of evidence,

412
00:42:03,030 --> 00:42:06,269
but other ones are the levels of evidence for anecdote based medicine.

413
00:42:06,270 --> 00:42:12,749
For example, level one is a very old jump from the Royal College level to is a doctor with an air of credibility, an honest face.

414
00:42:12,750 --> 00:42:14,130
I can see several around here.

415
00:42:15,630 --> 00:42:26,010
I'm only level three academic with a mad stare and level four as an NHS manager with their trust in financial crisis is the least reliable thing.

416
00:42:27,330 --> 00:42:34,710
Okay, where to now? You need to find out your rooms because we're now going to break up into small groups before we go for coffee.

417
00:42:35,250 --> 00:42:38,400
So just make sure you can work out where your room is.

418
00:42:39,420 --> 00:42:41,580
And I'm like it. The tutors to stand up.