Welcome to everyone. My name is Rafael PEREIRA, a medical statistician working in the Department of Primary Health Care.
I've been involved with the centre for the last five years, I think.
And today we're going to be talking mainly about rapid critical appraisal, particularly of controlled trials,
which is sort of the second or third step of your evidence based medicine steps.
So before I go into this, I hope you all received an envelope with something in it.
Please keep it short. Don't open it. The second thing I if you haven't, please ask only for one.
If you haven't got one,
second thing I need you to have is to have your workbooks because we're going to do a bit of work that involves reading and appraising.
You have it with you ask, all evolve has some spare copies as well that you can borrow if there's loads and those of you.
Then I'll ask you to work in groups of two or three.
All right. I'll need a lot of patient, patient participant participation.
Okay. So as Paul mentioned earlier on, we're really looking at five different steps in the evidence based medicine to do evidence based medicine.
Mike is on the OC.
Is the mike on the slide?
Can you hear me now? I'll predict, I hope.
Okay. Paul has mentioned or mentioned in the morning that the five different steps of evidence based medicine,
the first to formulating an answerable question and tracking down the best evidence.
The first one, sorry, was mentioned at the beginning of the first plenary session and was also part of the group sessions right now.
The second step tracking down the best evidence will be done this afternoon.
Okay. Today, we're going to right now we're going to be focusing mainly on critically appraising the evidence for the
validity of the evidence that we find with the studies that we we've identified are actually valid,
the impact of the size of the benefit. And finally, the applicability.
Well, the findings that we have in this particular study can be applied in our setting.
Now, when we're looking at using evidence based medicine, generally,
we think of a series of items that we need to check and decide whether they're actually there or not.
To define it, these particular studies are actually useful or not.
So if you type a critical appraisal checklist and randomised controlled trials in Google,
as you can see you get in 2005, you get 36,000 articles going up massively, 2006 it's getting a little bit better.
By 2008 we only have 97,000 articles that proposed different checklist of doing critical appraisal of randomised controlled trials.
And this is a sort of a typical checklist with 20 different items that you go
from the beginning to the end with your trial and with your study in hand,
in the other hand, and you go take one, take two, take three.
Probably by the time you finish reading all of them, we've forgotten what your question was in the first place.
Okay, so what are we going to try to do today is give you a framework of two simple acronyms
to try to remember how to do critical appraisal and first critical appraisal,
particularly of randomised controlled trials. And I think during the small group exercises you all looked at exercise,
one of your abstracts had to deal the dealt with this particular clinical question which is in people who take long haul flights,
does wearing graduated compression stockings prevent DVT?
So the first question I want to ask is how many of you have had a long haul flight in the last couple of years?
Just don't know how many of you up for stockings, compression stockings.
Okay. And those are you didn't let's see how many of you those of you that didn't, how many of you would think about wearing compression stockings?
Okay. Let's see if we can change that towards the end of the session, or maybe not.
So for those of you who don't know what compression is talking, so I have an example.
So these are the typical compression stockings worn by one of the teachers in this week's course.
And I need a volunteer to wear these stockings.
Washington. They've been very clean. Yeah.
Three or four can do it. And give the talk at the same time someone could.
Actually, you're in perfect position because it's very similar to what type of seats you were getting on an aeroplane as well.
Yeah. Two on the flight. Could you put both on? Thanks.
You. Okay. Now, in your workbooks, you can find in page 73 of your workbooks.
What search function to try to answer this particular question.
Okay. And they came up. Well, we came up with a particular study.
By Ska in 2001, and the full study of the full paper can be found page 95.
So if you go to page 95, what books? Okay.
You'll be able to see the full study as reported in The Lancet in 2001.
For the rest of the presentation, I'll be asking you to come up with the answers for the different parts of the quest of of the critical appraisal.
Should be able you should be able to find in the paper right there.
Okay. So the first thing to do is to think about whether this particular study that we found is relevant and can answer the question that we have.
And for that, we've already generated a structured question using a picture from a clinical setting.
And what we want to do is do exactly the same to generate the picture or the question that was trying to answer whoever wrote this particular paper.
So we weren't going to set up to find the high school for this particular trial.
So I want you to have a look. I'll give you 20, 30 seconds.
Search around in the and the article to identify the participants,
the intervention group, comparison group and the outcome that they were interested in.
It's not a solo exercise. We obviously can confer with your neighbours.
You're happy to talk, but it is a critical appraisal exercise.
So let's let's crack on. The other thing that I want you to notice here, I've set up the PI connects to a series of geometrical shapes,
and this is what is called the Gape framework developed by Rod Jackson.
And the way he came up with this series of figures is he was looking at where we are, the Xbox generation.
And this is pretty much what he's that's a lot of people that use play with Xbox and these are actually the figures that come in,
the experts, a triangle,
a circle, a square, and then the light from the triangle will represent precisely the population with unidentified target that you want to look at.
And then how it narrows down to who you actually included in your trial.
Then you have a circle which sort of creates the population that you encounter.
So we put into the trial into two groups, the intervention group, whoever you want to, to apply whatever therapy it is that you want to apply.
And then a control group or comparator group and the circle with dividing in the middle
sort of highlights the fact that actually looking at pretty much the same individuals,
so the same type of comparable groups with the only difference that these two groups have being the intervention as such.
So anything that we find different between these two groups will be down to the intervention.
And finally, an outcome which is a square just typical at two by two table with what we have,
whether they have the intervention or whether they were intervention of the
control group and whether they have the outcome or they didn't have the outcome.
So that's the framework. And it's very useful because as you go around, for example, through a paper and you find things about the population,
you can just put a little triangle next to to the, to the paper.
And they said, okay, this is something about the population, something about the population.
You find something about the intervention as such describes that you can just put
a little a little circle and identify the the intervention of the control groups,
that way of descriptions of intervention and the same thing with the outcome.
Okay, so look at this particular paper.
What was the population that were interesting in in looking at estimates of passengers compared with 41 years of age over 50 years,
the most history with no past history women.
Okay. Okay, so that's fairly specific. So passengers of long haul flights, they specify the this 8 hours.
Okay. Passengers of long haul flights longer than 8 hours over 50 returning to the UK in the next six months.
Six weeks. Okay. So it's fairly specific. Does that match up with the P that we have in our question, critical question.
Reminder people take long haul flights. Well, maybe it's a bit too specific because they're looking at over fifties.
Right. Maybe just the right the right time. So they're looking at a very fairly tight group of people by way of intervention.
What was the intervention that they're looking at? Didn't specify the type of stockings.
So you would be able to go and buy these stockings. Okay.
That's quite important as well in terms of whether we could actually generalise the findings or replicate the findings.
Finally, what was the comparator that they used did not work.
Okay. So nothing. All right. Okay. Again, I think it matches roughly a clinical setting.
And finally, what outcome do they look to? Symptomless, DVT.
How is that assessed? Okay.
We'll come back to that in a second. So all of these things that we've identified in a way,
the picture of the study that we've retrieved and what we need to decide is whether the
picture of this particular study matches the picture that we had in our structure.
Question. If it doesn't, we can take it away. We don't need to do anything else.
Okay. That said, if it does, then we can go to the next step.
We decided it can be relevant. So now we want to assess whether it is valid with internal validity or not.
Okay. So we have a call for relevance and we have another acronym for Internal Validity.
And this next acronym is Rambo. Okay.
And it stands for, in the case of therapy studies, recruitment, allocation, concealment,
maintenance measurements, and where these measurements were blinded and objective.
And we'll go through each one of these bits in more detail in a second.
Okay. These are the things we need to identify in order to determine whether the study study
was done to a high quality for then the results to be of any use to us whatsoever.
If we decide that any of these things are not done adequately, then we might or this particular study might be introducing some kind of bias,
which would mean that the results they find, no matter how big the trial was,
the results that they found might not be answering the relevant question.
And that's why we want to do this. Before we do, we look at any anything related to the actual results of the study.
And normally you would find all of these things in the methods side of of of the of the paper.
So you look at the methods side of the paper to find things about recruitment,
whether they did randomisation, how is the maintenance done and the type of measurements that were involved?
So Tamarod on Rambo is the thing that we're going to be focusing on for doing critical appraisal or validity,
internal validity of a randomised controlled trial. And just to break it into small bits.
First we look at recruitment,
try to identify in a way we're matching the Piku and the Rambo doing the internal validity and the relevance of the study.
So in parallel. So the first step is to find out who did the studies that the subjects represent.
Then we'll find out about a location, whether the assignment of treatments with randomise or not.
We'll explain why this is important, where the groups similar to the trials start.
Then we look at maintenance, which is where the groups are treated equally, where outcomes ascertain and analyse from those patients.
And finally,
the measurements and we will discuss why blinding is an issue and when blinding is an issue and what type of blinding is important or not.
And finally, whether the measurements were objective or standardised and the objectivity and the blinding of the of the outcomes are tied in.
So you might be more interested in blinding, particularly when the outcome is not objective or fairly subjective.
And finally, we look a little bit about the the results and study statistics.
So let's focus first on the steps one and two of the are the recruitment and allocation.
I want you to have a look at, again, the article and try to identify who they managed to recruit and.
What how do they actually work through their allocation. So how do they randomise and whether groups around the trial trial stuff.
And in many cases you might find these in figures and tables rather than lengthy descriptions.
So have a look at try to make it fairly. You'll find that in practice.
With practice, you might get this really, really fairly quickly. I'll give you again another 30 seconds.
Okay. Let's move on. So in terms of who they collect, who they identify, did anyone find?
Well, we already talked about what type of population they were interested in finding.
So these were people in long haul flights over 50 with no previous history of deep vein thrombosis.
Did they get those people to anyone?
I mean, it's not related to this, but they actually collected.
Because sometimes a study would say this is the type of population we want to identify and the people they actually get in is completely different.
Okay, so the pi coin, the original question might not be.
May not match the population or the people that weren't actually in the trial.
What about here? Do they actually match? 50%.
50%. That's okay.
The only one identified sort of how they recruited. Newspapers took.
Okay. And that was to determine whether they were okay.
So in a sense, it seems that, yes, that part was tight and fight was an issue to decide whether there were 50 probably.
What about long term, long haul flights? That was a third part of the inclusion criteria.
Okay. And why they included people on the aircraft.
I mean, I've just quickly gone through the. Okay. And I was trying to see why I think was the question, was prevention simple?
I mean, I'm I felt like they wanted to see a higher frequency indication that this expired high frequency goes above 50.
But knowing the opposite, this is why you chose specifically.
Okay. Okay.
So that they might that might be something that if you were to apply these findings to someone else, if they were under 50, you might disable it.
I'm not entirely sure. Yeah. Okay. Generalisability of the question.
Okay. Yeah. That that could be maybe one of the shortcomings.
Maybe you wanted to know further, but I think in a way, you've answered the question yourself.
But I think sometimes I should, because at the hospital, when they said that,
they think that that was not ideal authority to exclude individuals at highest risk.
So, again, I'm not really sure what are trying to say, but I expect that they did at one point a greater frequency,
but yet at the last part, they excluded girls who are at high risk.
And I think in this case, the reason for excluding those at higher risk is to in a way so that previous episodes.
Yeah. And questions in a previous. Yeah. Yes.
And also to make it more generalisable to the overall population, I guess, and to decide whether the long haul flight was positive or not.
That's right. Okay. Let's see how it is played.
The sample size is high sample size. I think they don't mention that.
Oh, sorry. They do mention that. We've got we'll go into that much later on.
So in terms of whether they collect the right sort of people,
I think they did at least identify a group that they decided or they said they were going to identify.
The next step is whether they actually did the right type of allocation.
That is, whether they randomise these individuals to the two groups in such a way that no systematic bias could be found.
That is, I will always watch all the men in one group and all the women in the other group that could potentially have a problem.
So to do they identified what did they say, how they chose the two groups.
Okay. So they were randomised the randomised volunteers by sealed envelope to one of two groups.
Okay. You're happy with that. I'm not sure I'll be randomised.
They just had an envelope. Okay. That was not unless this is related to our envelopes.
Okay. So we have our envelopes at hand.
Well, I need to now open them. Amanda and I.
So you should have one of two. You've been randomly allocated to fruit pastilles or fruit gums.
Right. Okay. And there were envelopes.
Did anyone switch their envelope around?
Okay. Did anyone open the envelopes before I asked? Okay.
A couple. So we might we might have some problems in terms of of how they were how randomisation was was.
Of course I don't want there. We have contamination swabbing.
Yeah. Okay. And these are all things that could potentially happen in a randomised controlled trial.
Okay. So thinking about what we've just done, we've done our own media randomisation.
I want to ask two, but we have two particular outcomes and is related to long haul flights who have been to Australia.
Those that have been to Australia can be raise your hands. Okay.
How many sorry? How many of those that have been to Australia have the gums, fighting gums.
One, two, three, four, five, six, seven, eight, nine, ten, 11, 12.
And how many of those have the still? Three, four, five, six, seven, eight, nine.
Okay. Okay. So in this case, I would maybe say that randomisation, roughly work.
We have roughly the same numbers in each one of the groups.
Okay. We have 12 and ten that if we want to do a proper statistical test, we probably will find that it's not statistically significant.
What about who how many of those who had the guns made us noble in the last 24 hours?
That comes. One, two, three. How about a positive?
One, two, three, four, five, six. Twice the number.
Okay, so here we have an outcome that's actually less likely to happen, and we have twice as many.
Probably if we were to test this thing,
maybe we actually show that there's a statistically significant difference between the two groups for that particular outcome.
And this was a completely random you have to agree.
And this is, again, this sort of illustrate the type of things that you might find in a trial,
even though you have randomisation, even even if you do randomisation properly,
just by chance, you might find that some characteristics in one group happen to be higher than another group of high school status.
So why should we be a double cross? Should be for arbitrary five.
Intention to treat as well. That's another thing that you need to. So there are some crossover here.
Okay. All right. So how do we ensure allocation, concealment?
There are several different methods, so you can try to deal with this.
And generally the one that's decided to be the best is to use some kind of central computer randomisation,
where when there's someone actually doing the randomisation for you that is outside that person, that's including the participants in the trial.
And generally, we for example, if you're entering someone to the trial, you phone a central person doing the randomisation.
They tell you, oh, it's labelled blah, blah, blah, two, three, four or five,
or it belongs to the placebo group, and therefore it has to receive this type of treatment.
And because there's that barrier between the person enrolling a participant and the person doing the randomisation,
then in a way you make sure that there's no problems, that the concealment is actually done adequately.
Now the second step would be used, and another approach would be to use, for example, envelopes, and this is seen as could be affected by tampering.
And here we have a little example with if someone is very interested in finding out what group they're going to be allocated to,
you might even look through the envelope, try to how many of you, for example, try to feel what the.
Okay. And maybe if you had this take away was fruit pastilles versus something completely different, you would have swap don't know.
So envelopes can be tampered with. And for that particular reason, they're seen as potentially suspect, but it's still using randomisation.
And finally, we have some studies that are actually not randomised when the allocation is is done by some characteristic of the participants.
I could be, for example, the date of birth when they come in at alternative alternate days.
And the problem with that is that you've already allocated the individual just just because of that particular characteristic.
And if that characteristic has an association with something else that might affect your findings in your trial,
in a way, what you're doing is you're creating two groups that are different.
They're not exactly the same. They're different because of one particular characteristic.
And in a randomised controlled trial, the objective is precisely to create two groups that are exactly the same except for the intervention.
So the question is, in this particular trial with a group similar to the trials, does they give us any information about that, whether they say.
So if you have a look at your table one. Are you happy with the information they provide you?
Do you believe that the two groups are two groups created on roughly the same?
Do you find any differences with gender?
Okay. Number of women. So the gender, of course, is actually about all this stuff.
Okay. So one thing that I should ask, how many of you always fly business class?
Okay, so you can all stay and this is relevant to everyone.
So that may be important. Why? Because the three of you know that there's space to move around.
Okay. So to say, what happened to those two women that were excluded?
Are you happy with that? Okay. That was part of the inclusion exclusion criteria.
Were those two in the same group? I remember. I think that I'm not sure if they specified.
So besides gender, was there any other thing that you identify in terms of the baseline characteristics that might be different?
But despite the. The last two the last to feel positive and positive now is that is that a genetic markers that right.
Yeah. Okay. Okay.
And again, those were done to try to identify if there was some one of these genetic markers where associated with deep vein thrombosis.
Um, the non stockings were slightly higher and one stockings were higher, slightly higher in the other.
I think a direct test might not show any fact, but it's similar to what we have here.
So why is this important?
We don't want to do a direct comparison between these two things or work out the p value of differences in the event rate of each one of these things.
Because that's not really what we're interested in, but we want to identify that potential differences between the two groups.
Why? Because we one need we might need to take that into account when we're interpreting our results.
So what are we doing? Finding out, doing our analysis. We might need to, for example, identify that women, slightly more women in the stockings group.
Does anyone know? Yeah, well, because.
Don't you think that. Because you want to. For that.
Four, two, one, two, one, one, six. And 45 out of 115 up the hole, I imagine, is a whole group.
Yep. And much as I would say good points, if they had more varicose veins in my head, I wouldn't have an impact.
So anyone know if women or men are more prone to having deep vein thrombosis?
So I'm not clinicians. Okay.
So if if women are more prone to having having the pain from both this and the ones in the stockings,
what we we expect in our findings to would be favouring stockings.
Going against stockings. Yes. Okay. Okay.
So, if anything, this imbalance that we find might be biasing against stockings.
Right.
And this this other type of things that you need to take into account, we need to be thinking about when you look at the baseline characteristics,
not only if there's a potential difference, but also in which direction those differences might bias.
Here, we felt. Okay. So.
So I could give you participation in that one quarter to the emails and all this.
Okay. So it may be that men were less likely to agree to participate.
That's true. That's true. Do you think that's just because men don't want to wear compression stockings?
But John, John, John was the only one that volunteered, so that was against it.
But it may be that participation was less an issue in terms of getting men to do it.
I think if we had a more efficient group would still be who would expect to do this to have more?
So, of course, that's right. So this is the damage was evident in the study.
It would if we if we find if our finding is the opposite, yet if our finding is the opposite,
that the stockings have less than sorry, this particular light, this particular issue would bias against stockings.
So if we find that stockings actually effective, then we might be more likely to believe it if it cuts out the characteristic.
So some people might be more willing to cooperate, theoretically saying, yeah, okay, now I know what the study is doing.
Yeah. Do I think they mentioned that, don't they, that they highlight the fact that because these are all participants.
Yes. So we've looked at the first two parts of our Rambo recruitment allocation.
Let's look at the third one maintenance. And for this,
we want to focus on whether the groups were treated equally or not and whether the outcomes were
ascertained in the same way for both groups and ascertained from most patients and most patients.
Why is this important? Okay.
Let me talk very briefly about what could happen if we have groups in a randomised controlled trial that are actually treated slightly differently,
not because of information but because of something else. There was a randomised controlled trial of vitamin E in preterm infants that's aimed to
check whether there was an effect of vitamin D preventing ritual landfills to replace it,
which is basically a condition with newborn.
Sorry, preterm infants basically go blind. That's that's the simple way to explain it.
And they found that there was an effect so that, yeah, vitamin E did prevent this.
However, looking more closely what was happening is that in order to give them vitamin D,
they had to be removed from the 100% oxygen chamber, and they were given that vitamin D and then put back in.
And it was actually that wasn't done to the control group.
Okay. So it was that aspect.
Rather than giving them or not giving them a vitamin B, vitamin D that was having the impact on the outcome that was preventing them going blind.
Okay. So whenever we do a trial, we need to make sure that apart from the intervention, everything else is done in the same way.
Otherwise it might be whichever other thing is not being done in the same way for for both groups,
that might be the one responsible for the effect that we have.
We actually getting rid of the, of the, of the main benefit of doing a randomised controlled trial.
So in the case of for example,
a drug trial is relatively straightforward because we have a drug trial and generally a placebo and everything else is done in the same way.
So you give, you give them medication, you tell them to, you tell whoever it is to take them two or three times a day.
You follow it up for however long, and then you find what the outcome is.
So everything is exactly the same, and then you can be sure if there's a difference,
then the active ingredient in the drug, in the medication is what's actually having the effect.
Okay. In some of the interventions like for example, this one.
Yeah, stockings it's a bit more different were even more difficult to identify that all the
participants did exactly the same thing or had slightly the same type of intervention.
But they do mention something and they talk about whether there was equal treatment in the two groups.
Okay. In order to do that, they they, for example, present something in table three.
And they they mentioned that looking at particularly had drugs that might affect the propensity of an individual to have deep vein thrombosis,
that both groups were actually roughly in the same types of medications aspirin, hormone replacement therapy, etc.
Whether it's anything else in terms of how these two groups were treated, do they mentioned that they were treated equally or differently?
Can you have a look? I'm going to wait 10 seconds for you to.
To identify anything else that was done differently or where they both treat in the same way.
No. You have to turn to donations for the research volunteers that have to travel to time ships before their travel.
Okay, I decided that it is just not practical to do that.
Okay. The rest of the volunteers will have. And it doesn't actually say which group which you belong to.
Okay. So it potentially could have made a slight difference.
And they also don't don't say whether they happen to be the ones that had the vein thrombosis or not.
But apart from that, I think that they do specify that they try to treat both groups in exactly the same way they gave the same type of advice.
Okay. What about the follow up?
And this is in terms, again, of maintenance, deciding whether they looked at it.
They follow all the individuals they try to ascertain everyone in and they analysed everyone that got into into the trial.
Now, what do you have on the right hand side? Is some sort of a flow diagram or flow diagram.
So everyone familiar with them who's not familiar with load diagrams?
Okay. Okay. So we have a total population that were considered and then how they, they, they go so how they excluded some, how many were excluded,
how many were randomised to each one of the groups and then what happened to them and if they were excluded,
the reasons why they were excluded and finally how many of them were analysed and we have hundred in each one of the groups.
This is important because. But particularly the second part of it after randomisation.
If we lose a lot of people if this.
A great loss of follow up after randomisation.
It might have a very important impact in how we interpret and analyse our findings.
So for starters, it could be that this type of intervention cannot be applied to everyone.
It could talk about it could be a proxy for the quality of the trial itself.
If they lose 50% of their lives, 60% of the ones that were randomised, maybe the overall quality of what they actually find might be put into doubt.
So this is this obviously a very important issue.
And not only that, but it might be that the actual loss to follow up might be linked to something related to the intervention.
So if we have differential attrition,
which I'll go back for a second that we lose more in the intervention group than in control group that might start ringing some alarm bells.
So there might be something wrong or interesting happening there.
In this particular trial, there were 231 included.
200 of them were analysed. So that was 87% of them.
And they do tell us why they lost the other 3127 one were unable to attend so they just couldn't be bothered or
they just didn't even come back to where it's clear from analysis because they were upgraded to business class.
We just mentioned about that and to exclude because they were taking anticoagulants medication so that they
were already likely to have probably likely to have from doses and therefore they were at higher risk.
Okay. Any questions about that, too? Are we happy? Yes, absolutely.
Okay. We'll talk that in a second. Any other witness?
There's just a number of accounts of how to be secure.
Number three shuts down. Okay.
They do? Yeah, they do mention in in the methods that they were able to work out a sample size,
a proper sample size calculation because they didn't have enough pilot studies.
And, in fact, they they say that this can be treated as a pilot study.
And what they had proposed is to keep doing it until they got 100 people recruiting into each other.
I think that's that's why you get those very convenient numbers.
But I think that's that's how they they set up to keep in terms of how many large groups they basically kept on recruiting.
Oh, sorry, 4/100 for the proposed sample size.
I can't remember if they mentioned specifically why one of the main objectives was to determine the prevalence of deep vein thrombosis.
And I'm not sure if that's one of the the reasons why they said, well, with 100, we should be able to at least get get a rough.
I think they do mention that if they if the vein thrombosis, the prevalence was fairly low about 2%,
that would give them fairly tight confidence intervals with 100. And that's why they came up with that number.
Remember that? Yes. That's how the study.
Yep. Is that in or out, though? No.
Conclusive. How? Yes. Well, there was a power calculation.
How about the study? Okay. Yeah.
The power of the study, in a way, tells you how likely you are to find a difference if one exists.
So find an effect, even if it's actually present. If you if your trial is very small and there is an effect, you're probably not likely to find it.
You could try this very, very large, even if the effect is small. Probably likely to find an effect.
And we'll go more into that when we look at how to deal with uncertainty.
So P values and confidence intervals. We'll talk about that in a second.
But going back to how many can we lose if really is actually dependent on the type of condition you're looking at?
So for some things, you want to have pretty much every one. For other things, you would be very, very happy if you have more than 20% followed up.
So it's actually just tying yourself to one number is actually not helpful.
But generally and as a rule of thumb, you can talk of the 5020 rule of thumb.
And that basically means that if you lose about 5% of your individuals in terms of follow up does slightly to to produce very little bias.
If you lose more than 20%, then actually have a strong implication in terms of the validity of the study.
You're probably losing too many generally. Again, this has to be taken in context.
Importantly, this depends, as I mentioned, in the outcome event rate and and also on comparative loss rates in the two groups.
So if we have that the loss of follow up rate exceeds the outcome event.
It's that we have a lot more that were lost to follow up than the number of people that benefited or or that did have or some believe vein thrombosis.
However important because all of those that we lost may have had be vein thrombosis and we don't know about them.
And probably more importantly, if the the attrition, the loss of follow up is differential,
we have more in one group than another that might have, again, implications as to whether the treatment itself,
the effect that we find is is is is real or it might be just being produced by the fact that we have differential follow ups.
So, for example, let's say that we have a medication that is very that reduces people to feel very nauseous and people stop taking it.
Therefore, you lose them from your analysis maybe,
and you find that the medication is brilliant because you have you haven't actually taken
into account of other people that actually couldn't take the medication in the first place.
Okay. So that's might be an extreme example of differential follow up, right?
And in this particular case, they do mention that there were losses, but these losses were equally distributed in the stock group.
They lost six men and women, roughly 15 in the non-smoking group.
So men and women with 16, do they have similar characteristics to those that were included?
Unfortunately, they didn't actually provide information about them and say it could have been just a sentence in the article saying the people
that were lost to follow up were roughly of the same age and and the medications taken were roughly the same as the other group.
They don't provide that type of information, but it would have been important to know about them.
And related to this is what I'm sure you've all heard of, is intention to treat principle,
which maintains that once a patient is randomised to one of the groups they should be analysed in the group they were randomised to.
So what Mike just did in terms of switching the wine gums and with pastilles would have still kept them into the gums initially.
I think, okay, he should have he would have still be kept in the commonest group, okay.
Because otherwise randomisation allows you randomisation means that if it's done properly, you get equivalent groups.
But if you have some kind of crossover because you don't like gums or because you don't like stockings, let's say,
then your findings might represent a completely different thing than the actually the actual effect of the intervention.
Okay. One exception to the intention to treat principle is that if the patient is found to have on blind reassessment to have been ineligible,
they should be excluded. And that's, for example,
what was mentioned earlier on in terms of the people that were flying in business class were upgraded
to business class because they were not they shouldn't be included as part of your population group.
They were part of the group that was part one of the exclusion criteria. Okay.
After after reassessment. So how do you assess the outcome for this trial or general belief?
Okay, that that's a very good question.
Intention to treat is very much seen as a gold gold standard way of doing the analysis.
And it might be very difficult to implement depending on what type of outcome you have.
If you have binary outcomes, where there's a very clear definition as to what with a positive outcome or what's a negative outcome,
let's say the vein thrombosis, if you were able to.
It's a simple scenario if you collected outcomes from everyone,
but you just have things like crossovers and people just not doing what they were told, then that's fine.
You just can keep them in the same group if you have not only people crossing over.
We're also missing values. For example, you don't know if they had the brain thrombosis or if they didn't have thrombosis.
You need to make some kind of decision as to what type of value you're going to give to that particular outcome,
and then try to analyse with that assumption and without that assumption to see if you
are actually increasing the or producing some kind of bias in your in your analysis.
But if you looking at other types of outcomes like continuous data and it becomes slightly more complicated,
you might need to use more sort of different statistical approaches to impute that missing information.
Okay. Finally measurement, the last part of a run when we do measurements.
What we're interested in is finding out whether the outcomes that we get are actually a true measure of the effect, if there is any.
And for that,
we're looking at things like whether the patients and clinicians were applying that or whether the measurements objective or not objective.
So in this particular study, if you looked at your paper, it says that blood was taken from all participants before travel.
Do they talk about the timer? Can anyone explain that to me?
No. It's just one of the focus that comes out of what you just talked about, the process.
Okay. And that that should be a fairly objective measure of the brain in the patients.
Okay. And the other type of of outcome or measurement of the outcome that they use was ultrasonography.
Okay. And they mentioned that all patients had ultrasonography once before travel, and then afterwards 30 of them had to.
When you consider ultrasonography to be objective as well. Yeah.
So whoever is doing the assessment, that's the comparable thing.
Temptation. Okay. The skill of whoever is doing the assessment.
Okay, so this so final, you see, we say we find that the outcome of all was measured using ultrasonography 24,
48 hours after the return of the flight. Now relating ultrasonography to to the assessment of the outcome,
here we have this one thing that we want to ask is whether blinding was done and when when for example,
we read this was a randomised double blind study.
Was that mean? So here we have an example where the we have the author being double blind versus single blinded.
Okay. So double blinded might mean nothing at all.
Okay. We need to identify who was actually blind.
Were the participants blinded with the investigators planet with the outcomes of blind or the analyst blinded?
That makes sense. Okay. Importantly, particularly for this type of study, we might need to identify if the person that was blind,
what the outcome assesses, if the outcome was not objective.
Why? Because in the worst case scenario,
if the if the whoever is doing the assessment for that particular outcome has a belief that stockings work or they don't work,
yeah, they might, for whatever reason, find more deep vein thrombosis in one group than another group.
Okay. And that's why it's crucial to identify when reading an article who was blind and who wasn't lining.
Do they mention who was blinded? I'll just want to get this from ultrasonography as we're blinded.
They give this, most passengers remove their stockings on completion of the journey,
and they don't say the nurse removes the stockings of those passengers who have continued to wear them.
A further duplex combination was undertaken with a technician unaware of the group to
which the volunteer has been randomised to generally want to take your stockings off.
I'm sure. How did you find them?
On pins and needles. Okay. So if someone were to analyse, to look at your legs right now, would they be able to tell if.
If you were wearing stockings or not. How you reckon?
You could describe. Yeah.
That sounds, you know, similar there.
Oh, yes. Okay. Thank you very much.
Thank God. Thank you. Okay.
So assessment was done in some cases when people were remove their stockings.
Not that long ago, not many of them.
Hopefully they don't specify how many of them. Okay. They don't mention any of this apart from that.
Just a remark in passing. What do you think? It is important to continue awareness.
Is there a benefit? Potentially maybe an issue.
So it's not only the flight, but wearing them after the flight. We don't know the exact examination subject to the question.
Which one do they. The ultrasound is subjective.
The best is subjective. Okay.
So if it's if it's objective, then we're not worried about the blinding.
So the more objective an outcome is, the less we worry about the blinding, the less objective.
So sample if it was a questionnaire that they had to fill in and it was a completely subjective assessment,
in a way, we might be more worried if there was no blinding done. Okay.
I'm going to go to talk finally about the results and when we have the results.
We've done our analysis, we've done around, but we've decided that this is worthwhile looking into.
Then there's the three basic when we look at the results of three basic explanations.
One is that the findings that we have could be explained through placebo, through chance, or through a real effect placebo.
So it's a very important explanation.
Here we have a trial in patients with chronic severe itching of two types of medications we have.
So perhaps adding and trimethoprim a pressing and we have a no treatment group and we have a score of an eating score.
Okay. So looking at this,
would you try one of those of these medications if you had a chronic severe itching or you would recommend to someone some say yes.
I'm saying now, if I were to look a bit further and I like to present this, that was actually the same.
If I am placebo group, then what do you would you do? Okay, so placebo is actually a very important explanatory intervention.
Okay. It shouldn't be taken lightly. Chance.
What about chance? And it's related to what John was mentioning in terms of sample sites, etc., etc.
What about if it's just chance that we find whatever we found with the effect that we found?
Well, there's two ways of trying to address that. That particular difference was done by chance, was happened to be there by chance.
And the two ways that we deal with it is to use hypothesis testing basically p values,
an estimation which is confidence intervals and many times they're given in conjunction.
So you give both a p value and a confidence interval. A pivot is just a way of estimating how likely it is that the null hypothesis is true.
So if the p value is very small and we would say the null hypothesis unlikely to be true generally a null hypothesis being there is no effect.
So you have a p value that's very, very small. We say, okay, there's some evidence that there is an effect.
The intervention in this case, stockings appear to work. On the second hand, we can have confidence intervals.
When confidence intervals, well, they do very, very, very simple terms in estimates of range of values where the true effect probably will lie.
Okay. So it gives you a number of a range of possible numbers where the effect is likely to be.
And in the case of that, what we're looking at is a reduction in deep vein thrombosis.
So how how are these two things integrated and presented, sorry, in this trial?
So in the case of looking at hypothesis testing, they looked at incidence of big vein thrombosis.
In the two groups, we have the stalking groups.
So they have none of them have deep vein thrombosis, non symptomatic bleeding from doses in the no stocking group we have 12%.
So that tells us there's a risk difference difference between the known stockings and the stockings.
So the stock. Yeah. They're not talking about stockings of 12% that has associated a p value.
That P value basically means if there was no effect, the chances of us seeing this difference is one in a thousand.
Okay. If there's no effect, the chance of seeing this is one in a thousand.
Therefore, it's very likely there is an effect. That's what that means.
What about confidence intervals? Again, we have the values in the stock in group zero, non-smoking group 12%.
The risk difference is again, 12%.
We can calculate a confidence interval, interval or a range of values where the true difference should lie, and it goes from about 6% to 20%.
That should be the difference that compression stockings probably would give you would give you an.
What does that mean? Well, because we're actually interested in comparing this to what would happen if there was no effect.
If there was no effect, then we would expect that the range of values that are possible to contain zero.
Okay, zero would mean that it's equally likely that this could be explained by chance.
And therefore, if series containing the confidence interval will say this is probably not statistically significant,
just that it's just as explainable to say that whatever you found in this particular test,
in this particular study could be due to just random variation.
So we decided there is potentially a real effect.
Okay. Given the two things that I just show you, the p value of one in a thousand and a confidence interval that does not obtain,
does not contain C or there appears to be a real effect.
So the next question I want to ask is now, would you consider wearing stockings on a long haul flight?
How many of you would? No.
No, many. You're still not completely convinced. If I were to tell you that there was a systematic review.
Okay, now there's a systematic review. I looked at a lot of it from the ten randomised controlled trials.
They did a critical appraisal of them. Nine of them looked at stockings in both legs versus not wearing then.
And they actually there's a lot of extra information of these trials.
Some of them were in low and medium risk, all of the others were in high risk participants getting away with this particular problem
of these are for this particular other particular trial they were all over 50 with no,
they were not at high risk for they were medium or low risk.
So some information on high risk participants and the results were that 50 out of 2637 participants had symptomless DVT,
three of them with stockings, and 47 with not with stockings.
There were no that no pulmonary emboli or symptomatic DVT reported.
Wearing stockings did have a significant impact on reducing oedema and no significant adverse events were reported.
I mean, if you were to look at the forest plot, this is the type of information that you would get.
We'll talk more about for plus later on. This is not really part of the main focus.
They ask again, how many of you will consider wearing stockings?
Half human. Okay. Thank you very much.
Now go back to the small.