Computer generated transcript

Warning!
The following transcript was generated automatically from the content and has not been checked or corrected manually.

So today what we're going to do, we're going to talk about PICO, a classic kind of acronym for talking about research studies and then building confidence in critically praising primary research. And we'll do so with an example paper, which, hopefully should have been sent up to you. But if not, I've got a little QR code. When we get to it, you can access it from the video. Okay, so let's crack on. So before we start talking about, um, critically appraising paper, we've got to think about why we're doing this. What is the point? Is it just for an interview? Just kind of show that you know what research is or you know why? Why are we doing it? Um, so some key ways that I like to think about why we bother doing it is peer review. So we look at papers, look how research has done, and we poke holes in it. We say that could have been done better. That's not great. Um, does this really answer the question? I want to ask, or does it answer Question? You're trying to ask yourselves, um, so it's a way that we can make sure that the evidence that the research that we put out is good evidence if we're testing each other on it, and we're sort of saying that's not so good, that could be done better. So that's a really important part of evidence based medicine is peer review, and that comes from critically raising the research that is put out, but also to inform clinical practice. So say, I have some patient with some really strange, weird condition. I want to know if this new treatment is going to make a difference to that patient, um, and say they are specific age range and you know there's no there's no clinical guidance so far on this is something rare. You can look to research papers and you can look okay. This patient kind of fits the demographic of the patients that were studied in this research article, saying that this was affected. Therefore, if I you know, if I agree with the findings of the paper and I think it's a good research paper, I think it's you know it's valid, then I can say right okay, I think I'm going to apply this knowledge from this paper into what I'm doing with this patient with the patient that I've got in situ. So it's very important to be able to read the paper and decide if it makes sense to what you're trying to solve with your own questions in clinical practice. And lastly, to inform further research So you can read an article. You can decide that. Okay, this makes sense, but it also raises more questions for me. Um, and that can then be a sort of jumping point to say that. Okay, I'm going to conduct research based on this one. Um, and I'm going to hopefully kind of follow it up and continue the line to find out more again, further improved clinical practice. So that's the main kind of reasons why we bother critically praising papers. Um, and it's a necessary evil, I guess, of anyone from academics to just normal clinicians that aren't even involved in research. Okay, so first of all, we've got to think What is the question? So, um, this is my first step of kind of critical appraisal things. And to do this there is a lovely acronym called PICO. Now, does anyone know what PICO stands for? That's it Don't be afraid. I won't bite. Is that p for population? Yeah, I got one down. Three to go. Any guesses? His, uh, intervention? Yes. I intervention when somebody is put in the chat. Uh, let's see if I can see that. Yeah, there we go. Thank Said population intervention, control and outcome or comparison and outcome is a bit more because you're always going to get a control in some studies. Um, yes. So basically, we're using this kind of summary. Tha does this paper ask the question. I'm asking. Sorry. So if I've got a 30 year old woman who has got rheumatoid arthritis and I want to know this new medicine is better than the best standard treatment, um, to treat her rheumatoid arthritis, I will look, and I'll look for a paper that's got the population that fits my 30 year old female in it. That's testing this new drug with intervention that's got a comparison. So, ideally, I want it to be the best standard treatment that is available at the moment. And see for instance, does it improve? Um, patient reported outcomes or improve stiffness, for instance, so that's kind of summarizing the question that I'm answer are asking and then apply this to the research paper and say, Does this paper answer the question that I want to know? Um And so when we put it into perspective, So we have that rheumatoid arthritis kind of example. But for instance, this is an example of something so the population could be British males ages 18 to 50. It could be the intervention is a vasectomy comparison. None. And you want to see if they get testicular cancer in 10 years. That's your outcome. Measure. So ignoring the title of research paper because that can sometimes be misleading. This is the real question of the paper, and this is the first thing you want to kind of do. When you look at the paper, what exactly is it trying to find out and what kind of people and what's it comparing it to? Okay, so that's our first step. And after we've gone through all the steps, we're going to apply it to the research paper that I sent so moving on next, we want to think of the types of study design. Um, so we want to look where in the hierarchy of evidence. Does this paper sit? You know what What's? What's the design of it? So does anyone know? Saying in terms of primary research was the gold standard of study design that we could be using any guesses? No. Okay, that's fine. So the top of our kind of pyramid of primary research is randomized controlled trials, which I'm sure everyone's heard about. And then we have on top of that kind of systematic reviews meta analysis, etcetera. But we have this hierarchy of evidence, as you guys can see here. So our our CTS are kind of the best, but you don't always have them. And you have cohort studies, case control, etcetera. Case reports, um, and each of them as you go down our low quality, and you're more likely to have bias in them. So that's why we look for that kind of peek to inform our proper clinical practice. And so the best you can find really is a meta analysis. But you kind of go down the list if you can't find your answer with each of the kind of higher up ones. Um, so we're mainly going to talk today about randomized control trials because that is kind of the as I say, the gold standard, most commonly the ones that you will be appraising as well. Um, but there are tons of others, and there are slightly different ways that you praise the different kinds of other ones. But a lot of things that will go through today are applicable to your other kind of types of research. Okay, so then once we've decided what the study design is, um, to say, for instance, it's a randomized controlled trial. Then we can then look at the methodology. So your first thing that you want to look for is you want to make sure that consent has been gained by the participants, and they're ethical approval. Um, it might seem unlikely that, uh can be published without ethical approval. But, you know, maybe some way back in the day from early 19 hundreds, when it wasn't so mandatory. Um, these things might not have happened, And so it's important to always look at that before you start analyzing the data. Next, we move on to our participants. So we want to know things about their demographics because, you know, say you're looking into, uh, I don't know mortality after a certain treatment or something like that. Obviously, someone who is very, very old is going to have less of life expectancy generally than someone who's younger. And so, if you're looking at who survives after 20 years after treatment, then obviously there's gonna be bias for the older people, um, or the more frail people. So you want to look at demographics, you want to look how they were recruited? So if, um, I don't know if you're looking for people who, uh, you're looking, how many people, for instance, are physically active in the UK or something like that? And you only place the advertisements for the study on like a fitness magazine, for instance, and obviously you're gonna get biased from there. You're going to get all of the people that actually sign onto the study. You want to look how many people are recruited because that's obviously very important. If you have two people in a study, um, then the findings are going to be very significant and very applicable for other people. But if you have thousands of people in the study, then you know that at least there's you know some. If you see a change in a significant improvement in whatever, then you know that that might be quite significant and applicable. And the inclusion and exclusion criteria. This kind of fits into the demographics. But you want to look that enough of the variables on these people have been controlled for, um And that's, you know, you're looking about a chance of getting a heart attack in 10 years and around in a group of people. Well, if they've had multiple heart attacks before, then they're more likely to have another one. So you want to make sure they've got strict inclusion exclusion criteria that helps them answer the question. Next we move onto Randomization. So this is obviously only really applicable and randomized controlled trials. You want them to have reported how they randomized people because they said they randomized. People have not really said how I used to know that they haven't been just someone saying I think this one should be in the arm one. I think this person should be armed, too, because I don't know, they have short hair, they have long hair. Let's just throw them into something. So we want very clear protocols of how things have been randomized. And we want to make sure that they've kind of check this and they they've seen that the demographics of Group A and Group B, um aren't, like, wildly dissimilar because obviously, then it's improper randomization, but they're kind of fairly equal. Okay? And next we move onto blinding. So obviously not every experiment is blinded. Um, you can have single blind, blinded studies. You can have double blinded studies. Um, so you want to make sure that you know what it is because it could just say blind is controlled randomized controlled trial and you would know single double and obviously double is the best when the sort of people conducting the study and the participants don't know what intervention is being received and who's getting it, Um, so that's a very important one. And then we look at the protocol, so this is very specific to the different kind of studies that you're looking at. Um, it requires a bit, sometimes a bit of inside knowledge, into the condition or into the treatment. But it's basically making sure that everything is very clearly laid out, and they try to control for as many variables as possible. And this brings us on to the issue potentially of internal versus external validity. Does anyone know what the difference between the two is? Any guesses, Guys? Go on, Jerry. You want me to go for it? Yeah. I was thinking external validity is to see how well we can use the result of this finding and apply it to all the settings I'm correct and internal validity. How the how it means just how we can apply to similar setting again. Kind of. So So you extend ability, bang on. But internal ability might might be what you're getting at. But the way I use to describe it is, um how well are we answering? The questions were answered. It essentially so if we're saying, does a lead to be, um regardless of everything else, just if you strip it down to a the equals be a plus B or C, for instance. Um then it's basically how well we answering that question. And if we stripped everything else away from it. So it's for instance, saying if you have a person taking statins every single day and exactly the way they should be the perfect physiology. It's like how well does that prevent? Um, an MRI, for instance, versus in real life people who don't always take their medicines, Uh, with people that you know don't always take them the right time with people that have different physiology is how well do they actually work in the grand scheme of things? So that's our external validity of the first ones are internal validity. So often we like to start with the studies that have very high internal validity because we want to make sure that a plus B equals C. But then we want to see, actually, in the real world, does it really work? So So it's always that kind of balance that you want between because the more real life you make it, the less internal validity it has because obviously real life isn't perfect. So that's our protocol. And in our protocol, we also want to think about, um, any control that there is there any placebo and how well that is that kind of works into are blinding as well if it's a placebo and it's supposed to look exactly the same as our intervention, So we want to clear description of that as well. And lastly, we want to look at the measures. So we want to see, you know, the outcomes. So say they are looking about, you know, am I like a chance of getting an MRI in the future? How are they looking into that? How they're measuring that. Are they looking through? Hospital records are using self reported outcomes. Um, you know, objective, subjective measures. Um, if they're using a survey, has that survey been validated to look what they're to see, what they're looking for? Or is the research has just come up with a bunch of questions. Um, and they think it's going to work and they haven't really said why they think it's going to work. So those are all kind of things you need to be thinking about. Really? Um, so an example of the subjective and objective method is, for instance, with a skin lesion. Skin lesions often very difficult to really get objectively like saying, Has this improved the skin lesion? This this cream on this treatment? Um, and so saying that, you know, one researcher evaluated, If it's got better or not, that's not very objective. Really, There's one person if they say they got five researchers who all went under underwent a training course in rating skin lesions, and they used a verified scale. Um, and they took photos and encountered the number of lesions or whatever. That's all more of a clear description of something that's a bit more objective, a bit more standardized. So that's always something you need to be looking for with that, so it's really kind of digging down. And as I said in some, um, some fields, it might take a bit of expert opinion to say that's the best way to measure that, or that's a bad way to measure that. So that's always sometimes a bit of a tricky area, so moving on, then have critiquing the statistics. So does anyone know what intention to treat means or per protocol means any ideas? It's often somewhere in an area where people can get a bit tripped up and often a nice way for interviews. Tha kind of test you so her protocol would be only including the they're kind of patients who go through the treatment kind of completely as as prescribed and get to the end, whereas intention to treat his, um, kind of but realistic and kind of takes into account everyone, no matter kind of whether they've been kind of taking the drug or whatever. Kind of absolutely rigidly or kind of other things have gotten away. Yeah, yeah, Hang on. So per protocol is, as you said, if everyone does it as exactly they're supposed to, um and therefore has a higher internal validity, but lower external validity. And then our intention to treat is once your randomized or assigned a group, then you're included in the analysis. So this is all about how we come up with a final result in our final findings of the study. Um, so intention to treat has much more external validity in that aspect. So very good. Um, and then we move onto power calculations. So this is, um, complex kind of statistical analysis that they used to find out what is the ideal sample size, Um, to basically say that their findings are significant and this will be different based on the different questions that they have. Um, and there should be evidence that they've done this power calculation, so usually they want to say that at least, um, if they have x number of participants, then that's 80% likely that the finding is significant. Um, and they'll take the number of participants that's needed. And then you can say, if they managed to recruit, maybe 1000 participants in their power calculations said that they need 1000 participants, then they're you know, they've reached that kind of target. Then if you look later and you see the actual they were going by the protocol analysis and only the data of maybe 800 were analyzed and they haven't really met their power calculation and therefore if they make any conclusions that this is a significant, it's probably not, and they've kind of shot themselves in the foot there. Um, so that's another thing you need to look at. And then the P value Does anyone know A P value means what what P value is used for? No. Okay, so the P value is, um, it's actually on my next slide. I can get up here, So this is basically explaining how likely is this result that's come up likely due to chance. So basically almost like out of thin air really, this mystical value of 0.5 has been used as the standard across a lot of research, and it's basically saying that it's 95% likely that this result is not due to chance. So anything under that is obviously high, as you can see. So, um, say it's 0.1 and it's 99.9% likely. So the lower the P value is, the more likely the result is significant and not due to chance. Okay, so you're going to really get the values between zero and one. And this isn't going to be a big talk about statistics, because I'm no expert on statistics, but it's good to have a little bit of knowledge and a bit of awareness of these things because, um, you need them to really interpret how confident you are about the results of the paper. So again, I said, not in no statistician, but you've also got a whole load of different statistics terms. So these are experimental event rate control, event rates, risk reduction, absolute risk reduction, relative risk reduction, um, odds of outcome, odds ratio, confidential intervals number needed to treat these are all things that are commonly used across research papers and kind of the big ones to really look at for Here are your confidence, intervals and the number needed to treat and maybe your odds ratio as well, because they will be able to help you. If you understand how these calculated they'll be able to help you with understanding, then once they present you with the result whether this is actually going to make a difference to your patients. Um, so an example is the number needed to treat. Does anyone know what that is? Any guesses? You get extra points if you know how it worked out as well. So basically, the number needed to treat is the amount of people that we need to give this treatment two for it to actually give the benefit that we want. So, um, say that were saying, um, if I don't know if, if dexamethasone um leads to survival after 10 years with coated or something like that, something strange like that. It's basically saying how many people do need to give dexamethasone, too, that they will survive more than 10 years, and obviously we want the number needed street being, we want to ideally be one, um, so everyone who gets it will survive. But it's not always that case, and it's that's very useful in kind of cost benefit analysis and, you know, strength and benefit analysis, because if it's a very high number need to treat. We have to treat like a million people for one person to get a benefit, and it's a very expensive treatment, or it comes with a lot of side effects. When you think about whether it's worthwhile, that's quite an interest in important one and also confidence intervals. So we'll go on to them a bit in a bit. But it is important in your own time to look into some of these terms. They frequently come up on papers. Um, and I think in in some interviews, Sometimes they like to ask you to work out some of them as well. Um, so it's important to know, um, but I said, this is not going to be a statistics lecture, so we'll talk about confidence intervals after this one. But then we look about critiquing the results so often, this is the part where people glaze over. They skip to the discussion where everything is summarized. But you want to actually look at this. You want to see all the outcomes that they've reported as the, you know, the primary outcomes and the secondary outcomes. Have they all been reported, or are they missing some things they choosing not to talk about? Certain things are the subject. Demographics reported as well. Um, so you want to see it clearly so you can say, Ah, these groups are well balanced. And what are the confidence? Intervals? What are the P value of the things that they're reporting? Um, you know, are the P value is less than 0.5. They're saying that this is significant. Are the confidence interval humongous? Um, and not really relevant and then kind of negate your significance of your value. Um, and did any participants drop out where they lost to follow up? Um, you know where they I don't know. Do they not get included in the analysis and why? And therefore does this? As I said before, have an impact on your power calculation. So these are all important things you need to ask yourselves when you're looking through the results and then we move forward and we're asking them about our confidence intervals. So brief summary of what this is. It's basically saying you're 95% likely to have the really true value of this. The answer to this question between these two values, Um, so they will give an upper limit and and lower limits. Um, this is most important, especially when you're thinking about things like odd ratios. Um, or if you're thinking about risk reduction, for instance, um so the thing with odds ratio if the odds ratio is one then giving them this medicine, for instance, is not going to give them an improvement. Okay, so that's like it's not going to give them an improvement or benefit. But for instance, if your confidence interval, if you'll say your mean is just above one, then you might think it does provide a benefit. But if your confidence intervals are below and also above one, then you have to question, actually, is this really significant there? So it's very important to also look at these things. But I said, I'm not going to go into much detail because we're not going to talk about stats specifically So the last thing we want to do is we want to critique our discussion. Are they saying that, for instance, the primary outcome measure it wasn't significant, But maybe some of the secondary outcome measures they had a P value that's close to 0.5. What exactly or lower are they basically then over playing this and kind of saying that this is This is great. This is amazing. Even though it wasn't significant, You want to really question that? I have. They reported the limitations have they have you come up with about 100 more limitations and they've actually said and acknowledged in the paper that are significant, um, and are therefore, through all of this, the conclusions valid. Um, you know, they have also insignificant results that they have lots of limitations. Anyway, they're still saying that, you know, this is the cure to the cancer. For instance, you want to really ask that because if they are over playing things that you want to find out why and therefore that will implement, it affects how you're going to take this further into your own research or into your own practice. And lastly we want to think about kind of in this area of discussion and conclusions, any conflict of interest. So these things like being sponsored by the pharmaceutical company or they just so happened to be the boss of this, um, company that's made this device that will I don't know how patients mobilized disabled or, you know, once upon a time they received a grant from, um someone to conduct this research. Or, you know, it's quite common with things like, for instance, smoking the tobacco industry and, um, sugar and, you know, soft drinks, industries and things like that. You want to see that first? Because obviously that will sway, um, the researchers bias towards the results. Okay, so lastly, then we want to summarize. We want to think about what the strength and limitations of this paper Think about it ahead, and therefore, take that forward. Say, Well, it actually change my practice. This is, I think, the most common question people get asked when they're critical appraisal of paper. At the end of the day, you come up with all of this. You know, you've look through it all. Is it actually going to change anything you do and why? Um, so that's the main question we want to know from taking this paper. And lastly, is there anything else you want to know? So has this research paper raises questions. If it's not going to change your practice, why is it what? What would you want to know That will make you change your practice. Um, so that's kind of the end of the steps, but lastly, we're going to do a quick bonus round before we take it practically. We do kind of a whistle stop tour through a a real paper. So we're going to look at some grass. So in the paper I'm giving you there aren't really that many charts or anything, But you often get different charts and papers, and you are supposed to know what they are and what they mean. So I'm going to give you a chart, and I would like someone to tell me what it is and what it means. So this one, we've got two of them side by side. Anyone know what this one is called? Any guesses? A forest plot, Correct? Yes. This is the forest plot. Do you know what kind of studies these mainly useful papers. They're used for type of paper. So if you've done a meta analysis correct. Very good. So these are basically to tell us taking into account all of these studies. How well are they going to answer the question of this meta analysis? Um, and taking into all of the results so we can see on this one. We've got one on the left. I think you can probably in my mouth. All the different studies we have in the center, that's probably around our mean of the results. And these are confident intervals. Either side, um, kind of sometimes called the dot and whisker plot as well. Um, have one scene called a blob program as well. Um, so this one is a bit different from the one on the right and one of the right. We've got different sized, um, little boxes, and they are sometimes used to describe You know how how important those results are. So maybe these ones will have will have more participants and some of the others. So therefore, the effect of it is kind of bigger. Um, and then at the end, you usually get this kind of summary line, Um, where it's basically sitting. But the important thing about this is what I said about continence intervals before our odds ratio. So if, for instance, if we have this one, which is the mean is over the odds ratio of one, it shows it's benefit. Then we've got this this confidence interval that's way below one. Then we can't really be sure that this one is showing showing an overall kind of benefit because it's 95% sure that's between these lines, the real results. So that's why we like to add them all up. This one's not got a summary one, but this one does, which shows that are also ratio is just above two, which is pretty good. So yes, this forest lot, and it's a nice way to summarize data from different studies. Next type. Anyone know what this one is called? Okay, this one's a bit more of a tough one. This one's a caplin Maya curve or Captain my A plot or chart, and it's basically saying, Um, it's, um it's also called like a survival chart survival plot survival curve, and it's you can test of different interventions that we can be just one intervention and you're looking along the line after maybe 15 days. What's your percentage of survival with this intervention So we can see often Maybe if you're looking for after maybe just 11 days, you have 50% survival with the control. For instance, in this one, Um, and the interesting thing about these is they again useful, predictable survival. But they're less useful the further away you get for predicting it, because your sample size decrease is if people are dying or participants or cells are dying. Um, depending on what you're doing, Um, so they're quite useful for this kind of end of things. They're a bit less reliable for this end of things, and they sometimes also can be used to reflect participant drop out or different things as well. But generally they used for survival. Um, yeah. So, for instance, here you if you have 100 participants by this point you have maybe that's around 40 participants. So how much this informs you further is much less reliable than when you have 100 participants and and the last one does anyone know what this diagram is called? Prisma flow chart correct. Very good. So this is used in systematic reviews or literature reviews, analyses and there to show that you've systematically, um, included directive papers and how much started off and how much ended up and how much were excluded and why they're excluded. Um, but you can also get these kind of similar flow charts talking about participants election as well. Um, and you see them often in papers where they you said at this stage this many were excluded because of this exclusion criteria and how much were followed up. And these many drops out all these many died. And this is the final ones that included for analysis. So it's called a prisoner flow chart. Specifically what when is talking about Met analysis? Systematic abuse? But you can get very similar ones with primary research as well. Okay, so now we're going to go on to our worked example. So if anyone who didn't get email with the paper here's a little QR code where you can get it. I thought I'd go for a bit more of an interesting one rather than your boring kind of certain medicine X that you've never heard of for treating condition. Why that you've never heard of, uh, it's a bit of a more of a fun one. We've got the sports medicine, So let's go from our first step one. Um, if anyone had a look at it already and is able to give me a picture of it, I'll be very happy, very impressed. Otherwise we'll work through it ourselves. Anyone had a look at this already, and I want to give it a go. That's okay, We'll we'll go through it. We'll go through it. So this was a study looking at experienced male and female gamers 18, 13, 30 years old. So we look at that. We can see that on the X inclusion criteria. I think from memory. They have to have a specific number of hours played, for instance. So that's how we can get their experience. We can see that they included male and female gamers, and they have to be of a certain age. So we get that from their inclusion and exclusion criteria, and it's good that they very clearly stated that intervention I want I want to have to go to the intervention. That's okay, so we have two interventions in this one. So three different arms of our trial. So we have six minutes of walking break after an hour, gaming or six minutes of a rest breaks and not an active break after one hour gaming and a further our gaming after. And this is compared against know rest so continuous scaling instead and outcomes. So, uh, here's a very clearly stated their primary and secondary outcomes. So our primary outcome is the executive function. Changes in the players and secondary is the performance and the players perceptions of that break or intervention. Okay. And so obviously I stated before here, and as we'll do later, we want to make sure we know our measures of those outcomes as well to make sure that we're happy with them and then, you know, quite valid in terms of assessing them so we can move forward to our type of study design. Can anyone tell me what study design was for this study? Any gas is. I flipped back a few slides and we look at the title. Gives us a little clue. Any guesses? It's a randomized controlled trial. Amazing. Amazing. How do you get it? Yeah, very good. So this is a randomized controlled trial. But quite specifically, um, in it is it's a repeated measures. Randomized controlled trial. Um, so does anyone know what a repeated measures part that part means of it? Any guesses? I want to know their thoughts on what repeated measures means as well for the study in terms of it's validity. So repeated measures is when you have a group and they do intervention a, then they do intervention be, and then they do intervention. See, for instance. And this one we have three arms. They do each of the different interventions, Um, and randomized means they can be randomized, which one they have first or in which order they have them. But all of all of the participants do all the interventions. Can anyone think of any issues potentially with that? So I think one issue is, uh, if you do one intervention after the other, um, 50 of the time in between different interventions. Not long enough. The for example, if they have intervention a an intervention, be the outcome measure from the intervention be might actually be a result of intervention A rather than intervention be or another case uh, it could be the result of intervention. Be, uh, not the intervention. A. But you have no way of telling, uh, which which one is, if the sort of the interval between the two interventions long enough to so clearly the clear the effect of the of the intervention. Yeah, totally. So that is one downside of it as well. Um, for this study, specifically, they're looking into questionnaires and things like that as well and and different tests of executive function. And if you're having to do those tests multiple times the first time you do it, you're probably gonna be a bit rubbish. You don't really get it the other times you can be a bit more familiar with it. And like that by the last time, you're gonna be a pro at this test. So regardless of the intervention, because you're getting better and more familiar with it. So, um, it's called, uh, order effects. That's the kind of bias that you get with things like this. Um, so it's It's like saying, for instance, um, the infamous situational judgment test. While it's something that they say you can't really revise for, you can definitely practice at it. And, um, I at least think that the more you practice, the more familiar with it. That may be the better you'll do, maybe marginally. But it will still improve your performance and so that this is the same thing. Regardless of whether you I know, take a caffeine pill before your sed 80 or if you, um, somehow slipped some amphetamines or something before us. JT, the fact that you have repeatedly done S j t will definitely impact your performance on it. So that's always a worry with repeated measures. Um, but the fact of this one is, although there are three different interventions and they could be, um, subject to this kind of order effect, there is a way that this study has counterbalanced that. So does anyone know maybe away in theory that you could kind of counter the effects of something like repeated order effects. With something like this, you have three different interventions. You're probably going to get better with each one. Does anyone know a way that you can maybe assign participants? That means that maybe the overall results might be a bit more reflective. Know So with this one, um, they have done. Three. They've assigned the participants to three different groups, and each group has a different order of doing each intervention. So the same amount of participants will have done a first. The same participants all done intervention. Be first, the same participants. Participants will have done intervention. See 1st and 2nd and 3rd and all of that. So hopefully, by the end, if you got better by the last study, then there will be someone who has done intervention a the last one. Someone who's done be someone has done. See, So your overall picture should be a bit more reflective. So they have made efforts to counterbalance this, although still repeated measures isn't the best. Then we look into the methodology. So this study has had consent and ethical approval, and they stated that that's very good. We're happy with that. They stated very clearly the patient of the participants demographics. They stated how they recruited them, and it's in my opinion, appropriate in the situation. They recruited them from places where game has kind of hang out. So discord twitch. Um, I think there's somewhere else where they said they use them for, um, but basically what places where you find gamers. They were looking for experience gamers. That's the right place to perfectly appropriate. Um, they stated the number of participants that they recruited and they've got a nice little flow Diagram of one's included, excluded and involved in the final analysis. And they said before they're very clearly stated that inclusion and exclusion criteria And I think that the ones that are used are very valid. Um, and they are good for assessing what they want to assess now. Randomization They've also stated how they randomized all of the participants. Um, it doesn't matter too much really with this one because, um, as long as they're just checked in different groups because they're all doing each intervention anyway. But if you're only doing one or the other, it's more important there. And they also have very clearly, um, stated the participants demographics. But again, it doesn't really matter for the randomization in this particular study, this study is not blinded. Uh, and so that definitely is a limitation of the study. But how you blind someone that they're taking an active versus resting break and not taking a break? I have no idea so I can't really get to an order The authors of the study for that Because I wouldn't know how to do that myself. Um, but it is something that might limit them. But however, another thing to add to this is one of the outcome measures. Was the participants, um, thoughts on how this affected them overall and just their their their views on things. And if you have, if they don't know what they're doing, then they might not be. They might not have a very clear view on how, uh, how they've enjoyed this thing. So the protocol, um, I think they've got a very clear protocol in this research. They're very specifically stated the words that were being said to the participants and how things were done and the times and it was very, very clear, well structured. They've tried to exclude lot of extraneous variables such as the participants diets during it. Um, so it's very, very clear with that, However, um, the participants, their in their home setting, they're not in, for instance, the lab. So they could have I don't know, throughout the game that they're playing, they could have a random cats walking across the screen, for instance, Um, and that can distract them. Or they could have, you know, the doorbell go off, and that can distract them. And so it's not very well controlled for internal validity. But the fact that they're doing it in their own home, where they normally be playing their games shows they have a good external validity and things like the computer set up in the game that was all standardized, however, noted later, is that the actual game play was real life gameplay. And so, you know, the other people that could be playing against might be better in certain players and things like that. So, um, there is a lot of external validity, but they've tried to be as good with controlling a strain of extraneous variables as they could do with that and lastly, um, measures that they've used. They've used a load of standardized and validated surveys and tests to measure what they were looking for. Um, so I don't really have any problems with that, Um and yeah, I'm very happy with that. But they haven't got any biometric data which might have been useful, um, to look for things like, um, kind of heart rate variability might be good. And to look for Maybe if they had eye tracking, for instance, they can look in concentrations. They could have a bit more metrics and bit more invasive metrics. Maybe if they wanted to, to properly answer that question. But this is much more from a coated kind of era when they can have patients come into the lab and do things in real life like that. So they kind of making do with what they could at the time. So that's our methodology. Critique. Um, next week, move on to the statistics, statistics. Um, so this one is a bit more of a difficult one in terms of intention to treat or per protocol, because they had people play the games and then they uploaded the data, and then that was that was basically it. It wasn't really following a treatment for a certain period of time. However, in our little flow diagram, it does state that certain participants were not included in the analysis. Um, that's because they didn't complete all of the different arms of the trial, or they can upload their data so you could say that if you're picking between to this is more per protocol. The ones that were analyzed because those ones weren't included in the analysis want to complete all of them. We can see very clearly they've got their power calculation that 20 participants would be enough to, uh, for their outcome measures. And that's again they stated in 80% likelihood. With that, um, and they had 21 participant. So I'm happy with that. And they're very clearly stated that significance of the study will set to the P value of less than 0.5, which I'm also very happy with. So they've been all very good with their statistics there, and they've also explained how they're going to be, including the statistical analysis of the program they're using. And so it's all very kind of replicable. So if I was to use the same system and analyze all the data given the raw data, then I could come up with the same kind of values the P values and that means and everything like that. So very nice. Very clear, very transparent. So next we move on to the results, um, so have they reported all the outcomes. I think they have reported all the outcomes. You can correct me if I'm wrong. Um and they also haven't lied about some of them being significant when the P value is, If it's over 0.5, quite clearly stated this wasn't significant or there was a significant difference is found in these things. So I was happy with that. The demographics were reported. We've got a nice long list, and something that you should do is obviously look through all of this as well. Um, I think they were looking for maybe was gender differences. And if you look quite clearly at their kind of reportable than most of them, males and females were fairly well balanced. However, there's a quite significant difference in the hours played weekly and the men versus the women. So therefore, the men probably more likely to be more experienced. And maybe that might have might have an impact in your analysis later on. So little things like that you can pick up on and you can see okay, and then lastly, they don't really report confidence intervals on here. They have. They have the range of values, so that kind of changes things slightly. I would have maybe quite liked some confidence intervals, But we'll stick with the Rangers for here. And lastly, we spoke about dropouts. Um, you know, we've got nine that didn't complete it, I think, Or eight. And they stated why and why they haven't been involved in the analysis. So we have a nice mixture of things that promote external validity and, uh, some things that promote internal validity here. Um, I think this probably was aimed to be something that would be done in the lab with high internal validity. Then Cove, it happens. They have to adapt things. Really? So lastly were critiquing our discussion conclusion. I don't think they ever played, um, insignificant results. However they did in the discussions start talking about physical health and how exercise, you know, break might be good physical health. Um, and kind of almost kind of sneak sneak that into their conclusion. But they didn't examine any of those parameters. So be wary of that. This this study isn't coming to any conclusion about this break being good for your physical health. Although a break after an hour is very good. We know from other studies. Um, they stated all the limitations very well. They've been very frank about them, which is very good, very honest. I couldn't come up with too many more limitations that they had, Uh, except maybe the sample size, even though the power calculation was 80%. With that sample size, I think anything that's reporting sample size of 20 isn't could do better. Really. Um, and I think their conclusions are quite valid. Really? Um, and lastly, none of the authors, none of them declared any conflicts of interest. If I was going to go really deep dive into really critiquing the paper, I could Google the authors and I could look for myself if if there's anything on there, but you would hope that all honest authors of would report they're conflicts of interest, which most do, and lastly, we got to summarize. So, um, as I said before, if I was saying That's my point of view with the strength of limitations, I could say that this one does have quite high elements of external validity in it. Um, I think they've been very clear with their methodology. I think they've got they've definitely done their best of what you could do in a coated. So no area to try and standardize things. Um, and that was good limitations. I thought, um, all of the ones they basically said in the paper, um, they were talking about, um, again not having that kind of internal validity side of things as much. Um, and I don't basically agree with all of those and all the things we've said so far, you know, the repeated measures, all of that kind of thing. Would it change my practice? So would I be advising every sports game and take a six minute break? I would, from a health point of view. But in terms of improving their performance, this study has shown that it won't necessarily improve their gaining performance, but it will improve their executive function. So what, I would want to know going forward in a lab setting, would this improve their performance? And also, um, is there a very clear link studied link between, um improved executive function and improve gaming performance? Because this is not something that is specifically stated. So if you can prove that gay the rest improves your executive function, then you can assume that it would improve performance and further studies. So I want this repeated in bigger samples. I want some more internal validity, and then you can then say, Oh, maybe there's a difference. If I just the brakes, will it work? So these are kind of the steps that you take to kind of taking and critically appraising a paper is kind of a whistle stop tool, um, and kind of rushed through kind of a specific example of a paper. But I do recommend you going and looking through similar papers or even this paper and seeing what you make yourself if you haven't already, um, and seeing if you come to the same conclusions as me. And if you don't, I'd love to know. I'd love to know if there's anything else that you think they could have done better as well or anything. They could have done different any strength, Um, and what you would take away from this. But these kind of steps are really the cornerstone of critical appraisal and and nothing I like to do also when I'm critically appraising an article is not only have the steps in my mind, but also There are lots of check lists online that you can use to follow for different types of studies. So what I would recommend is cast. It's C A S P. They have lots of different checklists on critical appraisal of different articles, and it's always helpful. Just look through it, um, to remind yourself of the things you should be looking for. Um, prisoners also got a very good one when you're looking for things for systematic reviews and then you get, like, a nice little score at the end as well. So thank you very much for listening.