Welcome to this webinar on how to critically appraise a paper. One of the key parts of veterinary practise, clinical practise or research, is our ability to be able to look at a published paper and more than just look at it, actually, appraise it quite critically and, determine, you know, what you think that that paper does or doesn't add to either your clinical practise or research. And talking to lots of veterinary practitioners, and other people involved within, the profession, it's quite common that people either look at just the abstract or only read the conclusions.
But that would be to miss, a, a potential of error or defect in a study, and, to miss out on that ability to properly, look at the published papers and, and, and gain from it. To start with, I just wanted to make a few acknowledgements. I wanted to acknowledge in particular, Doctor Gina Pinchbeck, who is an epidemiologist at the University of Liverpool, who I've worked with, for many years.
She's also a veterinary surgeon. And, this is a paper that we've recently published and quite a lot of this talk will be based on that paper and the checklist, that was devised. I also want to acknowledge some colleagues at the University of Liverpool, and particularly those who I worked with during my PhD, which was an epidemiology-based PhD, and also those who I've learned from over the years.
So I started in exactly the same way, looking at a paper in horror and thinking, well, where do I start with this, to now leading journal clubs and acting as a regular reviewer for various different journals. I also wanted to acknowledge Professor Celia Ma, who is the editor of the Equi Veterinary Journal, who's, given me really helpful, insights into the process of critical appraisal and particularly, from an editor, viewpoint, at that process of, editorial, review as well. In terms of declarations of interests, I am an unpaid director at Equine Veterinary Journal, Limited, and I am also a member of the Royal College of Veterinary Surgeons, Quality Improvements, committee as well.
So the aims of this talk are to try to assist you if you're new to the whole process of having to critically appraise a paper, or if it's something that you do but you don't feel particularly confident about doing. So, hopefully, I can help provide a logical approach to how you might be able to critically oppose a paper. This is just my approach.
There are also other approaches, and I think it's always quite helpful to be aware of how other people do sort of go through a paper. It's always a way of learning different tips and techniques. I think it's really important that you have an idea of what the key elements are of a good paper, and it's really helpful when not only when you're reviewing a paper, but also when you're, when you're going through those, sort of checklists of what should be there.
And it should hopefully help you gain some confidence in appraising general design and conduct of a study. And just being aware of some of the factors that commonly crop up, they shouldn't do, but can commonly crop up in some papers that can compromise. It may be only a small degree of compromise or it may be quite major, and it could be enough to completely invalidate a study's findings.
And then you then hopefully will have the skills to decide if you think whether the discussion and conclusions are justified based on what you've read about the study design and results. And to then consider if and how this paper alters your approach to looking after your clinical patients in veterinary practise, and that includes veterinary nursing practise as well, or to research in general. And critical appraisal is just one part of this process of evidence-based veterinary medicine.
And this is something that, in the medical field, nursing and, medicine has been going on, for many years and is defined as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. And that, of course, has been extended into the veterinary field. So we've got evidence-based veterinary medicine, and critical appraisal is just one part of that, because without the ability to look and appraise a paper, you can't really practise that evidence-based approach.
And within our veterinary profession, certainly in the UK and USA it's considered one of the key core professional competencies that is required of veterinary surgeons. And thinking about, well, why would you go to a paper? Why not just go to a textbook?
Well, textbooks become out of date relatively quickly in some fields, as new, research is published. So the ability to, appraise a paper and read that latest paper that's just been published last week. Can help you, update your, in your knowledge and approach to cases much more quickly than a textbook chapter that may not be updated for many years.
It may be that you don't have the time or the ability to access papers and one sort of helpful start, particularly in an area that you're not familiar with, is to look for reviews that, specifically look at published papers. And within the veterinary fields, we're increasingly seeing. Critically appraised topics that have a very methodical, very logical, appraisal of the literature to try and answer that question based on evidence and knowledge summaries, and some key resources, are in some of the following slides.
So this helps you take this new information to incorporate into your own clinical practise, or if you're a researcher, how you might take this into your research field. And ultimately, what we're trying to do is improve how we look after our patients and, getting the best outcome for them and their owners, or if you're based within research in attempting to improve the quality of that research that you're undertaking. So, when are we going to have a look at a paper?
Well, increasingly, individual practises, whether in academic or private practise, would be, hosting, formal journal clubs, either as part of formal training or an informal basis. And there's some very good resources. RCBS Knowledge is just one location of how you can set up a journal club and some sort of practical tips and, and ways of, undertaking these.
Appraising papers might be something that you're required to do as part of postgraduate study, for example, if you're undertaking a veterinary certificate, or it might be that you're, undertaking a research project, and this extends into, potentially those who are not yet graduated, so veterinary students, veterinary nurses. And for those graduates, or those with their professional qualifications already, continuing that process of, continued professional development. Or it may just be pure interest in keeping, up to date, with all the, the latest, and it might not be something that you do just, formally within a workplace, but it may be something that you do, out of the workplace and maybe even lying on the beach.
In terms of what type of papers and sort of scientific evidence you should appraise, I'm sure you'll be familiar with this pyramid of evidence, which is comes in sort of, various different sort of minor, differences in this, but this is the key, sort of core elements of the pyramids in that. Expert opinion and un sort of scrutinised or not based on any actual research per se, this sort of background information would be considered the lowest level of evidence. And then as you go up towards the top of the pyramids, this is where the evidence becomes increasingly robust, and, considered a better quality.
In, within the veterinary field, we would have quite a lot of expert opinion, and particularly, within some textbooks or articles, again, that are just based on one person's, experience. That's not to say, it's poor quality in any way, but when thinking about, evidence-based, veterinary medicine, personal opinion, without sort of the backing of scientific studies is, considered to be low quality. Within veterinary medicine, we have very few systematic reviews, and we're starting to see more critically appraised and topics and individual articles.
But for the most part, we would be looking at randomised controlled trials, cohort studies, . And case control studies. And within those, randomised controlled trials are relatively less frequent, a lot of, the information, that's available would be, cohort or case controlled.
But by far the most common would be case series and individual case reports. Again, they're all valuable, but they just have a slightly lower quality compared to some of those other, types in that pyramid. Where do I start?
And it can be quite easy where you've got a structured process where, for example, somebody selects a paper for discussion in a journal club, or you may just be starting out on your own, self-study or trying to work out for a research project or for a veterinary certificate, you know, that you've got to look up a certain topic. So. The world of scientific journals is something that can be increasingly sort of difficult to to navigate just because of the sheer multitude of journals and papers, and particularly with the advent of online only journals, rather than the more traditional paper-based ones.
And as I'm sure you'll have found by reading through various papers, some papers are obviously of higher quality than others, and there are various measures for this. One type of measure is what we call an impact factor, the higher the impact factor, the so-called, better standing, or better quality, potentially. Journal, but there are many other measures.
And it's important within veterinary medicine, I think, not to consider just impact factors, because there are many very good veterinary journals that for very, for various reasons, wouldn't necessarily have a particularly high impact factor. One thing that you're considering when thinking about the journal to choose from is whether papers have actually undergone a process of peer review. And this is really important because those journals that publish papers that have been unscrutined could potentially have some fairly major errors.
And the peer review process should be performed by colleagues who have particular skills or, and sort of knowledge and expertise in that area. Increasingly, we're seeing open access journals appear, which is great because it means that these journals can be downloaded directly off the internet rather than having to pay for access to these. And, we're increasingly seeing sort of transparency, increased transparency in the whole process of peer review and the stages that a journals been through this so-called open peer review.
And, this is an area that is continually sort of increasing and we'll see much more of in, in the following years, together with, I'm sure, a multitude of other journals appearing and maybe some journals disappearing. I think one thing that's really important when approaching critical appraisal of a journal, whether doing it during a group journal club or during individual study. Is to be realistic about, what is achievable.
And, and also within the, discussion of the sort of the reporting results within a study, because doing a perfect study is very difficult often to achieve, and it may simply be, impossible, or it may actually be something unethical. We'll touch on that a little bit later on. And, the plan, when conducting a study, can look very smooth and very neat.
But the reality of what performing a, a research study involves is often very different, with, bits that go to plan and other bits, that don't go to plan and can sometimes cause major issues. The bottom line is that the quality of the science must be good, for it to, add to sort of evidence, about how we, look after our patients. And as part of, science, you know, the reports, the results of those study, and conclusions, should be honest, .
And that can sometimes be very difficult to accurately, judge, and you rely on that honesty from the authors who've written that paper. It's important also to be, critical and questioning, but also to be constructive and realistic. So, when looking at a study, sometimes it might be, that the study design is quite poor, but when you actually think, well, what would be the ideal way to do this?
It's just not going to happen either because it wouldn't be fair, for example, for a horse, to receive a medication that is known to work versus having no medication at all, a, a sort of placebo that definitely doesn't work. And it may also be that, financially, the type of study would just not be doable, or it just wouldn't be acceptable for other ethical or logistical reasons. And there are lots of resources and checklists out there that can help you to further understand the process of critical appraisal, and these will have a really nice references that you can look up, articles on how to go through that process of appraisal and checklists as well and lots of tips and tricks.
So we've presented one type of checklist, and I'll run through it as a sort of a basis to the critical appraisal of a paper for the purposes of this talk. So find a checklist that you like, that you find very usable, and use that when you go through each of your, papers, to try and make sure that you have a sort of a logical and consistent approach each time. Some suggested readings, to help you, get started, as well as the resources that I mentioned and, I should have mentioned in the, the paper that I, that we've written on how to critically appraise the paper, within the reference list, there's some really, really useful, websites and other papers.
And some books that I think are really helpful, these, the two shown medical statistics at a glance and making sense of critical appraisal. They're one of a multitude of books. There are lots of other similar ones.
So, you know, these are not the the essential ones by any means. But I think, these two books provide a very easy way to sort of understand some of the basic, statistical tests, in a very easy to read format and the making sense of the critical appraisal, book. This is a really nice short, really easy to read book that goes through some of the key study designs.
As you get more in depth into study design and maybe undertaking research where you want to make sure your statistical methods are sound or your epidemiological methods are sound, there are a number of Really good, veterinary, epidemiology or statistics books, numerous, too numerous to mention in here, but there, there are lots of resources that you can go to, to, find a further read into that subject area. All right, so we're going to go through a paper that I've chosen. So I've just run through how the process of how I might go about finding or or picking a paper.
Now it might be that you're given a paper to appraise or you're given a topic, or it might just be that you've got a particular patient that's come in. That you want to, have a look in the literature, just to check whether there's anything new in that area, either in diagnosis, management, predicting outcomes, etc. And, you can use, various terms and various online resources.
Google Scholar is one that I'm sure most people are really familiar with, and is accessible to, most people. And the critical thing is just making sure that you put in, the appropriate, search terms and just being aware of any differences in, language that you might want to select. So for example, there are differences in the English or US spelling, for example, of oesophagus.
The one issue with Google Scholar is that it brings up loads of things. So it's then being able to philtre, through those to find, you know, a paper that you think is, of interest or use. You can then go to slightly more focused studies that will more reliably come up with published papers rather than just general articles written in the subject area.
And again, a lot of people will be very familiar with PubMed. The only thing just to be aware with PubMed, is that not all veterinary journals are recognised through PubMed, so you might miss some of these, for example, equine veterinary education. So again, Google Scholar can help you pull up some of those that you might otherwise miss.
And depending on where you're based, you may, if you're in an academic institution, you may have access to an academic library, or if you're doing some form of postgraduate study, you might have access, to this resource as well. And this allows you to access other, ways of finding papers and doing searches. So, for example, Web of Science might be one, but there are, many others that you can use.
So having had a little look, you might click on a paper and have a quick read of the title, and you should be able to read access the abstract as well to see whether it covers the area that you're actually interested in, or whether you, it's the area that you're needing to, to research. And then if you think it is suitable, then determining whether you can access the full paper or not. And it might be that you can't gain access easily, or you might need to find various ways, depending on where you're located, of ways in which you can find a published paper.
So, I'm going to run through a paper that I have randomly selected. So taking the scenario where your little kitten pictured here is going to undergo a spay procedure, you might be looking at papers, you might have a think about analgesia, in this cat, following. It being spades.
So, you might, then go and have a look for the papers and see what's being published. So I have randomly chosen, a paper from an open access journal. I did this, so that, those watching the webinar, would all be able to access and read, this paper.
I've got no conflicts of interest report. I don't know the, any of the, article authors, and, I, I do have, an interest in that I have previously had papers published within this journal. So this is one that I found as I was looking through analgesia of cats, following ovario hysterectomy.
And after finding this, you can then have a little look at the abstracts and having a read through it. It looks like an article that might be quite useful, for me, and it's an open access paper, so I know that I can read all of it. So this is just gonna be a run through of this paper.
It's one that I hadn't looked at until a few days ago. So, for me, this is the first time that I'll have seen and, and read through it. So this just runs through, as I would present it, in a journal club or as I would, sort of approach it as a journal reviewer.
So I have a quick look at as a start, looking at the journal that it's published in and when it was published. So you can see this isn't a really recent, article that was published, several years ago, but doesn't seem to be so long ago. So it's a slightly difficult area because some very good papers were may have been published a number of years ago, but are still very valid.
So, again, you have to have a look through the paper and just see whether you think it is slightly outdated or whether you feel it is still very relevant. And then you might have a look at the, the authors and see where this is being conducted. And in this case this was performed within a university veterinary school in Brazil.
If you want to have a little bit more of a look through this, you know, what this journal's all about, then, journals will have, their, their own web pages where they can give you, a bit of, backgrounds on what they sort of publishing, and you can find out what the process is for peer review, etc. Things like impact factors as well. So then I, the next thing I do when I've sort of got the paper and downloaded it is have a look at any conflicts of interest and funding.
So potentially, if you have got a study, for example, that looks at a drug, if it was funded by a drug company, and if those members of that company were included on the author list, a good study, that's on a well written paper, that shouldn't have any impact. But, it is always possible that, you know, that may influence, the reporting of, results. Something just to be aware of and decide yourself whether you think, you know, there are any conflicts of interest and whether the funding is an issue.
So this study was supported by a grant, and again, you might not be very familiar with the sources of funding, so you can just do a quick search on the internet, and you can see that this is a research foundation and it outlines what type of research it funds. So the next thing that I'll do is have a look at the ethical approval. And this is something that is really important and is becoming ever more sort of rigorously, sort of, .
Obtained within the good quality journals. And it's absolutely critical for any form of intervention study that animals that have been enrolled into a study, the owners of those animals have provided informed consent. It can be difficult in some retrospective case studies, particularly the data was that, you know, that was obtained, say, 1020 years ago.
But there should, in, in many journals be some sort of statement regarding ethics and whether it's been reviewed by a veterinary research ethics committee. So if we go to this paper, you can have a a look within the text and it outlines that, yes, this study has been approved by the university's ethical committee and it gives a protocol number as well. So then the next thing to have a look at, you might already have had a quick look at the abstract, but then it's time to take a little bit more of a detailed look at it.
And most journals will have a structure that you need to stick to with sets, headings, and will often and will usually have a word limit for this as well. And one of the important things I look for is that it is consistent with what's written in the main body of the paper. And occasionally when a paper has undergone sort of changes suggested by reviewers, sometimes a little bit of a mismatch can develop, sometimes that, sort of gets a little bit forgotten.
But it's an important thing, to at least have a read off and see, what you think of it. So I thought that this abstract was nicely written. Interestingly, the abstract actually provided a sort of more concise description about the study than, in the actual materials and and methods itself.
So they actually described here that it was a, a blinded, a double blind, perspective, study. And, just a very minor thing, not uncommon, where authors, primary language wouldn't be in English, some of the, editing. So if there's only very minor edits that, that, would be, required, that's usually taken care of, through the editing, process.
And sometimes, also submitting papers may actually, need to have a professional, . translators to check the quality of the English just to, help that process. OK, then it's time to go to the introduction or sort of background, depending on the journal, which title it would give.
And this should be a concise introduction to the subject area. It shouldn't be a book chapter. It should be ideally around about, probably 4 to 6 paragraphs long.
And it should reference relevant previous papers or review articles, especially if it's an area in which a lot has been published, and should provide you with an idea of what they're looking at and why and this sort of justification as to why they've performed the study. And this should be written in an unbiased way. So, looking at the backgrounds, here, it provides a nice sort of, overview.
And if you go to the reference list, this is, not a, an area that I am, particularly, experienced in. I'm an equine vet. I chose this study.
As, an example of how you can pick any type of paper, and even if you don't know much about the subject area, that background should still give you enough of an idea to be able to appraise that paper. And if it's an area that you're not very familiar with, and, you need to do research into it, this sort of background area, you know, will often give you some very useful, or it should give you some use useful key references or key summaries. And again, my comment on this paper would be that it, very nicely, introduces, the reader to the concept of the importance of, providing, analgesia, to, patients, undergoing a feline ovarian hysterectomy.
Going through the importance of pain and measurements and the key physiological, subjective, objective measurements and and justifying why they undertook this study. And then there's a final part of the introduction or background, the authors should then state the aims or the hypothesis or hypotheses of the study to make it clear what they're looking at. And this is detailed in this paper, and they clearly state that they're going to evaluate the effects of the degree of analgesia with pethedine compared to tramadol in cats undergoing ovarian hysterectomy.
And they also then outline some secondary sort of outcomes as well that they're looking at. So, then it's time to go into the materials and methods. And this is probably where I spend most time as a reviewer, and when I'm appraising a paper, because this is the crux of, the paper, because if you've got, a poor materials and methods, then potentially the whole study may be invalidated, and it then makes, study of the rest of the paper fairly pointless.
And, ideally, in a good peer review process, a paper shouldn't be published unless the materials and methods have been assessed by the peer reviewers and accepted as being suitable for publication. And said this is a really important part of the paper, and don't just bury your head in the sand and just go to the next section. It really is important for you to have a look through because the more you look through at the materials and methods, the more you'll get used to looking at these and recognising the different types of studies and tests that are performed.
And the key aim should be that somebody trying to do this study again should be able to be able to replicate the study and get the same results. And I put potentially obtain the same results because You could do a similar study, but maybe in a different population, and you might potentially get different results. But the, the, the key thing is that you should be able to read it, understand what's being done so that if you wanted to do the study, you could do it just from looking at that information.
So then we're gonna look at the type of study, and as I said, I would actually have asked the authors to actually state within the main body of the text that this was a, a double blind perspective study. And this was a, and the next thing we need to sort of think about is what sort of was the population they studied, when, how and where, and were there any inclusion or exclusion criteria. So this was a study that looked at 42 cats undergoing ovarian hysterectomy at one site.
It doesn't actually give the time scales, but one would assume that this was done fairly, fairly close to the time of ethical approval and sort of prior to publication. These were healthy, cats, and, they state that this was obviously based on physical examination and the medical history. And then some inclusion or exclusion criteria.
So these cats had to be well behaved and, biting cats were excluded, which I think is fair enough. So they then are going to look at you you're then going to look at, sorry, the what is it they're trying to look at? What was the exposure or the intervention.
So, and what were the outcomes that they were looking at and how was this measured? So we've already said that this was a blinded prospective study that was looking to see if particular analgesic groups would provide sufficient analgesia following ovarian hysterectomy. So the outcomes that they were looking at and how these were measured, consisted of a variety of things.
So, objective measurements, so, things where you get sort of a set figure, or those subjective, which, might be, slightly less easy to consistently, measure, and it runs through quite nicely how they measure each of the objective assessments. And then they run through the subjective pain scores and you can then go to the papers that they cite and know that these methods are ones that have been validated for use in cats. Now, one major area that crop can crop up in the materials and methods is bias.
And this is where there's a a systematic error, it's not, not completely random. There's something systematic in the design or how the study is conducted or analysed, which ultimately results in a mistaken result. And this could occur in many ways.
So for example, if the person doing the pain scoring knew which cat was in which group. They might have some form of bias if they think that one particular drug is better than another, that subconsciously, they may, pain score those cats in different ways. So that's just one example.
There are lots and lots of different types of bias, but that often comes up in the sort of, discussion and again, a good epidemiology, textbooks can provide you and, and other articles, can provide you with a really nice summary of the, the common ones that crop up. So, for example, recall bias would be if something happened 10 years ago, compared, you know, people being asked about their cats or dog or horse, you know, a year ago, compared to a week ago, it might be that the length of time for those in the year group, might be, they might be missing things because they've forgotten about them compared to the week before. So, we then want to have a look at these biases and one way in which you can avoid this systematic.
Sort of bias are two key things, which is, random assignments, which is what was performed in this case, although they don't say what system they use for randomly assigning them, and importantly, blinding. So those are really important things to try to avoid, this sort of systematic bias. Then we're going to look at the statistics.
And again, this is something that always results in cries of horror, but you don't need to be an expert in statistics to be able to evaluate a paper. One of those things that as you get used to having a look at them, you'll get an idea of what methods have been used again, this should have been through a period, a process of peer review where this has been analysed. So you want to look at what techniques they used, standards, and significance value.
And then you can go to some of the statistics textbooks that I've mentioned to get an idea of what a study is all about and what it's relevant for. For the more complex tests, there are lots of other sources, including Wikipedia, and if an author does use a more complex test within the body of the paper, they should justify the reasons for that with, with references. So the statistical methodology in this paper was fairly standard, and these are all tests that you can very easily look up in a basic statistics article, to get an idea whether they're suitable, depending on, the data that you're looking at, as well.
And as I said, don't feel that this is something that you've got to have particular expertise in. As I said, this is something that should have been looked at during the process of peer review in, in the, in the good journals. All right, so then coming on to the results, we then need to think, you know, are these logically presented?
What are the key results, what is actually statistically significant and which are not. At this point, I will mention that we're obviously talking about studies that are quantitative, but there are various qualitative, sort of social science-based, papers that are out there that are no less scientific. Just they looked at in a different way.
So most of the papers that we'll look at in veterinary medicine are what we call qualitative, where you have statistical tests on which usually have a a P value of usually less than 0.05 to be considered to be statistically significant. So we'll have a look at the tables and figures and then decide what the key take home message is.
So, looking at tables, some key things to look at is at the legends, but they should be fairly easy to follow. The legend should be a read alone thing so you can look at the legend and work out what it means. And any acronym.
Should be in the text. And as I said, they should be quite clear, and you should look at how they use so for example, a box and whisker plot, and how they mark whether something or denote whether something is statistically significant or not. So the key things that I will look for when I'm, I'm going through this is whether, what is stated in the aims and the materials and methods is actually what has been done, and whether I think that the results may have been affected by bias or what we call confounding, where the effects of actually a mixing of two or more factors.
And whether I think what they're sort of stating of the effect, whether I think that's clinically relevant or really strong, and whether they looked at all the sort of important factors that might be clinically relevant. So you can go through this paper yourself and decide for yourself. I thought it was quite logically presented.
Table one was quite big, but it was OK to read, and I thought that the figures were nicely presented, and I thought they were relevant and useful. There were some grammatical errors, and, some of the areas were not particularly, or could have been slightly reworded slightly to make it clearer that some of the results were different, but they weren't statistically significant. And part of this may have been due to lack of numbers.
They Didn't actually give any power calculations and having low numbers in, in groups, you may see some differences between groups, but those numbers may not be enough to actually detect a significant difference between them. And I thought they reported the results of it, quite honestly, for example, and the issues they had with interleukin 6 measurements. So the key take home messages from the paper, were that all the drugs and drug dosages used provided adequate analgesia.
There were some individual variations in the physiological parameters during the different time points, and particularly the pethedine group compared to the tramadol groups. And the main sort of finding was that the pethedine group was more likely to require additional analgesia, so called rescue analgesia than the tramadol before make the cake dosage. And then we come on to the discussion, and this should be a very honest and unbiased discussion about the study, which should be supported by the results.
So the study authors shouldn't go off on a tangent and talking about things that are really not directly relevant to the paper. So some key things that I always look for is discussion of non-significant findings as though they were significant. They're not significant, even if there was a trend, and they shouldn't be discussed as though they were significant.
And you shouldn't really have reporting of additional results. They should be in the main body of the text. But it may be that they did an additional statistical test to explain explain a finding.
The explanation of the results shouldn't be biassed. If, for example, the authors hold a particular, theory about things, and as I said, it should be supported by the results of the study and use, literature to sort of back that up or demonstrate where this study is different to previous studies. And just in terms of the ideal sort of makeup, it should have an introductory paragraph with the key findings and why this is novel or important.
The middle paragraphs, generally a paragraph per topic goes through, sort of key areas and how these agree or differ from other studies. Generally, the penultimate paragraph is the one where any limitations are discussed and anything that could have been differently or should have been avoided. And then the final paragraph is that sort of final key take home message, how this paper adds to the evidence and usually contains future areas for study.
So my appraisal of discussion of this paper was again, I thought it was logical, some errors in the grammar were fairly minor, and thought they could have stated the key findings a little bit more concisely. It is reasonably long, but I thought it was justified. They make suitable reference to existing literature where it agrees or differs from other studies.
They do briefly mention the limitations and the rationale, for example, for having a control group, which they, quite correctly, stated would not have been ethical. My only criticism was that there was no mention that some of the findings could have just been, due to a lack of numbers and no discussion about study power and power calculations. And I thought their conclusions were clear, it could be a little bit more concise, but there was a, a good sort of take home message.
So, finally, you need to decide if you think that the results of the paper that you've just appraised are reliable and how they relate to patients in your care. And as I've mentioned, use of a checklist doesn't have to be this one. There are other checklists.
They're good to sort of get you into a consistent pattern and make sure you haven't missed anything. So following reading this paper, I'm quite happy that, you know, I understand the main purposes of the study and how the study was designed and I thought it was appropriate. Then this sort of important, sort of internal, so-called internal validity, which is sort of a quality control, really, and, and talking about all those, thinking about all those things that I've gone through.
Because if a study is not ultimately well conducted and sort of robustly designed, then really, you know, you may not be taking anything from it at all. And then you might extrapolate that and think, well, how, you know, could this actually extend to my patients or other populations? But if a study isn't internally valid, then there's not much point in thinking about the external validity.
And then finally, the other factors, these have been forgotten in your sort of initial approach is thinking about those other important things, including ethical approval and potential, conflicts of interest. OK, so, to, summarise all of this, hopefully I've outlined why having critical appraisal skills is really important, and it's something that you have to be able to do to have an evidence-based veterinary medicine approach, to your cases or, to research in general, and, you know, but it is part of our professional responsibilities. And, it's time to not just, read the abstracts or just look at the conclusions, but actually have a logical and consistent approach to sort of seeing whether you actually think, this study has been properly conducted and what you think of, how it's been put together, conducted, and and how they've discussed those findings.
And the more you do it, the better you'll get at this, and you'll get other people's ideas, bounce, sort of ideas off each other, during journal clubs. And again, the same with personal study, the more you do, the better, you'll get it sort of recognising, particular, statistical tests, for examples, or particular approaches. All right, well, I hope you've enjoyed this sort of outline to critical appraisal of a paper and thank you very much for listening.
