Student Resources
Frequently Asked Questions

Q: I do not have any experience with research and have not taken any statistical courses yet. How will I know if the research design or the theory and concepts selected are appropriate?

A: As healthcare consumers, we are able to evaluate the relevance of a study from our own experience. Look over the design section in the book; then ask yourself: Does the design used in the article make sense? Is the concept important or has the researcher missed the boat? Is the rationale logical? As you gain experience with research methods, your appraisal skills will become more sophisticated. Nevertheless, your views about the appropriateness and relevance of the methods are important. Just as your clinical skills improve with practice, reading research will get easier and your competence will improve.

Q: If the report does not discuss any biases or limitations, how will I know what these are?

A: After you have read the report the first time, take a few moments to think globally about the study. Are there any questions in your mind? Does anything irritate you about the approach? Do the author’s opinions seem to influence the study unduly? Talk with others to see what they think. It is not uncommon for people to have different opinions about biases and merit of a study. This is especially true if the results are unconventional, new, or different.

Q: I find it hard to differentiate the problem and purpose of the study. How can I better figure these out?

A: It is true that many research reports use different terms for the research problem or the purpose of a study. It is common for the problem to be described in a background or introduction section. This section deals with the practice problem and is often about the health of the patient/client or the practice environment. The purpose of the study describes the researcher’s goals for examining the problem. Sometimes you will see research questions, objectives, or hypotheses instead of a study purpose. Sometimes the purpose is explained from an immediate as well as from a long-term perspective. A good place for you to start is to ask yourself these two questions: What was done in this research study? Why was it done? If you can answer them, then you are well on the way to sorting this out.

Q: How do I interpret the results? Maybe I should not even try to interpret the meaning of any research results without a statistics course under my belt?

A: A statistics course will certainly help you better understand the research results. However, if you have not taken a statistics course, do not give up. Many nurses and other healthcare researchers can use the results of research without formal statistical training. Look for the words statistically significant and clinically meaningful. For statistical significance, look for the symbol p < .05. The p stands for the probability. P < .05 means that the results are not likely due to chance in 95 out of 100 cases. You can have confidence in a statistically significant result. However, if the real difference in absolute numbers seems trivial from your viewpoint, then it is also important to think of the meaning of the results. Ask yourself: What do the differences found really mean for clinical practice?

Q: What is rigour and how is it determined?

A: The notion of rigour captures the different steps that have been taken in the study to ensure that the findings are sound and the interpretations will not be biased. The steps associated with rigour are related to the research approach and the purpose of the study. Also, there are ways that a researcher can collect and manage data that help convince us that the study was carried out according to accepted research standards. For example, in quantitative research, if the researcher was trying to compare two treatments, an experimental design would strengthen the study rigour. The experimental design helps make sure that any other factors that might influence the findings were fairly equal or were minimized before the data was collected. In qualitative research, some techniques the researcher may use are tape-recorded interviews, or provision of feedback to the study participants for them to validate the researcher’s interpretation. An important technique that can be used in a quantitative or a qualitative study is “triangulation”. With this technique, researchers use multiple data sources, methods, and/or analytic techniques to consider and validate the interpretation from different perspectives, in order to gain a better understanding of the topic under study.

Q: How would I know if the data collection tool is reliable and valid?

A: In some research reports, the authors will tell you exactly what they did to check out the reliability and validity of their tools, or they will reference other publications with the details about the measures. Be cautious of the results when there is no discussion of reliability and validity. With new questionnaires, frequently there are problems with at least a few items. If a pre-test of the tool or a pilot study was done, then it is likely that you can have more confidence in the results.

Q: What is a good sample size?

A: There is no one good sample size to fit all studies. What might be very appropriate for a qualitative study would be very inappropriate for a quantitative study. The answer about the size of a sample or the number of participants needed depends on the type of study and the research question. In most study reports, the author will justify the number of participants. Look for the terms sample size justification or the power of a study to determine statistically significant results for a specific outcome. In some cases, sufficient sample size is not possible, and a pilot or feasibility study is done to determine the sample size required. Justification of sample size is a key factor in determining the generalizability of results or, in plain language, in determining whether these results can be applied to other settings.

In qualitative research, you will see the terms informational redundancy or data saturation. These terms mean that there are no new ideas or concepts in data most recently collected and analyzed. This occurrence provides the researcher with the cue that it is time to stop increasing the sample size. As a rule, the size of a qualitative sample is smaller than a quantitative sample because no statistical analysis will be performed.

Q: How do you know that the study was ethical if the authors do not mention ethics?

A: There are a few studies published in reputable journals that have this information missing. Rather than assume it is all right, look under the “Acknowledgments” section for any indication that the study was funded by a government agency or other well-established health-care funding agency. These types of funding agencies require ethical approval before supporting the study. Then look at the research article paying particular attention to what the researchers did (the study purpose and the data collection), and also look at who was in the sample and how they were recruited. Be extra concerned if the sample consisted of vulnerable individuals, for example children, the homeless, or those with any mental incapacity. It is wise to consult with a researcher or a person with clinical ethics expertise in these situations. Also, it is often possible to contact the journal or author to ask questions about the ethics of a study.

Q: What things do you need to think about when considering the study findings for your own situation?

A: After you have decided that it is a sound, ethical study, consider how closely the study sample resembles the patients or clients under your care. If there are some reasonable similarities, find out if the change in practice is possible to do in your setting. Ask yourself and other involved professionals: Are there any other studies or clinical practice guidelines available on this subject? Do we have the necessary resources? How can we get them? Who can assist with putting this change into practice? What type of changes would need to be made to existing procedures or policies? Consider also who will support or oppose this change.

Q: Is it necessary to fill in every blank on the worksheet?

A: No, it is not necessary to fill in every blank. When you are learning a new skill, such as critical appraisal of research reports, you may find that you are able to fill in more items on the worksheet after you have read the research report a few times. Try to complete as much of the worksheet as you can. Sometimes it is possible to work in pairs or groups to facilitate worksheet completion.

Q: Is the ability to fill in the worksheet a reflection of the strength of a study?

A: No, the number of blanks filled in does not reflect the strength of a study. Please keep in mind that the details may not be in the report because the journal has restricted the length of the paper, or perhaps that item was not applicable to your selected study.