Link to HSC faculty home page faces

Data Analysis
  Example Frameworks for Critically Appraising Research Articles

The Introduction
Is there a clear statement about the topic being investigated?
Is there a clear rationale for the research?
Is there a clear statement about the limitations of the research?

The Literature Review
Do the researchers use contemporary material about the topic being investigated?
Do the researchers link their work to a wider body of knowledge through the references cited?
Do the researchers link the topic to the questions about theory?
Is there a clear link between the literature and the formulation of the research
question(s)?
Is the research question clearly stated?

The Methods Section
Is the research design clearly described?
Are the research methods appropriate for the topic being investigated?
Are any advantages or disadvantages of the design acknowledged by the researchers?

Is there a clear statement about who participated in the research?
Is there a clear statement about how the participants were selected?
Is the selection of participants appropriate to the design?
Is there a clear statement about the number of people taking part in the research?

Data Collection and Analysis
Is there a clear description about how the data was collected?
Was the data collected by appropriate people?
Is the approach to data analysis appropriate to the type of data collected?

Quantitative
Is there any explanation of sample size used?
Is the level of significance of the tests (alpha) used indicated, or implied to be the customary 5%?
If Pearson correlation coefficients are being calculated, is there any evidence of a check for a linear relationship?
If t-tests or analysis of variance (ANOVA) are to be preformed, is there any evidence of check(s) to demonstrate that the data follows a normal distribution, or of assumptions made?
Are reasons/assumptions re the level of measurement of the data given? (This affects the appropriateness of the descriptive statistics given and the tests used.)
Is there a clear statement describing how valid and reliable the measures are?
Are the type of statistical tests used appropriate for the sorts of data collected?
Is the use of any statistical analysis package, such as SPSS discussed?
Is there evidence of a statistician's input to the analysis?

Qualitative
Is there a clear reflexive statement about the researcher's role in the analysis?
Is the approach taken to data analysis clear?
Is the use of any electronic analysis package discussed?
Is there a clear statement about how the researcher validated interpretations?

Ethics
Is there a clear statement about ethical committee approval?
Is there a clear description about gaining consent, maintaining anonymity and or confidentiality?

The Results/ Findings
Are the results related back to the literature review?
Are the weaknesses in research design acknowledged?

Quantitative
Is the presentation of results clear and unambiguous?
Are all the results presented?
Do the tables and charts used give a clear picture of the sample data and results?
Are the charts used appropriate?
Are the tables easy to use?
If percentages are recorded, are actual numbers also clearly shown?
Are results of tests interpreted rightly?

Qualitative
Does the research present evidence of the data collected?
Does the data presented as part of a theme support the analysis suggested?
Is there a clear audit trail?

The Conclusions
Are the implications for further research acknowledged?
Are areas for further research identified?
Are further recommendations made for practice that come from the results/discussion?
© Moule, P. Pontin, D. Gilchrist. M. Ingram, R. (2003) Critical appraisal framework.

 

Further Examples

See Critical Appraisal Skills Programme (CASP) appraisal tool at:
http://www.phru.nhs.uk/casp/learning_resources.htm

And Polit, Beck and Hungler (2001)
http://www.sonoma.edu/users/n/nolan/n400/critique.htm