Example Frameworks for Critically Appraising
Research Articles
Review of: Evaluation
of the basic life support CD-ROM: Its effectiveness as a learning tool
and user experiences.
The following web address
contains the paper 'Evaluation of the Basic Life Support CD-ROM: Its effectiveness
as a learning tool and user experiences' http://www.ifets.info/journals/5_3/moule.html,
as the last paper listed.
The following is a suggested critical appraisal of this paper.
Introduction
The development of the
CD-ROM is presented and its importance as a nursing tool within the nursing
curriculum is given. The project aim clearly states that the effectiveness
of the CD-ROM as a learning aid will be measured, along with the experiences
of students and lecturers using the CD-ROM. The rationale for the research
is set in the context of a tool to support learning of a compulsory skill,
ie BLS, and also to enable development of IT skills. However, the need
for this tool is not stated in the introductory paragraphs.
Literature
Review
A brief review is included
in the introduction. This concentrates on referencing previous pilot evaluation
work and relates to the importance of developing BLS skills. It does not
identify other literature relating to BLS skill development or e-learning.
Some quite dated material is referred to in terms of IT skills. The literature
leads to a specific research aim rather than a research question
Methods
There is a statement
of the stages of data collection, but no formal discussion of research
design or alternatives. Strengths and weaknesses are not presented.
The testing of BLS knowledge
and skill attainment in the way conducted would seem to be a suitable way
to gain information about the impact of the resource on BLS knowledge and
skill attainment. Interviewing is a data collection method that is often
used by researchers to ascertain individual experiences of events and would
therefore be appropriate.
The sampling approach
is clearly laid out, showing institutional and individual involvement.
There was random selection of the research sites, although there is no
explanation as to why only 2 of the potential 13 sites were chosen for
testing. Some limitations relating to students' prior experience of BLS
skills are noted. The researcher states that students self-selected themselves
into the focus groups and
the research groups, which may lead to potential bias,
as identified in the study. It is not clear how lecturers were involved
in the research. Given the focus of developing BLS skills in the nursing
curriculum it would seem appropriate to use this sample group.
Data collection and analysis
The data was collected
by a member of university staff, with listed related publications, showing
they have research experience in the field. It is not clear what the status
of the researcher is or whether they hold a researcher role.
Quantitative
Data analysis is clearly
described, including measures taken to validate data entry. Data was assumed
to be at ordinal level, but no justification was given for this. However,
assuming this, the appropriate paired non-parametric test (Wilcoxon) was
used. No mention is made of the level of significance used, but it is assumed
that the researcher was working at the 5% level.
It is not clear how the
2 variables expired air respiration (EAR) and external chest compressions
(ECC) were arrived at, but since they are given as percentages, it seems
appropriate to carry out Pearson's correlation test. The linear relationship
also required for this calculation is indicated in Figure 3 and 4. There
is evidence of a statistician's input to the research, although there is
no reference to any computer package used.
Qualitative
The researcher doesn't provide reflexive comment about her role in the research,
though the approach to data preparation and analysis is clear. The researcher
followed a recognised published approach to data reduction, display and
verification, which on a positive note included independent researcher
and respondent verification, thus improving trustworthiness of
the data.
Ethics
The researcher states ethical approval was gained from each institution and
a protocol to maintain informed consent, confidentiality and anonymity
was applied, though unfortunately no detail of this is given.
Results/findings
Quantitative
The tables and charts are clearly presented, although a table of results to
include maximum, minimum and median values
may have made comparisons between groups easier. However, this information
is presented implicitly in the box-plots.
There was no evidence
of significant correlation between the variables EAR and ECC, with both
probabilities being > 0.05. Indeed, one of the P-values is
quoted as 1.738, which is clearly incorrect.
Qualitative
The discussion of the four emergent themes is supported by verbatim
extracts from respondents across both sites, to include lecturers and students,
though it is not clear which individual's views are being presented. This could
mean that the views of only a limited number of the sample are being presented.
The extracts clearly support the themes presented. A range of published literature
supports the themes and builds on the initial brief literature review. The
researcher does not explicitly relate the work to a framework for ensuring trustworthiness.
Conclusions
The final paragraphs
identify the limitations of the research before stating the key findings
and relates well back to the aims of the project. Key points for higher
education providers to consider are identified. The research fails to identify
further research issues/opportunities arising from the study.
| 
