A couple of people have recently asked about a tool I developed to teach critical reading, so I thought I’d blog about it to add a bit of context to what is basically a workshop handout.
Working in a one to one context as a Learning Developer with students on assignments like literature reviews has allowed me to see behind the scenes of how students approach this task. What I’ve noticed is a mismatch between some of the feedback on the written product “Unfocussed! Doesn’t flow! Needs to be structured better! Too descriptive!” can actually be traced back to issues around reading and note-taking, not writing.
Students understandably find critiquing the work of far more experienced and authoritative scholars very daunting, and that’s the first thing I address. I fear sometimes that the message that students should only use ‘quality, peer-reviewed sources” is over-egged, and can disempower students – if it’s been peer-reviewed by more experienced scholars, and passed for publication, what more can be said? This can lead to descriptive writing, as quality, peer-reviewed literature is assumed to be unassailable and takes on the authority of established fact. My first aim is to establish with the students is that there is still potential for meaningful critique, by them.
Actually performing a critical reading is also hard. Academic writing is by nature highly persuasive, and the illusion created by our typically objective, neutral, reasonable ‘academic writing style’ can mask this. Students often find themselves quite naturally sucked into the agenda of the writer – that this article or book is the most important, innovative and correct contribution to the debate that you could possibly be reading (after all, in the context of the REF, it’s in the interests of academic authors to persuade their readers of this!). A straightforward reading can often and quite naturally result in the response “well, I suppose so… ” Even I respond this way, if I read purposelessly or inattentively.
The best approach I’ve come across to help students really get their teeth into critical reading are the CASP (Critical Appraisal Skills Programme) tools, created to help students and professionals in the Health Sciences interrogate different kinds of study. The use of questions is a great way to help students find a bit of distance from the text, and a bit of purpose when reading- the CASP tools offer a script of such questions. However, the CASP tools focus on one element only – whether the text, on its own, in its own right, is sound. When critical reading is taught explicitly, this is often the dimension which is addressed. Students need confidence in their own understanding of research methodology in order to answer these questions, but it’s a good place to start.
However, following only this approach may still result in a literature review that is unfocussed, unstructured and ‘bitty’ – even if it’s a highly critical account of individual papers! To develop my own tool, then, I’ve expanded on the approach used by CASP, to add two more domains in which students can interrogate a text.
Domain 1: The first domain of critical reading, then, remains The Text On Its Own, In Its Own Right. This domain asks “on its own terms, is this text valid?” Engaging with the text in this domain allows for detailed critique on a level which allows the student to make decisions about how much confidence to put in the text’s conclusions or findings.
Domain 2: The second domain, however, looks at The Text In the Context of Other Texts, in other words, synthesis. It asks “how does it relate to other contributions to the debate?” For example, are the findings similar (in which case we’re seeing a weight of evidence building up), different (in which case we may have a quirky outlier or an innovative new take)? Is it a new answer to an established research question, or a new direction entirely? This helps students to start to see the overall shape of a debate, weigh up the contribution of each text, contextualise what they’re reading and ultimately synthesise it into their own account of the field.
Domain 3: The third domain, The Text As It Relates To Me, considers the text from the point of view of the student’s own agenda. It basically asks “how does this text relate to what I’m doing? Can I make use of it?” Academic writing being as persuasive as it is, it’s really easy for the student to lose sight of this and end up with an exhaustive set of notes, an inflated wordcount and a descriptive writing style. This dimension allows the student to make critical decisions about the text on their own terms. “Yes, this article is probably very important and interesting, but does it help me make my argument?” If not, the student can make that most invisible of critical decisions, to leave it out.
The starting point need not be the first dimension either. I very often start with the third! If a text doesn’t look like it’s going to help me, I’m not going to bother with an in-depth, laborious critique of its methods! Other readings might want to start with the second dimension, to spy out the lay of the land in a field, and spot main debates and trends in research. The domains aren’t a process or a hierarchy, just different dimensions.
To help students apply The Three Domains, I turned it into a framework to help students think about the kinds of specific questions they could ask in each area. Generalising the kinds of questions that CASP tools pose for medical sciences, I proposed five areas that students could ask questions about (these five aren’t necessarily exhaustive, but are one way to approach it!) They are:
- Context: discipline/profession, authors, currency, bias
- What are they doing? Research Question/Aims/Hypothesis
- How did they do it? Methods, Models and Materials
- How do they know? Argument, evidence, logic and reasoning
- What do they say? Findings and conclusions
Here’s the full thing: three domains of critical reading
This version of the the handout has indicative questions for each domain (again, not exhaustive, but hopefully therefore not overwhelming!). So for the question How did They Do It? You might ask:
- Domain 1: was the method appropriately chosen and used?
- Domain 2: is this a standard method for this field of research, or a novel one?
- Domain 3: can I borrow or adapt this method to help me answer my own research question? Or would a departure from it be my innovation?
The framework can help students focus their reading, give them a ‘way in’ with practical set of questions to interrogate a text, but also guide their note-taking, as they are encouraged not (just) to take notes of what the text says, but of their responses to the text – their answers to the questions. One of the nicest things that this framework has achieved for students is to validate the difficulty they are experiencing when reading critically. No, it’s not you, it is hard – you’re reading in three dimensions at once! This realisation can also help them ‘layer’ their reading, by reading for one dimension at a time, not all at once, and making it more manageable.