Imre Lakatos wrote somewhere, and I can’t remember where, that if we are to talk of positivism then we must define what we mean before we start. In thinking about the possibilities for experimentation and realism I start by defining positivism before going on to consider Andrew Hawkins’ (2013) thoughtful methodological account of an experiment in realist research.
I draw on four ‘fundamental rules’ of positivism from Cohen (1980). First, pheomenalism, positivists banish the essence of things from any rational discussion. Their focus is observable manifestations. Second, is the rule of the nominal. In denominating particular instances of things, these things share the same fact, but this term does not relate to some general property. Third, is an assertion that there is a unity of science, which adheres to a specified and mutually understood set of rules. This allows for the design of definable forms of enquiry (also know as protocols) that are developed before the investigation and must be adhered to through any experiment. Finally, the findings from any enquiry must be reducible to normative statements that describe a fact in such a way that prediction can be made from that fact that allows for technical prescriptions to be made.
For realists each of these rules is problematic. Phenomenalism is an expression of a flat ontology, an assertion that all that can be learnt of the social world is observable in an empirical domain. Pejoratively, positivism is empiricist. Realists contend that social phenomena are far richer and deeper than this. They are, as Andrew Hawkins (2014) points out, stratified in some way, with mechanisms as powers, liabilities, and dispositions (many of which are not amenable to direct observation or measurement) acting in actual ways on the empirical and observable phenomena we seek to investigate and explain.
The unwillingness in positivism to recognise (and name) the cause of things inevitably leads to nominalism. This, for realists, is an abstraction of that which can be observed (as a regularity or outcome for instance) from the ‘ugly circumlocution’ (Pawson, 2013:21) of context and mechanisms of which they are part. Nominalism is reduction. It is also reification.
Things (often called variables) shorn of their relations are essential for positivists to proceed (Byrne, 2002). In this state are they amenable to rule-laden investigation. There are no relationships to complicate investigation. Only that which can be acted upon and instrumentally measured in some way is permissible. Validity lies in the design of the procedure. Design, therefore, is valorised at the expense of creativity, ideas, and interpretation. For realists, the rigour of design is important but the direction of decision making is quite different. Experiments are well designed in the service of ideas to which they are subordinate.
Realists don’t produce normative statements from their research. They simply can’t because the real is independent of our knowing it and, try as we might, we will never adequately account for real social phenomena. There are reasons for this that positivism simply ignores—open social systems, mechanisms that are invisible, immeasurable, or obfuscating (and maybe dormant, but not latent—see below), and a recognition that mechanisms, contexts, and outcomes in the social world are only ever relatively enduring. Realists recognise how science progresses through the incremental accretion (and occasional punctuated equilibrium) of knowledge, not in the heroic leaps of Whigish positivism.
Now, the interesting thing for me about Andrew Hawkins’ paper is that it deals with these four fundamental rules of positivism well and elaborates the realist alternative. The experiment he has done is not positivist in the way outlined here, it is realist. The experiment is set up to test ideas. The design does not come first, but is shown to be fit for purpose to test ideas about how and why mentorship of students works. Identifying a quasi-experimental and a quasi-control group is quite acceptable because they are not groups, but cases developed to bring together particular configurations. They are purposefully selected to allow theories to be tested in each case and between cases. The use of ANOVA, Cohen’s d and t-tests, to measure variance, effect size, and fit, are not at odds with realist approaches, so long as the prior purposive work remains associated with these measures.
It is easy to be waylaid by the ‘cocked hat type’ distribution that is ‘sufficiently nearly normal’, which Student (1908: 18) suggested might be used for laboratory and biological experiments. Positivism and the use of experimental approaches can be conflated. Indeed it might be useful to avoid using terms normally associated with experimental trials, such as the experimental and control group to describe the mentored and un-mentored students in the experiment. This language of the trialist has particular meaning and with this implicit assumptions. Random allocation to experimental and control groups allows, it is claimed, for latent variables to be assumed to be latent for one case and therefore latent for all cases. They have no value (Bollen, 2002). (A reason not to use latent to describe mechanisms, I would suggest.) Even more sophisticated and recent techniques developed to account for multiple contexts in cluster, nested, partially nested, and multiple membership of context approaches to randomised control trials (Roberts and Walwyn, 2012) hold tight to a positivist model of the world. This nod at the complexity of lived experience is at first sight alluring, but remains significantly constrained through the assumption that the identification of a variable—the number of times a patient visits a therapist for a particular treatment, for instance—adequately explains the relationships that led to choices, however constrained, that conjoin treatment to outcome in some way. As Hawkins (2014) shows in his example of an experiment in realist research, realists add more than a few observable mediators and moderators to the descriptive assemblage of statistical tables, they seek to explain causal generative mechanisms through testing theories about these mechanisms.