Tag Archives: Research Excellence Framework

A Stern review for everyone?

The Stern reviewwas commissioned by the government to carry out the review of the Research Excellence Framework (REF) to ensure future university research funding is allocated more efficiently, offers greater rewards for excellent research and reduces the administrative burden on institutions’. In this post, I explain why no single policy can solve these problems uniformly: they affect scholars of different seniority, and different disciplines, very differently. The punchline is at the end.

My initial impression of the Stern review is that it has gone to great lengths to address the unintended consequences of the previous Research Excellence Framework. One of its key aims now is to try to anticipate the potential unintended consequences of its reduction of other unintended consequences! This is remarkably common in policymaking, perhaps summed up by Aaron Wildavsky’s phrase ‘policy as its own cause’: we enter a never-ending process of causing ripple effects when trying to fix previous problems.

The example of non-portability (#sternreview portable)

Take the example of one of the biggest problems:

Problem: there was a large incentive for Universities to ‘game’ the REF towards the end of the cycle: paying for 20% of the time of big name academics, or appointing them at huge salaries, to gain access to their 4 best publications. A policy of rewarding research excellence became a policy to (a) reward big transfers, undermining the efforts of other Universities and reducing their incentive to invest for the long term, and (b) boost the salaries of senior scholars (many of whom were already on 6-figure salaries), often to ridiculous levels.

Solution: non-portability. The idea is that you can move but your former employer gets to use the texts that you published while in your last job. So, there may now be less incentive to buy up the big names in the run up to the next REF.

Unintended consequence: uncertainty for early career researchers (ECRs) or scholars without permanent (open-ended) contracts. Many ECRs have expressed the concern that their incentives may suddenly change, from generating a portfolio of up to 4 excellent publications to secure a permanent post, to perhaps holding back publications and promising their delivery when in post. This could be addressed by the present review (ECRs could be included in the REF but be under no obligation to submit any publications), or perhaps by exempting ECRs since the policy is aimed at senior scholars (but the exemption would also have unintended consequences!).

Interpreting the problem through the lens of precarious positions

We will now enter a phase of debate driven by uncertainty and anxiety about the end result, and a lot of the discussion will be emotionally charged because many people will have spent maybe 8 years in education (and several years in low paid posts after it) and not know what to do next. It is relatively easy for people like me to say that the proposed new system is better, and for senior scholars to look on Stern as a big improvement, because people like me will be rewarded in either system (I will leave it to you to decide what I mean by ‘people like me’). It is more difficult for ECRs who are genuinely uncertain about their prospects.

The punchline: how does this look through the eyes of scholars in different disciplines?

The disciplinary lens is the factor that can often be most important but least discussed, for two key reasons:

  1. The general differences. Scholars operate in different ways. The ‘STEM’ (science, technology, engineering and mathematics) subjects are often described in these terms: you have large teams headed by a senior scholar; there is a hierarchy; you all work on the same research question; you publish many short articles as a team (with senior authors listed at the very start and end of the list of authors); you are increasingly driven by key metrics, such as personal citations and the ‘impact factor’ of journals . The humanities is often described like this: you are a lone scholar; you work on your own research question; you publish single-author work, and the big status symbol is the research monograph (book); these journal or other metrics do not work as well in your discipline. I am verging on caricature here (many ‘STEM’ scholars will work alone, and some humanities or social science scholars will operate laboratory-style teams), but you get the idea.

These differences feed into other practices: only in some subjects can it make sense for a University to ‘poach’ a whole team or unit; only in some subjects do ECRs need to develop their own portfolio of work; in some subjects, a PhD student or ECR effectively works for a senior scholar, while in others the PhD student has a supervisor but can set their own research agenda; in some subjects, it is automatic to include the senior scholar in an article you wrote, in others it would be seen as exploitation of the work of a PhD student or ECR. In some subjects, a CV with your name on team publications is the norm, while in others it would look like you do not have your own ideas.

  1. The REF reinforces these differences. You often find the impression that research exercises and metrics are there for the STEM subjects (or ‘hard sciences’) and not the humanities or social sciences: a process or review for Universities does not take into account the differences in incentives and practices across the disciplines, and some disciplines might lose out.

So, if you follow this debate on twitter, I recommend that you look at the bios of each participant to check their level of seniority and discipline because the Stern review is for all but it will affect us all in very different ways.

See also: James Wilsdon ‘The road to REF 2021: why I welcome Lord Stern’s blueprint for research assessment

 

3 Comments

Filed under Academic innovation or navel gazing, UK politics and policy

Academic Branding

The word ‘branding’ applied to academics is so pretentious that I feel dirty just typing it, but it’s a good way into thinking about how you are seen as an academic based on the decisions you make. I think that a lot of assume that we should brand ourselves according to our specialism: there’s Professor X, who knows the most about mutation, or there’s Professor Green who knows the most about mangling songs. However, there are now strongish incentives (in some disciplines – mostly social science?) to maintain some sort of generalist knowledge to further comparative, interdisciplinary and/ or ‘impact’ work. In each case, your audience may not necessarily benefit from your fiddly knowledge of a subject but, instead, appreciate how it fits into a wider picture. Let me give you some rationalised and glossed-over examples of my career to make the point:

  1. Comparative work. I’ve been paid to go to Japan 4 times (3 since 2011, 2 this year) on the back of my work on Scottish devolution. The most useful stuff here, even for academic colleagues, is the Scottish Politics textbook I did with Neil McGarvey. My most recent trip was to give talks to a public and practitioner audience where, again, few are interested in the Scottish navel.
  2. Interdisciplinary work. I’ve been working on collaborations with colleagues in subjects such as physics and psychology, The most useful stuff here is a public policy textbook I did in 2011.
  3. Specialist work. I’ve been making some good links with US colleagues, including Chris Weible (editor of Policy Studies Journal), who asked me to co-author the ‘Schlager chapter’ in the 3rd ed. of Theories of the Policy Process (you may have to trust me on how good that is). Again, this came out of work I did bringing together a discussion of theories after doing a textbook.

So, I’m convinced, based on some anecdotal evidence (which I’ve manipulated to suit my argument) that this more general textbook (and bloggy) work has been crucial to my career development – perhaps since it has helped me write in a more accessible/ less jargony way. Perhaps more importantly, it has allowed me to maintain a broad knowledge in particular fields, including policy theory and one particular area (Scotland/ UK). It is no substitute for the REF-type work we all have to do but, then again, it may not hurt either (my big book is underpinned by the policy theory work I analysed while writing a textbook).

2 Comments

Filed under Academic innovation or navel gazing