Better Metrics

This first appeared on Times Higher Education

So the UK HE sector, has just been evaluated again (the teaching excellence framework: TEF). This brought good news for some and bad news for others. And that is the problem with evaluation, it is divisive – there are winners and losers. This moves academia away from being a collaborative, team-effort with a free flow of ideas between individuals and a pooling of talents to a fight to the death for limited resources. However, regardless of your opinion about the validity of the process, external assessment of higher education is unlikely to disappear anytime soon. This means we need to think about what is being assessed and to shape it so that it builds rather than subdivides.

Whilst the assessors claim it drives up quality, assessment can put undue pressure on the people being assessed. And it changes the focus to the metrics being assessed. We should be in higher education because we love doing it. But, since the sector has moved away from the generation of gentleman scientists performing research on their country estates in their spare time, to be involved in higher education, you need a space to do it, income to support you while you do it and funding to pay for it. And to get these things you need a career. And to get a career you need to tick the boxes.

Call it what you want: gaming the system, focussing resources for maximum effect, metric based performance criteria, we all do things to progress our careers. If you don’t think you do, you are either: in denial, stuck in a scholarly Stockholm syndrome where you think this behaviour is the norm, a Nobel laureate or about to get sacked.

Changing the metrics is the easiest mechanism to deliver change, giving clear guidance and enabling senior staff to support people as they advance. But the new metrics need to be meaningful and critically, understandable to everyone involved. Poorly constructed metrics can lead to the loss of potential by cutting careers off at an early stage, perpetuate gender bias if they are worded in an overly aggressive fashion and can pile on unacceptable levels of stress, especially when used as a tool to manage out rather than support and develop.

The best metrics will align to support and deliver performance, scientific excellence, service and personal development. Easy to say, much harder to deliver. The San Francisco Declaration on Research Assessment (DORA) is a start, but focuses mainly on research output, without assessing the academic in the round. Here are my suggestions for underpinning principles for new metrics:

1. Holistic: we need to demonstrate that we are improving and growing, that the work we are doing is of value and that we are making a meaningful contribution to the community, both the greater community and also to the institutions in which we are based. Contributions to these communities – through teaching, service, outreach, mentoring need equal weighting to grant income and papers. Not just as boxes to be ticked, but actual equal weighting.
2. All informed: both the assessors and the assessed need to understand, accept and stick to the new metrics.
3. Supportive: It takes time to discover your academic niche – not all of us are great teachers, not everyone can be on TV, only 9 of us a year are going to get Nobel Prizes. There needs to be space to develop our talents and not to be cut off after three years because you failed to get a million pounds in grants and the cover of Cell. 
4. Simple.

If metrics can deliver academic excellence, personal development, community engagement and the greater good, then we might get the sector that we are all working hard towards.