Better Metrics

This first appeared on Times Higher Education

So the UK HE sector, has just been evaluated again (the teaching excellence framework: TEF). This brought good news for some and bad news for others. And that is the problem with evaluation, it is divisive – there are winners and losers. This moves academia away from being a collaborative, team-effort with a free flow of ideas between individuals and a pooling of talents to a fight to the death for limited resources. However, regardless of your opinion about the validity of the process, external assessment of higher education is unlikely to disappear anytime soon. This means we need to think about what is being assessed and to shape it so that it builds rather than subdivides.

Whilst the assessors claim it drives up quality, assessment can put undue pressure on the people being assessed. And it changes the focus to the metrics being assessed. We should be in higher education because we love doing it. But, since the sector has moved away from the generation of gentleman scientists performing research on their country estates in their spare time, to be involved in higher education, you need a space to do it, income to support you while you do it and funding to pay for it. And to get these things you need a career. And to get a career you need to tick the boxes.

Call it what you want: gaming the system, focussing resources for maximum effect, metric based performance criteria, we all do things to progress our careers. If you don’t think you do, you are either: in denial, stuck in a scholarly Stockholm syndrome where you think this behaviour is the norm, a Nobel laureate or about to get sacked.

Changing the metrics is the easiest mechanism to deliver change, giving clear guidance and enabling senior staff to support people as they advance. But the new metrics need to be meaningful and critically, understandable to everyone involved. Poorly constructed metrics can lead to the loss of potential by cutting careers off at an early stage, perpetuate gender bias if they are worded in an overly aggressive fashion and can pile on unacceptable levels of stress, especially when used as a tool to manage out rather than support and develop.

The best metrics will align to support and deliver performance, scientific excellence, service and personal development. Easy to say, much harder to deliver. The San Francisco Declaration on Research Assessment (DORA) is a start, but focuses mainly on research output, without assessing the academic in the round. Here are my suggestions for underpinning principles for new metrics:

1. Holistic: we need to demonstrate that we are improving and growing, that the work we are doing is of value and that we are making a meaningful contribution to the community, both the greater community and also to the institutions in which we are based. Contributions to these communities – through teaching, service, outreach, mentoring need equal weighting to grant income and papers. Not just as boxes to be ticked, but actual equal weighting.
2. All informed: both the assessors and the assessed need to understand, accept and stick to the new metrics.
3. Supportive: It takes time to discover your academic niche – not all of us are great teachers, not everyone can be on TV, only 9 of us a year are going to get Nobel Prizes. There needs to be space to develop our talents and not to be cut off after three years because you failed to get a million pounds in grants and the cover of Cell. 
4. Simple.

If metrics can deliver academic excellence, personal development, community engagement and the greater good, then we might get the sector that we are all working hard towards.

Lab culture: Part 1. What is lab culture?

What’s the difference between a Lab and E.coli … E. coli has culture.

Lab culture is surprisingly hard to describe, in part because it is so varied.

Lab culture is underpinned in the science. There are clear differences based on the discipline: a group waiting for their solitary slot of the year on the particle accelerator are going to have different priorities to a group conducting field work on the mating habits of the common sparrow. This seems obvious, but there are subtle differences in sub-speciality. Take the approaches to studying influenza virus as an example: a virology group working on viral replication are going to think about the problem differently to an immunology group working on flu vaccines who in turn are going to approach it differently to an epidemiology lab working on transmission dynamics.

There are also differences in the approach to science. Amongst others, there are n=3 is good enough labs and there are labs that need to iron out every wrinkle, even if it means never publishing. There are labs chasing the latest fashions and labs working on an obscure niche of the field. There are ‘translational’ labs working towards to a specific product as that will improve human life and ‘pure’ science labs seeking to discover some greater truth - each convinced their own approach is best. There are pile ‘em high sell ‘em cheap labs producing PhD likes sausages and bespoke hand crafted labs with only a single member. There are great labs to work in that generate nothing and shitty labs to work in that get Nature papers. And of course most labs are a mixture of everything.

But it doesn’t end there, lab culture extends into your social life. In some labs everyone stays at work till midnight but only a subset of those where people are actually working as opposed to messing about on Facebook looking busy till the boss leaves (presenteeism). In some labs everyone goes to the gym and power lifts whilst other labs binge drink. Some labs stop religiously for tea together at 10am and some labs no one talks to each other – simmering in resentment. More than anything else, the culture of a lab will shape your experience in the lab now and potentially going forward into the rest of your career (and possibly personal life: intra-lab weddings being not infrequent).

Since lab culture is key to your happiness and productivity, it is important to identify what works for you and then identify a lab that aligns as closely as possible to this. We would argue this is more important than the material detail of the project. Most scientific skills are transferrable; being miserable for 3 years and then quitting is not. John started in Drosophila lab and now work on human vaccines: Charlie started in a parasite lab (not entirely coincidentally in the same department) and moved through mast cells, asthma, pharma and charity. It is better to walk away from a lab that does not align with what you want than suffer for 3 years. This may be hard to imagine when you have spent an eternity looking for a PhD position: but a bad PhD is considerably worse than no PhD.

Identifying what you want takes some hard soul searching; harder still is finding the soul of a lab before you work there. There are some ways to sniff it out – it helps if you can do a 1+3 type PhD scheme and shop around for a lab, likewise masters or bachelors projects in the same lab or department can help. Some labs will have such a strong reputation (for good or bad) that it will precede them. It also gets easier as your network grows. Do some research, LinkedIn stalk the lab and work out how many people have worked there ever and what they have gone on to do, look at the number of publications and where they go. Failing knowledge you have to ask questions. Try and visit the lab before doing an interview, ask other people working in the lab what it’s like to work there but also snoop a bit in the lab and offices. Are they tidy, dirty, covered in ‘hilarious’ posters, is there evidence of communal food in the office, are there rotas for cake club or other social interactions. Interviews are a two way process, don’t waste questions about the start date, ask questions that probe lab culture. This is a tricky line to take as the questions need to be open without being confrontational and to make you sound employable: how often do you meet with your team is reasonably non-confrontational. Take some time to think about it, balancing the emotional with the rational.

At the end of the day it is always a bit of a punt and you may have to settle for good enough and paid rather than perfect but unemployed. However, once you’ve got your foot in the door there are ways to change things.

Lab Culture part 2: Shaping lab culture

We’ve established that the culture of the lab you work in is going to have a major impact on your life. Having taken the fantastic advice from our previous article, you went and picked a lab with the perfect culture for you and then lived happily ever after. The End, or is it?

Something’s changed.

The direction of the work can change; either driven by new funding opportunities (or lack thereof), or new techniques – with this week’s cutting edge technique becoming next week’s kit. The people doing the work also change; this is accelerated by the current climate where high-throughput, short-term contracts are so common. Changes to a lab’s culture can be positive; a coming together of like-minded individuals, with a common purpose at a similar stage of life. But they can be negative; it only takes one poorly managed psychopath to tear apart the fragile ecosystem of a lab.

I’ve got the power

These fluxes mean that lab culture isn’t set in stone and the power for change is in your hands. But in order to change lab culture you need to understand the different agents that shape it and the degree of leverage they have.

Institutional Change

The most remote agent influencing lab culture is the institute. Institutional influence is asserted through the principal investigators it hires and the expectations it has of them. The institute also influences broader interactions outside of your group through meetings, seminars, away days and social events. These extra-group interactions can be critical when you have hit a wall, both scientifically and socially. Whilst you can shape these wider interactions – departments often have unspent budgets for social events and are looking for an enthusiastic person to organise things – by  and large there isn’t much you as an individual can do to shape the broader policy:unless of course you are head of department!

Power to the people

The second driver is the group itself. People make the lab, they are the ones who go to lunch (and the pub) with you, spill stuff into your bench space, order replacement chemicals (before using the last aliquot), book key pieces of equipment for weeks on end or help you with that out of hours timepoint. Every member of the group can exert a positive or a negative effect on the culture. The degree of influence you have in the group will be a product of seniority, time served and personality type.

The simplest way to have a positive impact is to spend more time together: out of the context of work. This doesn’t have to mean going to the pub, simple social events – a cake rota or a bake-off – anything that gets everyone in one space and talking is beneficial. How you behave in the lab is also important; not being a dick is a good start. Contributing to communal tasks, for example tidying up, collecting parcels or reporting broken equipment, leads to a better lab culture. But think about how you let others know about your contribution. There is no point in being a silent martyr: you are only going to end up resentful. At the same time, don’t weaponise your contribution. Emptying bins then sending passive aggressive emails about full bins leads to a worse culture than not emptying bins in the first place.

Like a boss

The final (and main) driver of lab culture is the principal investigator (PI). Even seemingly unengaged PIs set the lab culture: through the staff members they choose, the bad behaviours they ignore, the field they work in and their approach to that field. Some of the things that can be done as a PI to build a good lab culture include:

1. Attitude. Spend a small amount of time thinking about what type of lab you want to run. If you are going to run a results-driven sweatshop, at least let it be a conscious decision rather than a default position based on your own postdoc experience.
2.  Altitude. Running a lab is not dissimilar to being a parent. There’s a balance between being too close (helicopter) or too remote (satellite). Different people need different things at different times. Only by getting to know your group will you know the level at which they work.
3. Break the bread. Take your group for a drink or an ice cream. Celebrate every win – papers, grants, vivas. When you have time, eat lunch with the group. Have an away day. Invite the group to your house. These small social interactions will help and energise you too. However, remember it helps to have some distance between you and your team: management’s not a popularity contest; you may have to make difficult decisions which can’t be done if you are always trying to be “besties with your crew”.
4.   Care. Take an interest in your team. Ask deeper questions about their lives outside the lab and actually listen to the answers. Remember what they said they were going to do at the weekend and then follow up on Monday. Even if you are not interested, fake it; 5 minutes of engagement can go a long way.
5.  Return to the lab. You may have got to the point where the majority of your effort is focussed on writing (papers or grants) and it is more efficient to get the people you employ to do the labwork, to do the labwork. However, not being in the lab, you will miss a substantial part of the group dynamic. Treat yourself to an experiment every now and then, you will get a much better sense of WHAT is and isn’t working and equally WHO is and isn’t working. As a side benefit, people tend to be much chattier when pipetting so you can catch up on the gossip.
6. Be the best you. Behaviour in your lab reflects you. This is partly because you have recruited people just like yourself (which we all do) and partly because as the figurehead of the lab, you are the main role model. Bitching about colleagues to your group, however tempting, will lead to a culture of bitchiness. Not sweating the small stuff will lead to a more relaxed atmosphere, but it may mean things get overlooked. Losing your rag every time a mistake is made won’t stop mistakes, but will mean people hide mistakes from you.

Whatever your role, the way you act will influence the culture of the lab; and whilst getting the science done is the priority, doing it in a way which is collegiate, supportive and fun makes it less painful for all involved.

This first appeared on digital science