"The Governor View" - The Big Conversation: shaping the future of higher education governance”, Measuring what matters

As part of the project Advance HE and CUC are running invite only workshops for governing body members, executives and senior governance professionals to facilitate proactive debate and consultation on key issues. The first of which asked, ‘Are governing bodies receiving the right information to focus on long-term strategy as well as immediate financial concerns, and to measure what matters?’.

In the complex and challenging times that universities are operating in, the quality and relevance of the information governing bodies receive has never been more important. 

In this environment, the immediate demands of assuring financial sustainability can take precedence over the longer-term strategic planning that is necessary to create the conditions for an institution to thrive in the future.

Evidence from Advance HE’s Governance Effectiveness Reviews points to the risk of governors being overwhelmed with paperwork and bogged down in routine updates and regulatory compliance issues rather than engaging with critical strategic matters. The timeliness of information, and how it is presented, is also an issue. 

At one London university, relatively simple changes made after an effectiveness review have made a positive difference, according to a board member.

The chair of each committee now gives a quick, five minutes report on the work undertaken since the board last met, with fuller papers contained in the meeting pack. Secondly, every paper has a cover note which links its content to the university’s strategic aims. 

“The committee chair updates bring to life the ongoing work and the cover notes answer the question that we all pose to ourselves from time to time: ‘why am I reading this?’,” said the board member.

For this governor, an effective committee system, where the heavy lifting of information and data scrutiny is carried out, acts as a conduit to the board, particularly the audit committee.

“It is where you see the data that is the DNA of the university,” she says. “It decides what area the internal auditors should look at it detail and that really is peeling away the layers of the onion and looking at what is inside.”

She cites a recent deep dive into the effectiveness of the personal tutorial system.

“We get a numerical indication of whether we can have reasonable assurance that the system is working, and an action plan and timescale if action is needed, which is reported back to the board,” she said. “This all feeds into longer-term strategic aims, such as retention and other regulatory key indicators.”

One “really important and underappreciated” aspect of information provision is how it is presented, according to a governor at an alternative provider. For him the analysis of and visual presentation of data is key to its usefulness.

“There are people who are experts in the numbers but are not necessarily experts at representing those numbers so people can follow them,” he said. “Information visualisation is super important. I think it should be one of generic life skills that we all learn. Tables of numbers are usually pretty useless; you almost always need to convert it into a graph of some description. Those providing the information need to be versed in the best ways of presenting it in order to convey meaning. Providing the information in a useless format is almost as bad as not providing the info at all. And if it is misleading, it is worse.”

Receiving timely, up-to-date intelligence is also an issue, and one that the Office for Students itself is trying to grapple with in its oversight of university finances.

Analysing graduate outcomes, for instance, necessitates a time lag.

“Students have to study for a few years, leave and then down the line you find out what they are doing,” said one governor. “The loop is not very tight; you have to wait years to get that information. It means that some of the things you might consider most consequential are not feeding into the day-to-day decision making.”

Providing governing bodies with information about context, trends and external influences, not just data, was identified by several governors as crucial to strategic planning.

“The issue is what questions should we be asking of the data?” says one. “You need more than a snapshot in time. You need data from previous years, maybe even the last decade, and comparisons with benchmarks. You also need the bigger picture in terms of the trends and the factors that are influencing the trends. What is the story that you want your data to be able to tell? How can that data be useful in influencing a shift in strategy or in delivering a kick up the backside, or whatever needs to be done?”

This need for context was cited by governors trying to make sense of the current focus on the skills agenda, and exactly what part universities play in it.

“How do we join the dots on the skills agenda?”, asked a board member at an alternative provider. “Here, some courses we launch don’t become particularly popular, so we wind them down again, while some really take off. It is an art as much as a science. I probably don’t feel sufficiently informed about what is going on out there in the wider skills agenda and it is something that we need to understand and appreciate.”

Strategic away-days were suggested by a number of governors as an effective tool in concentrating minds and providing the space to think about future directions.

Bringing in external speakers can also provide invaluable expertise on the wider landscape and future scenarios, according to a governor at a Russell Group university.

“Having a good grasp of the wider context can depend on all kinds of things: what kind of governor you are, where you are geographically, how plugged in your vice chancellor is to wider policy,” he said. “Being part of a mission group can help with that context and getting people in to talk about developments in the wider world can be really useful.”

One governor likened the balancing act of providing strategic overview at the same time as meeting short-term challenges to being in a helicopter and flying across a landscape.

“You see the meadows and fields, and the bridges and the roads and houses and you have to keep all of that in your mind. Then sometimes you have to dive down and land in a ditch and start to look at what is in it,” she said. “But you can’t stay in the ditch. You bring that learning about the ecosystem you have discovered back up into the helicopter, so it makes sense in terms of the whole landscape.”

The risk registers that all institutions operate are vital and help prioritise what needs to receive attention, but they are not fool proof.

“The sector is in a constant state of flux and governing boards need to reconcile themselves to that,” said a governor at a post-1992 institution. “We look at the traffic light system for risk. But actually, if something is red it is not necessarily something to worry about and if it is green, it isn’t necessarily something that you can be complacent about. Basically, the lesson is that denoting a risk with a numerical value and a colour should not be the end of the story.  Boards should keep their eyes open very wide.”

As well as the risk of information overload is the question of whether the right things are being measured. As one governor puts it: “The things that matter most are not easily measurable”.

She added: “Measurement forces you down a reductive way of thinking and to look at outcomes that are not necessarily the most important things. What matters is the emotional connection that student and staff have to an establishment – that is the life blood of the place but is very difficult to measure.”

An obsession with what the regulator is most interested in is understandable, and even desirable, but it is not sufficient, according to another governor.

“What matters in the end is: do our students end up living more fulfilling lives and end up as better, happier and more productive members of society – not just what salaries they end up earning,” he argues. “Our real obligation is to individual students, and the wider societies in which they live and work, and are we delivering, as a teaching university, on that.”

In the age of AI, new technology can be harnessed to collect and analyse information to help with those assurances.

“We are doing a lot of work on predictive analytics. If we can tie ultimate outcomes to early indicators, it becomes a useful, immediate feedback mechanism,” says a board member.

For example, if attendance to lectures, visits to the library or work completed, can be reliably linked to poor outcomes, it gives universities the opportunity to “head it off at the pass”. 

As well as a focusing on long-term indicators, boards need to be concerned about what institutions can do today to improve outcomes and “build that statistical bridge back to things it can more immediately effect”, he said.

In the past, varied and unstructured information, such as student feedback or student help requests, could be time consuming to collect and difficult to analyse. Now it can be reviewed at scale “really effectively” using AI.

“It’s not a magic wand and it has its limitations; you have to be careful about accuracy and using the data in a responsible and legal way. But in principle you can get a much better picture of how well things are working and what the main issues are,” said the board member. “It has the potential to tell us all sorts of things of a kind that would have been hard to measure in the past, for instance levels of engagement and motivation.”

With the right information and analysis, “universities can do better at writing their own stories and making the weather rather than just knee jerk reactions to it.”