The University of Portsmouth’s Inclusive Leadership Development Programme’s Evaluation

Background 

In 2022, Advance HE was commissioned by the University of Portsmouth to co-develop an Inclusive Leadership Programme, known as iLEAD. This programme is a central feature in delivering the University’s vision 2030 and strategy 2025 and demonstrating one of its core values; respecting and celebrating diversity and equal opportunity through an inclusive culture.   

iLEAD is a tailored programme targeting various leadership levels, including aspiring managers (AM), leaders and managers (LM), senior leaders (SL) and university executive board members​ (UEB)​. The University committed ​leadership engagement ​with the programme over a three-year delivery period, with participation taking place in waves. Given the substantial time and financial commitment that the roll-out of this programme has entailed, the University required evaluation of its impact on participants to be embedded into its delivery.  

 The learning outcomes for the iLEAD programme include:   

• Equipping participants with tools, skills, relations​hip (or connections)​, and knowledge they can use to practice inclusivity;  
• Embed inclusivity in all the institution does, with participants applying and sustaining what they have learnt in their daily practice;   
• Participants who are empowered to develop inclusivity and be active agents for change;   
• Participants who are ready to act as catalysts, stimulating change in others and the organisation.  

Method 

A bespoke pre- and post-survey was used to assess the impact that engagement with the programme has on participants’ perceived knowledge, confidence and ability to role model and empower others towards inclusivity. The content of the survey was based on the programme’s intended learning outcomes and included a mix of close-ended questions (e.g., agreement statements using 5-point Likert scales) and free-text responses (to allow participants to express their experiences and perceived gains in more detail).

In each cohort, participants have been asked to complete a bespoke online survey one week before the programme started (pre-test) and three months after the programme concluded (post-test), to allow time for learnings to be embedded and changes to manifest. We used unique identifiers to match participants’ pre- and post-survey responses, with the difference between these being used to evidence individual-level change. Impact was considered at an overall level, as well as across the individual leadership levels. 

Outcomes and impact 

Overall, data from 137 participants (41 AMs, 48 LMs and 48 SLs) over the last two years draw a clear picture of success: there were significant increases in 18 of the 19 items included in the pre- and post-survey. In other words, participating in the iLEAD programme significantly increased participants’ inclusive leadership knowledge, confidence and ability to role model and empower towards inclusivity (see Figure 1).   

Figure 1. Percentage increases in participants’ inclusive leadership knowledge, confidence, and role modelling and empowerment towards inclusivity.   

Across the whole sample, the greatest gains were seen in participants’ confidence – with average ratings on these items increasing by 21% pre- and post-programme. This is particularly positive as this was also the area rated the lowest in the pre-survey. Figure 2 outlines the five survey items in which participants’ average ratings showed the biggest improvement, before and after their engagement with iLEAD. These are all aspects of inclusive leadership confidence, and highlight how the programme boosted participants’ confidence to understand strategic finance, as well as use tools and techniques (e.g. mentoring, coaching and peer learning) to promote inclusivity. 

Figure 2. Survey items with the biggest change in average ratings before and after engagement with iLEAD.   

Although the programme benefited participants across all target areas (ie perceived knowledge, confidence and ability to role model and empower towards inclusivity), our evaluation allowed for impact to be interrogated further at different leadership levels:

• On the whole, the biggest increases were seen amongst LM (18.5% increase).  
• Out of all areas measured, the biggest change was in AM’s confidence to use their coaching skills to promote inclusivity (28% increase).  
• While AM and LM reported improvements in their confidence to lead difficult conversations, SLs’ ratings improved the most in terms of their ability to empower others to build an inclusive culture (a 16% increase).   

Alongside these significant changes, participants explicitly attributed gains and changes in current practice to their engagement with iLEAD. Overall, the three areas in which participants indicated that their engagement with iLEAD benefited them the most are:  

• Being curious about and open-minded to others’ perspectives and experiences.  
• Being honest and open to critique about one’s own potential biases and assumptions.  
• Being ready to adopt new ways of working in pursuit of inclusivity.  

Participants also described the programme’s positive impact in their free-text responses, mentioning: 

• Improvements in their general knowledge, skills and practices around leadership;  
• An increased awareness around what inclusivity means and how it plays out in practice; and   
• Engaging in self-reflection or being open to critique theirs’ and others’ own assumptions and biases.   

Participants valued particularly how iLEAD presented the opportunity for them to come together with colleagues across the University and to have protected time to focus on inclusivity and how to implement this in practice. Participants praised the leadership exploration groups and open discussions, as these allowed them to share their thoughts in a safe space, while learning from their peers. Peer-learning is by far the most common reported gain mentioned in participants’ open-ended responses, featuring in almost 30% of the total responses and in almost 50% of the responses of senior leaders.   

Finally, 81% of participants identified changes they have implemented in their current practice after engaging with iLEAD. The top three topics that emerged after thematically analysing these 111 responses are: 

• Engaging in reflective practice, mainly related to challenging own biases and assumptions and how these influence decision-making and actions (mentioned by 17% of participants).  
• Being more open towards other opinions, taking steps to include multiple voices in decision-making and being more responsive to challenge and questioning from colleagues (mentioned by 14% of participants).  
• Being attentive to practices, processes and opinions that are exclusive and trying to constructively ‘call them out’ and change them (mentioned by 14% of participants).  

The results of this impact evaluation have been essential in helping the University of Portsmouth develop an informed understanding around how the programme is influencing participants across leadership levels. The results have been used to showcase the success of the programme both internally and externally, raising their profile as an institution taking active and effective steps to foster an inclusive culture.