Designing programmes for learning: foundations and aspirations

Assessment practice remains an unsolved and persistent challenge across the sector. Traditional assessment practices dominate, despite the accumulation of NSS evidence for the past fifteen years that raise concerns in this area of practice. In parallel, discussions in the literature point to the design of curriculum and assessment at programme-level as a key to move forward. To date, these proposals, remain largely theoretical. Focusing on bridging the gap between theory and practice was a key objective of the conference that the University of Nottingham ran in January 2022: ‘Designing programmes for learning: foundations and aspirations’.

The aim of our conference was to grow awareness of solutions, showcase best cases of practice and set aspirations for the sector. We hosted the conference online which allowed 44 speakers to take part and a vibrant audience of more than 500 national and international participants. Our two-day online conference served to review progress across the sector and to consider how to affect institution-wide transformation.

If you are an institutional leader, a practitioner, a developer with an interest in how to design programmes of study, you will find the recordings of this conference of interest.  Recordings of all sessions and key resources are now available on the University of Nottingham conference website.

In this article we share key insights gained from students, practitioners, leaders in the field and institutional leaders of change.

What is programme level thinking for curriculum and assessment and why is it important?

A keynote by Professor Cees Van der Vleuten (Maastricht University) discussed the theoretical foundations for programme level design (programmatic assessment). Cees’ work is enlightening as he conjures validity, design, and student learning seamlessly in his theoretical description. Cees explained ‘programmatic assessment’ is not just an approach but a new paradigm to understand assessment design and learning. This paradigm offers a systems approach to think about assessment. Here are some key principles to guide our thinking and practice

• The whole is greater than the sum of its parts (systems thinking)
• The method or assessment type is not important. An assessment type is not intrinsically better or worse.  We should concentrate on the justification for its use in the context of the programme, the end goals and the other assessment methods
• Focus on high and low stakes in the decisions to be made about student learning and progression (and overcome the summative-formative dichotomy)
• Student learning requires constant reflection and dialogue in recognition that ‘self-directed’ learning needs direction (coaching).

Moreover, Cees shared experiences of the practical implementation of these ideas recognising the challenges. Professor Cees van der Vleuten has written extensively about the theory, practice and research in practice of programmatic assessment. Readers can find many resources at:  www.ceesvandervleuten.com.

So what does programme level design of assessment look like in practice?

Taking a programme level view of assessment has been discussed in the literature for over a decade. The ideas in the literature are new to most practitioners and are difficult to imagine without a tangible example. In practice, programmes of study that adopt a programme level and coherent view are more the exception than the norm. We invited practitioners to share their experiences. Two key frameworks for programme level design of assessment (PASS and TESTA) established over a decade ago, were revisited. These frameworks remain relevant and still offer practitioners different starting points to start considering how to take a programme level view.

Case studies of programme level design were presented to inspire practitioners. Cases from Science, Engineering, Medicine, Arts were showcased demonstrating that the principles can be applied across the range of disciplines.

How these different practitioners got to adopt this approach varies. Many were motivated by the NSS results about assessment. Others were new departments that thought actively about adopting a different model and we see the influence of professional bodies. Some had been inspired by the TESTA experience in their departments. Broadly speaking, there seemed to be overlap despite their different contexts and approaches.

• Changing processes of design to work as programme teams
• All cases illustrated well the principle of starting with the end in mind. Most exhibited clear and deliberate efforts to understand the needs for future professionals and place those at the heart of their design
• Reduced volume of assessment. A typical assessment load in programmes of study is of approximately 40 assessments in total (Jessop and Tomas 2017). In our experience, programmes of study that undergo a review, adopting a programme level view, have 20 or less summative assessments in total. Whilst quantity does not determine overall quality of the design, it is an important indicator as it can affect student stress and learning.
• Integration across modules. Most cases present hybrid models overall. Some modules kept a traditional approach with a focus on content and exams and possibly integrated across modules. Less traditional was the emergence of modules with a focus on skills (e.g. problem solving, project focus, team work) well integrated into the programmes of study. The result in most cases is a reduction of the presence of traditional examinations. The redesigned cases exhibited a balanced assessment diet with other assessments addressing key employability skills and from day one (e.g. project based, portfolios, team work). 

Moreover, radical approaches are starting to emerge. Colleagues from Software Design from the University of the Highlands and Islands, demonstrated the ultimate learning design. Their learning design, after careful consideration of the needs in industry and learning, engages students in project-based learning from day one. Also, students own their assessment as they address the learning outcomes, identify them in a given programme level rubric, and are assessed on those as part of the projects.

Students are also key stakeholders as co-designers of the curriculum. Students from the University of Nottingham are involved as partners and co-designers of programmes of study. Students presented on their experience and powerful impact on practice in their own Schools. Students are instrumental in providing insights into how to integrate across modules and also how learning could be paced.

Do institutions have a role to play?

A roundtable session on institutional transformation explored the challenges of institutional change with eight institutional leaders from across the sector. Despite differences, agreement on the principles was clear. Institutions have a role to play to lead and enable the changes to assessment practice and programme design. The designs described above require different approaches to work, an investment in areas such as employer engagement and marketing, working across professional services and academic teams, regulatory changes (Assessment and programme design).

Leaders spoke of 5 and 8-year journeys for institutions to progress indicating that the institutional effort is significant and executive support is paramount. Resources, flexibility, communication and support and guidance for the disciplines were key ingredients for the effectiveness in guiding and supporting transformation. The importance of systems and institutional processes is also key.

What are the future challenges?

New aspirations and targets for the next decade were also discussed:

• Values in programme design featured in a keynote by Jan McArthur and another session by the QAA with colleagues at the University of Nottingham.
• Analytics for programme design were discussed as a key area for development. The needs for development were discussed by Carmen Tomas sharing insights on proposals for the development of analytics based on outputs of a Jisc-funded project. Cases of institutional development of dashboards for staff and student programme level views were shared by Karen Barton, from the University of Hertfordshire.

Carmen Tomas is an Honorary Assistant Professor of Education at the University of Nottingham and a Director of Partner Support (Higher Ed Partners, HEP). Carmen led the Curriculum Transformation project between 2014 and 2022.

Katharine Reid is the Faculty of Science Associate Pro-Vice-Chancellor for Education and Student Experience and leading the institutional Curriculum Transformation Programme.