Transformative Assessment

Assessment is one of the most challenging aspects of designing curriculum. The debate about teaching and learning in MOOCs has exposed several issues related to online pedagogy, including formative and summative assessment. Designing Transform curricula has given us opportunities to re-think online assessment practices and learn from the world-wide MOOC experience.

The design principles underpinning Transform curriculum are to:

• be inquiry-based around real world challenges
• offer flexibility for learners
• allow for collaborative learning between peers (with varying degrees of synchronicity)
• allow for contact between learners and experts (with varying degrees of synchronicity)
• allow us to curate learning resources and experiences in meaningful and coherent ways
• enable learners to provide evidence of their learning
• provide appropriate guidance and support for learners

So, what does this mean for assessment? And how might the experience of assessment be transformative for learners and teachers?

First principles of assessment

Regardless of the mode of learning, best-practice assessment principles still apply. First principles of assessment are summarised in this graphic:

References:

Boud, D. (2000). Sustainable Assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167. 

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.

Herrington, J., Reeves, T., & Oliver, R. (2010). A guide to authentic e-learning. New York: Routledge. 

Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550.

For learning to address real world challenges, assessment should also provide real world opportunities for learners to enhance the quality of the products of their learning. In assessment theory, we talk about the close relationship between assessment and learning using terms like assessment FOR or AS learning. This means assessment is a learning experience in its own right, rather than being tacked on afterwards (i.e. assessment OF learning).  

Assessment FOR and AS learning involves both students and teachers making judgments in order to inform future learning (formative assessment). David Boud has captured this idea in his concept of  ‘sustainable assessment’. Sustainable assessment is where learners learn to make judgements about the quality of their own and other’s work, and to engage in reflective practice. This involves peer feedback and self-assessment, activities that require a good deal of scaffolding and support for learners.

Real world learning involves engaging with messy problems in order to develop higher order thinking, problem solving and decision-making skills. It’s easy to design assessment that measures things that are easy to measure. But perhaps we should be suspicious if something is easy to measure, as maybe it’s not designed to measure the type of real world learning that involves messiness!  This is where VALIDITY comes in. Are we measuring what we had intended to measure? Careful design is needed to match real world inquiry learning with valid and authentic assessment opportunities.

Real world learning also involves engaging with the student’s own context, interests, strengths and weaknesses. Inquiry-based learning involves students choosing a question/problem/project related to their own life/workplace. It involves deciding on a process or methodology to use to address their question, and the mode or format that they use to present the outcomes of their investigation. Like sustainable assessment, inquiry-based learning requires strong support and guidance for learners as they undertake the inquiry process.

The assessment design process

Design of authentic assessment starts with identifying the learning outcomes. The next step involves designing assessment that provides evidence of these learning outcomes – i.e. what does success look like and what opportunities are needed to create this evidence? Only then do we design the learning activities that develop the skills and knowledge needed for the assessment.

However, as academics, we tend to be very passionate about our field, and this means we often start with the content, then develop the learning activities, and then create the assessment. And then we realise that we need to design the learning outcomes!

An even greater trap is being seduced by technologies.

The challenge of assessment and modularisation

The Transform curriculum approach involves packaging learning into chunks undertaken as online modules that can be bundled together as part of a post-grad qualification. This creates challenges for assessment, as it’s easy to fall into the trap of designing atomistic, easy-to-measure assessment at the module level. It’s not easy to scaffold students’ learning to make complex judgements on the quality of their own and other’s work in a real world, inquiry-based approach. Herein lies our challenge. Some helpful questions to start with are:

• What will success look like?
• What opportunities do learners need to demonstrate success?
• How will learners know they have been successful?

The risks and opportunities for Transform can be seen in the MOOC assessment experience. The worst MOOCs provide simplistic, poorly written multiple-choice quizzes, while the best provide rich, authentic, collaborative assessment experiences. For us, the choice is easy!

Mandy Lupton -- Transformational Teaching Fellow / Senior Lecturer, Faculty of Education

Jill Willis -- Lecturer, Faculty of Education

Kirsty Kitto - Transformational Teaching Fellow / Postdoctoral Research Fellow

Kelli McGraw - Lecturer, Faculty of Education