TEASIG - AI and item writing for high stakes testing: insights from text and multiple-choice question generation

We now live in an age of myriad opportunities to harness the power of artificial intelligence (AI), especially large language models (LLMs), in a range of educational contexts. From a testing perspective this has included opportunities to develop secure, quality test content (including text passages and comprehension question for reading and listening tasks) using online platforms. In this session, we will discuss how one Awarding Organisation has implemented a process for item writing which exploits the available technology whilst adhering to best testing practice. We discuss how to use AI ethically and efficiently to generate test content, whilst keeping a place for human creativity and ingenuity. We consider the limitations and challenges that AI brings to the fore and consider a process of good practice for content development from the perspectives of both production and review.

Speakers:

Andrew Fleck

Andrew Fleck has over two decades' experience teaching EFL and EAP in Europe, South Korea and Britain. For the past five years he has been writing reviewing and, more recently, managing test content for Trinity College London. In 2023, he completed an MA in Language Testing at Lancaster University. His dissertation on task types in listening tests was highly commended by the Caroline Clapham IELTS Masters Award committee.

Philip Horne:

Philip Horne is the Head of Technical Content at Trinity College London with responsibility for item production and review. He has worked in various assessment-related roles in the UK, East Asia, and North Africa, including as teacher, item writer, examiner, examiner trainer, and teacher trainer. He has trained numerous professionals to produce content for diverse assessment contexts and has delivered workshops and presentations on language assessment literacy, statistical analyses for testers, and item writing. He has an MA in Language Testing from Lancaster University and is working towards a PhD in assessing pragmatics in online speaking tests.

To view the IATEFL event terms and conditions please click here