Lecture: AI-assisted short answer grading
We develop an open-source system for automated grading of short free texts. The system is being integrated with such tools as Jupyter Notebook and nbgrader. It can be used at schools and universities.
Assignment evaluation is one of the most time-consuming parts of teacher's work. Therefore, nowadays many assignment types, such as multiple-choice answers or fill-the-gap, are automated. Free text answer grading is a harder task. There are two types of free text assignments: automated essay grading (AES) and short answer grading (ASAG). AES systems are already successfully used by some organizations including Educational Testing Service (ETS). However, ASAG is still not popular, because the existing systems haven't shown sufficient robustness yet. This research concentrates on development of an AI assistant for grading assessments and exams in the fields of computer science, electrical engineering, physics and other technical disciplines. In this case answers are shorter than one paragraph and concrete. Furthermore, stylistics and spelling are not of interest, only the meaning of the answer should be taken into account. This study uses similarity between the students' and teachers' answers for grading. Such sentence similarity measures as BLEU, ROUGE and various k-skip-n-gram distances are considered. The system will be integrated with Jupyter Notebook and nbgrader.