Automatic evaluation of examination tasks in the form of function plot

 Abstract -

In this paper the approach to automatic exam sheets evaluation is presented. In our task the exam is written on paper forms and scanned. The bitmaps are processed to extract the specific task area and then the task is evaluated with aid of image processing and analysis. In this work we present the algorithm for automatic evaluation of function plot. The algorithm has been verified against the set of 47 sheets. Simultaneously the works have been graded be a qualified teacher. Keywords – e-evaluation, image analysis.

INTRODUCTION

 E-evaluation is a new idea in the field of didactics which aim is to enable automatic marking of the exams with aid of artificial intelligence and computer recognition systems, especially OCR and image understanding. E-evaluation software is a system which enables evaluation of examination answer sheets by the examiner on a computer screen rather than reading paper documents. In our previous paper we worked out the requirements for such a system . The advantage of the e-evaluation system is better organization of the examination session, because it performs segmentation of the answer sheets into individual tasks, distributes the tasks to selected examiners, specialized in their domain (for instance, physics), evaluates of the tasks and sends them back to the examination board and verifies the evaluation, gathers statistics, and presents the results. The e-evaluation method has been widely introduced in Great Britain and the USA.

The experience gained by Examination Boards like AQA, OCR and EDEXCEL in Great Britain and ETS in the USA suggests that introducing e-evaluation improves the quality and reliability of the exams. Studying the impact of changes in the process of evaluation to its quality has shown that if only this change does not involve exam preparation, it can keep its original accuracy and increase its reliability. For example, Williams and van Lent claim that introducing e-evaluation can improve exam quality by:

1. providing complete anonymity for task assessment;
2. suppress the halo effect – a way of solving by one student does not affect the way later tasks are evaluated;
3. steadily spreading potential errors in assessing – tasks to assess to the examiners are assigned at random. 

            Since 2007 in Poland the project „E-EVALUATION” is Tasks that appear in examination papers may be formulated as “closed” or “open” [6]. A closed task is either a single or multiple choice test, or it consists of filling gaps with specific words or numbers.

One of the incorrect answers presented as a choice in a multiple-choice test is known as distracter and the correct answer is called verstracter. Computer-aided evaluation of such a task is easy. In case of electronic form it is simply comparing the result to the key answer. In case of a paper form, the scanned sheet is processed to recognize some marks (shapes "X" or "V") put by a student on a form. Much more difficult is automatic evaluation of the open task, because the set of proper solutions may be infinite. Moreover, incomplete or inaccurate solutions should be graded with a reduced score. One of the problems that has been discussed in the literature is recognition of the mathematical formulae. Most often pattern recognition methods are utilized for this purpose.

The survey over the algorithms is presented in [2]. There exist only few commercial solutions of this problem. One of these systems is xMath Journal for TabletPC (with on-line recognition feature) which is a kind of scientific calculator. Most of the algorithms for mathematical formulae recognition work in two steps:

1. detection of individual symbols by image segmentation and object indexation with use of pattern recognition methods;

2. structural analysis of relations (for example spatial) between the selected segments. A classical approach known from OCR is used for recognition of some symbols. Initially the segmentation is performed and the classifier assigns the symbols to individual objects. In some publications the hidden Markov models method is applied to reduction of processing steps. The structural analysis of the expression (and the correction in case of recognition errors) is feasible by application of two dimensional context-free grammar

The type of the open task that we take into account in this paper is drawing a plot of a given math function. The type of task often appears in final high-school or maturity exams of mathematics. The student should draw the plot in a printed coordinate system. Because the plot is drawn manually, the teacher checks if the drawn curve crosses some characteristic lattice points. In a case of complex plot (few curves or lines) the curve may be discontinuous and the student should mark the endings of the segments.