Evaluating the quality of the ontology-based auto-generated questions
Demaidi, Mona and Gaber, Mohamed Medhat and Filer, Nick (2017) Evaluating the quality of the ontology-based auto-generated questions. Smart Learning Environments, 4 (7). ISSN 2196-7091
|
Text
Evaluating the Quality.pdf - Published Version Available under License Creative Commons Attribution. Download (614kB) |
Abstract
An ontology is a knowledge representation structure which has been used in Virtual Learning Environments (VLEs) to describe educational courses by capturing the concepts and the relationships between them. Several ontology-based question generators used ontologies to auto-generate questions, which aimed to assess students' at different levels in Bloom's taxonomy. However, the evaluation of the questions was confined to measuring the qualitative satisfaction of domain experts and students. None of the question generators tested the questions on students and analysed the quality of the auto-generated
questions by examining the question's difficulty, and the question's ability to discriminate between high ability and low ability students. The lack of quantitative analysis resulted in having no evidence on the quality of questions,
and how the quality is a�affected by the ontology-based generation strategies, and the level of question in Bloom's taxonomy (determined by the question's stem templates). This paper presents an experiment carried out to address the drawbacks mentioned above by achieving two objectives. First, it assesses the auto-generated questions' difficulty, discrimination, and reliability using two
statistical methods: Classical Test Theory (CTT) and Item Response Theory (IRT). Second, it studies the effect of the ontology-based generation strategies and the level of the questions in Bloom's taxonomy on the quality of the questions. This will provide guidance for developers and researchers working in the field of ontology-based question generators, and help building a prediction
model using machine learning techniques.
| Item Type: | Article | ||||||
|---|---|---|---|---|---|---|---|
| Identification Number: | https://doi.org/10.1186/s40561-017-0046-6 | ||||||
| Dates: |
|
||||||
| Uncontrolled Keywords: | ontology; ontology-based question generator; ontology-based generation strategy; Bloom's taxonomy; Classical Test Theory; question difficulty index; question discrimination index; reliability; discrimination power; Item Response Theory | ||||||
| Subjects: | CAH11 - computing > CAH11-01 - computing > CAH11-01-01 - computer science | ||||||
| Divisions: | Faculty of Computing, Engineering and the Built Environment Faculty of Computing, Engineering and the Built Environment > School of Computing and Digital Technology |
||||||
| Depositing User: | Ian Mcdonald | ||||||
| Date Deposited: | 13 Nov 2017 13:16 | ||||||
| Last Modified: | 22 Mar 2023 12:01 | ||||||
| URI: | https://www.open-access.bcu.ac.uk/id/eprint/5304 |
Actions (login required)
 |
View Item |
Tools
Tools