Automatic Acquisition of Knowledge for Constraint-based TutorsAcquiring the domain knowledge is a task that requires a major portion of the time and effort when building an ITS. Researchers have been exploring ways of automating the knowledge acquisition process since the inception of ITSs with limited success. All past research attempts have focussed on acquiring knowledge for procedural domains. Our goal is to develop an authoring system that acquires knowledge for procedural as well as non-procedural domains. We propose a four phase approach: composing an ontology of the domain, extracting syntax constraint from it, learning semantic constraints from the examples provided by the domain expert and finally verifying the generated constraints.
We propose a four-stage process to infer constraints automatically.
- Intially the the domain expert composes an ontology of the instructional domain.
An ontology contains a lot of information about the domain and it is much easier to create than the final domain model. Ontology defines the concepts of the domain and the relationships between them. Each concept has a set of attributes that describe it. The range of the attributes specified in terms of minimum and maximum values or a set of distinct values can be directly translated to constraints. Furthermore, the minimum and maximum number of instances that can participate in a relationship (cardinality) can also be translated directly to constraints.
- At the completion of the ontology, the system analyses the ontology and extracts syntax constraints directly from it.
The authoring system analyses the ontology composed by the domain expert and generates constraints during the second phase. Since all the restrictions specified in the ontology deal with the syntax of the domain, the generated constraints are also syntactic in nature. As an example consider an ontology for the domain of punctuation in the English language. The ontology would contain a ‘sentence’ concept and a ‘period’ concept. The ‘sentence’ concept would be involved in a relationship with the ‘period’ concept with a minimum and maximum cardinality of 1. This translates directly to a constraint that specifies that a ‘sentence’ must have exactly one ‘period’.
- The third phase involves learning from examples. During this phase the system generates constraints by identifying commonalities between solutions provided by the domain expert.
The domain expert has to specify the representation for solutions prior to entering problems and sample solutions. The solution representation is a decomposition of the solution into components consisting of a list of instances of concepts. For example, a sentence in English consists of a list of words and a list of punctuation marks.
The domain expert enters problems and solutions in the next phase which uses learning from examples techniques to induce semantic constraints. After a solution is specified by the author, the system evaluates it against its collection of automatically generated syntax constraints. When a discrepancy is identified the expert is alerted and they may chose to alter the solution or alter the ontology. The expert is encouraged to enumerate all correct solutions to demonstrate different ways of solving a problem. While the expert enters in alternative correct solution, the system attempts to match each component of the solution to components of the initial solution. These matches are later used to compose a set of semantic constraints that compare the student’s solution against the system’s ideal solution. The expert is also called upon to provide typical erroneous solutions for the system to generate semantic constraints that identify typical student errors.
- Finally the constraint set is validated with the assistance of the domain expert. The system would generate examples to be labelled as correct or incorrect by the expert.
The validation phase involves ensuring the correctness of the generated constraints. The expert can go through the generated constraint set and ensure that it does not contain an redundant or erroneous ones. The expert may either directly modify erroneous constraints or provide new examples to illustrate the reasoning for disputing the constraint. The expert may also opt to validate the constraints by labelling system generated examples as correct or incorrect.
The architecture of the knowledge acquisition system for CBM consists of an ontology workspace, ontology checker, problem/solution manager and syntax and semantic constraint generators.