A Heuristic for General Rule Extraction From a Multilayer Perceprtron

Rule extraction from Artificial Neural Networks (ANN's) is an essential step towards the integration of ANN's and Knowledge-based Systems (KBS's). Two central questions addressed in this paper are what is a suitable format embodying ANN knowledge correctly and efficiently; and how is the knowledge extracted. A General Rule is defined in an efficient format to represent the knowledge from ANN's. General Rules are extracted from a trained Multilayer Perceptron (MLP). The inputs of the MLP correspond to the premises and the outputs , to the conclusion of the rules. Two criteria are used to ascertain the significance of input components for the construction of rules. The first criterion, the Potential Default Set (PDS) is drawn up from the weighted connections combined with the input/output correspondence of a training pattern. A subset of the inputs in the training pattern, which is possibly redundant, is defined as the PDS. The second criterion, the Feature Salient Degree (FSD) embodies the casual bond of the changes on each input bit and on each output bit. The system using both PDS and FSD is demonstrated by application to typical logic problems such as AND, OR,XOR and by interpretation for unknown data. Clinical data have also been used to assess the performance of the method in the real-world. The rules derived here have been evaluated by a domain expert and are found to conform with his view of the problem. The computational complexity is a third order polynomial of the problem size.