A MOF-Based Framework for Defining Metrics to Measure the Quality of Models

Abstract

Controlled experiments in model-based software engineering, especially those involving human subjects performing modeling tasks, often require comparing models produced by experiment subjects with reference models, which are considered to be correct and complete. The purpose of such comparison is to assess the quality of models produced by experiment subjects so that experiment hypotheses can be accepted or rejected. The quality of models is typically measured quantitatively based on metrics. Manually defining such metrics for a rich modeling language is often cumbersome and error-prone. It can also result in metrics that do not systematically consider relevant details and in turn may produce biased results. In this paper, we present a framework to automatically generate quality metrics for MOF-based metamodels, which in turn can be used to measure the quality of models (instances of the MOF-based metamodels). This framework was evaluated by comparing its results with manually derived quality metrics for UML class and sequence diagrams and it has been used to derive metrics for measuring the quality of UML state machine diagrams. Results show that it is more efficient and systematic to define quality metrics with the framework than doing it manually.