Specification-Based Oracles
"Specification-based Test Oracles for Reactive Systems",
by D. Richardson, S. Aha and O. O'Malley
in Proceedings of the Fourteenth International Conference on
Software Engineering
Abstract
The testing process is typically systematic in test data selection and
test execution. For the most part, however, the effective use of
test oracles has been neglected, even though they are a critical
component of the testing process. Test oracles prescribe acceptable
behavior for test execution. In the absence of judging test results
with oracles, testing does not achieve its goal of revealing failures
or assuring correct behavior in a practical manner; manual result checking
is neither reliable nor cost-effective.
We argue that test oracles should be derived from specifications and
in conjunction with
testing criteria, represented in a common form, and their
use made integral to the testing process. For complex, reactive
systems, oracles must reflect the multiparadigm nature of the
required behavior. Such systems are often specified using multiple
languages, each selected for its utility in specifying a particular
computational paradigm. Thus, we are developing an approach for
deriving and using oracles based on multiparadigm and multilingual
specifications to enable the verification of test results for reactive
systems as well as less complex systems.
Top: Owen O'Malley