Test Generation. We extended a popular random test generator, Randoop [Xxxxxxx and Xxxxx 2007], to collect only relevant tests. Note that whether a test is relevant is decided at runtime while Randoop is generating tests. In our initial experiment, it took too long (almost five minutes in some instances) for Randoop to start generating relevant tests. We made a couple of simple changes to Randoop to alleviate the problem. First, our test generator selects the seed method with a 50% chance from specified target methods, unlike the original Randoop that selects the seed method from all legal methods that are in the scope of the tool. As target methods, we used either: (i) the target method m of a change contract if m is public or (ii) public callers of m if m is not public. Such target method specification can be automated with the help of static analysis. The reason for assigning a 50% chance to the target methods (as opposed to assigning 100% chance) is that, otherwise, Randoop does not consider other method calls that may be necessary for constituting a relevant test. The second change we made to Randoop is to address the following problem we found in our initial experiments. It took particularly long for Randoop to generate relevant tests in a case where the update condition of a change contract is satisfied only if void- type methods are called to change the program state properly before the target method is called. For example, if a target method is m2(int i), then the unmodified Randoop opts for generating a sequence that ends with “m2(var2);” preceded by a sequence of statements that ends with a statement to assign a value to variable var2, such as “var2=m1(var1);”. This statement is again preceded by another statement to assign a value to var1. Such a style of Xxxxxxx’s sequence generation tends to exclude void-type method calls in the middle of a sequence. To address the previous issue, we intersperse a statement sequence with random void-type method calls. We also transform statements like “var1.m1(); var2.m2();” into “var2.m1(); var2.m2();” to merge the receivers. We let such a transformation take place with an 80% chance in our experiments. Note that generally there is no guarantee that executing a relevant test in the updated system will execute the target method with isomorphic input because only the previous version was considered when constructing relevant tests. Obviously, by considering the updated system as well, this problem can be avoided in exchange fo...
Test Generation. Whereas the theory of testing equivalences and preorders is used to define semantic relations over LTS using all possible tests, actual testing turns this around: given an LTS s (the specification) and a relation imp over LTS (the implementation relation), determine a (minimal) set of tests Timp(s) that characterizes all implementations i with i imp s. First steps towards systematically generating such test suites (sets of tests) from a specification LTS were made by Xxxxxxxx et al. in [11, 12, 17] leading to the canonical tester theory for the implementation relation conf . The intuition of conf is that after traces, i.e., sequences of actions, that are explicitly specified in the specification LTS, the implementation LTS shall not unexpectedly refuse actions, i.e., the implementation may only refuse a set of actions if the specifi- cation can refuse this set, too. This introduces under-specification, in two ways. First, after traces that are not in the specification LTS, anything is allowed in the implementation. Second, the implementation may refuse less than the spec- ification. In this approach, models were represented using the process algebraic specification language LOTOS [7, 16, 42].
