Algorithms Used Clause Samples

Algorithms Used. We use Adaboost to create the original ensemble for pruning. Following ▇▇▇▇▇ et al. (2006) the ensemble size was set to one hundred. If the training error converges to zero before the ensemble sizes reaches one hundred, then all the subsequent members will be replications of the last member because no instance weight is changed. Therefore in such cases, the AdaBoost.M1 procedure is repeated using a different random seed. Note that the original AdaBoost weights are used to weight the predictions of the selected members. As for the induction algorithm that was used for training the base classifier, we have examined the C4.5 decision tree algorithm (▇▇▇▇▇▇▇, 1993) and decision ▇▇▇▇▇. The C4.5 algorithm is frequently used for comparing ensembles. Decision ▇▇▇▇▇ is a weak learner consisting of a one-level tree which known to be benefit from boosting strategy (▇▇▇▇▇▇▇▇▇▇ et al., 2006). The new algorithm is compared to the following ensemble pruning methods: 1. GASEN-b - genetic-algorithm (number of generation=200 and population size=50). The GASEN employs a wrapper evaluator, in which a candidate subset is evaluated by repeatedly sampling the training set and measuring the accuracy of the subset ensemble over a holdout validation dataset. 2. Kappa members ranking (as used by ▇▇▇▇▇▇▇▇▇▇▇ and ▇▇▇▇▇▇▇▇▇▇, 1997). Kappa members ranking can prune the ensemble to any pre-set size. Thus, in order to make a fair comparison, we set it to the same size obtained by our algorithm which makes. We also evaluated the following configurations of the proposed approach: 3. CAP-F-K – Using Forward-Selection search strategy with kappa statistics as the agreement measure. 4. CAP-BF-K – Using Best First search strategy with kappa statistics as the agreement measure. 5. CAP-GA-K – Using genetic algorithm search strategy (number of generation=200 and population size=50) with kappa statistics as the agreement measure. 6. CAP-BF-SU – Using Best First search strategy with symmetrical uncertainty as the agreement measure. 7. CAP-GA-SU – Using genetic algorithm search strategy (number of generations=200 and population size=50) with symmetrical uncertainty as the agreement measure.