Random Forest. The literature review performed before this pilot project identified a method that used Random Forest classification on LIDAR metrics to predict the presence of snags in various size classes (Martinuzzi, et al., 2009). This method was attempted to see how results compared to the presence/absence logistic regression model used above, with the results presented in Table 14 and Table 15. The size classes identified by the researchers in that paper were tested, but only two of those classes are reported here, the ≥ 6” class (15cm) and the ≥ 10” class (25cm). The ≥ 6” class, is similar to the 5” minimum diameter cutoff used in our model. The ≥ 10” class is reported because it has approximately equal numbers of plots with and without snags. For small size classes, nearly all plots had snags, while for large size classes, almost no plots had snags, making the presence/absence classes unbalanced. Random Forest tends to work better with balanced presence/absence classes. It is expected that a model predicting all absence or all presence, would be very accurate, but have limited utility, in that it only provides information that is already known. The ≥ 10” class provides the most realistic and useful model results.
Random Forest. Random forest is an adaptation of the decision tree ensemble technique. It is often seen as an improvement of the bootstrap aggregation or bagging method. It works the same way as bagging, as it also takes multiple subsets out of the training data set, runs the decision tree algorithm on each subset and then average all results to get the final prediction. For each observation, the predicted class by each of the trees is recorded and the most common class is chosen for that observation. This is called the majority vote. Using bagging, the variance will be reduced from 𝜎2 to 𝜎2 (with 𝑛 the number of trees) compared to the decision tree assuming independent and identically distributed data. However, as the trees use similar data, they will actually be correlated leading to a higher variance than the formula mentioned. Furthermore, the correlation will also increase bias. (Xxxxxx et al., 2008). Random forest allows to improve both variance and bias compared to the regular bagging method . It is able to decrease the correlation between trees by randomly taking m out of the p predictors at each split. The value of m is often √𝑝 . This is done to inhibit that the same very strong predictor is chosen at each split. Averaging many less correlated trees will reduce variance much more than averaging many highly correlated trees. The bias will decrease thanks to the lower correlation as well. In summation, the random forest model allows to drastically reduce the variance of the decision tree, while only receiving a minimal increase in bias. Thus, the overall performance is improved. The biggest disadvantage is the huge decrease in interpretability, as it is difficult to visualize for a huge number of trees (Xxxxx et al., 2017). For this classifier, it will also be investigated whether scaling the features has any impact on the prediction performance. It is true that RF is based on tree partitioning algorithms, where a collection of partition rules is obtained which should not change with scaling (the trees thus only see ranks in the features). However, RF will tend to favour highly variable continuous predictors to split, since there are more opportunities to partition the data (even if only a subset of the variables is used in each individual tree). This leads to some highly variable features to get an unjustified large importance. Since we might want to take a look at the importance of each individual feature in the prediction, it was decided to try scaling (St...
Random Forest. Random Forest (RF) is a well-known ensemble learning method consisting of a number of decision trees [6]. Decision trees consist of combinations of Boolean decisions on a different random subset of attributes of input data (called bootstrap sampling). For each node of each tree, the best split is taken among these randomly chosen attributes. RF is a stochastic algorithm because of its two sources of randomness: bootstrap sampling and attribute selection at node splitting. The most important hyper-parameter to tune is the number of trees in the forest (we do not limit the tree size nor use pruning methods).
Random Forest. Random Forest is an ensemble, supervised machine learning method used for classification. It constructs a multitude of decision trees at training time and outputs the mode of the classes (the most repeated value) of the individual trees as the final class [6]. Essentially, each tree’s prediction is counted as a vote for one class and the final label is predicted to be the class which receives the most votes (majority vote) (Figure 13). The algorithm applies the general technique of bootstrap aggregation (or bagging) to tree learners, leading to a better performance model by decreasing the variance, without increasing the bias. Random forest is considered one of the best-performing ML algorithms, mainly because of its ability to remove decision trees' habit of overfitting the training set (being too much dependent on the training set and not performing so well in the testing set) and of its excellent classification accuracy compared to current algorithms [7]. In the case of network traffic classification, the datasets are usually unbalanced since the majority class (normal traffic) is usually orders of magnitude higher than the minority classes (attack flows). Therefore, classifiers are overwhelmed by the dominating class and tend to ignore the flows related to malicious activity. Random forest is of no exception, thus techniques like cost-sensitive learning and oversampling of the minority class are leveraged to tackle this issue.
