Introduction to Random Forest Machine Learning

Multiple recent studies have used random forest machine learning for classifying wetlands (Liu et al. 2011; Millard & Richardson, 2013; Corcoran et al. 2015; Franklin & Ahmed, 2017; Halabisky et al. 2017; Amani et al. 2019; Bourgeau-Chavez et al. 2017;  Mahdianpari et al. 2019, Merchant et al. 2019). It is simpler than some other classifying machine learning techniques and still provides high accuracy. Other benefits of machine learning are that it reduces the risk of overfitting that comes from using single decision trees and can maintain accuracy when a large portion of data is missing.

What exactly is machine learning?

El Bouchefry & de Souza (2020) and this website provide a good introduction to machine learning and the various types. To summarize, machine learning is a type of artificial intelligence that learns from doing and can improve its results this way. It uses algorithms (a series of decisions made by a computer or a set of rules followed to come to a decision) to build a model that can recognize patterns and classify high-dimensional data. There are two types of machine learning techniques: supervised and unsupervised. Supervised means the algorithm is shown data prior to making predictions and can classify new data based on the previous data. Unsupervised means that it is making predictions without a training data set. In the case of wetland classification where there are pre-determined wetland classes, supervised is the preferred method. This requires a training dataset, which may have to be collected from the field before hand. In their review of wetland remote sensing, both Ozesmi and Bauer (2002) and Guo et al. (2017) provide a good overview of supervised versus unsupervised classification methods.

Figure taken from: https://www.section.io/engineering-education/introduction-to-random-forest-in-machine-learning/

What is Random Forest?

As many decision trees as you want. Random forest (RF) machine learning is a supervised learning algorithm that uses multiple decision trees and then takes the majority vote for the final output. When using decision trees you have to be cautious not to overfit your model with too many parameters; however, if there are enough trees using the random forest technique, this is less likely to happen. If you have too many trees, the model will be too slow and ineffective. This post has some good info about what a random forest is.

Figure taken from: https://www.analyticsvidhya.com/blog/2020/05/decision-tree-vs-random-forest-algorithm/

You can modify various hyperparameters to increase your predictive power. Hyperparameters include the number of trees, the maximum number of features, the minimum number of leaves required to split an internal node, the number of processors RF is allowed to use, the randomness of the model (to make it replicable or not), and whether or not you want to cross validate on your dataset. One great feature of Random Forests is that you can get an output showing feature importance, which tells you how much each feature (or parameter) reduces the impurity of your data across all trees – basically, how effective your parameter is at classifying your data. Using this you can fine tune your model to eliminate useless features or parameters that might cause overfitting.

There are a few common terms that come up when using RF modelling that I will define now. First is the Gini Index, which is a way to calculate how good each node classifies (or splits) the data. Second is bagging (bootstrapped aggregation). RF is a type of bagging algorithm that reduce complexity of models that overfit the training data. Several bootstrap subsets of data are used to train the model and the remaining data is referred to the out-of-bag (OOB) dataset. The OOB dataset is used by the model to evaluate its own performance. Specifically for RF, random subsets of features are also used in addition to the bootstrap samples for training the individual trees. This allows each tree to be more independent, which improves predictive performance. This is compared to just bagging, where each tree is provided with the full set of features. Note that in traditional bagging, the trees are not independent of each other since all the original predictors are considered at every split of every tree. Underlying relationships cause the trees to be very similar to one another. Random forest eliminates the similarities but introducing randomness into the sampling.

A good run through of random forest can be found here: Random Forests · UC Business Analytics R Programming Guide (uc-r.github.io)

Classification Accuracy

Supervised image classification methods require a training data set to distinguish between classes. It is important that each class is represented sufficiently in the training dataset. If one or more untrained classes exist on the landscape, the accuracy of a classification or resulting map may be compromised. G.M. Foody (2021) explores this phenomenon and the hazards of classification accuracy in greater detail in his paper Impacts of ignorance on the accuracy of image classification and thematic mapping.

…more to come in next post: random forest modelling in R

References