RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

Figure 23: example of a confusion matrix with 2 classes: positive and negative source: https://towardsdatascience.com/visual-guide-to-the-confusion-matrix-bb63730c8eba The most classic value that measures the performance of a classification is the accuracy (eq?). It calculates the percentage of results that have been correctly classified. The main disadvantage of this method is when the classes are not equally present. For example, if 80% of the data were on class A a dummy classifier would always predict A and the accuracy of this model is 80%. This example shows the importance of comparing different indices to better understand the performance of a model. An alternative is the balanced accuracy (eq?(Pb reference). It is not as intuitive as the accuracy, but the most important is that as the model better identifies the classes the value grows closer to one, and a dummy classifier will always score 0.5. Two important values are the precision (eq?) and the recall (eq?). The precision is the probability that the event occurred when the model predicts it will happen. The Recall is the probability that the model predicted an event given it occurred. On operational terms for road management, augmenting the recall augments safety, as more rock falls will be predicted, and enhancing the precision will be economically important, as it will reduce the days the road is closed without any occurence of rock falls. It is important to have those concepts in mind as enhancing one will generally diminish the other. = + = + = (recall + ?)/2 = + + + ( ) /2 An interesting indice to represent the performance of a classification model is the β (eq?). It is based on the precision and recall explained earlier and the analyst can tune the value 31