RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

Decision tree model Operational Train Test accuracy 0.85 0.86 0.84 balanced accuracy 0.63 0.59 0.60 precision 0.73 0.48 0.35 recall 0.28 0.22 0.27 F1 0.40 0.30 0.30 Table 4: Indices calculated based on the confusion matrix to quantify performance of the decision tree model, for the test and train set, and the operation model. The results for the train set are superior to the one for the test set. This is normal as the model performs better on the data it was trained on. Both models have similar accuracies and balanced accuracies, but the training data is much more precise than the test set leading to a superior score. This happens even though a small depth was adopted. Possibly an even β=1 smaller depth could be adopted or even a larger number of samples per leaf, but this result is considered satisfactory. Similar to the nearest neighbors this model has a smaller balanced accuracy as it predicts less rock falls than the operational model, but it also has a higher precision, meaning that when it predicts that there will be a rock fall there is a higher chance it will be true. In order to increase the safety on the road the importance of predicting rock falls should be increased. This means that the model will favor predicting rock falls even if it will have a negative impact on the precision leading to more days the road is closed having a negative economic impact and becoming an inconvenience to drivers. So, it is proposed to weight the predictions and it will be adopted that the weight of predicting an event be 3 times greater than the ladder. The heat maps are re-calculated with the adjusted weights. 37