RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

model for the training and test sets. The indice considered to measure the performance of the model is the balanced accuracy. The number of neighbors is varied between 1 and 50 and the result is presented by Figure 25. Figure 25: Balanced accuracy of the models for training and testing data and its variation with the number of neighbors. With one neighbour we have the maximum performance of the training data but the difference between both data sets is significant indicating that the model is overfitting. As the number of neighbors decreases the performance of the training data also decreases, becoming closer to the one of the test data. The balanced accuracy for the test data set also decreases with a larger number of neighbors, but much slower maintaining it’s value approximately 0.6. The value of 15 is adopted as the model does not overfit. Comparing figure 24 and figure 26 we observe that the Operational code is capable of correctly predicting 8 additional rock falls, but at the cost of incorrectly predicting 77 additional events.(indiquer les conséquences pour la gestion de la route) By comparing the indices calculated for this model and the operational model it is noticed that the accuracy has increased and is even slightly superior to the dummy classifier, which has an accuracy of 0.866. The precision also increased being superior to 0.5, meaning the majority of the times the model predicts a rock will fall it actually happened. A higher precision is important as less false predictions will happen leading to less days the road would be closed without any risk to drivers. On the other hand, the accuracy and the recall have fallen. This means that the operational system is safer as it predicts more events than this first model. 34