RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

CNN actual 0 1 predicted 0 805 80 1 133 62 Figure 36 : confusion matrix for DNN model using 3 classes adapted to 2 classes CNN Operational accuracy 0.80 0.84 balanced accuracy 0.65 0.60 precision 0.32 0.35 recall 0.44 0.27 F1 0.37 0.30 Table 9: indices for the CNN model and the Operational model 5.5. Other models As described in the methodology Random forest classifier and Histogram gradient boosting classifier were also calculated. The same processo of optimization and analysis of the confusion matrix were done. Those results are present on the annex of this document. The random forest model had a similar result to the bagging classifier with a slightly worse result for the test set, but superior for the training set indicating overfitting. The parameters for the generation of this model were: 5.6. Discussion The tested models return varied performances [Table 10]. Generally there is a trade off of some characteristics in order to improve in others. The general tendency is that with stronger rains there is a greater chance that an event will happen. The different models find this general rule. Some will be safer for drivers meaning a smaller amount of rain will trigger the model. This is reflected by a high recall. An example are the NN models which tend to favor security. On the other hand they also have a lower precision meaning the road will be closed more frequently. The most economically interesting models are those with the highest precision for which the models will predict a rock will fall with a greater accuracy avoiding closures on days an event is not present. An example of this are the nearest neighbors model and the decision tree model. Both of those have the highest precision but the smallest recall. The NN models were tuned to give great importance to the safety (not tuned, but data-driven with unbalanced dataset) while the nearest neighbors and simple decision tree gave 44