RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

2.3. NEURAL NETWORKS (NNS) 21 train dataset. Starting from model configuration θ n , we can learn from one sample of labeled data ( X, Y ) better weights θ n +1 : • a Forward Pass : given the input X and the actual model configuration θ n , a prediction ˆ Y = f θ n ( X ) is compute, • a Backward Pass : given the label Y , the loss L θ and its gradient ∇ L θ with respect to all trainable parameters θ are computed. Model configuration θ n is upgraded, in the opposite direction of gradient to lower loss function (therefore errors between prediction and labels) : θ n +1 = θ n − lr ∇ L θ where lr , the learning rate is a scaling factor, tuned during learning. These two points, forming a learning step, are repeated over the entire dataset and are called a training epoch. After one epoch, model capabilities are evaluated on the validation dataset. Depending on the results, some hyper-parameters could be adjusted like the learning rate. To learn correctly, we repeat many epochs to reach a good predictive model. There are many parameters that can be optimized for each problem like the structures for the intermediate layers, changing the activation function, the error function and also the learning rate. 2.3.2 Forward Propagation: from input to predictions To better understand their structure and how they works, we will illustrate it through a simple Dense Neural Network (DNN). Besides, this example will be tested on RN1 Réunion data in next chapter. Definition 4. DNNs are composed by different layers of many neurons, all the neurons from a layer being connected to all neurons from the previous and following layer. Example. Let’s formalise a simple classification DNN with one hidden layer with M neurons and one last layer with K neurons the number of classes (see example on Figure 2.1 and scheme of one neuron). For inputs X = ( x i ) ∈ R N , the j-th neuron h j from dense hidden layer integrate in a linear way all neurons ( x i ) from previous layer: h j = N X i =1 w i,j x i + b i where: