RINA - Démonstrateur d’utilisation de l’intelligence artificielle pour la gestion opérationnelle des risques naturels gravitaires géologiques | FEREC 2020

22 CHAPTER 2. TOWARD FORMALISATION • w i,j ∈ R is the connection weight between this hidden neurons and the i-th neuron from previous layer (input), • b j ∈ R is the bias , • classically h j is followed by an relu activation to to model non linearity: relu(x)=max(0,x). In a matrix way, hidden layer H = ( H j ) ∈ R M is given by: H = W H X + B H where W H ∈ R N × M and B H ∈ R M are respectively connection weights and biases. If W last ∈ R M × K and B last ∈ R K denote weights and biases of last dense layer, output vector is W last · relu ( H ) + B last (with relu point-wise extended). Classically, this last layer is followed by a sof tmax ( . ) activation define from R K to simplex [0 , 1] K by: sof tmax ( c i ) = exp ( c i ) P K i =1 exp ( c i ) that transform output vector to probability vector ˆ Y on the K classes ( ∀ 1 ≤ i ≤ K, ˆ Y i ∈ [0 , 1] and P K i =1 ˆ Y i = 1 ), This give the Forward Propagation from input X to ˆ Y : ˆ Y = sof tmax ( W last · relu ( WX + B ) + B last ) The predicted class is finaly the maximum argument of this output vector argmax 1 ≤ i<K ( Y i ) . In that case, trainable parameters are θ = [ W H , B H , W last , B last ] , and the mapping between inputs and label is the parametric non linear function ˆ Y = f θ ( X ) Example. As example, if model compute ˆ Y = [0 . 2 , 0 . 1 , 0 . 7] then predicted class is the last class "3". 2.3.3 Backward Propagation: from loss function to weights update Given a labelled data sample ( X, Y ) , to modify appropriately model weights θ , one should measure errors between prediction ˆ Y and true label Y , through a loss function L ( Y, ˆ Y ) . As ˆ Y = f θ ( X ) , for a given input X and Y is known, loss became a function of weights θ . As we mention before, an simple gradient descent (like in optimizer SGD) θ n +1 = θ n − lr ∇ L θ permit to decrease iteratively loss function. For classification task in our tested networks , we used the classical Cross-Entropy for the loss function. L ( Y, ˆ Y ) = L θ = − K X i =0 Y i log ˆ Y i = − Y T · log( ˆ Y )