神经网络公式推导

2023-05-16

1.规定

y i j y_{ij} yij为第 i i i层网络第 j j j个神经元的输出.
t i t_i ti为输出层第 i i i个输出.
n i n_i ni为第 i i i层网络的神经元数量.
激活函数 σ ( x ) = S i g m o d ( x ) = 1 1 + e − x \sigma(x)=Sigmod(x)=\frac{1}{1+e^{-x}} σ(x)=Sigmod(x)=1+e−x1，因此 ∂ σ ( x ) ∂ x = σ ( x ) [ 1 − σ ( x ) ] \frac{\partial \sigma(x)}{\partial x}=\sigma(x)[1-\sigma(x)] ∂x∂σ(x)=σ(x)[1−σ(x)].
E E E代表误差，即 E = ∑ i = 1 2 ( y 3 i − t i ) 2 E=\sum_{i=1}^{2}(y_{3i}-t_i)^{2} E=∑i=12(y3i−ti)2.
∇ i j k \nabla_{ijk} ∇ijk为第 i i i层网络第 j j j个神经元，它对上一层网络的第 k k k个神经元的 ω \omega ω的梯度值.

2.公式推导

现在以 11 → 21 → 31 11\rarr 21 \rarr 31 11→21→31该过程为例，同时导出通项公式。

n e t i j net_{ij} netij

n e t 21 = ∑ i = 1 2 ( ω 21 i y 1 i ) net_{21}=\sum_{i=1}^{2}(\omega_{21i}y_{1i}) net21=∑i=12(ω21iy1i).

n e t 31 = ∑ i = 1 3 ( ω 31 i y 2 i ) net_{31}=\sum_{i=1}^{3}(\omega_{31i}y_{2i}) net31=∑i=13(ω31iy2i).

因此通项公式为： n e t i j = ∑ k = 1 n i − 1 ( ω i j k y i − 1 , k ) net_{ij}=\sum_{k=1}^{n_{i-1}}(\omega_{ijk}y_{i-1,k}) netij=∑k=1ni−1(ωijkyi−1,k), y i j = σ ( n e t i j ) y_{ij}=\sigma(net_{ij}) yij=σ(netij).

3. y i j y_{ij} yij

y 21 = σ ( n e t 21 ) y_{21}=\sigma(net_{21}) y21=σ(net21).

y 31 = σ ( n e t 31 ) y_{31}=\sigma(net_{31}) y31=σ(net31).

因此通项公式为： y i j = σ ( n e t i j ) y_{ij}=\sigma(net_{ij}) yij=σ(netij).

4.误差 E E E

E = ∑ i = 1 2 ( y 3 i − t i ) 2 E=\sum_{i=1}^{2}(y_{3i}-t_{i})^2 E=∑i=12(y3i−ti)2.

5. ∇ 3 i j \nabla_{3ij} ∇3ij（输出层各 ω \omega ω的梯度值）

∂ E ∂ n e t 31 = E ∂ y 31 ⋅ ∂ y 31 ∂ n e t 31 = 2 ( y 31 − t 1 ) y 31 ( 1 − y 31 ) \frac{\partial E}{\partial net_{31}}=\frac{E}{\partial y_{31}}\cdot \frac{\partial y_{31}}{\partial {net_{31}}}=2(y_{31}-t_1)y_{31}(1-y_{31}) ∂net31∂E=∂y31E⋅∂net31∂y31=2(y31−t1)y31(1−y31).

∴ E ∂ n e t 3 i = 2 ( y 3 i − t i ) y 3 i ( 1 − y 3 i ) \therefore \frac{E}{\partial net_{3i}}=2(y_{3i}-t_i)y_{3i}(1-y_{3i}) ∴∂net3iE=2(y3i−ti)y3i(1−y3i).

∇ 311 = ∂ E ∂ n e t 31 ⋅ ∂ n e t 31 ∂ ω 311 = ∂ E ∂ n e t 31 ⋅ y 21 \nabla_{311}=\frac{\partial E}{\partial net_{31}} \cdot \frac{\partial net_{31}}{\partial \omega_{311}}=\frac{\partial E}{\partial net_{31}}\cdot y_{21} ∇311=∂net31∂E⋅∂ω311∂net31=∂net31∂E⋅y21.

因此通项公式为： ∇ 3 i j = ∂ E ∂ n e t 3 i ⋅ ∂ n e t 3 i ∂ ω 3 i j = 2 ( y 3 i − t i ) y 3 i ( 1 − y 3 i ) y 2 i \nabla_{3ij}=\frac{\partial E}{\partial net_{3i}}\cdot \frac{\partial net_{3i}}{\partial \omega_{3ij}}=2(y_{3i}-t_{i})y_{3i}(1-y_{3i})y_{2i} ∇3ij=∂net3i∂E⋅∂ω3ij∂net3i=2(y3i−ti)y3i(1−y3i)y2i.

6. ∇ 2 i j \nabla_{2ij} ∇2ij（隐藏层各 ω \omega ω的梯度值）

∇ 211 = ∂ E ∂ ω 211 = ∂ E ∂ n e t 31 ⋅ ∂ n e t 31 ∂ y 21 ⋅ ∂ y 21 ∂ n e t 21 ⋅ ∂ n e t 21 ∂ ω 211 + ∂ E ∂ n e t 32 ⋅ ∂ n e t 32 ∂ y 21 ⋅ ∂ y 21 ∂ n e t 21 ⋅ ∂ n e t 21 ∂ ω 211 = [ ∑ i = 1 2 ( ∂ E ∂ n e t 3 i ⋅ ∂ n e t 3 i ∂ y 2 i ) ] ⋅ ∂ y 21 ∂ n e t 21 ⋅ ∂ n e t 21 ∂ ω 211 = [ ∑ i = 1 2 ( ∂ E ∂ n e t 3 i ⋅ ω 3 i 1 ) ] ⋅ y 21 ( 1 − y 21 ) ω 21 \nabla_{211}=\frac{\partial E}{\partial \omega_{211}}=\frac{\partial E}{\partial net_{31}}\cdot \frac{\partial net_{31}}{\partial y_{21}}\cdot \frac{\partial y_{21}}{\partial net_{21}}\cdot \frac{\partial{net_{21}}}{\partial \omega_{211}}\\+\frac{\partial E}{\partial net_{32}}\cdot \frac{\partial net_{32}}{\partial y_{21}}\cdot \frac{\partial y_{21}}{\partial net_{21}}\cdot \frac{\partial{net_{21}}}{\partial \omega_{211}}\\=[\sum_{i=1}^{2}(\frac{\partial E}{\partial net_{3i}}\cdot \frac{\partial net_{3i}}{\partial y_{2i}})]\cdot \frac{\partial y_{21}}{\partial net_{21}} \cdot \frac{\partial net_{21}}{\partial \omega_{211}}\\=[\sum_{i=1}^{2}(\frac{\partial E}{\partial net_{3i}}\cdot \omega_{3i1})]\cdot y_{21}(1-y_{21})\omega_{21} ∇211=∂ω211∂E=∂net31∂E⋅∂y21∂net31⋅∂net21∂y21⋅∂ω211∂net21+∂net32∂E⋅∂y21∂net32⋅∂net21∂y21⋅∂ω211∂net21=[∑i=12(∂net3i∂E⋅∂y2i∂net3i)]⋅∂net21∂y21⋅∂ω211∂net21=[∑i=12(∂net3i∂E⋅ω3i1)]⋅y21(1−y21)ω21

因此通项公式为： ∇ 2 i j = ∂ E ∂ ω 2 i j = [ ∑ k = 1 2 ( ∂ E ∂ n e t 3 k ) ⋅ ω 3 k i ] ⋅ y 2 i ( 1 − 2 i ) ω 2 i j \nabla_{2ij}=\frac{\partial E}{\partial \omega_{2ij}}=[\sum_{k=1}^{2}(\frac{\partial E}{\partial net_{3k}})\cdot \omega_{3ki}]\cdot y_{2i}(1-2_i)\omega_{2ij} ∇2ij=∂ω2ij∂E=[∑k=12(∂net3k∂E)⋅ω3ki]⋅y2i(1−2i)ω2ij.

本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)