本文给出了四维张量卷积的表达式，卷积输出大小的表达式，以及Matlab和PyTorch下卷积实例。
直观地理解卷积
设有 N i N_i Ni 个二维卷积输入 I ∈ R N i × C i × H i × W i {\bm I} \in {\mathbb R}^{N_i × C_i \times H_i \times W_i} I∈RNi×Ci×Hi×Wi, C k × C i C_k \times C_i Ck×Ci 个二维卷积核 K ∈ R C k × C i × H k × W k {\bm K} \in {\mathbb R}^{C_k \times C_i \times H_k \times W_k} K∈RCk×Ci×Hk×Wk, N o N_o No 个卷积输出记为 O ∈ R N o × C o × H o × W o {\bm O} \in {\mathbb R}^{N_o × C_o \times H_o \times W_o} O∈RNo×Co×Ho×Wo, 在经典卷积神经网络中, 有 C k = C o , N o = N i C_k = C_o, N_o = N_i Ck=Co,No=Ni, K {\bm K} K 与 I \bm I I 间的二维卷积运算可以表示为
O
n
o
,
c
o
,
:
,
:
=
∑
c
i
=
0
C
i
−
1
I
n
o
,
c
i
,
:
,
:
∗
K
c
o
,
c
i
,
:
,
:
=
∑
c
i
=
0
C
i
−
1
Z
n
o
,
c
o
,
c
i
,
:
,
:
\begin{aligned} {\bm O}_{n_o, c_o, :, :} &= \sum_{c_i=0}^{C_i-1} {\bm I}_{n_o, c_i, :,:} * {\bm K}_{c_o, c_i, :,:} \\ &= \sum_{c_i=0}^{C_i-1}{\bm Z}_{n_o, c_o, c_i, :, :} \end{aligned}
Ono,co,:,:=ci=0∑Ci−1Ino,ci,:,:∗Kco,ci,:,:=ci=0∑Ci−1Zno,co,ci,:,:
其中,
∗
*
∗ 表示经典二维卷积运算, 卷积核
K
c
o
,
c
i
,
:
,
:
{\bm K}_{c_o, c_i, :,:}
Kco,ci,:,: 与输入
I
n
o
,
c
i
,
:
,
:
{\bm I}_{n_o, c_i, :,:}
Ino,ci,:,: 的卷积结果记为
Z
n
o
,
c
o
,
c
i
,
:
,
:
∈
R
H
o
×
W
o
{\bm Z}_{n_o, c_o, c_i, :, :}\in {\mathbb R}^{H_o \times W_o}
Zno,co,ci,:,:∈RHo×Wo, 则
Z n o , c o , c i , h o , w o = ∑ h = 0 H k − 1 ∑ w = 0 W k − 1 I n o , c i , h o + h − 1 , w o + w − 1 ⋅ K c o , c i , h , w . {\bm Z}_{n_o, c_o, c_i, h_o, w_o} = \sum_{h=0}^{H_k-1}\sum_{w=0}^{W_k-1} {\bm I}_{n_o, c_i, h_o + h - 1, w_o + w - 1} \cdot {\bm K}_{c_o, c_i, h, w}. Zno,co,ci,ho,wo=h=0∑Hk−1w=0∑Wk−1Ino,ci,ho+h−1,wo+w−1⋅Kco,ci,h,w.
记卷积过程中, 高度与宽度维上填补(padding)大小为 H p × W p H_p \times W_p Hp×Wp, 卷积步长为 H s × W s H_s \times W_s Hs×Ws, 则卷积输出大小满足
H o = ⌊ H i + 2 × H p − H k H s + 1 ⌋ W o = ⌊ W i + 2 × W p − W k W s + 1 ⌋ \begin{array}{ll} H_{o} &= \left\lfloor\frac{H_{i} + 2 \times H_p - H_k}{H_s} + 1\right\rfloor \\ W_{o} &= \left\lfloor\frac{W_{i} + 2 \times W_p - W_k}{W_s} + 1\right\rfloor \end{array} HoWo=⌊HsHi+2×Hp−Hk+1⌋=⌊WsWi+2×Wp−Wk+1⌋
卷积神经网络中的卷积操作, 实际上是相关操作, 因为在运算过程中, 未对卷积核进行翻转操作
下图所示为二维卷积操作示意图.
设有 N i N_i Ni 个二维卷积输入 I ∈ R N i × C i × H i × W i {\bm I} \in {\mathbb R}^{N_i × C_i \times H_i \times W_i} I∈RNi×Ci×Hi×Wi, C k × C i C_k \times C_i Ck×Ci 个二维卷积核 K ∈ R C k × C i × H k × W k {\bm K} \in {\mathbb R}^{C_k \times C_i \times H_k \times W_k} K∈RCk×Ci×Hk×Wk, 高度与宽度维上填补(padding)大小为 H p × W p H_p×W_p Hp×Wp, 膨胀(dilation)大小为 H d × W d H_d×W_d Hd×Wd, 卷积步长为 H s × W s H_s×W_s Hs×Ws, 在经典膨胀二维卷积神经网络中, 有 C k = C o , N o = N i C_k = C_o, N_o = N_i Ck=Co,No=Ni, 则卷积后的输出为 O ∈ R N o × C o × H o × W o {\bm O} \in {\mathbb R}^{N_o × C_{o}\times H_{o} \times W_{o}} O∈RNo×Co×Ho×Wo, 其中
H o = ⌊ H i + 2 × H p − H d × ( H k − 1 ) − 1 H s + 1 ⌋ W o = ⌊ W i + 2 × W p − W d × ( W k − 1 ) − 1 W s + 1 ⌋ \begin{array}{ll} H_{o} &= \left\lfloor\frac{H_{i} + 2 \times H_p - H_d \times (H_k - 1) - 1}{H_s} + 1\right\rfloor \\ W_{o} &= \left\lfloor\frac{W_{i} + 2 \times W_p - W_d \times (W_k - 1) - 1}{W_s} + 1\right\rfloor \end{array} HoWo=⌊HsHi+2×Hp−Hd×(Hk−1)−1+1⌋=⌊WsWi+2×Wp−Wd×(Wk−1)−1+1⌋
可以发现当膨胀大小为 H d × W d = 1 × 1 H_d×W_d = 1×1 Hd×Wd=1×1 时, 膨胀卷积退化为经典卷积.
更多二维卷积示意图参见 A technical report on convolution arithmetic in the context of deep learning.
二维转置卷积是一种解卷积方法, 设有二维卷积核 K ∈ R C o × H k × W k {\bm K} \in {\mathbb R}^{C_o\times H_k \times W_k} K∈RCo×Hk×Wk, 二维卷积输入 I ∈ R N i × C i × H i × W i {\bm I} \in {\mathbb R}^{N_i × C_{i}\times H_{i} \times W_{i}} I∈RNi×Ci×Hi×Wi, 高度与宽度维上填补(padding)大小为 H p × W p H_p×W_p Hp×Wp, 膨胀(dilation)大小为 H d × W d H_d×W_d Hd×Wd, 卷积步长为 H s × W s H_s×W_s Hs×Ws, 则卷积后填补(output-padding)大小为 H o p × W o p H_{op}×W_{op} Hop×Wop, 则卷积后的输出为 Y ∈ R N × C o × H o × W o {\bm Y} \in {\mathbb R}^{N × C_{o}\times H_{o} \times W_{o}} Y∈RN×Co×Ho×Wo, 其中
H o = ( H i − 1 ) × H s − 2 × H p + H d × ( H k − 1 ) + H o p + 1 W o = ( W i − 1 ) × W s − 2 × W p + W d × ( W k − 1 ) + W o p + 1 \begin{array}{ll} H_{o} &= (H_{i} - 1) \times H_s - 2 \times H_p + H_d \times (H_k - 1) + H_{op} + 1 \\ W_{o} &= (W_{i} - 1) \times W_s - 2 \times W_p + W_d \times (W_k - 1) + W_{op} + 1 \end{array} HoWo=(Hi−1)×Hs−2×Hp+Hd×(Hk−1)+Hop+1=(Wi−1)×Ws−2×Wp+Wd×(Wk−1)+Wop+1
以二维卷积为例, 设有矩阵 a , b {\bm a}, {\bm b} a,b
a = [ 1 2 3 4 5 6 7 8 9 ] {\bm a} = \left[ {\begin{array}{ccc} 1&2&3\\ 4&5&6\\ 7&8&9 \end{array}} \right] a=⎣⎡147258369⎦⎤
b = [ 1 2 3 4 ] {\bm b} = \left[ {\begin{array}{ccc} 1&2\\ 3&4 \end{array}} \right] b=[1324]
则有卷积 a ∗ b {\bm a}*{\bm b} a∗b
a ∗ b = [ 1 4 7 6 7 23 33 24 19 53 63 42 21 52 59 36 ] {\bm a}*{\bm b} = \left[ {\begin{array}{cccc} 1&4&7&6\\ 7&{23}&{33}&{24}\\ {19}&{53}&{63}&{42}\\ {21}&{52}&{59}&{36} \end{array}} \right] a∗b=⎣⎢⎢⎡171921423535273363596244236⎦⎥⎥⎤
互相关 a ⋆ b {\bm a}\star{\bm b} a⋆b
a ⋆ b = [ 4 11 18 9 18 37 47 21 36 67 77 33 14 23 26 9 ] {\bm a}\star{\bm b} = \left[ {\begin{array}{cccc} 4&{11}&{18}&9\\ {18}&{37}&{47}&{21}\\ {36}&{67}&{77}&{33}\\ {14}&{23}&{26}&9 \end{array}} \right] a⋆b=⎣⎢⎢⎡41836141137672318477726921339⎦⎥⎥⎤
在 Matlab 环境中, 输入如下代码, 求解卷积 a ∗ b {\bm a} * {\bm b} a∗b 与相关 a ⋆ b {\bm a}\star{\bm b} a⋆b
a = [1 2 3;4 5 6;7 8 9];
b = [1 2;3 4];
disp(a)
disp(b)
% convolution
disp(conv2(a, b))
% cross-correlation
disp(xcorr2(a, b))
MATLAB中的2D卷积和相关结果为
1 2 3
4 5 6
7 8 9
1 2
3 4
1 4 7 6
7 23 33 24
19 53 63 42
21 52 59 36
4 11 18 9
18 37 47 21
36 67 77 33
14 23 26 9
在 Python 环境中, 输入如下代码, 求解卷积 a ∗ b {\bm a} * {\bm b} a∗b
import torch as th
a = th.tensor([[1., 2, 3], [4, 5, 6], [7, 8, 9]])
b = th.tensor([[1., 2], [3, 4]])
a = a.unsqueeze(0) # 1x3x3
a = a.unsqueeze(0) # 1x1x3x3
b = b.unsqueeze(0) # 1x2x2
b = b.unsqueeze(0) # 1x1x2x2
print(a, a.size())
print(b, b.size())
c = th.conv2d(a, b, stride=1, padding=1)
print(c)
PyTorch中的2D卷积结果为
tensor([[[[1., 2., 3.],
[4., 5., 6.],
[7., 8., 9.]]]]) torch.Size([1, 1, 3, 3])
tensor([[[[1., 2.],
[3., 4.]]]]) torch.Size([1, 1, 2, 2])
tensor([[[[ 4., 11., 18., 9.],
[18., 37., 47., 21.],
[36., 67., 77., 33.],
[14., 23., 26., 9.]]]])
对比结果可以发现, PyTorch中的2D卷积实际上是2D相关操作, 与此类似, Tensorflow等深度神经网络框架中的卷积均为相关操作. 但这并不影响网络的性能, 这是因为卷积核是通过网络学习的, 通过学习得到的卷积核可以看作是翻转后卷积核.