图像的上采样(up-sampling)和下采样(down-sampling)
缩小图像(或称为下采样(subsampled)或降采样(downsampled)的主要目的有两个:
1.使得图像符合显示区域的大小,
2生成对应图像的缩略图.
放大图像(或称为上采样(upsampling)或图像插值(interpolating)的主要目的是放大原图像,从而可以显示在更高分辨率的显示设备上,对图像的缩放操作并不能带来更多关于该图像的信息,因此图像的质量将不可避免地受到影响.然而,确实有一些缩放方法能够增加图像的信息,从而使得缩放厚的图像质量超过原图质量的.
上采样原理:图像放大几乎都是采用内插值方法,即在原有图像像素的基础上在像素点之间采用合适的插值算法插入新的元素.
为什么要进行上采样?
上采样是为了将特征图采样到指定分辨率大小,比如一张(416,416,3)的图片经过一系列卷积池化操作后,得到一个特征图,维度(13,13,16), 为了把这个特征图和原图进行比较,需要将这个特征图变成(416,416,3)大小.这个就称为上采样.
上采样的过程类似于一个卷积的过程,只不过在卷积之前将输入特征值插到一个更大的特征图然后进行卷积,
在神经网络中,扩大特征图的方法,即upsample/上采样的方法
1)unpooling,恢复MAX的位置,其余部分补0
2)deconvolution(反卷积):先对input补零,再conv
3)插值方法,双线性插值等,
pytorch 有相应API:
torch.nn.UpsamplingBilinear2d(size=None,scale_factor=None)
#使用方法
class UnetUp_2(nn.Module):
def __init__(self):
self.up=nn.UpsamplingBilinear2d(scale_factor=(2,2))
def forward(self, input):
output=self.up(input)
return output
- 对输入特征图进行上采样，
- 参数
- size - 输出尺寸,可以自己指定比如（512，418）
- scale_factor - 空间尺寸放大倍数
-
备注：size和scale_factor两个参数不能同时指定，只能指定一个
再来看其在tensorflow中的实现,可以看出,就是对输入图像做了一个resize()操作,
class UpSampling2D(_UpSampling):
"""Upsampling layer for 2D inputs.
Repeats the rows and columns of the data
by size[0] and size[1] respectively.
# Arguments
size: int, or tuple of 2 integers.
The upsampling factors for rows and columns.
data_format: A string,
one of `"channels_last"` or `"channels_first"`.
The ordering of the dimensions in the inputs.
`"channels_last"` corresponds to inputs with shape
`(batch, height, width, channels)` while `"channels_first"`
corresponds to inputs with shape
`(batch, channels, height, width)`.
It defaults to the `image_data_format` value found in your
Keras config file at `~/.keras/keras.json`.
If you never set it, then it will be "channels_last".
interpolation: A string, one of `nearest` or `bilinear`.
Note that CNTK does not support yet the `bilinear` upscaling
and that with Theano, only `size=(2, 2)` is possible.
# Input shape
4D tensor with shape:
- If `data_format` is `"channels_last"`:
`(batch, rows, cols, channels)`
- If `data_format` is `"channels_first"`:
`(batch, channels, rows, cols)`
# Output shape
4D tensor with shape:
- If `data_format` is `"channels_last"`:
`(batch, upsampled_rows, upsampled_cols, channels)`
- If `data_format` is `"channels_first"`:
`(batch, channels, upsampled_rows, upsampled_cols)`
"""
@interfaces.legacy_upsampling2d_support
def __init__(self, size=(2, 2), data_format=None, interpolation='nearest',
**kwargs):
normalized_size = conv_utils.normalize_tuple(size, 2, 'size')
super(UpSampling2D, self).__init__(normalized_size, data_format, **kwargs)
if interpolation not in ['nearest', 'bilinear']:
raise ValueError('interpolation should be one '
'of "nearest" or "bilinear".')
self.interpolation = interpolation
def call(self, inputs):
return K.resize_images(inputs, self.size[0], self.size[1],
self.data_format, self.interpolation)
def get_config(self):
config = super(UpSampling2D, self).get_config()
config['interpolation'] = self.interpolation
return config
