本项目需要搭建一个简单的卷积神经网络（CNN）来对 CIFAR-10 数据进行图片分类。本文记录了这个项目的一些注意事项。
１.数据的预处理：对于CIFAR-10 的图片数据，首先要做归一化处理。对于 Label 数据，要做 one-hot-encoder 处理。
One-hot-encoder 可以利用 sklearn 中的 Preprocessing 中的 LabelBinarizer 函数：
也可以利用 numpy 中的 eye 函数：
def one_hot_encode(x):
"""
One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
: x: List of sample Labels
: return: Numpy array of one-hot encoded labels
"""
# TODO: Implement Function
np_classes = 10
one_hot_labels = np.eye(np_classes)[x]
return one_hot_label
２．卷积层，最大池化层，扁平化层，全连接层，输出层的代码如下：
def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
"""
Apply convolution then max pooling to x_tensor
:param x_tensor: TensorFlow Tensor
:param conv_num_outputs: Number of outputs for the convolutional layer
:param conv_ksize: kernal size 2-D Tuple for the convolutional layer
:param conv_strides: Stride 2-D Tuple for convolution
:param pool_ksize: kernal size 2-D Tuple for pool
:param pool_strides: Stride 2-D Tuple for pool
: return: A tensor that represents convolution and max pooling of x_tensor
"""
# TODO: Implement Function
depth = x_tensor.get_shape().as_list()
padding = 'SAME'
conv_ksize2 = [conv_ksize[0], conv_ksize[1],depth[-1], conv_num_outputs]
conv_strides2 = [1, conv_strides[0], conv_strides[1], 1]
pool_ksize2 = [1, pool_ksize[0], pool_ksize[1],1]
pool_strides2 = [1, pool_strides[0], pool_strides[1], 1]
filter_weights = tf.Variable(tf.truncated_normal((conv_ksize2),0 ,0.1))
filter_bias = tf.Variable(tf.zeros(conv_num_outputs))
filter_output = tf.nn.conv2d(x_tensor, filter_weights, conv_strides2, padding)
filter_output = tf.nn.bias_add(filter_output, filter_bias)
filter_output = tf.nn.relu(filter_output)
filter_output = tf.nn.max_pool(filter_output, pool_ksize2, pool_strides2, padding)
#print (filter_output.get_shape().as_list())
return filter_output
def flatten(x_tensor):
"""
Flatten x_tensor to (Batch Size, Flattened Image Size)
: x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
: return: A tensor of size (Batch Size, Flattened Image Size).
"""
# TODO: Implement Function
flattened_image_size = np.prod(x_tensor.get_shape().as_list()[1:])
flat_inputs = tf.reshape(x_tensor,[-1,flattened_image_size])
#flat_inputs = tf.contrib.layers.flatten(x_tensor)
return flat_inputs
def fully_conn(x_tensor, num_outputs):
"""
Apply a fully connected layer to x_tensor using weight and bias
: x_tensor: A 2-D tensor where the first dimension is batch size.
: num_outputs: The number of output that the new tensor should be.
: return: A 2-D tensor where the second dimension is num_outputs.
"""
# TODO: Implement Function
#output = tf.contrib.layers.fully_connected(x_tensor, num_outputs)
weights_shape = list((x_tensor.get_shape().as_list()[-1], ) + (num_outputs, ))
weights = tf.Variable(tf.truncated_normal(weights_shape, 0, 0.1))
bias = tf.Variable(tf.zeros(num_outputs))
return tf.nn.relu(tf.add(tf.matmul(x_tensor, weights), bias))
def output(x_tensor, num_outputs):
"""
Apply a output layer to x_tensor using weight and bias
: x_tensor: A 2-D tensor where the first dimension is batch size.
: num_outputs: The number of output that the new tensor should be.
: return: A 2-D tensor where the second dimension is num_outputs.
"""
# TODO: Implement Function
image_shape = x_tensor.get_shape().as_list()[1]
weights = tf.Variable(tf.truncated_normal([image_shape, num_outputs],0,0.01))
bias = tf.Variable(tf.zeros(num_outputs))
outputs = tf.add(tf.matmul(x_tensor, weights),bias)
return outputs
３．构建模型及参数选择
本项目的难点在于模型的参数选择。具体参数包括：
卷积层滤波器的 size 以及 stride：
经过反复测试，滤波器 选择 4*4，stride 选择 1 的效果较好
最大池化的 size 以及 stride:
池化的 size 选择 8*8， stride 选择１的效果较好
全连接层的输出 size：
输出层为 384 个输出的效果较好
keep-probability:
经过测试，keep-probability 宜小一点，可以更好的防止 overfitting。但是太小的 keep-probability 也会导致运算速度慢，以及模型预测不准的问题。
各个层级的 weight 的初始化问题：
我在搭建模型初期，遇到一个很大的问题就是，初始化参数设置不合理，导致收敛太慢。一开始注意到收敛太慢的问题时，我采用的方法是增大 optimizer 的 learning-rate ， 设置：
optimizer = tf.train.AdamOptimizer(learning_rate = 0.01).munimize(cost)但是效果仍然不好。通过查询资料，发现应该在初始化变量时，调整正态分布的标准差，将默认的标准差“1”修改为 “0.1” 或 “0.01”。
最终搭建神经网络的代码如下：
def conv_net(x, keep_prob):
"""
Create a convolutional neural network model
: x: Placeholder tensor that holds image data.
: keep_prob: Placeholder tensor that hold dropout keep probability.
: return: Tensor that represents logits
"""
# TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
# Play around with different number of outputs, kernel size and stride
# Function Definition from Above:
# conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
output1 = conv2d_maxpool(x, 18, (4,4),(1,1),(8,8),(1,1))
output1 = tf.nn.dropout(output1, keep_prob)
#output2 = conv2d_maxpool(output1,200,(2,2),(2,2),(2,2),(2,2))
#output2 = tf.nn.dropout(output2, keep_prob)
# TODO: Apply a Flatten Layer
# Function Definition from Above:
# flatten(x_tensor)
output3 = flatten(output1)
# TODO: Apply 1, 2, or 3 Fully Connected Layers
# Play around with different number of outputs
# Function Definition from Above:
# fully_conn(x_tensor, num_outputs)
output4 = fully_conn(output3, 384)
#output5 = fully_conn(output4,50)
output4 = tf.nn.dropout(output4, keep_prob)
# TODO: Apply an Output Layer
# Set this to the number of classes
# Function Definition from Above:
# output(x_tensor, num_outputs)
logits_output = output(output4, 10)
# TODO: return output
return logits_output
4.显示结果：
注意在输出结果时，计算 accuracy 要使用 validation 数据集来计算。
def print_stats(session, feature_batch, label_batch, cost, accuracy):
"""
Print information about loss and validation accuracy
: session: Current TensorFlow session
: feature_batch: Batch of Numpy image data
: label_batch: Batch of Numpy label data
: cost: TensorFlow cost function
: accuracy: TensorFlow accuracy function
"""
# TODO: Implement Function
loss = session.run(cost, feed_dict={x:feature_batch, y:label_batch, keep_prob:1.0})
valid_acc = sess.run(accuracy, feed_dict={
x: valid_features,
y: valid_labels,
keep_prob: 1.})
print('Loss: {:>10.4f} Validation Accuracy: {:.6f}'.format(
loss,
valid_acc))最终结果为，一个 batch 的准确率为 63% 左右。 全部五个 batch 的准确率在 70% 左右。结果较为满意。