在 Tensorflow 中实现暹罗网络

我想实现一个连体卷积神经网络，其中两个图像在卷积层中共享权重，然后在通过全连接层之前进行连接。我尝试过一种实现，但它似乎是一个“黑客”解决方案。特别是，我已将张量上的操作定义为简单的 Python 函数，并且我不确定这是否允许。

这是我尝试过的代码：

images = tf.placeholder(tf.float32, shape=[None, 64 * 64])
# Convolutional layers
# ...
# ...
# Results in pool3_flat, which is the flattened output of the third convolutional layer
pool3_flat = tf.reshape(pool3, [-1, 8 * 8 * 128])

# Now, merge the image pairs, where each pair is composed of adjacent images in the batch, with a stride of 2
def merge_pairs():
  # Create a tensor to store the merged image pairs
  # The batch size is 128, therefore there will be 64 pairs (64 in the first dimension of this tensor)
  merged_pairs = tf.Variable(tf.zeros([64, 8 * 8 * 128]))
  # Split the images into 64 pairs
  pairs = tf.split(0, 64, pool3_flat)
  # For each pair, concatenate the two images across dimension 1, and set this tensor in the appropriate row of merged_pairs
  for pair_num, pair in enumerate(pairs):
      merged_pair = tf.concat(1, pair)
      merged_pairs[pair_num] = merged_pair
  return merged_pairs


# Proceed with operations on the merged_pair tensor, as if the batch size is 64
fc4 = tf.matmul(merge_pairs(), weights4)
# ...
# ...

虽然它可以编译并且运行良好，但结果并不真正符合预期。所以，我想知道是否有更好的方法来使用 TensorFlow 中的内置操作来实现 Siamese 网络？

您可以利用tf.pack https://www.tensorflow.org/versions/r0.7/api_docs/python/array_ops.html#pack and tf.解包 https://www.tensorflow.org/versions/r0.7/api_docs/python/array_ops.html#unpack，有点像：

pairs = tf.pack(tf.split(0, 64, pool3_flat))
left, right = tf.unpack(tf.transpose(pairs, perm=[1,0,2]))
merged_pairs = tf.concat(1, [left, right])

一种更简洁的方法是从一开始就将各对分开，以便您可以定义两个网络并在每个网络中使用相同的可训练变量。

你会得到类似的东西（跳过卷积层）：

image_left = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])
image_right = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])

pool_left = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
pool_right = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

pool_flat_left = tf.reshape(pool_left, [-1, 32*32])
pool_flat_right = tf.reshape(pool_right, [-1, 32*32])

然后简单地在维度 1 中左右连接。

concat_layer = tf.concat(1, [pool_flat_left, pool_flat_right])

这样您也可以稍后改变批量大小。 确保每个尺寸（左和右）使用相同的权重和偏差。