如何访问 keras 中单个自定义损失函数的所有输出

我正在尝试重现中提出的网络架构本出版物 https://www.sciencedirect.com/science/article/pii/S0031320319302031在张量流中。作为一个完全的初学者，我一直在使用本教程 https://www.tensorflow.org/tutorials/images/transfer_learning作为工作的基础，使用tensorflow==2.3.2。

为了训练这个网络，他们使用了一个损失，这意味着同时来自网络的两个分支的输出，这让我开始关注 keras 中的自定义损失函数。我知道您可以定义自己的函数，只要函数的定义如下所示：

def custom_loss(y_true, y_pred):

我还明白你可以给出其他论点，如下所示：

def loss_function(margin=0.3):
    def custom_loss(y_true, y_pred):
        # And now you can use margin

然后您只需在编译模型时调用它们即可。当谈到使用多个输出时，最常见的方法似乎是建议的方法here https://github.com/keras-team/keras/issues/12553#issuecomment-519124729，您将给出几个损失函数，每个输出都会调用一个损失函数。 但是，我找不到为损失函数提供多个输出的解决方案，而这正是我在这里需要的。

为了进一步解释它，这里是一个最小的工作示例，展示了我所尝试的内容，您可以自己尝试这次合作 https://colab.research.google.com/drive/1e_P7gbJGULbfKRjfhYJc3qb7KUbk9goA?usp=sharing.

import os
import tensorflow as tf
import keras.backend as K
from tensorflow.keras import datasets, layers, models, applications, losses
from tensorflow.keras.preprocessing import image_dataset_from_directory

_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'
path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)
PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')

train_dir = os.path.join(PATH, 'train')
validation_dir = os.path.join(PATH, 'validation')

BATCH_SIZE = 32
IMG_SIZE = (160, 160)
IMG_SHAPE = IMG_SIZE + (3,)

train_dataset = image_dataset_from_directory(train_dir,
                                             shuffle=True,
                                             batch_size=BATCH_SIZE,
                                             image_size=IMG_SIZE)

validation_dataset = image_dataset_from_directory(validation_dir,
                                                  shuffle=True,
                                                  batch_size=BATCH_SIZE,
                                                  image_size=IMG_SIZE)

data_augmentation = tf.keras.Sequential([
  layers.experimental.preprocessing.RandomFlip('horizontal'),
  layers.experimental.preprocessing.RandomRotation(0.2),
])
preprocess_input = applications.resnet50.preprocess_input
base_model = applications.ResNet50(input_shape=IMG_SHAPE,
                                               include_top=False,
                                               weights='imagenet')
base_model.trainable = True
conv = layers.Conv2D(filters=128, kernel_size=(1,1))
global_pooling = layers.GlobalAveragePooling2D()
horizontal_pooling = layers.AveragePooling2D(pool_size=(1, 5))
reshape = layers.Reshape((-1, 128))

def custom_loss(y_true, y_pred):
    print(y_pred.shape)
    # Do some stuffs involving both outputs
    # Returning something trivial here for correct behavior
    return K.mean(y_pred)

inputs = tf.keras.Input(shape=IMG_SHAPE)
x = data_augmentation(inputs)
x = preprocess_input(x)
x = base_model(x, training=True)

first_branch = global_pooling(x)

second_branch = conv(x)
second_branch = horizontal_pooling(second_branch)
second_branch = reshape(second_branch)

model = tf.keras.Model(inputs, [first_branch, second_branch])
base_learning_rate = 0.0001
model.compile(optimizer=tf.keras.optimizers.Adam(lr=base_learning_rate),
              loss=custom_loss,
              metrics=['accuracy'])
model.summary()

initial_epochs = 10
history = model.fit(train_dataset,
                    epochs=initial_epochs,
                    validation_data=validation_dataset)

这样做时，我认为赋予损失函数的 y_pred 将是一个列表，包含两个输出。但是，在运行它时，我在标准输出中得到的是：

Epoch 1/10
(None, 2048)
(None, 5, 128)

我从中了解到的是，损失函数是对每个输出一个接一个地调用的，而不是对所有输出调用一次，这意味着我无法定义同时使用两个输出的损失。有什么办法可以实现这一点吗？

如果我不清楚，或者您需要更多详细信息，请告诉我。

我在尝试实施时遇到了同样的问题Triplet_Loss功能。

我参考了 Keras 的实现具有三重态损失函数的连体网络 https://keras.io/examples/vision/siamese_network/但有些事情没有成功，我不得不自己实现网络。

def get_siamese_model(input_shape, conv2d_filters):
    # Define the tensors for the input images
    anchor_input = Input(input_shape, name="Anchor_Input")
    positive_input = Input(input_shape, name="Positive_Input")
    negative_input = Input(input_shape, name="Negative_Input")

    body = build_body(input_shape, conv2d_filters)
    # Generate the feature vectors for the images
    encoded_a = body(anchor_input)
    encoded_p = body(positive_input)
    encoded_n = body(negative_input)

    distance = DistanceLayer()(encoded_a, encoded_p, encoded_n)
    # Connect the inputs with the outputs
    siamese_net = Model(inputs=[anchor_input, positive_input, negative_input],
                        outputs=distance)
    return siamese_net

而“错误”是在DistanceLayerKeras 发布的实现（也在上面的同一链接中）。

class DistanceLayer(tf.keras.layers.Layer):
    """
    This layer is responsible for computing the distance between the anchor
    embedding and the positive embedding, and the anchor embedding and the
    negative embedding.
    """

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def call(self, anchor, positive, negative):
        ap_distance = tf.math.reduce_sum(tf.math.square(anchor - positive), axis=1, keepdims=True, name='ap_distance')
        an_distance = tf.math.reduce_sum(tf.math.square(anchor - negative), axis=1, keepdims=True, name='an_distance')
        return (ap_distance, an_distance)

当我训练模型时，损失函数只取其中一个向量ap_distance or an_distance.

最后，修复是将向量连接在一起（沿着axis=1在这种情况下），在损失函数上，将它们分开：

    def call(self, anchor, positive, negative):
        ap_distance = tf.math.reduce_sum(tf.math.square(anchor - positive), axis=1, keepdims=True, name='ap_distance')
        an_distance = tf.math.reduce_sum(tf.math.square(anchor - negative), axis=1, keepdims=True, name='an_distance')
        return tf.concat([ap_distance, an_distance], axis=1)

关于我的自定义损失：

def get_loss(margin=1.0):
    def triplet_loss(y_true, y_pred):
        # The output of the network is NOT A tuple, but a matrix shape (batch_size, 2),
        # containing the distances between the anchor and the positive example,
        # and the anchor and the negative example.
        ap_distance = y_pred[:, 0]
        an_distance = y_pred[:, 1]

        # Computing the Triplet Loss by subtracting both distances and
        # making sure we don't get a negative value.
        loss = tf.math.maximum(ap_distance - an_distance + margin, 0.0)
        # tf.print("\n", ap_distance, an_distance)
        # tf.print(f"\n{loss}\n")
        return loss

    return triplet_loss