我能想到的最简单的方法是定义一个忽略输出层结果的损失。
完整的 Colab 在这里:https://colab.research.google.com/drive/1Rp3EUWnBE1eCcaisUju9TwSTswQfZOkS https://colab.research.google.com/drive/1Rp3EUWnBE1eCcaisUju9TwSTswQfZOkS
损失函数。请注意,它假设输出层是 Dense(activation='softmax') 并且忽略y_pred
。因此,在使用损失的训练/评估期间,密集层的实际输出是 NOP。
进行预测时使用输出层。
class SampledSoftmaxLoss(object):
""" The loss function implements the Dense layer matmul and activation
when in training mode.
"""
def __init__(self, model):
self.model = model
output_layer = model.layers[-1]
self.input = output_layer.input
self.weights = output_layer.weights
def loss(self, y_true, y_pred, **kwargs):
labels = tf.argmax(y_true, axis=1)
labels = tf.expand_dims(labels, -1)
loss = tf.nn.sampled_softmax_loss(
weights=self.weights[0],
biases=self.weights[1],
labels=labels,
inputs=self.input,
num_sampled = 3,
num_classes = 4,
partition_strategy = "div",
)
return loss
Model:
def make_model():
inp = Input(shape=(10,))
h1 = Dense(16, activation='relu')(inp)
h2 = Dense(4, activation='linear')(h1)
# output layer and last hidden layer must have the same dims
out = Dense(4, activation='softmax')(h2)
model = Model(inp, out)
loss_calculator = SampledSoftmaxLoss(model)
model.compile('adam', loss_calculator.loss)
return model
tf.set_random_seed(42)
model = make_model()
model.summary()
请注意,SampledSoftmaxLoss 强制最后一个模型层的输入必须具有与类数相同的维度。