我正在使用张量流2.1.1
并尝试使用 Attention 构建序列到序列模型。
latent_dim = 300
embedding_dim=100
batch_size = 128
# Encoder
encoder_inputs = tf.keras.Input(shape=(None,), dtype='int32')
#embedding layer
enc_emb = tf.keras.layers.Embedding(x_voc, embedding_dim,trainable=True)(encoder_inputs)
#encoder lstm 1
encoder_lstm = tf.keras.layers.LSTM(latent_dim,return_sequences=True,return_state=True,dropout=0.4,recurrent_dropout=0.4)
encoder_output, state_h, state_c = encoder_lstm(enc_emb)
print(encoder_output.shape)
# Set up the decoder, using `encoder_states` as initial state.
decoder_inputs = tf.keras.Input(shape=(None,), dtype='int32')
#embedding layer
dec_emb_layer = tf.keras.layers.Embedding(y_voc, embedding_dim,trainable=True)
dec_emb = dec_emb_layer(decoder_inputs)
decoder_lstm = tf.keras.layers.LSTM(latent_dim, return_sequences=True, return_state=True,dropout=0.4,recurrent_dropout=0.2)
decoder_output,decoder_fwd_state, decoder_back_state = decoder_lstm(dec_emb,initial_state=[state_h, state_c])
# Attention layer
attn_out, attn_states = tf.keras.layers.Attention()([encoder_output, decoder_output])
# Concat attention input and decoder LSTM output
decoder_concat_input = tf.keras.layers.Concatenate(axis=-1, name='concat_layer')([decoder_output, attn_out])
#dense layer
decoder_dense = tf.keras.layers.TimeDistributed(Dense(y_voc, activation='softmax'))
decoder_outputs = decoder_dense(decoder_concat_input)
# Define the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
model.summary()
当我运行这个时,我在创建注意层时遇到错误TypeError: Cannot iterate over a tensor with unknown first dimension.
.
我检查了尺寸encoder_output
and decoder_output
他们都是(None, None, 300)
所以认为这可能是问题所在。但我检查了注意力示例张量流示例 https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention并且他们还拥有None
注意力层输入参数的维度。
我想知道我错过了什么?请建议。
EDIT
添加堆栈跟踪
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-49-d37cd48e626b> in <module>()
28
29 # Attention layer
---> 30 attn_out, attn_states = tf.keras.layers.Attention()([encoder_output, decoder_output])
31
32 # Concat attention input and decoder LSTM output
~/anaconda3/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py in __iter__(self)
546 if shape[0] is None:
547 raise TypeError(
--> 548 "Cannot iterate over a tensor with unknown first dimension.")
549 for i in xrange(shape[0]):
550 yield self[i]
TypeError: Cannot iterate over a tensor with unknown first dimension.