我有以下列表数组(每部电影的演员):
partial_x_train_actors=array([list([b'victor mclaglen', b'jon hall', b'frances farmer', b'olympe bradna', b'gene lockhart', b'douglass dumbrille', b'francis ford', b'ben welden', b'abner biberman', b'pedro de cordoba', b'rudy robles', b'bobby stone', b'nellie duran', b'james flavin', b'nina campana']),
list([b'jessica biel', b'ben barnes', b'kristin scott thomas', b'colin firth', b'kimberley nixon', b'katherine parkinson', b'kris marshall', b'christian brassington', b'charlotte riley', b'jim mcmanus', b'pip torrens', b'jeremy hooton', b'joanna bacon', b'maggie hickey', b'georgie glen']),
list([b'gr\xc3\xa9gori derang\xc3\xa8re', b'anouk grinberg', b'aur\xc3\xa9lien recoing', b'niels arestrup', b'yann collette', b'laure duthilleul', b'david assaraf', b'pascal demolon', b'jean-baptiste iera', b'richard sammel', b'vincent crouzet', b'fred epaud', b'pascal elso', b'nicolas giraud', b'micha\xc3\xabl abiteboul']),
...,
list([b'jason schwartzman', b'mickey rourke', b'brittany murphy', b'john leguizamo', b'patrick fugit', b'mena suvari', b'chloe hunter', b'elisa bocanegra', b'julia mendoza', b'china chow', b'nicholas gonzalez', b'debbie harry', b'josh peck', b'charlotte ayanna', b'eric roberts']),
list([b'fred kirschenmann', b'daniel salatin', b'joel salatin', b'paul willis', b'chuck wirtz']),
list([b'jan sebastian', b'tray loren', b'paul muzzcat', b'brad koepenick', b'jerry armstrong', b'ben sebastian', b'reyn hubbard', b'levita gros', b'betty flemming', b'randolph parro', b'susan serigny', b'keith gros', b'rocky dugas', b'sid larrwiere', b'jocelyn boudreaux'])],
dtype=object)
由于我想将其用作 Keras 模型的输入,因此我必须将列表数组转换为数组数组。为此,我运行以下代码,取自这个问题 https://stackoverflow.com/questions/57760510/converting-array-of-lists-to-keras-input
partial_x_train_actors_array=[]
for i in range(len(partial_x_train_actors)):
partial_x_train_actors_array.append(np.array(list(x for x in partial_x_train_actors[i])))
partial_x_train_actors_array = np.asarray(partial_x_train_actors_array)=
type(partial_x_train_actors_array[0])
所以现在我明白了:
array([array([b'victor mclaglen', b'jon hall', b'frances farmer',
b'olympe bradna', b'gene lockhart', b'douglass dumbrille',
b'francis ford', b'ben welden', b'abner biberman',
b'pedro de cordoba', b'rudy robles', b'bobby stone',
b'nellie duran', b'james flavin', b'nina campana'], dtype='|S18'),
array([b'jessica biel', b'ben barnes', b'kristin scott thomas',
b'colin firth', b'kimberley nixon', b'katherine parkinson',
b'kris marshall', b'christian brassington', b'charlotte riley',
b'jim mcmanus', b'pip torrens', b'jeremy hooton', b'joanna bacon',
b'maggie hickey', b'georgie glen'], dtype='|S21'),
array([b'gr\xc3\xa9gori derang\xc3\xa8re', b'anouk grinberg',
b'aur\xc3\xa9lien recoing', b'niels arestrup', b'yann collette',
b'laure duthilleul', b'david assaraf', b'pascal demolon',
b'jean-baptiste iera', b'richard sammel', b'vincent crouzet',
b'fred epaud', b'pascal elso', b'nicolas giraud',
b'micha\xc3\xabl abiteboul'], dtype='|S19'),
...,
array([b'jason schwartzman', b'mickey rourke', b'brittany murphy',
b'john leguizamo', b'patrick fugit', b'mena suvari',
b'chloe hunter', b'elisa bocanegra', b'julia mendoza',
b'china chow', b'nicholas gonzalez', b'debbie harry', b'josh peck',
b'charlotte ayanna', b'eric roberts'], dtype='|S17'),
array([b'fred kirschenmann', b'daniel salatin', b'joel salatin',
b'paul willis', b'chuck wirtz'], dtype='|S17'),
array([b'jan sebastian', b'tray loren', b'paul muzzcat',
b'brad koepenick', b'jerry armstrong', b'ben sebastian',
b'reyn hubbard', b'levita gros', b'betty flemming',
b'randolph parro', b'susan serigny', b'keith gros', b'rocky dugas',
b'sid larrwiere', b'jocelyn boudreaux'], dtype='|S17')],
dtype=object)
但这都不足以摆脱输入张量的类型,因为我收到此错误:
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).
我的模型拟合过程
# import the pre-trained model
model = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1"
hub_layer = hub.KerasLayer(model, output_shape=[20], input_shape=[], dtype=tf.string, trainable=True)
# create the neural network structure
model = tf.keras.Sequential(name="English_Google_News_130GB_witout_OOV_tokens")
model.add(hub_layer)
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(i, kernel_regularizer=regularizers.l2(neural_network_parameters['l2_regularization']),
activation=neural_network_parameters['dense_activation']))
model.add(tf.keras.layers.Dropout(neural_network_parameters['dropout_rate']))
model.add(tf.keras.layers.Dense(y_val.shape[1],
activation=neural_network_parameters['output_activation']))
#model.name("English Google News 130GB witout OOV tokens")
print(model.summary())
#instantiate Optimizer
optimizer = optimizer_adam_v2(len(partial_x_train_actors_array), validation_split_ratio, i)
model.compile(optimizer=optimizer,
loss=neural_network_parameters['model_loss'],
metrics=[neural_network_parameters['model_metric']])
plot_model(model, to_file=os.path.join(os.getcwd(), 'model_three\\network_structure_english_google_news_without_OOV_model_{0}.png'.format(version_data_control)))
history = model.fit([partial_x_train_features, partial_x_train_plot, partial_x_train_actors_array, partial_x_train_reviews],
partial_y_train,
steps_per_epoch=int(np.ceil((len(partial_x_train_actors_array)*0.8)//16)),
epochs=100,
batch_size=16,
validation_split=0.2
verbose=0,
callbacks=callback("english_google_news_without_oovtokens", model))
[编辑] - 04.07.2020
我想补充一点,我已经为另一个实验完成了序列的填充,上面列出的演员列表已转换为下面的列表
partial_x_train_actors=array([[ 2024, 3228, 451, ..., 18119, 0, 0],
[ 3230, 7889, 12357, ..., 0, 0, 0],
[20001, 20001, 20001, ..., 0, 0, 0],
...,
[ 6887, 20001, 15352, ..., 20001, 20001, 20001],
[10206, 20001, 3426, ..., 20001, 0, 0],
[ 2969, 5903, 447, ..., 0, 0, 0]])
但是,当我在神经网络的 .fit() 中应用此列表时,出现以下错误
ValueError: Error when checking input: expected keras_layer_4_input to have 1 dimensions, but got array with shape (39192, 17)
(39192, 17) 是 actor 数组的形状
[编辑 2] - 2020 年 7 月 5 日
Trial 1(失败的)
根据提供的答案的一些建议,我尝试更改 hub.Keraslayer 的输入形状:
hub_layer = hub.KerasLayer(model, output_shape=[20], input_shape=[len(y_train)], dtype=tf.string, trainable=True)
我让它等于我的训练输入长度#39192 每个演员、情节、功能、评论的数据。
Error produced:
From the error, I can guess that the input_shape should be []?
Trial 2(失败的)
#list of actors (training data) tensors
actors_training_tensors=np.array([tf.convert_to_tensor(partial_x_train_actors[i]) for i in range(len(partial_x_train_actors))])
actors_testing_tensors=np.array([tf.convert_to_tensor(x_val_actors[i]) for i in range(len(x_val_actors))])
又报错:
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type tensorflow.python.framework.ops.EagerTensor).
我将演员的输入列表转换为张量。请注意,只有演员的列表有问题,因为它们作为名称存储在列表 [[name1, name2, name3]] 中。我对情节、特征或评论输入都没有问题,因为它们被保存为语料库列表。
Trial 3(失败的)
根据评论,我同样使用了数据 API:
data_tf=tf.data.Dataset.from_tensor_slices([partial_x_train_features, partial_x_train_plot, partial_x_train_actors_array, partial_x_train_reviews])
我又收到一个错误:
ValueError: Can't convert Python sequence with mixed types to Tensor.
所以我搜索了一下,我发现了这个question https://stackoverflow.com/questions/49824872/convert-python-sequence-with-multiple-datatypes-to-tensor和文档 https://www.tensorflow.org/guide/tensor#data_types,
我做了以下更改(添加了 tf.constant):
data_tf=tf.data.Dataset.from_tensor_slices([tf.constant(partial_x_train_features), tf.constant(partial_x_train_plot), tf.constant(partial_x_train_actors_array), tf.constant(partial_x_train_reviews)])
此外,我似乎无法将 NumPy 字符串数组转换为浮点数张量。也许这就是序列填充发挥重要作用的地方。但是,如果您遵循这个链接 https://www.tensorflow.org/hub/tutorials/tf2_text_classification在我从中得到这个想法的张量流文章中,您会注意到用户仅提供字节字符串而不是填充序列作为输入。
请注意,解决所有这些问题的方法就是使用“”.join() 命令来展平参与者列表。然而,演员只是名字的文本,而不是单独的名字。尽管它有效,但我认为为了获得更好的结果,应该为演员提供单独的名称,因为神经网络无法自行区分名称。
[输入数据进行调试 - 问题复制]
如果有人想要复制和调试问题,下面我代表我的 4 个输入层(数据样本)和来自 Tensorflow 的文章 https://www.tensorflow.org/hub/tutorials/tf2_text_classification我已经遵循了。
这是问题的GitHub 链接 https://github.com/tensorflow/tensorflow/issues/41109发布了问题。当我在本地运行代码时,除了附加的 GitHub 问题中出现的 EarlyStopping 错误之外,一切看起来都很好。我将重新检查我使用的数据,因为 GitHub 链接中提供的数据是要使用的正确数据。