如何使用 tf.data.Dataset.from_generator() 将参数发送到生成器函数？

我想创建多个tf.data.Dataset使用from_generator()功能。我想向生成器函数发送一个参数（raw_data_gen）。这个想法是生成器函数将根据发送的参数产生不同的数据。这样我想raw_data_gen能够提供训练、验证或测试数据。

training_dataset = tf.data.Dataset.from_generator(raw_data_gen, (tf.float32, tf.uint8), ([None, 1], [None]), args=([1]))

validation_dataset = tf.data.Dataset.from_generator(raw_data_gen, (tf.float32, tf.uint8), ([None, 1], [None]), args=([2]))

test_dataset = tf.data.Dataset.from_generator(raw_data_gen, (tf.float32, tf.uint8), ([None, 1], [None]), args=([3]))

当我尝试致电时收到的错误消息from_generator()这样就是：

TypeError: from_generator() got an unexpected keyword argument 'args'

这里是raw_data_gen函数，虽然我不确定你是否需要这个，因为我的预感是问题出在调用from_generator():

def raw_data_gen(train_val_or_test):

    if train_val_or_test == 1:        
        #For every filename collected in the list
        for filename, lab in training_filepath_label_dict.items():
            raw_data, samplerate = soundfile.read(filename)
            try: #assume the audio is stereo, ready to be sliced
                raw_data = raw_data[:,0] #raw_data is a np.array, just take first channel with slice
            except IndexError:
                pass #this must be mono audio
            yield raw_data, lab

    elif train_val_or_test == 2:
        #For every filename collected in the list
        for filename, lab in validation_filepath_label_dict.items():
            raw_data, samplerate = soundfile.read(filename)
            try: #assume the audio is stereo, ready to be sliced
                raw_data = raw_data[:,0] #raw_data is a np.array, just take first channel with slice
            except IndexError:
                pass #this must be mono audio
            yield raw_data, lab

    elif train_val_or_test == 3:
        #For every filename collected in the list
        for filename, lab in test_filepath_label_dict.items():
            raw_data, samplerate = soundfile.read(filename)
            try: #assume the audio is stereo, ready to be sliced
                raw_data = raw_data[:,0] #raw_data is a np.array, just take first channel with slice
            except IndexError:
                pass #this must be mono audio
            yield raw_data, lab

    else:
        print("generator function called with an argument not in [1, 2, 3]")
        raise ValueError()

您需要根据以下内容定义一个新函数raw_data_gen这不需要任何争论。您可以使用lambda关键字来执行此操作。

training_dataset = tf.data.Dataset.from_generator(lambda: raw_data_gen(train_val_or_test=1), (tf.float32, tf.uint8), ([None, 1], [None]))
...

现在，我们将一个函数传递给from_generator不需要任何参数，但它只是充当raw_data_gen将参数设置为 1。您可以对验证集和测试集使用相同的方案，分别传递 2 和 3。