使用 Keras 的 python 生成器线程安全

我正在使用 Keras 进行一些机器学习，并使用此生成器来生成数据和标签：

def createBatchGenerator(driving_log,batch_size=32):
    batch_images = np.zeros((batch_size, 66, 200, 3))
    batch_steering = np.zeros(batch_size)
    while 1:
        for i in range(batch_size):
            x,y = get_preprocessed_row(driving_log)
            batch_images[i]=x
            batch_steering[i]=y
        yield batch_images, batch_steering

当我在本地使用它时，它运行良好，但是当我在带有 GPU 的 AWS g2.2xlarge 上运行它时，我收到此错误“ValueError：生成器已在执行”。有人可以帮我解决这个问题吗？

你需要做一个可以支持多线程的生成器 http://anandology.com/blog/using-iterators-and-generators/确保生成器同时被两个线程调用：

import threading

class createBatchGenerator:

    def __init__(self, driving_log,batch_size=32):
        self.driving_log = driving_log
        self.batch_size = batch_size
        self.lock = threading.Lock()

    def __iter__(self):
        return self

    def __next__(self):
        with self.lock:
           batch_images = np.zeros((batch_size, 66, 200, 3))
           batch_steering = np.zeros(batch_size)

           for i in range(self.batch_size):
               x,y = get_preprocessed_row(self.driving_log)
               batch_images[i]=x
               batch_steering[i]=y
           return batch_images, batch_steering