TensorFlow - 为什么这个 softmax 回归没有学到任何东西？

我的目标是用 TensorFlow 做大事，但我正在尝试从小事做起。

我有一些小的灰度方块（有一点噪音），我想根据它们的颜色对它们进行分类（例如 3 个类别：黑色、灰色、白色）。我编写了一个小 Python 类来生成正方形和 1-hot 向量，并修改了它们的基本 MNIST 示例以将它们输入。

但它不会学到任何东西 - 例如对于 3 个类别，它的猜测总正确率约为 33%。

import tensorflow as tf
import generate_data.generate_greyscale

data_generator = generate_data.generate_greyscale.GenerateGreyScale(28, 28, 3, 0.05)
ds = data_generator.generate_data(10000)
ds_validation = data_generator.generate_data(500)
xs = ds[0]
ys = ds[1]
num_categories = data_generator.num_categories

x = tf.placeholder("float", [None, 28*28])
W = tf.Variable(tf.zeros([28*28, num_categories]))
b = tf.Variable(tf.zeros([num_categories]))
y = tf.nn.softmax(tf.matmul(x,W) + b)
y_ = tf.placeholder("float", [None,num_categories])
cross_entropy = -tf.reduce_sum(y_*tf.log(y))
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

# let batch_size = 100 --> therefore there are 100 batches of training data
xs = xs.reshape(100, 100, 28*28) # reshape into 100 minibatches of size 100
ys = ys.reshape((100, 100, num_categories)) # reshape into 100 minibatches of size 100

for i in range(100):
  batch_xs = xs[i]
  batch_ys = ys[i]
  sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

xs_validation = ds_validation[0]
ys_validation = ds_validation[1]
print sess.run(accuracy, feed_dict={x: xs_validation, y_: ys_validation})

我的数据生成器如下所示：

import numpy as np
import random

class GenerateGreyScale():
    def __init__(self, num_rows, num_cols, num_categories, noise):
        self.num_rows = num_rows
        self.num_cols = num_cols
        self.num_categories = num_categories
        # set a level of noisiness for the data
        self.noise = noise

    def generate_label(self):
        lab = np.zeros(self.num_categories)
        lab[random.randint(0, self.num_categories-1)] = 1
        return lab

    def generate_datum(self, lab):
        i = np.where(lab==1)[0][0]
        frac = float(1)/(self.num_categories-1) * i
        arr = np.random.uniform(max(0, frac-self.noise), min(1, frac+self.noise), self.num_rows*self.num_cols)
        return arr

    def generate_data(self, num):
        data_arr = np.zeros((num, self.num_rows*self.num_cols))
        label_arr = np.zeros((num, self.num_categories))
        for i in range(0, num):
            label = self.generate_label()
            datum = self.generate_datum(label)
            data_arr[i] = datum
            label_arr[i] = label
        #data_arr = data_arr.astype(np.float32)
        #label_arr = label_arr.astype(np.float32)
        return data_arr, label_arr

对于初学者，尝试使用随机值（而不是零）初始化 W 矩阵 - 当所有输入的输出全为零时，您不会为优化器提供任何可处理的内容。

代替：

W = tf.Variable(tf.zeros([28*28, num_categories]))

Try:

W = tf.Variable(tf.truncated_normal([28*28, num_categories],
                                    stddev=0.1))