tensorflow报错：InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [12]

tensorflow报错：InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]

我是在执行tensorflow/examples/speech_commands/freeze.py时报的错，具体错误如下：

(tensorflow) ubuntu@VM-0-13-ubuntu:~/tf3$ python tensorflow/examples/speech_commands/freeze.py \
> --start_checkpoint=/tmp/speech_commands_train/conv.ckpt-18000 \
> --output_file=/tmp/my_frozen_graph.pb
2018-09-13 14:34:38.589857: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-09-13 14:34:38.699275: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-09-13 14:34:38.699780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
name: Tesla P40 major: 6 minor: 1 memoryClockRate(GHz): 1.531
pciBusID: 0000:00:06.0
totalMemory: 22.38GiB freeMemory: 22.21GiB
2018-09-13 14:34:38.699807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
2018-09-13 14:34:39.003121: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-13 14:34:39.003185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 
2018-09-13 14:34:39.003194: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N 
2018-09-13 14:34:39.003625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21544 MB memory) -> physical GPU (device: 0, name: Tesla P40, pci bus id: 0000:00:06.0, compute capability: 6.1)
Traceback (most recent call last):
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
    return fn(*args)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@final_fc_bias"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](final_fc_bias, save/RestoreV2/_1)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 877, in run
    run_metadata_ptr)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    run_metadata)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@final_fc_bias"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](final_fc_bias, save/RestoreV2/_1)]]

Caused by op 'save/Assign', defined at:
  File "tensorflow/examples/speech_commands/freeze.py", line 208, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "tensorflow/examples/speech_commands/freeze.py", line 134, in main
    models.load_variables_from_checkpoint(sess, FLAGS.start_checkpoint)
  File "/home/ubuntu/tf3/tensorflow/examples/speech_commands/models.py", line 154, in load_variables_from_checkpoint
    saver = tf.train.Saver(tf.global_variables())
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
    self.build()
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1293, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
    build_save=build_save, build_restore=build_restore)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
    restore_sequentially, reshape)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
    assign_ops.append(saveable.restore(saveable_tensors, shapes))
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 112, in restore
    self.op.get_shape().is_fully_defined())
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 216, in assign
    validate_shape=validate_shape)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 60, in assign
    use_locking=use_locking, name=name)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@final_fc_bias"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](final_fc_bias, save/RestoreV2/_1)]]


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tensorflow/examples/speech_commands/freeze.py", line 208, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "tensorflow/examples/speech_commands/freeze.py", line 134, in main
    models.load_variables_from_checkpoint(sess, FLAGS.start_checkpoint)
  File "/home/ubuntu/tf3/tensorflow/examples/speech_commands/models.py", line 155, in load_variables_from_checkpoint
    saver.restore(sess, start_checkpoint)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1759, in restore
    err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@final_fc_bias"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](final_fc_bias, save/RestoreV2/_1)]]

Caused by op 'save/Assign', defined at:
  File "tensorflow/examples/speech_commands/freeze.py", line 208, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "tensorflow/examples/speech_commands/freeze.py", line 134, in main
    models.load_variables_from_checkpoint(sess, FLAGS.start_checkpoint)
  File "/home/ubuntu/tf3/tensorflow/examples/speech_commands/models.py", line 154, in load_variables_from_checkpoint
    saver = tf.train.Saver(tf.global_variables())
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
    self.build()
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1293, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
    build_save=build_save, build_restore=build_restore)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
    restore_sequentially, reshape)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
    assign_ops.append(saveable.restore(saveable_tensors, shapes))
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 112, in restore
    self.op.get_shape().is_fully_defined())
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 216, in assign
    validate_shape=validate_shape)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 60, in assign
    use_locking=use_locking, name=name)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/ubuntu/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Assign requires shapes of both tensors to match. lhs shape= [12] rhs shape= [6]
         [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@final_fc_bias"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](final_fc_bias, save/RestoreV2/_1)]]

(tensorflow) ubuntu@VM-0-13-ubuntu:~/tf3$ 

具体描述一下我遇到此错误的情形，我是在traning自己的数据时，traning时执行的这个命令：

python tensorflow/examples/speech_commands/train.py \
--data_url= \
--data_dir=../traning \
--wanted_words=jia,yu,wo,yi

导出时执行的这个命令：

python tensorflow/examples/speech_commands/freeze.py \
--start_checkpoint=/tmp/speech_commands_train/conv.ckpt-18000 \
--output_file=/tmp/my_frozen_graph.pb

然后就报错了，网上搜索了好久，有说需要把之前traning之后的数据删除，重新traning，我照做后再次traning（对于Tesla P40 GPU来讲，每次traning要花费100分钟左右）还是这个错误。最后认真看了一下，似乎是数量对应不上而已。我改了一下导出命令，就成功了。

python tensorflow/examples/speech_commands/freeze.py \
--start_checkpoint=/tmp/speech_commands_train/conv.ckpt-18000 \
--output_file=/tmp/my_frozen_graph.pb \
--wanted_words=jia,yu,wo,yi