项目场景:
有关深度强化学习过程中使用torch.cat遇到的问题
问题描述
在学习MEC,使用深度强化学习改变卸载率的过程中,要将action和state通过torch.cat合并的时候遇到的问题
第一个问题:torch.cat当中直接使用transitions.state
state = torch.cat(transitions.state)
报错提示:
TypeError: expected Tensor as element 0 in argument 0, but got numpy.ndarray
第二个问题:torch.cat当中直接使用[torch.tensor(a) for a in transitions.action]
action = [torch.tensor(a) for a in transitions.action]
action = torch.cat(action)
报错提示:
RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated
第三个问题: torch.cat当中直接使用next_state = [torch.from_numpy(s) for s in transitions.next_state]
next_state = [torch.from_numpy(s) for s in transitions.next_state]
next_state = torch.cat(next_state)
报错提示:
TypeError: expected np.ndarray (got int)
原因分析:
分析:torch.cat需要的参数必须是以下格式ndarray当中是tensor,tensor当中又必须是ndarray这种嵌套的形式,举个例子:[tensor([5.2626, 3.4518, 4.4650, 6.1861, 3.0479]), tensor([5.2572, 3.4194, 4.4557, 6.2838, 3.0352]), tensor([5.2626, 3.4338, 4.4604, 6.2512, 3.0444])],像是这种形式
根据以上分析,查看上面两个数据的transitions.state和action长什么样。
transitions.state的样子:(array([5.34502891, 3.50129482, 4.39213256, 6.07983465, 2.73172546]), array([5.33968388, 3.50129482, 4.39676183, 6.10588068, 2.75003088]), array([5.34502891, 3.49770036, 4.39213256, 6.09285766, 2.73477636]))
action和next_state的样子:[tensor(8), tensor(0), tensor(0), tensor(7), tensor(5), tensor(0), tensor(3)]
解决方案:
第一个问题的解决方案:
使用state = [torch.from_numpy(s) for s in transitions.state]
将每一个numpy转化为tensor类型
第二个问题的解决方案:
使用action = [torch.tensor([a]) for a in transitions.action]
将其封装为数组类型之后再转化为tensor类型
第三个问题的解决方案:
将from_numpy更改为tensor,记得在迭代变量当中加入中括号,将其变为数组next_state = [torch.tensor([a]) for a in transitions.next_state]