有多种方法可以做到这一点:
import numpy.linalg as la
from scipy.spatial import distance as dist
# Manually
def method0(x, y):
dotprod_mat = np.dot(x, y.T)
costheta = dotprod_mat / la.norm(x, axis=1)[:, np.newaxis]
costheta /= la.norm(y, axis=1)
return np.arccos(costheta)
# Using einsum
def method1(x, y):
dotprod_mat = np.einsum('ij,kj->ik', x, y)
costheta = dotprod_mat / la.norm(x, axis=1)[:, np.newaxis]
costheta /= la.norm(y, axis=1)
return np.arccos(costheta)
# Using scipy.spatial.cdist (one-liner)
def method2(x, y):
costheta = 1 - dist.cdist(x, y, 'cosine')
return np.arccos(costheta)
# Realize that your arrays `x` and `y` are already normalized, meaning you can
# optimize method1 even more
def method3(x, y):
costheta = np.einsum('ij,kj->ik', x, y) # Directly gives costheta, since
# ||x|| = ||y|| = 1
return np.arccos(costheta)
(n, m) = (1212, 252) 的计时结果:
>>> %timeit theta = method0(x, y)
100 loops, best of 3: 11.1 ms per loop
>>> %timeit theta = method1(x, y)
100 loops, best of 3: 10.8 ms per loop
>>> %timeit theta = method2(x, y)
100 loops, best of 3: 12.3 ms per loop
>>> %timeit theta = method3(x, y)
100 loops, best of 3: 9.42 ms per loop
时间差异随着元素数量的增加而减小。对于 (n, m) = (6252, 1212):
>>> %timeit -n10 theta = method0(x, y)
10 loops, best of 3: 365 ms per loop
>>> %timeit -n10 theta = method1(x, y)
10 loops, best of 3: 358 ms per loop
>>> %timeit -n10 theta = method2(x, y)
10 loops, best of 3: 384 ms per loop
>>> %timeit -n10 theta = method3(x, y)
10 loops, best of 3: 314 ms per loop
但是,如果您省略np.arccos
步骤,即假设您可以通过costheta
,并且没有need theta
本身,那么:
>>> %timeit costheta = np.einsum('ij,kj->ik', x, y)
10 loops, best of 3: 61.3 ms per loop
>>> %timeit costheta = 1 - dist.cdist(x, y, 'cosine')
10 loops, best of 3: 124 ms per loop
>>> %timeit costheta = dist.cdist(x, y, 'cosine')
10 loops, best of 3: 112 ms per loop
这是针对(6252, 1212)的情况。所以实际上np.arccos
占用了80%的时间。在这种情况下我发现np.einsum
is much比...快dist.cdist
。所以你肯定想使用einsum
.
Summary:结果theta
大部分相似,但是np.einsum
对我来说最快,特别是当你没有额外计算规范时。尽量避免计算theta
并与刚刚合作costheta
.
Note:我没有提到的重要一点是浮点精度的有限性可能会导致np.arccos
给予nan
价值观。method[0:3]
为以下价值观而努力x
and y
当然,这还没有得到适当的标准化。但method3
给了一些nan
s。我通过预归一化解决了这个问题,这自然会破坏使用中的任何收益method3
,除非您需要对一小组预归一化矩阵进行多次计算(无论出于何种原因)。