有几种方法
选项1]列表理解
In [3631]: pd.Series([x[0].union(x[1])
for x in zip(tokens, tokens.shift(-1).fillna(''))],
index=tokens.index)
Out[3631]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
选项2] map
In [3632]: pd.Series(map(lambda x: x[0].union(x[1]),
zip(tokens, tokens.shift(-1).fillna(''))),
index=tokens.index)
Out[3632]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
选项3] Using concat
and apply
In [3633]: pd.concat([tokens, tokens.shift(-1).fillna('')],
axis=1).apply(lambda x: x[0].union(x[1]), axis=1)
Out[3633]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
Timings
In [3647]: tokens.shape
Out[3647]: (60000L,)
In [3648]: %timeit pd.Series([x[0].union(x[1]) for x in zip(tokens, tokens.shift(-1).fillna(''))], index=tokens.index)
10 loops, best of 3: 35 ms per loop
In [3649]: %timeit pd.Series(map(lambda x: x[0].union(x[1]), zip(tokens, tokens.shift(-1).fillna(''))), index=tokens.index)
10 loops, best of 3: 40.9 ms per loop
In [3650]: %timeit pd.concat([tokens, tokens.shift(-1).fillna('')], axis=1).apply(lambda x: x[0].union(x[1]), axis=1)
1 loop, best of 3: 2.2 s per loop
不相关并且为了一个数字diff
In [3653]: %timeit tokens.diff()
10 loops, best of 3: 10.8 ms per loop