Use filter http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.filter.html with abs http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.abs.html and all http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.all.html用于创建mask
进而boolean indexing http://pandas.pydata.org/pandas-docs/stable/indexing.html#boolean-indexing:
mask = (df.filter(like='Value').abs() < 1).all(axis=1)
print (mask)
0 False
1 True
2 False
3 True
4 False
5 True
6 False
dtype: bool
print (df[mask])
A_Name B_Detail Value_B Value_C Value_D
1 BB Y1 0.76 -0.7 0.8
3 DD L1 0.90 -0.5 0.4
5 FF N1 0.70 -0.8 0.9
所有组合在timings:
#len df = 70k, 5 columns
df = pd.concat([df]*10000).reset_index(drop=True)
In [47]: %timeit (df[(df.filter(like='Value').abs() < 1).all(axis=1)])
100 loops, best of 3: 7.48 ms per loop
In [48]: %timeit (df[df.filter(regex=r'Value').abs().lt(1).all(1)])
100 loops, best of 3: 7.02 ms per loop
In [49]: %timeit (df[df.filter(like='Value').abs().lt(1).all(1)])
100 loops, best of 3: 7.02 ms per loop
In [50]: %timeit (df[(df.filter(regex=r'Value').abs() < 1).all(axis=1)])
100 loops, best of 3: 7.3 ms per loop
#len df = 70k, 5k columns
df = pd.concat([df]*10000).reset_index(drop=True)
df = pd.concat([df]*1000, axis=1)
#only for testing, create unique columns names
df.columns = df.columns.str[:-1] + [str(col) for col in list(range(df.shape[1]))]
print (df)
In [75]: %timeit ((df[(df.filter(like='Value').abs() < 1).all(axis=1)]))
1 loop, best of 3: 10.3 s per loop
In [76]: %timeit ((df[(df.filter(regex=r'Value').abs() < 1).all(axis=1)]))
1 loop, best of 3: 10.3 s per loop
In [77]: %timeit (df[df.filter(regex=r'Value').abs().lt(1).all(1)])
1 loop, best of 3: 10.4 s per loop
In [78]: %timeit (df[df.filter(like='Value').abs().lt(1).all(1)])
1 loop, best of 3: 10.1 s per loop