I would like to drop dataframe rows by time condition (ignoring date). My data contains around 100 million rows. I have around 100 columns and each column has different sampling frequency.
我准备了以下考虑到不同采样频率的代码片段:
import pandas as pd
# leave_duration=0.01 seconds
# drop_duration=0.1 seconds
i = pd.date_range('2018-01-01', periods=1000, freq='2ms')
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='3ms'))
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='0.5ms'))
df = pd.DataFrame({'A': range(len(i))}, index=i)
df=df.sort_index()
print(df)
# drop by duration....
在这个简单的示例中,数据持续约 1 秒,并具有 3 个不同的采样频率。目标是删除持续时间(例如)0.1 秒的行并留下持续时间(例如)0.01 秒的行。我怎样才能用一条线做到这一点?
by df=df.loc['2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500 ']
你将得到新的 df ,数据介于两者之间2018-01-01 00:00:00.000000
and 2018-01-01 00:00:00.000500
现在您可以应用过滤器来查找所需的日期
import pandas as pd
# leave_duration=0.01 seconds
# drop_duration=0.1 seconds
i = pd.date_range('2018-01-01', periods=1000, freq='2ms')
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='3ms'))
i=i.append(pd.date_range('2018-01-01', periods=1000, freq='0.5ms'))
df = pd.DataFrame({'A': range(len(i))}, index=i)
df=df.sort_index()
print(df)
#filter data between 2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500
df=df.loc['2018-01-01 00:00:00.000000 ':'2018-01-01 00:00:00.000500 ']
print(df)
输出:
应用数据过滤器之前
A
2018-01-01 00:00:00.000000 0
2018-01-01 00:00:00.000000 2000
2018-01-01 00:00:00.000000 1000
2018-01-01 00:00:00.000500 2001
2018-01-01 00:00:00.001000 2002
... ...
2018-01-01 00:00:02.985000 1995
2018-01-01 00:00:02.988000 1996
2018-01-01 00:00:02.991000 1997
2018-01-01 00:00:02.994000 1998
2018-01-01 00:00:02.997000 1999
[3000 rows x 1 columns]
应用日期过滤器后:
A
2018-01-01 00:00:00.000000 0
2018-01-01 00:00:00.000000 2000
2018-01-01 00:00:00.000000 1000
2018-01-01 00:00:00.000500 2001
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)