我需要从 html 解析一个表,该表在较大的表中嵌套有其他表。如下所示pd.read_html
,每个嵌套表都会被解析,然后作为行“插入”/“连接”。
我希望将这些嵌套表分别解析为自己的表pd.DataFrames
并将插入的对象作为相应列的值。
如果这是not可能的话,将嵌套表的原始 html 作为字符串放在相应的位置就可以了。
测试代码:
import pandas as pd
df_up = pd.read_html("up_pf00344.test.html", attrs = {'id': 'results'})
Screenshot of output:
Screenshot of table as rendered in html:
文件链接:https://gist.github.com/smsaladi/6adb30efbe70f9fed0306b226e8ad0d8#file-up_pf00344-test-html-L62 https://gist.github.com/smsaladi/6adb30efbe70f9fed0306b226e8ad0d8#file-up_pf00344-test-html-L62
你不能使用read_html https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.read_html.html读取嵌套表,但您可以使用自己的 html 阅读器并使用read_html
对于表格单元格:
import pandas as pd
import bs4
with open('up_pf00344.test.html') as f:
html = f.read()
soup = bs4.BeautifulSoup(html, 'lxml')
results = soup.find(attrs = {'id': 'results'})
# get first visible header row as dataframe headers
for row in results.thead.find_all('tr'):
if 'display:none' not in row.get('style',''):
df = pd.DataFrame(columns=[col.get_text() for col in row.find_all('th')])
break
# append all table rows to dataframe
for row in results.tbody.find_all('tr', recursive=False):
if 'display:none' in row.get('style',''):
continue
df_row = []
for col in row.find_all('td', recursive=False):
table = col.find_all('table')
df_row.append(pd.read_html(str(col))[0] if table else col.get_text())
df.loc[len(df)] = df_row
的结果df.iloc[0].map(type)
:
<class 'str'>
Entry <class 'str'>
Organism <class 'str'>
Protein names <class 'str'>
Gene names <class 'str'>
Length <class 'str'>
Cross-reference (Pfam) <class 'str'>
Cross-reference (InterPro) <class 'str'>
Taxonomic lineage IDs <class 'str'>
Subcellular location [CC] <class 'str'>
Signal peptide <class 'str'>
Transit peptide <class 'str'>
Topological domain <class 'pandas.core.frame.DataFrame'>
Transmembrane <class 'pandas.core.frame.DataFrame'>
Intramembrane <class 'pandas.core.frame.DataFrame'>
Sequence caution <class 'str'>
Caution <class 'str'>
Taxonomic lineage (SUPERKINGDOM) <class 'str'>
Taxonomic lineage (KINGDOM) <class 'str'>
Taxonomic lineage (PHYLUM) <class 'str'>
Cross-reference (RefSeq) <class 'str'>
Cross-reference (EMBL) <class 'str'>
e <class 'str'>
奖励:因为您的表行有一个id
,您可以使用它作为数据框的索引df.loc[row.get('id')] = df_row
代替df.loc[len(df)] = df_row
.
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)