为什么我用 beautifulSoup 刮的时候有桌子，但没有 pandas

尝试抓取条目页面转换为制表符分隔格式（主要拉出序列和 UniProt 登录号）。

当我跑步时：

url = 'www.signalpeptide.de/index.php?sess=&m=listspdb_bacteria&s=details&id=1000&listname='    
table = pd.read_html(url)
print(table)

I get:

Traceback (most recent call last):
  File "scrape_signalpeptides.py", line 7, in <module>
    table = pd.read_html(url)
  File "/Users/ION/anaconda3/lib/python3.7/site-packages/pandas/io/html.py", line 1094, in read_html
    displayed_only=displayed_only)
  File "/Users/ION/anaconda3/lib/python3.7/site-packages/pandas/io/html.py", line 916, in _parse
    raise_with_traceback(retained)
  File "/Users/ION/anaconda3/lib/python3.7/site-packages/pandas/compat/__init__.py", line 420, in raise_with_traceback
    raise exc.with_traceback(traceback)
ValueError: No tables found

于是我尝试了美丽的汤法：

import requests
import pandas as pd
import json
from pandas.io.json import json_normalize
from bs4 import BeautifulSoup

url = 'http://www.signalpeptide.de/index.php?sess=&m=listspdb_bacteria&s=details&id=1000&listname='
res = requests.get(url)
soup = BeautifulSoup(res.content, "lxml")
print(soup)

我可以看到那里有数据。有谁知道为什么我不能用 pandas.read_html 解析此页面？ 编辑1：根据下面的建议我运行了这个：

from bs4 import BeautifulSoup
import requests
s = requests.session()
s.headers['User-Agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36'
res = s.get('https://www.signalpeptide.de/index.php?sess=&m=listspdb_bacteria&s=details&id=2&listname=')
print(res)

....我将 URL 更改为全部 www、http 和 https；对于所有我得到与连接错误相关的错误，例如

urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x1114f0898>: Failed to establish a new connection: [Errno 61] Connection refused

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.signalpeptide.de', port=443): Max retries exceeded with url: /index.php?sess=&m=listspdb_bacteria&s=details&id=2&listname= (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x1114f0898>: Failed to establish a new connection: [Errno 61] Connection refused'

ConnectionRefusedError: [Errno 61] Connection refused

尝试这个：

from bs4 import BeautifulSoup as bs
import requests
import pandas as pd

url = 'http://www.signalpeptide.de/index.php?sess=&m=listspdb_bacteria&s=details&id=1000&listname='    
r = requests.get(url)

tabs = soup.find_all('table')
my_tab = pd.read_html(str(tabs[0]))
my_tab[0].drop(my_tab[0].columns[1], axis=1).drop(my_tab[0].index[0])

这应该在以“id 1000”开头的页面上输出主表。