我正在尝试提取内容href
在班上a
,里面是<td class="DataZone">
。它适用于下面的示例
from bs4 import BeautifulSoup
text = '''
<td class="DataZone"><div id="Content_CA_DI_0_DataZone">
<div style="font:bold 8pt 'Courier New';letter-spacing:-1px">
<a href="Browse-A">A</a> <a href="Browse-B">B</a> <a href="Browse-C">C</a> <a href="Browse-D">D</a>
</div>
</div></td>
'''
soup = BeautifulSoup(text, 'html.parser')
[tag.attrs['href'] for tag in soup.select('td.DataZone a')]
,结果是['Browse-A', 'Browse-B', 'Browse-C', 'Browse-D']
。当我把它应用到真实的时候url https://www.thefreedictionary.com/,不幸的是它不起作用
import requests
session = requests.Session()
from bs4 import BeautifulSoup
url = 'https://www.thefreedictionary.com'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0'}
r = session.get(url, headers = headers)
soup = BeautifulSoup(r.content, 'html.parser')
[tag.attrs['href'] for tag in soup.select('td.DataZone a')]
它返回一个错误
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-12-0a06dde2d97b> in <module>
4 soup = BeautifulSoup(r.content, 'html.parser')
5
----> 6 [tag.attrs['href'] for tag in soup.select('td.DataZone a')]
<ipython-input-12-0a06dde2d97b> in <listcomp>(.0)
4 soup = BeautifulSoup(r.content, 'html.parser')
5
----> 6 [tag.attrs['href'] for tag in soup.select('td.DataZone a')]
KeyError: 'href'
显然,url的来源与示例类似
您能解释一下为什么会出现这样的错误吗?
Update:这对我来说很奇怪[x['href'] for x in soup.select('td.DataZone a[href^=Browse]')]
工作正常,但不是[x['href'] for x in soup.select('td.DataZone a')]
。还请您详细说明一下这个问题。