所以我试图抓取下图中方框内的所有音乐会:
https://i.stack.imgur.com/7QIMM.jpg https://i.stack.imgur.com/7QIMM.jpg
问题是列表仅显示前 10 个选项,直到您在该特定 div 中向下滚动到底部为止,然后它会动态显示更多选项,直到没有更多结果。我尝试按照下面的链接回答,但无法让它向下滚动以显示所有“音乐会”:
如何使用 Puppeteer 在 div 内滚动? https://stackoverflow.com/questions/52030394/how-to-scroll-inside-a-div-with-puppeteer
这是我的基本代码:
const browser = await puppeteerExtra.launch({ args: [
'--no-sandbox'
]});
async function functionName() {
const page = await browser.newPage();
await preparePageForTests(page);
page.once('load', () => console.log('Page loaded!'));
await page.goto(`https://www.google.com/search?q=concerts+near+poughkeepsie&client=safari&rls=en&uact=5&ibp=htl;events&rciv=evn&sa=X&fpstate=tldetail`);
const resultList = await page.waitForSelector(".odIJnf");
const scrollableSection = await page.waitForSelector("#Q5Vznb"); //I think this is the div that contains all the concert items.
const results = await page.$$(".odIJnf"); //this needs to be iterable to be used in the for loop
//this is where I'd like to scroll down the div all the way to the bottom
for (let i = 0; i < results.length; i++) {
const result = await (await results[i].getProperty('innerText')).jsonValue();
console.log(result)
}
}
尝试用此方法向下滚动音乐会列表。您可以继续循环,直到结果数量停止增加,或者找到您正在寻找的音乐会:
await page.evaluate(()=>{
document.querySelector("#Q5Vznb").scrollIntoView(false);
});
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)