Scrapy Selenium in a Jupyter Notebook
发布于2021-02-28 16:50 阅读(318) 评论(0) 点赞(8) 收藏(3)
When using Scrapy, I like to experiment with the scrapy response object inside a Jupyter notebook like this:
import requests
from scrapy.http import TextResponse
res = requests.get(my_url)
response = TextResponse(res.url, body=res.text, encoding='utf-8')
For my current project I must use the scrapy-selenium middleware, which allow you to replace the built-in scrapy request like this:
from scrapy_selenium import SeleniumRequest
yield SeleniumRequest(url=url, callback=self.parse_result)
I am wondering if there is any way to interact with the scrapy-selinium response object in a Jupyter notebook, like I have done with regular scrapy above.
解决方案
暂无回答
所属网站分类: 技术文章 > 问答
作者：黑洞官方问答小能手
链接： https://www.pythonheidong.com/blog/article/863992/a82fc5d26f7a4cc5e6e7/
来源： python黑洞网
任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任
昵称:
评论内容：(最多支持255个字符)