python3.6爬虫需要安装的模块

  • 时间:
  • 浏览:0

                链接:https://pan.baidu.com/s/1oAh2Dse 密码:okk0

            不可能 下载失败

            pip3 install beautifulsoup4

            http://npm.taobao.org/mirrors/chromedriver/

            pip3 install redis

            pip3 install wheel

        chromedriver

            2. lxml

            pip3 install pymysql

        beautifulsoup4(依赖lxml)

    框架:

            pip3 install  selenium

            pip3 install jupyter

            首先安装

        requests

        jupyter(强大的记事本)

            pip3 install pymongo

        django

                https://pypi.python.org/pypi/pyOpenSSL#downloads

                软件和经常突然出现难题修复脚本,

    D:\Program Files\MongoDB\Server\3.6\bin>mongod --bind_ip 0.0.0.0 --logpath C:\data\logs\logs.txt  --logappend --dbpath C:\data\db --port 27017 --serviceName "mongodb" --serviceDisplayName "mongodb" --install

            4. Twisted

            http://phantomjs.org/

本文转自 天道酬勤VIP 51CTO博客,原文链接:http://blog.51cto.com/tdcqvip/2073149

    存储库:

        pyquery

            jupyter notebook(启动服务)

        phantomjs(无界面浏览器)

        pymysql

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

            https://pypi.python.org/pypi/lxml

            pip3 install django

        pymongo

            下载,但会 配置环境变量

                pip3 install scrapy

        selenium

        urllib

        lxml

    还要安装的库

            下载驱动,贴到 配置好的环境变量下

        redis

            1. wheel

启动服务:

            用下面的法律措施安装

图形化管理页面软件

            6. Scrapy

            3. PyOpenssl

        scrapy

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/

MongoDB 的安装:

    建立日志文件在c盘创建C:\data\db文件夹和C:\data\logs\logs.txt文件

                链接:https://pan.baidu.com/s/1mj2VjxI 密码:ls54

        flask(代理IP的库)

            pip3 install flask

    http://blog.51cto.com/tdcqvip/2072845

参考上篇博客:    

  内置库

配置可视化服务:

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/pywin32-221.win-amd64-py3.6.exe/download

redis数据库安装:

            pip3 install lxml

        re

                pip install wheel

            但会 载pip3 install 下载的文件路径和名字

        PySpider

            pip3 install requests

            5. Pywin32

            pip3 install PySpider

    https://download.robomongo.org/1.2.1/windows/robo3t-1.2.1-windows-x86_64-3e400a65.exe

    D:\Program Files\MongoDB\Server\3.6\bin>"D:\Program Files\MongoDB\Server\3.6\bin\mongod.exe" --dbpath ../data/db

            pip3 install pyquery