当前位置: > 主机教程 > Centos 系统下安装百度云爬虫-爬取百度云

Centos 系统下安装百度云爬虫-爬取百度云

运行环境
  • MySQL
  • Python 2.7
  • Mysql-python

1、安装MySQL
安装依赖

yum install libaio

安装MySQL

wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
yum localinstall mysql-community-release-el7-5.noarch.rpm
yum install mysql-community-server

启动MySQL

systemctl start  mysqld

设置MySQL密码

mysql_secure_installation;

2、防火墙设置
安装iptables

yum install iptables-services

开放3306端口

vi /etc/sysconfig/iptables

添加

-A RH-Firewall-1-INPUT -m state –state NEW -m tcp -p tcp –dport 3306 -j ACCEPT
-A RH-Firewall-1-INPUT -m state –state NEW -m udp -p udp –dport 3306 -j ACCEPT

重启iptables

service iptables restart

3、安装MySQL-python

yum install MySQL-python

4、设置程序

wget https://github.com/x-spiders/baiduyun-spider/archive/master.zip
unzip master.zip
cd baiduyun-spider-master

设置连接数据库的账号密码

打开 bin/spider.py ,修改 DB_HOST、DB_PORT、DB_USER、DB_PASS

首次运行爬虫

python bin/spider.py --seed-user

运行爬虫

python bin/spider.py