北邮python爬虫学堂在线

爬取学堂在线动态页面

【实例内容】包括源文件、说明、可执行程序

采集的数据保存到xuetang.csv文件中了，如下图：

from scrapy import cmdline
cmdline.execute（“scrapy crawl xuetang“.split（））

 属性            大小     日期    时间   名称
----------- ---------  ---------- -----  ----

     文件        174  2020-11-21 16:15  xuetang\.idea\inspectionProfiles\profiles_settings.xml

     文件        273  2020-11-21 16:15  xuetang\.idea\modules.xml

     文件        291  2020-11-21 16:15  xuetang\.idea\xuetang.iml

     文件         75  2020-11-21 17:43  xuetang\begin.py

     文件        257  2020-11-21 16:14  xuetang\scrapy.cfg

     文件        209  2020-11-21 21:09  xuetang\xuetang\items.py

     文件       3650  2020-11-21 16:14  xuetang\xuetang\middlewares.py

     文件        450  2020-11-21 23:00  xuetang\xuetang\pipelines.py

     文件       2962  2020-11-21 22:10  xuetang\xuetang\settings.py

     文件       2145  2020-11-21 22:54  xuetang\xuetang\spiders\spider.py

     文件        161  2020-11-13 20:02  xuetang\xuetang\spiders\__init__.py

     文件       2086  2020-11-21 22:54  xuetang\xuetang\spiders\__pycache__\spider.cpython-37.pyc

     文件        164  2020-11-21 18:35  xuetang\xuetang\spiders\__pycache__\__init__.cpython-37.pyc

     文件          0  2020-11-13 20:02  xuetang\xuetang\__init__.py

     文件        418  2020-11-21 21:16  xuetang\xuetang\__pycache__\items.cpython-37.pyc

     文件        958  2020-11-21 21:50  xuetang\xuetang\__pycache__\pipelines.cpython-37.pyc

     文件        325  2020-11-21 22:11  xuetang\xuetang\__pycache__\settings.cpython-37.pyc

     文件        156  2020-11-21 18:35  xuetang\xuetang\__pycache__\__init__.cpython-37.pyc

     文件       3398  2020-11-21 22:54  xuetang\xuetang.csv

     目录          0  2020-11-21 22:54  xuetang\xuetang\spiders\__pycache__

     目录          0  2020-11-21 16:15  xuetang\.idea\inspectionProfiles

     目录          0  2020-11-21 22:54  xuetang\xuetang\spiders

     目录          0  2020-11-21 22:11  xuetang\xuetang\__pycache__

     目录          0  2020-11-21 16:15  xuetang\.idea

     目录          0  2020-11-21 23:00  xuetang\xuetang

     目录          0  2020-11-21 20:46  xuetang

----------- ---------  ---------- -----  ----

                18152                    26