59 lines
2.3 KiB
Markdown
59 lines
2.3 KiB
Markdown
|
crawler:
|
|||
|
|
# Python 可执行文件路径（Windows 建议指向 python.exe；如已在 PATH，可直接用 "python"）
|
|||
|
|
pythonPath: F:/Environment/Conda/envs/schoolNewsCrawler/python.exe
|
|||
|
|
# 爬虫脚本根目录（NewsCrawlerTask 的工作目录）
|
|||
|
|
basePath: F:/Project/schoolNews/schoolNewsCrawler
|
|||
|
|
|
|||
|
|
# 下面为原有的定时任务清单（保持不变，仅修正到正确文件）
|
|||
|
|
crontab:
|
|||
|
|
items:
|
|||
|
|
- name: 人民日报新闻爬取
|
|||
|
|
methods:
|
|||
|
|
- name: 关键字搜索爬取
|
|||
|
|
clazz: newsCrewerTask
|
|||
|
|
excuete_method: execute
|
|||
|
|
path: crawler/RmrbSearch.py
|
|||
|
|
params:
|
|||
|
|
- name: query
|
|||
|
|
description: 搜索关键字
|
|||
|
|
type: String
|
|||
|
|
value: ""
|
|||
|
|
required: true
|
|||
|
|
- name: total
|
|||
|
|
description: 总新闻数量
|
|||
|
|
type: Integer
|
|||
|
|
value: 10
|
|||
|
|
required: true
|
|||
|
|
- name: 排行榜爬取
|
|||
|
|
clazz: newsCrewerTask
|
|||
|
|
excuete_method: execute
|
|||
|
|
path: crawler/RmrbHotPoint.py
|
|||
|
|
- name: 往日精彩头条爬取
|
|||
|
|
clazz: newsCrewerTask
|
|||
|
|
excuete_method: execute
|
|||
|
|
path: crawler/RmrbTrending.py
|
|||
|
|
params:
|
|||
|
|
- name: startDate
|
|||
|
|
description: 开始日期
|
|||
|
|
type: String
|
|||
|
|
value: ""
|
|||
|
|
required: false
|
|||
|
|
- name: endDate
|
|||
|
|
description: 结束日期
|
|||
|
|
type: String
|
|||
|
|
value: ""
|
|||
|
|
required: false
|
|||
|
|
- name: yesterday
|
|||
|
|
description: 是否是昨天
|
|||
|
|
type: Boolean
|
|||
|
|
value: true
|
|||
|
|
|
|||
|
|
|
|||
|
|
1. 改成系统表配置。
|
|||
|
|
2. 通过system模块的config配置crawler的基本信息
|
|||
|
|
3. 创建定时任务meta表存放定时任务的元数据
|
|||
|
|
4. 创建定时任务meta邮件人员模板表，存放定时任务的邮件人员模板
|
|||
|
|
5. 创建定时任务邮件接收人员表，存放定时任务的邮件接收人员
|
|||
|
|
6. 修改dto、service、mapper、controller等，实现定时任务meta的查询。定时任务meta邮件人员模板的增删改查。定时任务邮件接收人员的增删改查。
|
|||
|
|
7. 修改定时任务的param等内容，实现定时任务的执行。
|
|||
|
|
8. 修改前端接口请求逻辑，实现定时任务和邮件人员的增删改查。
|